OpenAI president Greg Brockman explains how the company allocates GPUs internally: ‘pain and suffering’

“Pain and suffering” is how OpenAI’s Greg Brockman describes the internal battle for GPU allocation.

Errich Petersen/Getty Images for SXSW

OpenAI’s president described the internal battle for GPU allocation as “pain and suffering.”
Greg Brockmain said compute drives the productivity of entire teams — and the stakes are high.
He also explained how OpenAI shuffles its GPUs internally.

OpenAI’s president, Greg Brockman, said deciding which teams get graphic processing units inside the company is an exercise in “pain and suffering.”

Brockman said on an episode of the “Matthew Berman” podcast published Thursday that managing the crucial resource is emotional and exhausting.

“It’s so hard because you see all these amazing things, and someone comes and pitches another amazing thing, and you’re like, yes, that is amazing,” he said.

He explained that the company divides its computing power between research and applied products. The company’s chief scientist and research head decide allocations within the research side. Senior leadership — CEO Sam Altman and the CEO of applications, Fidji Simo — decide the overall split between research and applied teams.

At the operational level, a small internal team focuses on shuffling GPU assignments, including Kevin Park, who is responsible for redistributing hardware as projects wind down.

“You go to him and you’re just like, ‘OK, like we need this many more GPUs for this project that just came up,'” Brockman said. “And he’s like, ‘All right, there’s like these five projects that are sort of winding down,'” he added.

The internal GPU shuffle reflects the broader scarcity that OpenAI has warned about for months. Brockman said compute drives the productivity of entire teams — and the stakes are high.

“People really care,” he said. “The energy and emotion around, ‘Do I get my compute or not?’ is something you cannot understate.”

Brockman and OpenAI did not respond to a request for comment from Business Insider.

The race for GPUs

OpenAI has been vocal about its insatiable demand for computing power.

“Every time we get more GPUs, they immediately get used,” OpenAI’s chief product officer, Kevin Weil, said on an episode of the “Moonshot” podcast published in August.

Weil said the need for compute is simple: “The more GPUs we get, the more AI we’ll all use.” He highlighted that adding bandwidth made the explosion of video possible.

Altman said last month that OpenAI is launching “new compute-intensive offerings.” Because of the costs involved, some features will initially be limited to Pro subscribers, while certain new products will have extra fees, he added.

Altman framed the push as an experiment in stretching AI infrastructure to its limits: “We also want to learn what’s possible when we throw a lot of compute, at today’s model costs, at interesting new ideas,” he wrote on X.

Other tech giants have also been blunt about their appetite for GPUs.

Mark Zuckerberg said on an episode of the “Access” podcast published last month that Meta is making “compute per researcher” a competitive advantage. He said the company is outspending rivals on GPUs and the custom infrastructure needed to power them.

Read the original article on Business Insider

Content Accuracy: Keewee.News provides news, lifestyle, and cultural content for informational purposes only. Some content is generated or assisted by AI and may contain inaccuracies, errors, or omissions. Readers are responsible for verifying the information. Third-Party Content: We aggregate articles, images, and videos from external sources. All rights to third-party content remain with their respective owners. Keewee.News does not claim ownership or responsibility for third-party materials. Affiliate Advertising: Some content may include affiliate links or sponsored placements. We may earn commissions from purchases made through these links, but we do not guarantee product claims. Age Restrictions: Our content is intended for viewers 21 years and older where applicable. Viewer discretion is advised. Limitation of Liability: By using Keewee.News, you agree that we are not liable for any losses, damages, or claims arising from the content, including AI-generated or third-party material. DMCA & Copyright: If you believe your copyrighted work has been used without permission, contact us at dcma@keewee.news. No Mass Arbitration: Users agree that any disputes will not involve mass or class arbitration; all claims must be individual.