--- title: "CEO of OpenAI Says Making Models Bigger Is Already Played Out" description: "Each iteration of OpenAI's GPT has been way bigger than the last. But the era of simply scaling up AIs may already be at an end, according to Sam Altman." date: "2023-04-18" modified: "2023-04-18" authors: - name: "Frank Landymore" job_title: "Contributing Writer" link: "https://futurism.com/authors/flandymore" url: "https://futurism.com/the-byte/ceo-openai-bigger-models-already-played-out" categories: - "Artificial Intelligence" - "OpenAI" tags: - "chatgpt" - "gpt-4" - "OpenAI" - "the digest" --- # CEO of OpenAI Says Making Models Bigger Is Already Played Out ![Each iteration of OpenAI's GPT has been way bigger than the last. But the era of simply scaling up AIs may already be at an end, according to Sam Altman.]() *\Image: Drew Angerer via Getty / Futurism\* ## Big News Large language models (LLMs) like OpenAI's are getting bigger and better with each new iteration. Last month, the company unveiled its long awaited [GPT-4](), a beefy and substantially larger upgrade to its chatbot's underlying LLM that's so impressive it immediately inspired a massive group of experts and tech CEOs — including Elon Musk — to sign [a letter calling for a moratorium on experimenting on AI]() more advanced than OpenAI's latest model. With results like that, you'd think OpenAI would want to keep digging its heels in to push out even larger models than before. But its CEO Sam Altman is now cautioning that the age of simply scaling up AI to make them more powerful may already be over. From here, the approach will have to be decidedly less size-focused. "I think we're at the end of the era where it's going to be these, like, giant, giant models," Altman said at an MIT event last week, [as quoted by ]()*[Wired]().* "We'll make them better in other ways." ## Diminishing Returns Generally speaking, when it comes to AIs and LLMs in particular, bigger has been better. OpenAI's first landmark model GPT-2, released in 2019, boasted around 1.5 billion parameters, the adjustable variables that connect the neurons of an AI that help it "learn" and refine itself based on its input data. By the time GPT-3 rolled out the next year, it boasted a whopping 175 billion parameters, and by GPT-4, one trillion, according to some [outside estimates](). But crucially, as *Wired* notes, OpenAI itself has not shared GPT-4's exact size, which is perhaps emblematic of the company's pivot away from simply enlarging its models. While each of these increases in parameters has been met with an uptick in the capabilities of the GPT models, this approach may now be yielding diminishing returns, according to the [findings of OpenAI's own technical report]() — like how you can't just keep adding more cylinders to a car engine to make it more powerful. ## Still Heading Up It's worth mentioning that Altman conceded that parameter counts may trend up regardless — diminishing returns, after all, are still returns — but he maintains that the metric gets "way too much focus." "This reminds me a lot of the gigahertz race in chips in the 1990s and 2000s, where everybody was trying to point to a big number," Altman said, [as quoted by *TechCrunch*](). "What we want to deliver to the world is the most capable and useful and safe models," he added. "We are not here to jerk ourselves off about parameter count." **More on AI:** *[Google Surprised When Experimental AI Learns Language It Was Never Trained On]()* ## Author At Futurism, my work has often centered on bringing a sense of clarity and insight to complex topics ranging from the regulation of emerging technologies to the esoteric ideologies of Silicon Valley executives, while striving not to lose the poetic sense of awe inspired by often-obscure fields like astrophysics and quantum computing. I broke the story of CNET using AI to produce articles that turned out to be riddled with factual errors and plagiarism — a dam-breaking inflection point, as I've reported, that's inspired copycats and endless discourse while beguiling stakeholders ranging from tech giants to purveyors of spam around the web. My work at Futurism has been cited by publications including CBS News, the Los Angeles Times, Vice, Gizmodo, Engadget, the Verge, and Vanity Fair. I grew up in locales ranging from India to China, and now live in the exotic suburbs of Virginia. In my free time, I'm an avid reader of weird sci-fi literature, an aficionado of East Asian cinema, and, regrettably, a relapsed gamer. Allegedly, I’m working on a debut novel, currently untitled. ### Author social links [Bluesky]()