--- title: "OpenAI Reportedly Hitting Law of Diminishing Returns as It Pours Computing Resources Into AI" description: "It seems that OpenAI has hit the wall when it comes to computing power for its much-hyped large language models (LLMs)." date: "2024-11-12" modified: "2024-11-12" authors: - name: "Noor Al-Sibai" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/nooralsibai" url: "https://futurism.com/the-byte/openai-diminishing-returns" categories: - "Artificial Intelligence" - "OpenAI" tags: - "ilya sutskever" - "llms" - "OpenAI" - "the digest" --- # OpenAI Reportedly Hitting Law of Diminishing Returns as It Pours Computing Resources Into AI ![It seems that OpenAI has hit the wall when it comes to computing power for its much-hyped large language models (LLMs).]() *\Image: Andrej Sokolow / picture alliance via Getty\* ## Age of Wonder Reports are emerging that OpenAI is hitting a wall as it continues to pour more computing power into its much-hyped large language models (LLMs) like ChatGPT in a bid for more intelligent outputs. AI models need loads of training data and computing power to operate at scale. But in an [interview with *Reuters*](), recently-exited OpenAI cofounder Ilya Sutskever claimed that the firm's recent tests trying to scale up its models suggest that those efforts have plateaued. "The 2010s were the age of scaling, now we're back in the age of wonder and discovery once again," Sutskever, a [staunch believer]() in the forthcoming arrival of so-called artificial general intelligence (AGI) or human-level AI, told *Reuters*. "Everyone is looking for the next thing." While it's unclear what exactly that "next thing" may be, Sutskever's admission — which comes, notably, just under a year after he [moved to oust]() OpenAI CEO Sam Altman and was [subsequently sidelined]() until his [eventual departure]() — seems to dovetail with other recent claims and conclusions: that AI companies, and OpenAI specifically, are butting up against the law of diminishing returns. ## Bounded Leaps Over the weekend, [*The Information* reported]() that with each new flagship model, OpenAI is seeing a slowdown in the sort of "leaps" users have come to expect in the wake of its game-changing [ChatGPT release]() in December 2022. This slowdown seems to test the core belief at the center of the argument for AI scaling: that as long as there's ever more data and computing power to feed the models — which is a big "if," given that firms have already [run out of training data]() and are [eating up electricity]() at unprecedented rates — those models will continue to grow or "scale" at a consistent rate. Responding to this latest news from *The Information*, data scientist [Yam Peleg teased on X]() that another cutting-edge AI firm had "reached an unexpected HUGE wall of diminishing returns trying to brute-force better results by training longer & using more and more data." While Peleg's commentary could just be gossip, researchers *have* been [warning for years]() now that LLMs would eventually hit this wall. Given the [insatiably high demand]() for [powerful AI chips]() — and that firms are now [training their models]() on [AI-generated data]() — it doesn't take a machine learning expert to wonder whether the low-hanging fruit is running out. "I think it is safe to assume that all major players have reached the limits of training longer and collecting more data already," [Peleg continued](). "It is all about data quality now.. which takes time." **More on AI crises:** [*Sam Altman Says the Main Thing He’s Excited About Next Year Is Achieving AGI*]() ## Author At Futurism, I've often been drawn to unpacking the narratives that underlie technological, scientific and medical progress, with a special interest in areas of conflict and ambiguity that end up setting agendas and steering the fates of both elites and the hoi polloi. I'm a committed generalist, but I often find myself returning to work involving NASA and the private space sector, the effects of AI on media and society, and the mechanics of the pharmaceutical industry, with a specific focus on the spread of GLP-1 drugs like Ozempic and Wegovy. Prior to Futurism, I worked for publications ranging from Media Matters and Truthdig to Raw Story and Bustle. I'm also the author of "Myspace Scene Queens," a 2024 title in Instar Books' acclaimed "Remember the Internet" series. My work at Futurism has been cited by outlets including the New Yorker, Slate, Nieman Lab, the Verge, the MIT Technology Review, the Sunday Times, and the Daily Beast. I grew up in North Carolina, attended the University of North Carolina at Asheville, and now live in Brooklyn, New York. In my free time, I'm an avid reader and music fan; you can probably find me at a local poetry reading, concert, underground rave, or DJ set. I'm the proud parent of an ineffable orange cat named Mee-Mow. ### Author social links [Bluesky]()