--- title: "There’s a Humongous Problem With AI Models: They Need to Be Entirely Rebuilt Every Time They’re Updated" description: "New research shows that AI models need to be completely retrained to learn new concepts — which is an expensive problem for AI companies." date: "2024-08-24" modified: "2024-08-24" authors: - name: "Maggie Harrison Dupré" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/mharrison" url: "https://futurism.com/the-byte/ai-models-rebuilding-problem" categories: - "Artificial Intelligence" tags: - "ai" - "ai training" - "deep learning" - "the digest" --- # There’s a Humongous Problem With AI Models: They Need to Be Entirely Rebuilt Every Time They’re Updated ![New research shows that AI models need to be completely retrained to learn new concepts — which is an expensive problem for AI companies.]() *\Image: Getty / Futurism\* ## Learning Gaze A new study highlights a glaring hole in AI models' ability to learn new information: turns out, they can't! According to the study, conducted by a team of scientists at Canada's University of Alberta and [published this week in the journal *Nature*](), AI algorithms trained via deep learning — in short, AI models like large language models built by finding patterns in heaps of data — fail to work in "continual learning settings," or when new concepts are introduced to a model's existing training. In other words, if you want to teach an existing deep learning model something new, you'll likely have to retrain it from the ground up — otherwise, according to the research, the artificial neurons in their proverbial minds will sink to a value of zero. This results in a loss of "plasticity," or their ability to learn at all. "If you think of it like your brain, then it'll be like 90 percent of the neurons are dead," University of Alberta computer scientist and lead study author Shibhansh Dohare [told *New Scientist*](). "There's just not enough left for you to learn." And training advanced AI models, as the researchers point out, is a cumbersome and wildly expensive process — making this a major financial obstacle for AI companies, which burn through a ton of cash as it is. "When the network is a large language model and the data are a substantial portion of the internet," reads the study, "then each retraining may cost millions of dollars in computation." ## Obstacles Course This phenomenon of plasticity loss is also a major moat between current AI models and the imagined "artificial general intelligence," or a theoretical AI that would be considered generally as intelligent as humans. After all, in human terms, this would be like if we had to fully reboot our brains from scratch every time we took a new college course, lest we nuke most of our neurons. If there's any bright spot for AI companies? Excitingly, the study authors were able to create an algorithm with the power to randomly revive certain damaged or "dead" AI neurons, which showed some success in countering the plasticity problem. Still, as it stands, a practical solution is still out of reach. "A solution to continual learning is literally a billion-dollar question," Dohare told *New Scientist*. "A real, comprehensive solution that would allow you to continuously update a model would reduce the cost of training these models significantly." **More on AI training:** [*When AI Is Trained with AI-Generated Data, It Starts Spouting Gibberish*]() ## Author At Futurism, I've reported extensively on the rise of AI as a cultural and business force shaping the media industry, and more broadly how those dynamics are changing how we all consume and share information and relate to one another. I'm also fascinated by public health policy and ethics, the role of emerging tech in politics and governance — and the powerful people and forces at those intersections — climate change, and the environment. My investigation on Sports Illustrated's use of AI-generated authors with fictional biographies won a 2024 Mirror Award for "Best Story on Media Coverage of Artificial Intelligence in Journalism and the Media" from Syracuse University's SI Newhouse School, I contributed to Niemen Lab's 2025 Predictions for Journalism series, and I've discussed my work for Futurism during appearances on NPR, CNN, the BBC, the CBC, and more. I grew up in rural Pennsylvania and attended the University of Massachusetts Amherst, where I played Division I field hockey for the Minutewomen as a midfielder. Since then, I've lived in New Orleans, Louisiana and Manhattan, New York. I spend my free time running, reading, perusing archival fashion, and searching for the world’s best negroni. I also have a debonair tuxedo cat, Westley, who's named after "The Princess Bride." ### Author social links [Bluesky]()