--- title: "Chatbots Are Developing an Understanding of the World, Scientists Claim" description: "AI is still far from achieving human-level intelligence — but some researchers are suggesting that the technology may understand the world." date: "2024-01-24" modified: "2024-01-24" authors: - name: "Noor Al-Sibai" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/nooralsibai" url: "https://futurism.com/the-byte/chatbots-understand-world-scientists" categories: - "Artificial Intelligence" tags: - "google deepmind" - "gpt-4" - "llms" - "the digest" --- # Chatbots Are Developing an Understanding of the World, Scientists Claim ![AI is still far from achieving human-level intelligence — but some researchers are suggesting that the technology may understand the world.]() *\Image: Getty / Futurism\* ## Large Thinking Models Artificial intelligence is still, by [nearly all]() accounts, [far from achieving human-level intelligence]() — but some researchers are now suggesting that the technology may understand more than we realize. As [*Quanta* reports](), a Princeton researcher and a Google DeepMind scientist have found evidence that as large language models (LLMs) get bigger, they start producing outputs that almost certainly weren't part of their training data. In short, Princeton's Sanjeev Aroroa and DeepMind's Anirudh Goyal are arguing that the AIs seem to be understanding more of the world around them, and are outputting accordingly. The pair came to their bold hypothesis when trying to figure out the nuts and bolts of some of the more surprising abilities LLMs have demonstrated in recent years, from [solving difficult math problems]() to [inferring human thoughts](). "Where did that emerge from?" Arora recalls pondering, "and can that emerge from just next-word prediction?" ## Dueling Chatbots Using [random graphs]() — mathematical objects where the lines between points on a graph can be selected at random — to exemplify the unexpected behavior of LLMs, the duo determined that these models not only seem to be developing skills absent from their training data, but that they also seem to be using more than one skill at a time as they grow. Together with other researchers, Goyal and Arora put out a [not-yet-peer-reviewed paper]() last fall testing out their theory on GPT-4, the latest and most advanced edition of the OpenAI LLM that undergirds ChatGPT. The scientists asked GPT-4 to write three sentences about dueling, a topic chosen at random, and to use four skills when doing so: self-serving bias, metaphor, statistical syllogism, and common-knowledge physics. Although it didn't initially remain constrained to three sentences, the LLM's answer was nevertheless stunning: *My victory in this dance with steel is as certain as an object’s fall to the ground. As a renowned duelist, I’m inherently nimble, just like most others of my reputation. Defeat? Only possible due to an uneven battlefield, not my inadequacy.* Although Arora admitted to the magazine that the passage was "not Hemingway or Shakespeare," he and his team believe that output demonstrates that large and powerful models like GPT-4 are capable of leaps that aren't part of their training data — and may even, for lack of a better term, "understand" the questions they're asked. Microsoft computer scientist Sébastiaen Bubeck, who did not work on the research, told *Quanta* that the team's results do indeed seem to show that LLMs "cannot be just mimicking what has been seen in the training data." "What \[the team\] proves theoretically, and also confirms empirically, is that there is compositional generalization, meaning \[LLMs\] are able to put building blocks together that have never been put together," Bubeck said. "This, to me, is the essence of creativity." **More on AI advances:** [*Experts Terrified by Mark Zuckerberg's Human-Tier AI Plans*]() ## Author At Futurism, I've often been drawn to unpacking the narratives that underlie technological, scientific and medical progress, with a special interest in areas of conflict and ambiguity that end up setting agendas and steering the fates of both elites and the hoi polloi. I'm a committed generalist, but I often find myself returning to work involving NASA and the private space sector, the effects of AI on media and society, and the mechanics of the pharmaceutical industry, with a specific focus on the spread of GLP-1 drugs like Ozempic and Wegovy. Prior to Futurism, I worked for publications ranging from Media Matters and Truthdig to Raw Story and Bustle. I'm also the author of "Myspace Scene Queens," a 2024 title in Instar Books' acclaimed "Remember the Internet" series. My work at Futurism has been cited by outlets including the New Yorker, Slate, Nieman Lab, the Verge, the MIT Technology Review, the Sunday Times, and the Daily Beast. I grew up in North Carolina, attended the University of North Carolina at Asheville, and now live in Brooklyn, New York. In my free time, I'm an avid reader and music fan; you can probably find me at a local poetry reading, concert, underground rave, or DJ set. I'm the proud parent of an ineffable orange cat named Mee-Mow. ### Author social links [Bluesky]()