--- title: "Political Pollsters Are Trying to Save Money by Polling AI Instead of Real People, and It’s Going About as Well as You’d Expect" description: "Pollsters are surveying AI instead of real people to save time and money — but the responses, per new research, are not up to snuff." date: "2025-08-23" modified: "2025-08-23" authors: - name: "Noor Al-Sibai" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/nooralsibai" url: "https://futurism.com/ai-polling-inaccuracy" categories: - "Artificial Intelligence" tags: - "artificial intelligence" - "politics" - "polling" - "surveys" --- # Political Pollsters Are Trying to Save Money by Polling AI Instead of Real People, and It’s Going About as Well as You’d Expect ![Pollsters are surveying AI instead of real people to save time and money — but the responses, per new research, are not up to snuff.]() *\Image: Getty / Futurism\* As if the institution of political polling weren't [already fraught enough](), pollsters are now [surveying AI instead of real people]() to cut costs and save time. As new research demonstrates, AI is [clearly not up to the job]() — but that probably won't stop any [firms who've bought in]() to [continue doing so](). In a [white paper]() about the topic for the survey platform Verasight, data journalist G. Elliott Morris found, when comparing 1,500 "synthetic" survey respondents and 1,500 real people, that large language models (LLMs) were overall very bad at reflecting the views of actual human respondents. Using six OpenAI models — GPT-4.1, GPT-4.1 nano, GPT-4.1 mini, GPT-4o, GPT-4o mini, and o4-mini — Morris instructed each LLM to respond as various demographics. In one example, the researcher prompted the LLMs to respond as a white 61-year-old woman in Florida who makes between $50,000 and $75,000 per year and who considers herself a moderate voter. Using typical real-world political survey questions, the LLMs were asked things like "Do you approve or disapprove of the way Donald Trump is handling his job as president?" and given a five-point scale ranging from "strongly approve," "slightly approve," "slightly disapprove," strongly disapprove" and "don't know/not sure." The results weren't exactly inspiring. The worst-performing model, which was not specified, was 23 points off from the real respondents overall, while the best-performing model, GPT-4o-mini, was 4 points off. And the more closely Morris zoomed in, the worse things looked. As this graphic from the study shows, even the "voters" generated by the best-performing model, 4o-mini, veered further from reality as they were instructed to respond as groups who are less well represented in the United States population, like Black, Asian and Pacific Islander respondents. ![A graph visualizing the difference between human responses minus AI-generated responses, with the modeled percentage and true percentage of each group giving each response category, for each race/ethnic and age group in the data.]() For any pollster looking to do a good job, this is a big deal. Imagine, for instance, a presidential campaign crafting its messaging for Black voters using the data above. When it comes to that cohort's disapproval rating of Trump, there was a 15 percentage point difference between what actual people responded versus what 4o-mini predicted, with the AI significantly exaggerating that bloc's disapproval rate for Trump. Obviously, any campaign that used only that AI-generated data would miss the mark — instead of looking at the views of real respondents, it would be looking at a funhouse mirror reflection of a demographic cooked up by a language model with no access to actual data. "The performance of our 'synthetic sample' is too poor to be useful for all of our research questions," Morris wrote. "In computing overall population proportions, the technique above produces error rates at a minimum of several percentage points, too large to tolerate in academic, political, and most market-research contexts." "Synthetic samples generate such high errors at the subgroup level," he continued, "that we do not trust them at all to represent key groups in the population." These results, while not entirely unexpected, fly in the face of the recent push to use AI-generated responses in political polling regardless of accuracy. One AI polling startup, Aaru, [told *Semafor*]() after last November's presidential election that even though it incorrectly predicted Kamala Harris would win, its methods were, somehow, still superior to traditional polling. "A coin flip is a coin flip. 53-47 is not significantly different from 48-52," Aaru cofounder Cameron Fink told the website. "Statistically speaking, we’re within the margin of error — so we did well." Heck, maybe that's a good enough attitude to get hired by a political campaign — but it's not going to do much help for any candidate trying to win. **More on AI and politics:** [*AI Powering MAGA Botnet Confused by Trump's Connections to Epstein, Starts Contradicting Itself*]() ## Author At Futurism, I've often been drawn to unpacking the narratives that underlie technological, scientific and medical progress, with a special interest in areas of conflict and ambiguity that end up setting agendas and steering the fates of both elites and the hoi polloi. I'm a committed generalist, but I often find myself returning to work involving NASA and the private space sector, the effects of AI on media and society, and the mechanics of the pharmaceutical industry, with a specific focus on the spread of GLP-1 drugs like Ozempic and Wegovy. Prior to Futurism, I worked for publications ranging from Media Matters and Truthdig to Raw Story and Bustle. I'm also the author of "Myspace Scene Queens," a 2024 title in Instar Books' acclaimed "Remember the Internet" series. My work at Futurism has been cited by outlets including the New Yorker, Slate, Nieman Lab, the Verge, the MIT Technology Review, the Sunday Times, and the Daily Beast. I grew up in North Carolina, attended the University of North Carolina at Asheville, and now live in Brooklyn, New York. In my free time, I'm an avid reader and music fan; you can probably find me at a local poetry reading, concert, underground rave, or DJ set. I'm the proud parent of an ineffable orange cat named Mee-Mow. ### Author social links [Bluesky]()