--- title: "AI Researcher Slams OpenAI, Warns It Will Become the “Most Orwellian Company of All Time”" description: "Cognitive scientist and prominent AI researcher Gary Marcus believes that OpenAI will pivot into the surveillance business." date: "2024-10-09" modified: "2024-10-09" authors: - name: "Frank Landymore" job_title: "Contributing Writer" link: "https://futurism.com/authors/flandymore" url: "https://futurism.com/the-byte/openai-most-orwellian-company" categories: - "Artificial Intelligence" - "OpenAI" tags: - "generative ai" - "OpenAI" - "surveillance" - "the digest" --- # AI Researcher Slams OpenAI, Warns It Will Become the “Most Orwellian Company of All Time” ![Cognitive scientist and prominent AI researcher Gary Marcus believes that OpenAI will pivot into the surveillance business.]() *\Image: Getty / Futurism\* ## Grave New World OpenAI is far from its days [of being an altruistic, non-profit company](). It is now the face of a booming AI industry and is effectively [for-profit in all but name](), steaming ahead with little regard for its technology's [environmental toll](), or for the potentially [existential risks]() it poses to society. But according to Gary Marcus, a cognitive scientist and prominent AI researcher, the worst is yet to come. In his assessment, OpenAI could soon take an even more dystopian pivot, à la George Orwell's novel "1984," by getting into the business of spying on you. "My guess is that OpenAI is going to become the most Orwellian company of all time," Marcus said [during a seminar]() at Stanford's Center for Human-Centered Artificial Intelligence last week, [as quoted by *Business Insider*](). "What they're going to be pressed to do is become a surveillance company." ## Fast Cash The reason that this will happen is simple: OpenAI will need a way to make money, Marcus said. "OpenAI sold the dream of universal AI for all purposes, and in 2023 practically every big company ran pilot studies on that premise," Marcus later told *BI*. "But in 2024, a lot of reports from the field are about disappointment: the technology isn't reliable enough yet for production, because it is plagued with problems like hallucinations and boneheaded errors." "So many businesses are starting to be cautious," he added. That echoes [what other analysts have said]() about the [AI boom-slash-bubble]() (and indeed, what Marcus has [harped on before]()). Despite the hype and massive amounts of investment, how these AI models will generate enough profits [remains uncertain](). For now, the technology's [obvious pitfalls]() and [commercial]() [shortcomings]() are being overlooked for the promise of it eventually being far more capable and transformative than it currently is. ## A Date With Data A startup like OpenAI going down the surveillance road wouldn't be unheard of. Surveillance, in some shape or form, is a [sidestream of income]() for many [big tech companies](). Platforms like X-formerly-Twitter, for example, [sell user data to third parties](). Those third parties can be surveillance firms, who then sell their services to government agencies. But as leaders of the generative AI industry — which is predicated on the mass consumption of data from all corners of the web — OpenAI will be uniquely positioned to deploy powerful surveillance capabilities of its own. Some of the warning signs may already be apparent. This summer, for example, OpenAI controversially [appointed a former director of the National Security Agency](), Paul Nakasone, to its board. Marcus once held out hope that OpenAI would course correct. But he's since grown cynical about its CEO Sam Altman's leadership, especially as the company [drops the non-profit act](). "In the last few months, the mask has really come off, and I seriously doubt \[Altman\] will ever return to the mission," Marcus told *BI*. "The promise to investors to turn into a for-profit basically means that ship has sailed." **More on AI:** *[Google Is Stuffing Annoying Ads Into Its Terrible AI Search Feature]()* ## Author At Futurism, my work has often centered on bringing a sense of clarity and insight to complex topics ranging from the regulation of emerging technologies to the esoteric ideologies of Silicon Valley executives, while striving not to lose the poetic sense of awe inspired by often-obscure fields like astrophysics and quantum computing. I broke the story of CNET using AI to produce articles that turned out to be riddled with factual errors and plagiarism — a dam-breaking inflection point, as I've reported, that's inspired copycats and endless discourse while beguiling stakeholders ranging from tech giants to purveyors of spam around the web. My work at Futurism has been cited by publications including CBS News, the Los Angeles Times, Vice, Gizmodo, Engadget, the Verge, and Vanity Fair. I grew up in locales ranging from India to China, and now live in the exotic suburbs of Virginia. In my free time, I'm an avid reader of weird sci-fi literature, an aficionado of East Asian cinema, and, regrettably, a relapsed gamer. Allegedly, I’m working on a debut novel, currently untitled. ### Author social links [Bluesky]()