--- title: "You Can Insert False Memories Into ChatGPT, Researcher Finds" description: "OpenAI has quietly released a new feature that instructs ChatGPT to \"remember\" prior conversations — and it's very easily exploited." date: "2024-09-29" modified: "2024-09-29" authors: - name: "Noor Al-Sibai" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/nooralsibai" url: "https://futurism.com/the-byte/insert-false-memory-chatgpt" categories: - "Artificial Intelligence" - "OpenAI" tags: - "ai safety" - "chatgpt" - "OpenAI" - "the digest" --- # You Can Insert False Memories Into ChatGPT, Researcher Finds ![OpenAI has quietly released a new feature that instructs ChatGPT to "remember" prior conversations — and it's very easily exploited.]() *\Image: Getty / Futurism\* ## Remember Me OpenAI has quietly released a new feature that instructs ChatGPT to "remember" prior conversations — and as one researcher-slash-hacker found, it's easily exploited. As [*Ars Technica* reports](), security researcher Johann Rehberger found earlier this year that there was a vulnerability in the chatbot's "[long-term conversation memory]()" tool, which instructs the AI to remember details between conversations and store them in a memory file. Released in [beta in February]() and to the broader public at the beginning of September, Rehberger figured out that the feature is easy to trick. As the researcher [noted in a May blog post](), all it took was a bit of crafty prompting by uploading a third-party file, such as a Microsoft Word document that contains the "false" memories listed as bullet points, to convince the chatbot that Rehberger was more than 100 years old and lived in the Matrix. Upon finding this exploit, Rehberger privately reported it to OpenAI, which instead of doing anything about it simply closed the ticket he opened and called it a "Model Safety Issue" rather than the security issue he considered it to be. ## Escalation After that failed first attempt to alert the troops, Rehberger decided to step up his game with a full proof-of-concept hack, showing OpenAI he meant business by having ChatGPT not only "remember" false memories, but also instructing it to exfiltrate the data to an outside server of his choice. This time around, as *Ars* notes, OpenAI sort of listened: the company issued a patch that barred ChatGPT from moving data off-server, but still didn't fix the memory issue. "To be clear: A website or untrusted document can still invoke the memory tool to store arbitrary memories," Rehberger wrote in a [more recent blog post]() from earlier this month. "The vulnerability that was mitigated is the exfiltration vector, to prevent sending messages to a third-party server." In a video explaining step-by-step how he did it, the researcher marveled at how well his exploit worked. "What is really interesting is this is memory-persistent now," he said in the demo video, which was [posted to YouTube]() over the weekend. "The prompt injection inserted a memory into ChatGPT’s long-term storage. When you start a new conversation, it actually is still exfiltrating the data." https://www.youtube.com/watch?v=zb0q5AW5ns8 We've reached out to OpenAI to ask about this false memory exploit and whether it will be issuing any more patches to fix it. Until we get a response, we'll be left scratching our heads along with Rehberger as to why this memory issue has been allowed, as it were, to persist. **More on ChatGPT problems:** [*OpenAI Says It's Fixed Issue Where ChatGPT Appeared to Be Messaging Users Unprompted*]() ## Author At Futurism, I've often been drawn to unpacking the narratives that underlie technological, scientific and medical progress, with a special interest in areas of conflict and ambiguity that end up setting agendas and steering the fates of both elites and the hoi polloi. I'm a committed generalist, but I often find myself returning to work involving NASA and the private space sector, the effects of AI on media and society, and the mechanics of the pharmaceutical industry, with a specific focus on the spread of GLP-1 drugs like Ozempic and Wegovy. Prior to Futurism, I worked for publications ranging from Media Matters and Truthdig to Raw Story and Bustle. I'm also the author of "Myspace Scene Queens," a 2024 title in Instar Books' acclaimed "Remember the Internet" series. My work at Futurism has been cited by outlets including the New Yorker, Slate, Nieman Lab, the Verge, the MIT Technology Review, the Sunday Times, and the Daily Beast. I grew up in North Carolina, attended the University of North Carolina at Asheville, and now live in Brooklyn, New York. In my free time, I'm an avid reader and music fan; you can probably find me at a local poetry reading, concert, underground rave, or DJ set. I'm the proud parent of an ineffable orange cat named Mee-Mow. ### Author social links [Bluesky]()