--- title: "Accounting Firm’s AI Caught Telling Customers About Each Others’ Financial Records" description: "Sage Group temporarily suspended its AI model after it started leaking information, which allegedly included customer transactions." date: "2025-01-21" modified: "2025-01-21" authors: - name: "Frank Landymore" job_title: "Contributing Writer" link: "https://futurism.com/authors/flandymore" url: "https://futurism.com/the-byte/sage-accounting-firm-ai-financial-records" categories: - "Artificial Intelligence" tags: - "ai models" - "generative ai" - "privacy" - "the digest" --- # Accounting Firm’s AI Caught Telling Customers About Each Others’ Financial Records ![Sage Group temporarily suspended its AI model after it started leaking information, which allegedly included customer transactions.]() *Man deconstruction. This is entirely 3D generated image. \Image: Getty Images\* ## Blabbermouth Sage Group, a self-described "leader" in accounting and financial tech for businesses, temporarily suspended its Sage Copilot AI assistant after it was caught giving out customer's financial records, [*The Register* reports](). All you had to do to get this information, which allegedly included recent transactions, was ask. "A customer found when they asked \[Sage Copilot\] to show a list of recent invoices, the AI pulled data from other customer accounts including their own," a source told *The Register* last week. Thankfully, the customer reported the issue right away. A Sage spokesperson told the outlet that its currently early-access AI assistant was taken offline for several hours on Monday to address the issue. ## No Big Deal This incident would justifiably be considered a major breach of trust, both in the organization and the experimental technology it's deploying. But Sage doesn't see it that way. In a statement to *The Register*, its spokesperson described the AI blunder as a "minor issue" that only involved a "small amount" of customers. Sage Group also denied that its AI leaked sensitive customer information. "The issue showed unrelated business information to a very small number of customers," the spokesperson told *The Register*. "At no point were any invoices exposed. The fix is fully implemented and Sage Copilot is performing as expected." ## Model Behavior This certainly wouldn't be the first time that an AI model has divulged information it shouldn't have. Data privacy remains one of the most pressing concerns surrounding the technology as it's deployed by businesses both large and small. Such mishaps are a symptom of AI models' difficulty to control, and a consequence of the vast amounts of data used in their training stage, ingested during conversations and interactions, and that they're given access to by the organizations using them. (Fearful of leaks, many large companies, [including the likes of Amazon]() and [major banks](), have instructed employees not to use AI chatbots.) There's a mountain of evidence to justify these worries. Google Deepmind scientists showed over a year ago that an older version of ChatGPT could be [manipulated into leaking phone numbers and emails](). Demonstrating the reckless streak inherent in these AI models, researchers at Anthropic, the creators of the Claude chatbot, recently revealed that even the industry's smartest LLMs could easily be [tricked into ignoring their own guardrails]() by feeding them a prompt with randomly capitalized letters and typos. We are clearly still in a very dodgy stage of the technology. But that evidently hasn't stopped countless companies from trusting AI models with their customers' data. **More on AI:** *[Large Publisher Lays Off More Than 100 Employees After Striking Deal With OpenAI]()* ## Author At Futurism, my work has often centered on bringing a sense of clarity and insight to complex topics ranging from the regulation of emerging technologies to the esoteric ideologies of Silicon Valley executives, while striving not to lose the poetic sense of awe inspired by often-obscure fields like astrophysics and quantum computing. I broke the story of CNET using AI to produce articles that turned out to be riddled with factual errors and plagiarism — a dam-breaking inflection point, as I've reported, that's inspired copycats and endless discourse while beguiling stakeholders ranging from tech giants to purveyors of spam around the web. My work at Futurism has been cited by publications including CBS News, the Los Angeles Times, Vice, Gizmodo, Engadget, the Verge, and Vanity Fair. I grew up in locales ranging from India to China, and now live in the exotic suburbs of Virginia. In my free time, I'm an avid reader of weird sci-fi literature, an aficionado of East Asian cinema, and, regrettably, a relapsed gamer. Allegedly, I’m working on a debut novel, currently untitled. ### Author social links [Bluesky]()