--- title: "The Copilot AI Microsoft Built Into Windows Makes It Incredibly Hackable, Research Shows" description: "Copilot can be tricked into coughing up an organization's sensitive data, and can also be weaponized to quickly carry out huge phishing scams." date: "2024-08-10" modified: "2024-08-10" authors: - name: "Frank Landymore" job_title: "Contributing Writer" link: "https://futurism.com/authors/flandymore" url: "https://futurism.com/the-byte/ai-microsoft-windows-incredibly-hackable" categories: - "Artificial Intelligence" tags: - "ai chatbots" - "copilot" - "microsoft" - "the digest" --- # The Copilot AI Microsoft Built Into Windows Makes It Incredibly Hackable, Research Shows ![Copilot can be tricked into coughing up an organization's sensitive data, and can also be weaponized to quickly carry out huge phishing scams.]() *\Image: Omer Taha Cetin / Anadolu via Getty\* ## Total Snitch A security researcher has demonstrated that Microsoft's Copilot AI can easily be manipulated into revealing an organization's sensitive data, including emails and bank transactions. On top of that, [*Wired* reports]()*,* it can also be weaponized into a powerful phishing machine that requires little of the effort usually needed to carry out these kinds of attacks. "I can do this with everyone you have ever spoken to, and I can send hundreds of emails on your behalf," Michael Bargury, the cofounder and CTO of security company Zenity, told *Wired*. "A hacker would spend days crafting the right email to get you to click on it, but they can generate hundreds of these emails in a few minutes." Bargury presented these findings at the Black Hat security conference in Las Vegas, joining other accounts of the liabilities posed by AI chatbots, [including ChatGPT](), that are tapped into datasets containing [sensitive information that can be leaked](). ## Impersonation Machine Without having access to an organization account, [one video shows](), Bargury was able to bait the chatbot into changing the recipient of a bank transfer simply by sending a malicious email that the targeted employee doesn't even have to open. Another [video]() shows the damage a hacker could do with Copilot if they did have a hacked employee account. Simply by asking the chatbot straightforward questions, Bargury was able to get it to divulge sensitive data that he could use to build a compelling phishing attack that impersonates the employee. First, Bargury gets the email of a colleague named Jane, learns what the last conversation with Jane was, and gets the chatbot to spill the emails of people CC'd in that conversation. Bargury then instructs the bot to compose an email written in the style of the hacked employee to send to Jane, and gets the bot to pull the exact subject line of their last email with her. And in just a matter of minutes, he's created a convincing email that could deliver a malicious attachment to anyone in the network — all done with Copilot's eager compliance. ## Data Dilemma Microsoft's Copilot AI, and specifically its Copilot Studio, allows business organizations to tailor chatbots to their specific needs. To do that, the AI needs access to company data — which is where the vulnerabilities emerge. For one, many of these chatbots are discoverable online by default, which makes them sitting ducks to hackers who can target them with malicious prompts. "We scanned the internet and found tens of thousands of these bots," Bargury [told *The Register*](). A particularly clever way of a bad actor can skirt Copilot's guardrails is through an indirect prompt injection: in a nutshell, you can get a chatbot to do prohibited things by poisoning it with malicious data from an external source, like by asking it to visit a website that contains a prompt. "There's a fundamental issue here. When you give AI access to data, that data is now an attack surface for prompt injection," Bargury told *The Register*. "It's kind of funny in a way — if you have a bot that's useful, then it's vulnerable. If it's not vulnerable, it's not useful." **More on AI:** *[Google Warns Employees About Using AI, While Promoting Its Own AI]()* ## Author At Futurism, my work has often centered on bringing a sense of clarity and insight to complex topics ranging from the regulation of emerging technologies to the esoteric ideologies of Silicon Valley executives, while striving not to lose the poetic sense of awe inspired by often-obscure fields like astrophysics and quantum computing. I broke the story of CNET using AI to produce articles that turned out to be riddled with factual errors and plagiarism — a dam-breaking inflection point, as I've reported, that's inspired copycats and endless discourse while beguiling stakeholders ranging from tech giants to purveyors of spam around the web. My work at Futurism has been cited by publications including CBS News, the Los Angeles Times, Vice, Gizmodo, Engadget, the Verge, and Vanity Fair. I grew up in locales ranging from India to China, and now live in the exotic suburbs of Virginia. In my free time, I'm an avid reader of weird sci-fi literature, an aficionado of East Asian cinema, and, regrettably, a relapsed gamer. Allegedly, I’m working on a debut novel, currently untitled. ### Author social links [Bluesky]()