When Amazon Web Services suffered an outage on Monday morning, it practically took down the internet with it. There went all of Amazon’s services, from its shopping hub to its Ring doorbell cameras. ChatGPT went quiet. “Smart” mattresses became unsleepable. Video games like Fortnite blinked out, as did platforms like Snapchat and banking apps.
An even greater source of alarm was how long it took to fix the outage. Issues related to the crash were first reported at 3:11 AM EST. Three hours later, the AWS dashboard said the underlying issue had been “fully mitigated.” But it wouldn’t be until 6:53 PM that Amazon announced that all its services were returned to “normal operations.” Over half a day had passed, causing an estimated billions of dollars in lost productivity.
The cause was blamed on a DNS resolution issue. DNS, or the Domain Name System, translates URLs like “Amazon.com” into IP addresses, acting as a sort of phonebook — using a common analogy — that lets everything connect online. But if the phone numbers are outdated or wrong, then you’re screwed, and you have a resolution issue on your hands.
Amazon has suffered major AWS outages before. But the timing and impact of this one comes just months after an eyebrow-raising personnel decision by the e-commerce giant. In July, its cloud computing unit cut at least hundreds of jobs — and perhaps more — following a warning from CEO Andy Jassy that the adoption of generative AI would lead to layoffs.
“As we roll out more Generative AI and agents, it should change the way our work is done,” Jassy said in a note to employees in June, per Reuters. “We will need fewer people doing some of the jobs that are being done today, and more people doing other types of jobs.”
It remains unclear which roles were affected by the layoffs. But if Amazon is relying on AI to pick up the slack in the wake of the AWS layoffs, it could be a stunning example of how efforts to replace human employees with unreliable AI tools and AI agents have backfired.
Technical jobs are some of the most targeted for AI replacement amid the rise of AI coding assistants. Even titans like Google and Microsoft, who can easily afford to pay exorbitant salaries to attract armies’ worth of top talent, are using the tech to get an edge. Twenty-five percent of Google’s new code is written with AI, according to CEO Sundar Pichai; Microsoft CEO Satya Nadella claims at his company this figure is nearly a third.
While AI may speed up workflows — and the jury’s still out on that premise, with several studies, including ones focused on programming tasks, finding that it actually slows them down — one thing it most certainly does not bring to the table is experience.
In a writeup for The Register, Corey Quinn, a cloud computing expert and author of the “Last Week in AWS” newsletter, blasted Amazon for what he saw as a clear dearth of veteran knowhow at AWS amid the outage, scrutinizing the layoffs at the company.
“They legitimately did not know what was breaking for a patently absurd length of time,” wrote Quinn.
“You can hire a bunch of very smart people who will explain how DNS works at a deep technical level,” Quinn continued, “but the one thing you can’t hire for is the person who remembers that when DNS starts getting wonky, check that seemingly unrelated system in the corner, because it has historically played a contributing role to some outages of yesteryear.”
“When that tribal knowledge departs, you’re left having to reinvent an awful lot of in-house expertise,” Quinn wrote. “This doesn’t impact your service reliability — until one day it very much does, in spectacular fashion. I suspect that day is today.”
More on Amazon: Oops! The AWS Outage Took Down Everybody’s Bored Apes