NYC Mayor Defends AI System That Tells Business Owners to Commit Wage Theft

Just another tech fail under Mayor Eric Adams.

Spin City

Even though an official New York City chatbot has been telling business owners and landlords to break the law, CTV News reports that Mayor Eric Adams rose to the chatbot's defense on Tuesday, saying officials are still hammering out bugs.

"It's wrong in some areas, and we've got to fix it," Adams said on Tuesday about the MyCity chatbot. "Any time you use technology, you need to put it into the real environment to iron out the kinks."

City officials announced the Microsoft-powered MyCity chatbot in October, touting the platform as a way for small businesses and landlords to ask "business-related questions." But explosively, reporters from The Markup found last week that the chatbot was giving answers that are against the law, such as telling business owners it's okay to take tips from workers and that landlords could discriminate against people who use rental assistance. Yikes!

The chatbot now has warnings on its website that "it's a beta product" and "its responses may sometimes be inaccurate or incomplete." And the website also tells people to double check the city's official business information portal — leading us to ponder what good the broken chatbot is doing at all.

Microsoft told Reuters that the company was fixing the chatbot, while New York City's Office of Technology and Innovation released a statement saying that "as soon as next week, we expect to significantly mitigate inaccurate answers."

Setting Fail

Adams clearly longs to be a tech leader.

Another of his notable fails was the deployment and mothballing of a 400-pounds surveillance robot in city subways. Other high tech equipment that Adams has pushed for include drones to rescue drowning swimmers at city beaches, an NYPD robot dog that alarmed residents, and driverless cars — making New York City and its residents unwilling guinea pigs for often untested and unreliable technology.

The AI chatbot seems like an especially egregious waste of resources. AI chatbots built on large language models are notoriously prone to spitting out errors, a shortcoming that some experts think will never be fully fixed.

And what's the point of using a chatbot if it's going to give you incorrect information that may lead you to breaking the law? Couldn't the money for this tech go towards, say, something decidedly low tech and yet beloved by all New Yorkers, like shoring up the city library system?