Humans Teamed up With a New Kind of AI to Crush a Notoriously Difficult Video Game

Teamwork is Very Important

A team of scientists at Uber’s artificial intelligence lab just pulled off one of the major outstanding challenges in AI research — something so advanced and difficult that some wondered whether an AI system would ever figure it out.

That’s right. The team, the MIT Technology Review reports, built an algorithm that’s really, really good at old-school video games “Pitfall!” and “Montezuma’s Revenge.”

And while it did take some help from humans, according to Uber’s blog post on the soon-to-be-published research, the algorithm learned to rack up scores no human player could ever reach.

Yes, Seriously

When AI learns how to play and subsequently master a video game, it’s often by way of reinforcement learning, a type of algorithm that learns to seek out things that it has been programmed to treat as a reward. An ever-rising video game score makes for eager reinforcement learning-driven AI gamers, but games like “Pitfall!” and “Montezuma’s Revenge” don’t immediately (or often) increase the player’s score, MIT Tech Review mentions.

In games like the classic “Super Mario Bros.” titles, at which AI systems have long excelled, it’s easy to boost your score little by little — collecting coins and jumping on goombas give obvious and immediate rewards.

But even reinforcement learning algorithms that were explicitly instructed to randomly explore and try as many new ideas as possible floundered when trying to figure out the many sequential steps necessary to increase one’s score in either game.

Save State

Uber’s new algorithm is able to remember past attempts, so it can learn from its own mistakes better than any other AI gamer that’s tried to crack the two challenging games. But still, the algorithm needed a nudge from human supervisors who encouraged it to revisit promising areas or strategies that it may have overlooked.

So rest easy, streamers and speed runners — you’re safe from automation for now. It would seem, if this research is an indication of things to come, that the high scores (and solving of other complex problems) will probably be set by human-AI collaborations, not just machines working alone.

READ MORE: Uber has cracked two classic ‘80s video games by giving an AI algorithm a new type of memory [MIT Technology Review]

More on AI memory: New Artificial Intelligence Does Something Extraordinary — It Remembers