MIT’s New Robot Can Visually Understand Objects It’s Never Seen Before

After it learned to pick up one shoe, the system could do the same with shoes it had never seen before.

A FIRST LOOK.

Computer vision — a machine's ability to "see" objects or images and understand something about them — can already help you unlock your fancy new iPhone just by looking at it. The thing is, it has to have seen that information before to know what it's looking at.

Now, researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a computer vision system that can identify objects it has never seen before. Not to mention it's an important step towards getting robots to move and think the way humans do.

They published their research on Monday and plan to present it at the Conference on Robot Learning in Zürich, Switzerland, in October.

A UNIQUE PERSPECTIVE.

The MIT team calls its system "Dense Object Nets" (DON). DON can "see" because it looks at objects as a collection of points that the robot processes to form a three-dimensional "visual roadmap." That means scientists don't have to sit there and tediously label the massive datasets that most computer vision systems require.

FAMILIAR BUT NOT IDENTICAL.

When DON sees an object that looks like one it's already familiar with, it can identify the various parts of that new object. For example, after researchers showed DON a shoe and taught it to pick the shoe up by its tongue, the system could then pick other shoes up by their tongues, even if it hadn't seen the shoes previously or they were in different positions than the original.

"Many approaches to manipulation can't identify specific parts of an object across the many orientations that object may encounter," researcher Lucas Manuelli said in a press release. "For example, existing algorithms would be unable to grasp a mug by its handle, especially if the mug could be in multiple orientations, like upright, or on its side."

TOMORROW'S ROBOTS.

This advanced form of computer vision could do a lot that robots can't do now. We could someday get a robot equipped with the system to sort items in a recycling center as they move down a conveyor belt without needing to train it on huge datasets. Or maybe we could even show one of these "self-supervised" robots an image of a tidy desk and have it organize our own.

Ultimately, this is another step forward on the path to machines that are as capable as humans. After that, we'll just have to see how much more capable than us they can get.

 

READ MORE: Robot Can Pick up Any Object After Inspecting It [EurekAlert]

More on smarter bots: DARPA Is Funding Research Into AI That Can Explain What It’s “Thinking”