This Robot Can Pick Up Objects and Recognize What They Are

Hand-Eye Coordination

Robots are increasingly populating — and in some cases taking over — the workplace, performing tasks that humans aren't capable of, or making others more efficient. But they can't do it all just yet. One seemingly simple job that robots still struggle with is identifying objects and picking them up.

But researchers from MIT and Princeton have come up with a solution to improve the dexterity of robotic limbs.

The "pick-and-place" system is comprised of a standard robotic arm outfitted with a custom gripper and suction cup. An algorithm was written to allow the robot to grab an item from a bin of random objects and hold it securely without knowing anything about it beforehand.

It's this algorithm that allows the system to go beyond what current technology is capable of. Most robots that are used on production lines are carefully calibrated for one very specific movement. The researchers used a deep neural network to teach their arm how to handle any given object. The system recorded a 54 percent success rate using suction, a 75 percent success rate while grasping, and recognized objects with 100 percent accuracy during the Amazon Robotics Challenge.

Learning by doing

The versatile artificial limb tackles the same challenge in four different ways: applying its suction cup vertically, applying it from the side, gripping vertically, and gripping vertically while using a separate tool to separate the object from a wall. During the experiment, the robot picked up object after object, while the researchers recorded successes and failures and fed the data into the neural network to teach the system how best to pick up various types of object.

A similar approach was used to develop the algorithm that manages visual perception. The researchers used stock photos to teach the robots to recognize each object in real life. The robot matched the pixels of each image with the right object using a set of on-board cameras.

The creators of the robot suggest that it could be used for organizing stock in a warehouse, grabbing things from a kitchen cabinet, or even digging through debris after an accident, to name a few. The idea is that the robotic arm — powered by machine learning — can tailor its approach to any situation, so its potential applications are very broad.