Robot learns to recognize objects by its own

 

HERB, a robot butler under development at Carnegie Mellon University, can discover objects by itself

When all the humans went home for the day, a personal-assistant robot under development in a university lab recently built digital images of a pineapple and a bag of bagels that were inadvertently left on a table – and figured out how it could lift them. 

The researchers didn't even know the objects were in the room.

Instead of being frightened at their robot's independent streak, the researchers point to the feat as a highlight in their quest to build machines that can fetch items and microwave meals for people who have limited mobility or are, ahem, too busy with other chores. 

The robot, a two-armed machine called HERB (the Home Exploring Robot Butler), uses color video, a Kinect depth camera and non-visual information to build digital images of objects, which for its purposes are defined as something it can lift. 

The depth camera is particularly useful, as it provides three-dimensional shape data. Other information HERB collects include the object's location – on the floor, on a table, or in a cupboard. It can determine if it moves, and whether it is in a particular place at a particular time – say, mail in a mail slot. 

The video below illustrates how the process, called Lifelong Robotic Object Discovery (LROD), works.

At first, everything in the video lights up as a potential object, but as HERB uses the information it gathers – domain knowledge – to discriminate what is and isn't an object, the objects themselves become clearer, allowing the robot to build digital models of them.