A Robot with Its Head in the Cloud Tackles Warehouse Picking

Hidden inside a busy industrial building in Somerville, Massachusetts, a robot arm spends its day picking up seemingly random objects—bottles of shampoo, onions, cans of shaving foam—from a conveyor belt that goes in a circle about 10 meters in diameter.

The odd-looking setup is a test bed for a system that could take on many of the mundane picking tasks currently done by hand in warehouses and fulfillment centers. And it shows how advances in robotic hardware, computer vision, and teleoperation, along with the ability for machines to learn collaboratively via the cloud, may transform warehouse fulfillment in coming years.

The new robotic picking platform, which uses a combination of a hybrid gripper and machine learning, and which was developed by a startup called RightHand Robotics, can handle a wide variety of objects faster and more reliably than existing systems.

The company launched its platform, called RightPick, at a supply chain industry event earlier this month. It is targeting fulfillment for the pharmaceutical, electronics, grocery, and apparel industries.

When I visited RightHand Robotics early this year, the company’s cofounders, Yaro Tenzer and Leif Jentorf, showed me several prototypes they had developed. Besides the conveyor-belt scenario, these included a setup designed to match that of a company that sends packages of cosmetics tailored to individual customers. The company’s system could pick a customer’s items from several bins attached to a circular carousel. They also showed me a system learning to grasp a particular object by trying, over and over again, to move items piled up in one bin to another bin.

Picking different types of objects piled into a bin may sound simple, but it remains a huge challenge for robots, especially if the objects are unfamiliar. Humans are able to guess how an occluded object looks and feels, and we apply years of grasping experience to the task. Fulfillment centers typically handle a range of products, making them difficult to automate. Amazon, for example, has only been able to automate parts of its centers so far.

RightHand’s system grabs objects using a compliant fingered hand with a suction cup at its center. A camera is  embedded in the hand to help figure out which appendage  to use and how to grasp the item. The company employs machine learning to refine its control algorithm over time, and the tricks learned by one robot are fed back to a cloud server so that they can be shared with others. It is also possible for RightHand’s engineers to log into a system remotely to solve problems, or to help a company train the robot to pick a new object.

It is difficult to gauge the reliability and speed of such a system, or to tell how it might deal with any number of awkward new objects, but it appeared capable of picking up common objects you might find in a grocery store about as fast as a person could.

Ken Goldberg, a professor at UC Berkeley and an expert on robot vision, manipulation, and learning, says it remains very difficult for robots to rummage for items in a cluttered bin. He says he is impressed by the hybrid  gripper and adds that applying machine learning via the cloud, so that every robot deployed by the company gets smarter over time, makes a lot of sense. “This is a clever mechanism,” Goldberg says. “These guys are smart.”

At the start of this month, RightHand received $8 million in Series A  funding. The company’s early investors include Playground Global. This Palo Alto incubator and venture fund was created by Andy Rubin, who led the creation of Google’s Android smartphone operating system and who later managed the company’s foray into robotics with the acquisition of a number of startups working on various robot technologies. 

Tenzer and Jentorf both studied in Harvard’s Biorobotics Lab, and early company employees come from robotics labs at Yale and MIT.

Over the past year or so, the company has been working with a number of large logistics companies and retailers to prove the reliability of its system. “When we saw the tech and the progress they’ve made on the business side, we got really excited,” says Mark Valdez, a partner at Playground Global. “There’s an opportunity to build a virtuous cycle and a network effect for some of these software-defined hardware products.”

Besides Amazon, many other companies are trying to develop robots capable of grasping a range of objects from a disordered pile. “This is a major frontier for robotics right now,” says Goldberg of UC Berkeley.