In a collaborative effort between computer scientists at Columbia University and the Toyota Research Institute (TRI), a team of researchers are using a combination of physics, psychology, and geometry to develop robots capable to carrying out more complex tasks than they are presently able to do.
Teaching a robot to do multiple tasks at once in the lab. (Credit: Zhenjia Xu/Columbia Engineering)
One such obstacle the researchers are attempting to overcome is permanence, which is a psychological concept that involves knowing the existence of an unseen object, which is key for robots to learn in our dynamic, ever-changing world. However, most computer vision applications ignore this concept, which leads to robots losing track of objects that are no longer in direct view.
"Some of the hardest problems for artificial intelligence are the easiest for humans," said Dr. Carl Vondrick, who is an associate professor of computer science at Columbia University, and a recipient of the Toyota Research Institute Young Faculty award.
For this aspect, Dr. Vondrick and his team trained a machine to learn how an environment might change by watching videos, much like a child learns as they watch their surroundings change, as well.
"I have worked with images and videos before, but getting neural networks to work well with 3D information is surprisingly tricky," said Basile Van Hoorick, who is a third-year PhD student at Columbia University, and who worked with Dr. Vondrick on occlusions and how environments change.
The second aspect of this research involves overcoming what’s known as the rigid body assumption, meaning that robots assume objects are completely solid and don’t change shape. But new research conducted by Dr. Shuran Song, who is an assistant professor in computer science at Columbia and leads the Columbia Artificial Intelligence and Robotics (CAIR) Lab, has created an algorithm that lets robots learn through their own actions.
"One of the interesting strategies the system developed with the bag-opening task is to point the air a little above the plastic bag to keep the bag open," said Zhenjia Xu, who is a fourth-year PhD student at Columbia and who works with Dr. Song in the CAIR Lab. "We did not annotate or train it in any way; it learned it by itself."
Going forward, this collaboration hopes to allow robots to maneuver in environments where they are initially unfamiliar but learn as they go, which includes identifying tasks and subtasks, along with knowing what to do if they encounter failure on a specific task, as well.
“The progress that Carl Vondrick and Shuran Song have made with their research contributes directly to Toyota Research Institute's mission," says Dr. Eric Krotkov, advisor to the University Research Program. "TRI's research in robotics and beyond focuses on developing the capabilities and tools to address the socioeconomic challenges of an aging society, labor shortage, and sustainable production. Endowing robots with the capabilities to understand occluded objects and handle deformable objects will enable them to improve the quality of life for all.”
The goal of the research between Dr. Vondrick and Dr. Song will be to develop robots capable of helping people at home, and they both plan to collaborate further to make this a reality.
Sources: Columbia Engineering
As always, keep doing science & keep looking up!