SCS researchers have developed a learning method for robots that allows them to learn directly from human-interaction videos and apply that knowledge to new tasks.
Novel Method Developed by CMU Researchers Allows Robots To Learn in the Wild
The robot watched as Shikhar Bahl opened the refrigerator door. It recorded his movements, the swing of the door, the location of the fridge and more, analyzing this data and readying itself to mimic what Bahl had done.
It failed at first, missing the handle completely at times, grabbing it in the wrong spot or pulling it incorrectly. But after a few hours of practice, the robot succeeded and opened the door.
“Imitation is a great way to learn,” said Bahl, a Ph.D. student at the Robotics Institute (RI) in Carnegie Mellon University’s School of Computer Science. “Having robots actually learn from directly watching humans remains an unsolved problem in the field, but this work takes a significant step in enabling that ability.”
Bahl worked with Deepak Pathak and Abhinav Gupta, both faculty members in the RI, to develop a new learning method for robots called WHIRL, short for In-the-Wild Human Imitating Robot Learning. WHIRL is an efficient algorithm for one-shot visual imitation. It can learn directly from human-interaction videos and generalize that information to new tasks, making robots well-suited to learning household chores. People constantly perform various tasks in their homes. With WHIRL, a robot can observe those tasks and gather the video data it needs to eventually determine how to complete the job itself.
The team added a camera and their software to an off-the-shelf robot, and it learned how to do more than 20 tasks — from opening and closing appliances, cabinet doors and drawers to putting a lid on a pot, pushing in a chair and even taking a garbage bag out of the bin. Each time, the robot watched a human complete the task once and then went about practicing and learning to accomplish the task on its own. The team presented their research this month at the Robotics: Science and Systems conference in New York.
“This work presents a way to bring robots into the home,” said Pathak, an assistant professor in the RI and a member of the team. “Instead of waiting for robots to be programmed or trained to successfully complete different tasks before deploying them into people’s homes, this technology allows us to deploy the robots and have them learn how to complete tasks, all the while adapting to their environments and improving solely by watching.”
Current methods for teaching a robot a task typically rely on imitation or reinforcement learning. In imitation learning, humans manually operate a robot to teach it how to complete a task. This process must be done several times for a single task before the robot learns. In reinforcement learning, the robot is typically trained on millions of examples in simulation and then asked to adapt that training to the real world.
Both learning models work well when teaching a robot a single task in a structured environment, but they are difficult to scale and deploy. WHIRL can learn from any video of a human doing a task. It is easily scalable, not confined to one specific task and can operate in realistic home environments. The team is even working on a version of WHIRL trained by watching videos of human interaction from YouTube and Flickr.
Progress in computer vision made the work possible. Using models trained on internet data, computers can now understand and model movement in 3D. The team used these models to understand human movement, facilitating training WHIRL.
With WHIRL, a robot can accomplish tasks in their natural environments. The appliances, doors, drawers, lids, chairs and garbage bag were not modified or manipulated to suit the robot. The robot’s first several attempts at a task ended in failure, but once it had a few successes, it quickly latched on to how to accomplish it and mastered it. While the robot may not accomplish the task with the same movements as a human, that’s not the goal. Humans and robots have different parts, and they move differently. What matters is that the end result is the same. The door is opened. The switch is turned off. The faucet is turned on.
“To scale robotics in the wild, the data must be reliable and stable, and the robots should become better in their environment by practicing on their own,” Pathak said.
Original Article: Robots Learn Household Tasks by Watching Humans
More from: Carnegie Mellon University
The Latest on: Robot learning
- UWC Coding & Robotics experts stress importance of early exposure to technologyon August 7, 2022 at 4:25 am
The University of the Western Cape's Science Learning Centre for Africa on Saturday launched its Coding and Robotics Club.
- Segway Robotics from Kitchen to Your Dining Table with Tony Hoon August 5, 2022 at 10:08 am
Artificial intelligence and machine learning are driving innovations in robotics to meet needs across multiple sectors in virtually every industry. Segway, know ...
- Primrose Schools® Junior Robotics Challenge Winners Celebrate Success with Local Charity Donationson August 4, 2022 at 8:57 am
Children at Primrose schools across the country recently put their robotics skills to the test by solving problems through coding, design thinking and engineering during its first-ever "Ready, Set, ...
- Robots Need More Human Skillson August 4, 2022 at 12:01 am
Robots need freedom. At least if they are working intelligently with humans. Out of the cage, into the open shop floor of a factory. However, with freedom, great challenges await robots, which ...
- Allowing social robots to learn relations between users' routines and their moodon August 3, 2022 at 6:40 am
Social robots, robots that can interact with humans and assist them in their daily lives, are gradually being introduced in numerous real-world settings. These robots could be particularly valuable ...
- Hotel Robots Trigger Mixed Feelings of Joy and Fear — New Studyon August 3, 2022 at 6:00 am
Joy, fear and sadness: these are just some of the emotions hotel guests feel when they encounter a hotel robot, a new study claims.
- Safetrust and Cobalt Robotics expand partnershipon August 2, 2022 at 9:41 am
Cobalt robots provide safety and security primarily through corporate office spaces, automating security workflows more reliably than humans. With this deeper integration between Safetrust and Cobalt, ...
- Safetrust and Cobalt Robotics Expand Partnership Integrating Cobalt’s Robotics with Safetrust Sensor Ecosystemon August 2, 2022 at 6:24 am
Safetrust, a pioneer in secure and virtual credential solutions, and Cobalt Robotics, the only machine learning startup to replace security guard posts with autonomous robots around the world, today ...
- Gupta earns prestigious NSF CAREER award to build predictive models that can improve robot navigation and object interactionon July 31, 2022 at 5:00 pm
Champaign Electrical & Computer Engineering Assistant Professor Saurabh Gupta received a coveted NSF CAREER award grant for young faculty to conduct research that will improve robots’ abilities to ...
- 5 Trends Dominating the Robotics Sectoron July 30, 2022 at 7:28 pm
What is the state of the robotics and automation sector at 2022's midpoint? ABI Research’s Lian Jye Su has a few thoughts, five in fact.
via Bing News
The Latest on: Robot learning
via Google News