AI experts establish the “North Star” for domestic robotics field

Robots that can do everything from helping people get dressed in the morning to washing (and putting away) dishes have been a dream since humans uttered the word “artificial intelligence”. However, in an area where the technical level is currently below this level of sophistication, a fundamental challenge has emerged: namely, what even “success” will look like, if the day comes robots can perform these important tasks. duty towards human standards? To perform these mundane but surprisingly complex tasks, a robot must be able to perceive, reason, and operate with full awareness of its own physical dimensions and capabilities, as well as the world. and the objects around it. In robotics, this combination of ability and physical and situational awareness is known as embodied AI. Today, a multidisciplinary team of researchers at Stanford University released Benchmarks for Household Daily Activities in Virtual, Interactive, and Ecological (BEHAVIOR) Environments. This is a catalog that includes the physical and intellectual details of 100 daily household chores, washing dishes, picking up toys, mopping floors, and more. and deploy these tasks in several simulated houses. A paper describing BEHAVIOR was recently accepted at the Conference on Robotics (CoRL). BEHAVIOR imbues a set of realistic, varied, and complex activities with a new logical and symbolic language, a fully functional 3D simulator with a virtual reality interface, and a set of success metrics drawn from the performance of humans doing the same tasks in virtual reality. Taken as a whole, BEHAVIOR delivers a breadth of tasks and a level of detailed descriptions about each task that were previously unavailable in AI.

While any one of those tasks is already highly complex in its own right, imagine the challenge of creating a single robot that can do all of these things, creating these benchmarks now, before the field has evolved too far, will help to set up potential common goals for the community.”                                                                                                                                                                                                                                                              Jiajun Wu, assistant professor of computer science and a senior author on the paper.

A monumental task

Imagine the multiple problems a robot must overcome to achieve a simple task like cleaning a countertop. The robot not only has to perceive and understand what a countertop is, where to find it, that it needs cleaning, and the counter’s physical dimensions, but also what tools and products are best used to clean it and how to coordinate its motions to get it clean. The robot would then have to determine the best course of action, step by step, needed to clean the counter. It even requires a complex understanding of things humans think nothing of, such as what tools or materials are “soakable” and how to detect and declare a countertop “clean.” In BEHAVIOR, this level of complexity is achieved in 100 activities performed in multiple different simulated houses. Each of these steps (navigation, search, grasping, cleaning, evaluating) may require hours or even days of training in simulation to be learned far beyond the capabilities of current autonomous robots. “Deciding the best way to achieve a goal based on what the robot perceives and knows about the environment and about its own capabilities is an important aspect in BEHAVIOR,” Roberto Martin-Martin, a postdoctoral scholar in computer science who worked on the planning aspects of the benchmark. “It requires not only an understanding of the environment and what needs to be done, but in what order they need to be done to achieve a task. All this for 100 tasks in different environments!”

Sim to real

In creating the BEHAVIOR benchmark, the team, led by Stanford Institute for Human-Centered AI co-director and computer scientist Fei-Fei Li, together with experts from computer science, psychology, and neuroscience, has established a “North Star,” a visual reference point by which to gauge the success of future AI solutions, which might also be used to develop and train robotic assistants in virtual environments that are then migrated to operate in literal ones a paradigm known in the field as “sim to real.” “Making this leap from simulation to the real world is a nontrivial thing, but there have been a lot of promising results in training robots in simulation and then putting that same algorithm into a physical robot,” Sanjana Srivastava, a doctoral candidate in computer science who specializes in the task definition aspects of the benchmark. “I got involved specifically to see how far we can push simulation technology,” Michael Lingelbach, a doctoral candidate in neuroscience. “Sim to real is a big area in robotic research and one we’d like to see develop more fully. Working with a simulator is just a much more accessible way to approach robotics.” Next up, the BEHAVIOR team hopes to provide initial solutions to the benchmark while extending it with new tasks not currently benchmarked. According to the team, that effort will require contributions from the entire field: robotics, computer vision, computer graphics, cognitive science. Other researchers are invited to try their own solutions; to that end, the current version of BEHAVIOR is open source and publicly available at behavior.stanford.edu. “If you think about these one hundred activities at the level of detail we provide, you begin to comprehend how difficult and important benchmarking is,” Chengshu Li, a doctoral candidate in computer science. “In that regard, BEHAVIOR is not final. We will continue to iterate and add new tasks to our list.”