Researchers at Carnegie Mellon University (CMU) have developed H2O – Human2HumanOid, a reinforcement learning-based framework that allows humans to remotely control life-sized humanoid robots in real time using only RGB cameras. The following questions arise: Will physical work soon be performed remotely?
Remotely controlled humanoid robots can be used to perform complex tasks that, at least at this stage, are too complex for robots to perform independently. However, full-body control of a human-sized humanoid to replicate its movements in real time is a challenging task. This is where reinforcement learning (RL) comes into play.
RL is a machine learning technique that mimics the human trial and error approach to learning. Using the reward and punishment paradigm of RL, the robot learns from feedback about each task it performs and discovers for itself the best processing path to achieve the desired outcome. Unlike machine learning, RL does not require humans to label pairs of data to direct the algorithm.
Remote Remote Controlled Humanoid: Example 1
“H2O telemanipulation is a reinforcement learning (RL)-based framework that facilitates real-time full-body telemanipulation of humanoid robots using only RGB cameras.” Tairan He, director of CMU's LeCAR (Learning and Control for Agile Robotics) lab and one of the researchers, said: Lead researchers on the project told Tech Xplore. “This process begins with retargeting human behavior to humanoid features through new ‘sim-to-data’ methodologies, ensuring that the behavior is actionable under the physical constraints of the humanoid. This refined motion dataset trains an RL-based motion imitator in simulation, which is then transferred to the real robot without any further adjustments.”
Remote Remote Controlled Humanoid: Example 2
Using this approach, researchers can mimic the movements of a human teleoperator with H2O by utilizing images captured by an RGB camera and a 3D pose estimator that collects visible light and converts it into a color image that replicates normal human vision. .
Remote Remote Controlled Humanoid: Yes 3
The results seen in the video taken by the researchers speak for themselves. In it, H2O can be seen kicking a ball, emptying a box, stepping aside in a boxing stance, and walking with a stroller (albeit like an unsteady toddler). To the researchers' knowledge, this is the first demonstration of real-time full-body humanoid teleoperation.
Remote Remote Controlled Humanoid: Yes 4
CMU researchers described their approach in a paper posted at: arXiv Preprint website.
The lengthy video below from the CMU LeCAR Lab shows more of H2O's capabilities, including hitting an Amazon box while wearing boxing gloves and giving a victory salute (you can't help but wonder if it's the robot's revenge). , showing toughness by kicking humans in the back.
Human-to-humanoid real-time full-body teleoperation learning
Further research will investigate how H2O's functionality can be improved by introducing inputs other than human teleoperators, such as force feedback, language, and conversational feedback. And a promising direction for future research is to incorporate lower body tracking to enable humanoids to perform more skilled human movements, such as sports or dancing.
This goes back to the previous question. Could H2O or something similar enable remote work? CMU's announcement on H2O sparked thought-provoking discussion on Reddit's r/Futurology subreddit.
![An example of the r/Futurology subreddit discussion generated by CMU's H2O announcement.](https://assets.newatlas.com/dims4/default/d86a1f8/2147483647/strip/true/crop/2000x404+0+0/resize/1440x291!/quality/90/?url=http%3A%2F%2Fnewatlas-brightspot.s3.amazonaws.com%2F3f%2F42%2Fb6ea5cd740a99e65667f754665ae%2Freddit-post.jpg)
Obviously, using remote controlled humanoids means humans can keep their jobs (positive). Another advantage is the use of these humanoids for hazardous work or search and rescue missions in remote and/or inaccessible areas.
However, there are also some disadvantages, the most notable of which are labor-related. As Reddit user lughnasadh points out, “[I]“Rich countries don’t open more avenues for overseas job creation.” Then there is the slippery slope argument. Remotely operated robots now, fully autonomous (read: job-stealing) robots later.
Interestingly, a 2022 study aimed at measuring public acceptance of using remote-controlled robots to perform labor tasks found that many people are open to remote work with lower salaries than on-site roles. Again, this is one of those ‘time will tell’ situations we have become accustomed to in the rapidly evolving field of AI/robotics.
Source: CMU, Tech Xplore