Training Robots Virtually

Freddo, a robot, walks across an office and reaches out to accept a plastic bottle offered by a staff member.

Considering that a robot recently beat Usain Bolt’s 100-meter sprint record, Freddo’s performance may not sound extraordinary. But the speed at which he learned to walk, recognize the bottle, and grasp it is impressive. It took only a few minutes to develop those skills and upload them to Freddo. His developers say competing systems could take days to achieve similar results.

I’m visiting Vsim, a British startup based in Cambridge. Founders Michelle Lu and Kier Storey hope their software will eventually control robots capable of navigating homes and workplaces and performing useful everyday tasks.

There is still a long way to go.
“It’s a weird situation with robotics because actually the stuff that we find as humans to be incredibly difficult, like gymnastics, you can get robots to do reasonably well. The stuff that humans are really good at, like fine dexterity, is really hard in robots,” Storey says.

Teaching Robots in Virtual Worlds

Freddo’s abilities were developed in a virtual environment, where a particular task can be performed in a computer simulation millions of times. Once the system discovers the most effective solution, known as a policy, it can be uploaded to the physical robot.
Virtual simulations have become a common method for training robots. Nvidia, for example, has a system called Isaac Sim that uses this approach. Lu and Storey both worked on an early version of the technology.
In 2022, they founded Vsim to build their own robotic training environment and a collection of supporting tools.
Because they were starting from scratch, Lu and Storey were able to optimize their software around the powerful computer chips used in AI, known as graphics processing units, or GPUs.
“The underlying algorithms that we were using for most of these robotic simulations they hark back to the 1970s and 1980s, but those algorithms are not really brilliant fits for GPUs,” Storey says.
Within months, the founders realized their system could operate far faster than anything they had previously seen.

“Eighteen months in and we actually have a completely functional, super high-performance simulator,” Lu says.
The software is efficient enough to run on the hardware carried by Freddo. That allows the robot to conduct tens of thousands of simulations while moving through its surroundings.
“It can look about a second, or so, ahead into the future for 20,000 different kind of combinations of things that might happen,” Storey explains.
That capability could be crucial for robots operating in unpredictable environments such as homes.

Preparing for the Unexpected

A household is filled with variables that a robot cannot control. People, pets, and other robots can suddenly move or change what they are doing, forcing a machine to adjust its strategy.
“Things outside of the robot’s control, like humans, animals or even other robots, could do things that require a change of strategy. These unexpected events could happen very quickly and the robot needs to be able to quickly adapt to ensure its actions remain safe and on-mission,” Lu says.
Vsim currently has 10 engineers working on its technology.
At the other end of the industry is Nvidia, which dominates the market for computer chips used in AI and has a major robotics software division staffed by hundreds of engineers.
Nvidia does not manufacture robots. Instead, it develops software designed to help organizations train and control them.
Its technology includes virtual simulation systems as well as a so-called world model called Cosmos. The system is designed to help robots understand real-world physics and anticipate how their surroundings could change as they move.
Even with Nvidia’s enormous computing resources, however, developing a detailed understanding of reality remains difficult.
“Manipulation, where I just grab a bottle, that’s not too hard. The problem is when you start doing long-horizon tasks, where I say: ‘I want you to take the bottle and I want you to fill it up and I want you to go pour,'” says Spencer Huang, Nvidia’s director of product for robotics.

The Challenge of Reality

Yet virtual environments remain imperfect representations of the physical world, limiting what robots can learn through simulation alone.
“There are certain things that are hard to model in simulation, like highly deformable objects and cutting,” Antonova says.
Closing that gap between simulation and reality remains a major challenge for Nvidia and the team at Vsim.
Lu says Vsim’s approach has “reduced approximation, using accurate simulations to train models that genuinely work in reality as well as they do in simulations.”
Soon, a second robot named Nacho will join the project and help the team develop the technology.
Lu says Nacho should accelerate the development process while helping ensure that Vsim’s software can operate across different machines.
And, of course, Freddo will finally have some company.