Heriot-Watt research team: Helping train robots to collaborate
About Heriot-Watt University
Based in Edinburgh, Heriot-Watt University was founded in 1821 with a focus on engineering, business, and science. It serves 29,000 students across five campuses.
Tell us your challenge. We're here to help.
Contact usGoogle Cloud Research Innovator runs graphic neural network models on eight GPUs simultaneously.
Google Cloud results
- Run graph neural network models for as many as 10 million timesteps
- Faster results, more iteration and experimentation
- Extra credits, resources, and network through Research Innovator program
Solving real-world problems in robotics
Working in robotics may inspire science-fiction scenarios, as if one day we'll wake up and our streets will suddenly be filled with R2-D2s. But advances in the field are really more of an evolution, with the design and development of "learning agents," as engineers call the robots, occurring in many tiny, gradual steps.
Ignacio Carlucho, Assistant Professor in the School of Engineering and Physical Sciences at Heriot-Watt University, is committed to this iterative process of carefully and gradually developing learning agents that can function efficiently alongside humans. That means the robots must be able to collaborate with each other, or other humans, on specific tasks. During his postdoc at the University of Edinburgh, together with Dr. Stefano Albrecht and PhD student Arrasy Rahman, as well as his external collaborator, Niklas Höpner at the University of Amsterdam, Carlucho focused on the problem of open ad hoc teamwork—to develop robots that can collaborate with any number of teammates without prior coordination mechanisms.
According to Carlucho, planning for open ad hoc teamwork creates two major difficulties for engineers. First, the teams are of unknown sizes, meaning that robots can leave or enter the environment at any point. Second, in real world applications the robot can only observe part of the environment, which is called partial observability. "Think about a situation like self-driving cars," Carlucho explains. "You don't know how many other cars will be on the road so you can't train the robots in advance to work together with a fixed number. And your robot also won't have all the information about everything that is happening. The car will have a few cameras, but objects or other cars may block the view, so your robot needs to be able to make decisions using only this limited available information. These are challenging problems for the future of robotics in solving real-world applications."
To train the robots in ad hoc teamwork, Carlucho and his colleagues borrowed techniques from graph analysis, and developed a class of reinforcement learning algorithms that use graph neural networks to analyze the behavior of the robot-agents. They called this learning framework Graph-based Policy Learning (GPL) and demonstrated that it could improve robot learning in different ad hoc teamwork problems.
Accelerating results with Google Cloud
But these training simulations required a lot of compute power: "We have to run these models for as many as 10 million timesteps, doing independent runs, typically utilizing eight Cloud GPUs simultaneously (VM series N1 with 8 x NVIDIA Tesla K80)," Carlucho says. "It can take a few days to finish the training. I started using Google Collab from the moment it was released. I was a PhD in Argentina at that point, and the fact that I had access to free compute power was incredible. But once we started this project it was clear that the compute power from Colab was not enough for us, and we started to look for alternatives. That's when we started using Google Cloud. We applied for research credits to start testing the platform and we have been using it ever since. Now, we don't need to schedule services on campus servers, or anything like that. We just find a computer with internet and we do it whenever we want. That's much more convenient."
The speed and ease of cloud computing was a major factor for Carlucho's team, but there were other benefits as well. Carlucho points out that training models faster means he and his team can experiment and iterate more easily than when they were competing for campus resources using servers. "It's much easier to do this research on Google Cloud," Carlucho says. "When you have an idea, you can try it and if it works then you're happy with that. But then as you go on and you do deeper analyses, you have new insights, and that leads you to the drawing board again, and then you think further and you come up with new ideas—It's like this constant evolution. And having the ability to run constant experiments without distraction is ideal for us."
Imagining the future of robotics
In 2022 Carlucho joined Google Cloud Research Innovators program and met with colleagues across the world who are tackling similar challenges with cloud computing. In the longer term, Carlucho hopes to develop an autonomous agent that can collaborate with a set of teammates on the fly, in real world applications. He is well aware of how complicated that will be: in real-life robots conditions can change unexpectedly, robots have limited sensing capabilities, and any mistake can cause real damage to humans or other robots. But Carlucho is confident these challenges will be solved with enough patient experimentation, and he knows the problem is worth solving: "robotics has broad applications not just for autonomous cars and warehouse automation, but also for space and ocean exploration. These robots can go places humans can't, and if they are able to work together and collaborate it opens exciting opportunities."
"Robotics has broad applications not just for autonomous cars and warehouse automation, but also for space and ocean exploration. These robots can go places humans can't, and if they are able to work together and collaborate it opens exciting opportunities."
—Ignacio Carlucho, Assistant Professor, School of Engineering and Physical Sciences, Heriot-Watt UniversityTo learn more about Carlucho's research, read the full article in the Journal of Machine Learning Research, or consult their open-source code on GitHub. Learn more about the current Google Cloud Research Innovators' cohort . To find out how you can get started with generative AI for higher education, download the new 10-step public sector guide. With domain-specific use cases and customer stories from the city of Memphis, the state of Minnesota, the U.S. Department of Defense, and more, it offers a comprehensive guide to kickstart your gen AI journey.
Tell us your challenge. We're here to help.
Contact us
About Heriot-Watt University
Based in Edinburgh, Heriot-Watt University was founded in 1821 with a focus on engineering, business, and science. It serves 29,000 students across five campuses.