Interview Transcript
Q1: Hi, Francesco. Thank you so much for coming today. Could you tell us when did you first begin working on humanoid robots?
Francesco Ferro: I started working in humanoid robotics in 2004 when we started in PALs. So I’m one of the co-founders, and we just had an idea to make a full humanoid biped robot that had to be able to play chess. So we built a robot that was going to the table and then just start playing chess.
Q2: What was the idea behind this design of a humanoid playing chess?
Francesco Ferro: Actually, it was quite clear because 20 years ago there was still a lot of energy around chess players. At that time, it was the first computer that was playing against humans. The idea came from one of the companies that helped us grow at the beginning from UAE, from Abu Dhabi. They committed to a project in order to make a robot that was moving the pieces on the chessboard in the same way as a human chess player. So that was the idea. It was quite original. It was probably the old style of artificial intelligence. Instead of doing a simple arm movement for this kind of task, we decided to create a full humanoid robot.
Q3: How did your work evolve over the years after this chess robot?
Francesco Ferro: Actually, from the beginning, we always tried to be different. We tried to do service robotics. Our idea was that we didn’t want to have a robot that just stayed in a cage because it had to be far away from humans. We still believe that the important thing is interaction. The contact is the important thing. Robots are there to serve humanity, to help us improve our quality of life in different ways.The idea is to have robots that can collaborate with humans. We should not be scared of robots; we have to collaborate with them. This was our leitmotif, and it is one of the important reasons behind our goal of having more and more robots around us, helping us with our daily jobs and making things easier and more efficient.
Q4: Could you walk us through a rough timeline of your life related to humanoid robotics, from the moment you started until today?
Francesco Ferro: Yes, this is a good question. The roadmap of PAL Robotics started with different robots. We started with REEM-A, which we built for engineers in 14 months. In one year, that was the chess player robot. Then we had another commitment to create something even better. We developed REEM-B, which was a robot capable of autonomous navigation. It was a strong biped robot that could carry up to 15 kilograms in its hand. From this project, the crucial point was around 2009 when we started saying that humanoid biped robots were something that would happen. We still believe that is happening right now, but it still requires a lot of research and a lot of robustness, especially on the balance side. In order to have something that could give us our first robot, we also developed some platforms with wheels. From this idea, we started developing the REEM-H and REEM robots while continuing the development of biped robots. Actually, we developed REEM-C, which was the evolution of REEM-B, and we still have many robots and many universities working with them. From that point, we also started having different business units. We opened the mobile interaction business, which was different from the legged robotics business unit we had inside the company.
Doing this, we realized that even robots designed for interaction could support physical human-robot interaction as well as social robotics. We did a lot of work with social robotics, creating different platforms that could help in healthcare, industry, reception, and guiding applications. We used to call them dynamic information points — robots that could help make people’s experiences a little bit better. This was our goal. Then, working more on mobile platforms, we opened another business unit. We developed many mobile bases, always for indoor environments. We created a vertical called stock, which is an autonomous inventory robot. This robot has a mobile base with RFID antennas. Using autonomous navigation, it can move through shops and perform inventory tasks. We had great success in the retail domain. We have different robots deployed in real stores with people. Technically, it is autonomous navigation in crowded environments. For example, with Decathlon, which is probably one of the biggest sports fashion retailers in the world, we opened operations in 15 countries with more than 200 robots performing inventory every day. Even in Singapore, we have a 24-hour shop where the robot performs inventory during opening hours, with people inside the store. These kinds of projects give us many challenges that we try to solve using the best technology and the best engineering ideas we can develop. At the same time, they also give us the possibility to improve our other business units. For example, during the years after REEM-A, we developed TIAGo, which is now a reality because we have sold more than 200 robots all around the world. TIAGo represents modularity, not only from the software point of view but also from the hardware perspective. It is a robot that can be updated, modified, and equipped with different sensors. Our idea is to allow researchers to do the best research possible using the best sensors they want while keeping the same hardware platform that is designed to last. The idea is to use the platform for as long as possible. We started selling the first robots in 2015, and they are still working today. That represents more than ten years of success, and we continue providing support. Another important point is that from the beginning, we adopted ROS, the Robotics Operating System middleware, in order to make this possible for the community.
Behind these robots, especially the TIAGo robots, which stands for “Take It And Go,” we have different pillars. On one side, there is navigation; on the other side, perception, manipulation, and human-robot interaction. It is very difficult to find a research team that has all these capabilities at the same time. With these robots, we provide software solutions that are open source, allowing researchers to use existing tools and easily modify the parts they want to improve. This creates a win-win strategy. It helps researchers, and it also helps us receive feedback to improve what we are developing. Nowadays, TIAGo is modular. We have versions with one arm, two arms, and we even developed a version with three arms instead of a head in order to place hand sensors. We also developed TIAGo Pro, which is the evolution with new arms that are slimmer, faster, and have a larger workspace. We continue improving every day and developing different solutions to better support our partners and clients.
Q5: What are the biggest challenges you’ve encountered so far in these real-world applications?
Francesco Ferro: The biggest challenge is the challenge that we will solve tomorrow. During these more than 20 years in the robotics community, we have faced many different challenges. On one side, there was the computational cost of these systems. For example, many people thought that the cloud would be the future. We always believed that robots have to be autonomous because you cannot trust a 100% wireless connection. On another side, there was sensor technology. Advances in sensors such as actuators, cameras, 3D sensors, time-of-flight cameras, and other technologies allowed us to introduce better solutions. Recently, artificial intelligence has also become important in different forms. We now have generative AI, embodied AI, and reinforcement learning. We are very strong in model-based development because we believe that in order to have systems that are secure and safe for human collaboration, you need models that are transparent and allow you to understand what the robot is doing at any moment.
On the other hand, there are applications where it may not be as important, and learning methods such as imitation learning or reinforcement learning can provide faster solutions compared to model predictive control. The idea is to manage and understand these technologies properly in order to create good and safe solutions for the real world. The reason I say that the biggest challenge is always the next one is because every time we put robots into real-world applications, we discover new problems. We do not simply create beautiful videos to show success. We try to solve real problems through hard work and engineering in order to find the best solutions.
Q6: The real-world applications you mentioned have a lot of complex and unexpected moments or problems. Can you recall any surprising success you had in terms of problem-solving, and also a surprising failure that you never anticipated before when you brought robots into the real world?
Francesco Ferro: One of the things that personally surprised me a lot was using social robots in a healthcare environment. We wanted to place a robot in a residence for elderly people and people with diseases. The purpose was to make the robot navigate, guide people, and interact with them. We had already done this many times, but during this specific situation, one person started approaching the robot, talking with it, taking the robot’s hand, and trying to walk around the residence with it. For me, it was something I had seen before, so I thought it could happen. But then I noticed something strange because all the doctors around were shocked. They told me that this person had not spoken for ten years, and thanks to the robot, something opened up. The person started talking again and formed a kind of friendship with the robot. Even the doctors did not understand exactly what had happened. For me, these moments show that robotics could have a very important future in healthcare.
Regarding failures, there are many every day. Let me explain it this way because I do not know which one was the biggest. We worked on many projects involving robots in real environments. One example was when we developed speech recognition systems several years ago to create robots that could socialize. We deployed this solution in a shopping mall around 2005 or 2006. We realized that it was almost impossible because the noise filtering that humans can naturally do is extremely difficult for robots. There can be many different types of noise, and creating a robot that understands exactly what is happening in that environment is very challenging. It was not exactly a failure, but it was something we did not expect when we started. Because of this, instead of using human voice interaction in noisy environments, we moved toward simpler solutions, such as touchscreens, joysticks, or buttons. For example, in industrial environments, we developed mobile robots that could be controlled through simple and user-friendly interfaces, even through a mobile phone. We invested a lot of effort into making these systems multi-platform, secure, and robust. In one case, after all the development, testing, and cybersecurity work, the users told us: “It doesn’t work.” We asked why. They explained that humans would need to open the application, search for the correct option, and then change their minds. So we simplified everything. We put one button on the robot. You press the button, the robot already knows what it has to do, completes the task, and comes back. It became successful.From this failure, we learned the importance of listening.
The most difficult part is really understanding the problem and creating a disruptive solution that solves it easily. Sometimes even the users do not know what the best solution is. Our role is to work through co-design, listening to different stakeholders during projects, and finding solutions that are sometimes not obvious at the beginning. The goal is always to find the proper solution that actually works.
Q7: Like what you just described, it is real hard work on top of engineering. Recently, people have been saying that generative AI and large language models are going to be game changers. How do you see these developments affecting the work you have been doing?
Francesco Ferro: AI is helping a lot now. The challenge is moving toward having more engineering work to collect more data that can be generalized with large language models. Large language models are progressing quickly because sometimes, in a stochastic way, they find solutions. They may not always find the optimal solution, but they can find good solutions. The open research question, in my opinion, is how we can use these solutions in the real world. Sometimes these systems do not have enough traceability, stability, or trustworthiness to be used safely in real-world applications. This is even more challenging in Europe because we have many regulations regarding robots operating in real environments. The question is how we can create enough trust to use these AI models in daily-life robots that interact with people. This is still an open research area.
There are already some solutions in specific areas, but with language models you cannot guarantee 100% that something is safe. When you are only dealing with speech, the consequences are usually less serious. But when a robot physically interacts with the real world, mistakes can become much more dangerous. For example, our latest humanoid robots are large machines. A robot can weigh around 95 kilograms and carry several kilograms with its arms extended. If a robot falls or behaves unexpectedly, it can have a negative impact or even harm people. So we need to find ways to use these technologies properly and safely in order to deploy useful robotic systems.
Q8: Just Google falling down is not fun. It is serious. Can you recall any funny episodes from your deployments, or any funny stories from participants or people you worked with?
Francesco Ferro: I have tons of very good stories. I have had the opportunity to work with many very good researchers. I have also supervised many interns who are now working in big companies. The journey has been very, very good. We always try to joke and take things easily because it is important to understand that work is one part of life, but at the same time we have to be realistic because everyone has their own life. I have no complaints at all about the team that I have had the opportunity to lead. The idea is how we can push forward and share the same vision in order to make these things happen. It is very difficult because you cannot do everything alone. You need a lot of cooperation and collaboration, not only with colleagues inside the company but also with people outside. Sometimes different companies, universities, and technology centers have different ideas and objectives, so finding something that works together can be difficult. The biggest challenge nowadays is time management. We have to complete things within a certain timeframe because if you lose that timeframe, you are done.
A funny story from the real world happened in 2006 when we were in Los Angeles for an event with our biped robots. We brought the robot there, and it was walking through the exhibition area with people around. At that time, Uber had just started with a different type of interface, and we were walking around with a humanoid robot in the crowd while people were looking at it. It was quite funny. I can show and share some videos about this. Another thing that was funny, but also politically incorrect, involved face recognition. We had a robot and created a demonstration where people could enter and the robot could recognize their faces. During the demonstration process, which we performed hundreds of times with many people, we encountered an issue. When there was not enough light to register a face, we had programmed the robot to say: “Please move to the left side because your face is too dark.” Saying this to a person with darker skin was obviously not the best message. I take full responsibility for that. There was no bad intention; it was simply an engineering decision made to make the system work. We laughed about it afterward, but it was also a lesson. From an engineering perspective, we sometimes create solutions to solve technical problems, but those solutions can have unintended consequences. We have to consider these aspects carefully.
Q9: What is considered politically correct is always evolving, and I assume robotic design also has to adapt to that.
Francesco Ferro: This is a difficult question because political correctness is not always in our hands. It depends on different factors. One factor is geography. I am Italian, I live in Spain, and I opened a branch in France, so I have a more European mindset. The United States has a different perspective, and Eastern countries can also have different viewpoints. However, behind all these differences, there is something called ethics. If you try to do things ethically and avoid harming others, in my opinion, that is the best way to be politically correct. It is not only about doing exactly what the law requires, but about doing what is ethically right. This is a big concept, but I believe it is something that religions and cultures around the world also try to promote. Love is not only between people; it is also reflected in what you do, what you create, and what you try to give to humanity.
Q10: What do you think is your contribution to humanity?
Francesco Ferro: My personal contribution is something very, very small. My idea is that we should always try to do our best in whatever we are trying to accomplish. I started by writing code, doing different tasks, and taking different jobs, and I always tried to approach them with motivation. The main idea behind my work is passion for robotics. Passion is something we should never lose. You can do many different things, but if you do them with passion, energy, conviction, and motivation, it makes a real difference. Sometimes it is difficult, but in the end, you can find that you have changed something, even if it is only with the people around you. The important thing is having more people who try to do the right things, not only for money or for recognition, but by doing everyday work with passion and in an ethical way. In my opinion, this is what makes the difference. We need more people who are not only doing marketing about their work, but who are simply doing their jobs with passion. That is the good part of humanity and what it means to be human.
Q11: What do you think is fueling your passion for this specific field of robotics?
Francesco Ferro: Actually, it is the whole experience that I have had using and developing these new robots. This gives me the energy to continue because we have also experienced many difficult moments. Depending on the period, there were times when we had to question our direction or consider changing our approach. But the idea is always to try to solve these kinds of problems. I always say that passion has to come from yourself. It is not the company that has a vision and then transfers passion to you. It is something that you personally have to find within yourself. Nowadays, my job as a CEO is to put together a big puzzle. My goal is to find the right people, with the right competencies and the right passion, and organize them in the proper way so they can work together. When you achieve this, you can create a wonderful team like the one we have in robotics.
Q12: I would like to change the subject a bit. How do you see the relationship between the robot’s body and its controller?
Francesco Ferro: What do you mean by controller?
Q12: This question is about how you see the relationship between a robot’s body and the control units that operate it.
Francesco Ferro: Okay, I will try to interpret the question. Most of the time, engineers with different experiences try to optimize their own area. For example, robotics is one of the biggest challenges of integration. We have to integrate mechanics, electronics, control systems, high-level control, software, embedded systems, and different types of sensors. Robotics is fundamentally an integration challenge. What happens is that when people try to find the optimal solution only within their own field, sometimes the final system does not work well as a whole. The big difference in robotics teams is having people with different expertise who can design together properly. The goal is not to have the best mechanical design, the best electronics, or the best controller individually. Sometimes we need to accept a suboptimal solution in one area in order to create a system that works better overall. In robotics, we see many examples of this. Nowadays, at exhibitions, you can see robots that work extremely well, but sometimes it is because they are using a very specific controller or software architecture. If you try to use the same robot in a different way, the performance can change completely. This happens every day because we provide many robots to researchers, and often they do not use them exactly as we suggest. They explore different methods and find new ways to use the robots. This is challenging for us because researchers sometimes show us new possibilities. They say, “You designed this robot for one purpose, but it can also be used in another way that you did not consider.” This feedback helps us improve. The important thing is not to be stuck using only one approach. We have to stay open-minded, try to solve problems, accept failure many times, and recover from those failures as quickly as possible.
Q13: Humanoid robotics goes back many years. Have there been any research labs, researchers, or institutions that inspired you or influenced your work?
Francesco Ferro: Actually, we are very close to researchers, so we have a lot of contact with universities. At the beginning of the 2000s, when I started looking around after finishing university and during my PhD, the most inspiring universities were in Japan, especially for humanoid biped robots. Nowadays, it is not only Japan. We can find many excellent universities around the world, and especially in Europe, we try to work with the right researchers. I can mention some examples. For instance, we work with LAAS in Toulouse through a joint laboratory called ANR. We collaborate directly with them, and there are many things we can learn from their research. We also have the opportunity to work with many other excellent researchers through different projects and through the robots we provide. We consider researchers our partners. We try to work with them as much as possible in order to move humanoid robotics forward. I can also mention Christian Ott, who is now at the University of Vienna but previously worked at DLR in Germany for many years. We have a large network of researchers that we collaborate with. Many of them are in Europe, but we also look toward the United States and Eastern countries because there are many opportunities to work together. The difficult part is taking research results and making them work on real platforms.
Q14: That is amazing. Thank you for your answer. I have one last question. We are at the Humanoids Conference today. What are you most excited about in humanoid robotics in Korea this year and at the Humanoids Conference?
Francesco Ferro: For me, the Humanoids Conference is an honor because there is a lot of knowledge and experience in robotics gathered here. Yesterday, we had a very good keynote from Juho, who was also part of the team that won the robotics challenge with robots. The most exciting thing for me is seeing our latest development, which is Kangaroo, competing downstairs with other robots. Even though we did not focus only on making the robot for competition purposes, it is always exciting to see the robot working in that environment. Another thing that I am proud to show at the Humanoids Conference is our teleoperation system with TIAGo Pro. If you have not seen it yet, please visit the booth today because it is the last day. You can see how, in a very simple and intuitive way, humans can control humanoid-form robots using the same hardware and perform different tasks. From an organizational point of view, I think the conference organizers did a great job. It was very challenging to coordinate this conference with other events, including the conference that finished yesterday and the joint day that also happened yesterday. It was very nice to see everything coordinated together. The best thing, I think, will be tomorrow when we have the workshops. So stay tuned.
Interviewer: Very exciting. Thank you so much for your time and for all your answers.
Francesco Ferro: Thank you