Interview with James Kuffner

James Kuffner: Right. My name is James Kuffner. I was born in New York but grew up in Portland, Oregon and went to school in California at Stanford University.

Q1: And how did you become familiar with robots? Do you remember some of the first times you had contact with them or saw them?

James Kuffner: I think, like many of my colleagues, the first robot that I really became fascinated with, I saw in Star Wars. R2-D2 and C-3PO were two of my favorite fictional robots and the movies brought them to life in a way that fascinated me. And I think that has always stuck with me with my work later on humanoids.

Q2: I know if it’s going to bug you.

James Kuffner: Oh, no. Oh, oh, oh, thank you. Thank you. Thank you.

Q2: Just the second part, about the –

Q1: Oh. Could you tell us a little bit about like what were the first times that you remember seeing or hearing about robots?

James Kuffner: I remember seeing the robots portrayed in Star Wars in 1978. I was seven or eight years old and really, really became fascinated with building machines that could not only be useful but also friends with humans.

Q1: So how did you get into actually working with robots?

James Kuffner: I think my journey was, you know, building LEGOs with motors and exploring mechanisms and RC cars and other things. But I also had a TRS 80 color computer from Radio Shack that had 4K of RAM that I started playing with and teaching myself how to program mostly in basic or assembly. And my goal was actually to mostly create games. I loved video games. But I think, you know, combining those two things together didn’t happen until university times. 2

Q1: So what happened at university?

James Kuffner: There were robot competitions. And, you know, we had teams where we would try to program small wheeled robots. They were basically RC cars that, you know, had a small microcontroller. And it was so much fun. And this was as an undergraduate at Stanford. I had come in thinking I was going to do astrophysics or something related to sciences. But I really got fascinated with computer science and then making these RC cars was so fascinating to me.

Q1: So what did you do after your undergraduate at Stanford?

James Kuffner: I had applied to graduate school to get a master’s degree in computer science. And I was doing a lot of computer graphics. So in particular, at the time, it was 1993. Most of the animations of human motion in movies, like the first full length computer graphics film came out, the Toy Story that Pixar had, all of that was hand animated. And many games had human characters with 3D, but it’s all hand animated. And it’s hard to get it natural looking. And, at the same time, computer vision was coming along and motion capture became possible, where tracking with cameras, optical motion capture, and then you would recover not only the kinematics of the motion, but it captures the dynamic and expressivity of the motion. Now, of course, we use optical motion capture and other capture techniques for films and video games, but at the time it was new. So I had entered Stanford as a graduate student studying computer graphics and robotics. And really it was about algorithms and methods to generate motion. Robot motion planning became my main thesis topic. And that was really fascinating for me.

Q1: Who did you work with?

James Kuffner: I worked with Jean-Claude L’Eton. He was the department chair at the time. And he led a lab that was really focused on motion planning. And what was really fortunate for me was, at the time, there had been work in the 80s on combinatorial motion planning and a lot of theory around configuration space formulations for motion planning for robots. But Barakhand and L’Eton had published a paper using gradient descent and random walks, the potential field method for scaling to large configuration spaces where you couldn’t store everything in memory and do optimal planning like A-star search. And that became really fascinating. And that eventually became sampling-based motion planning, which we use a lot today in order to plan high-dimensional motions for robot arms or humanoids. But at the time it was brand new, and many people were looking at ways of adapting new algorithms that were practical that could generate motion that was collision-free and feasible. And that became my main interest and research topic at the time.

3

Q1: And what kind of robots did you work with then?

James Kuffner: At that time, we had many different robots in the lab. It was the Stanford Computer Science Robotics Lab. And Professor Osama Khatib was on my committee, so I got a mentoring from many, many wonderful roboticists at the time. We had Puma robots that were mounted on nomadic bases that were mobile manipulators. There were also other different arms robots. And at the time, legged robots were just starting to appear in Japan. And that was about midway through my PhD. I got blown away when I saw Honda’s P2 and then later Asimo, which reignited the images of the Star Wars humanoids that I had seen and fell in love with when I was a kid. And I decided at that point that I needed to learn Japanese and go to Japan.

Q1: How did that go?

James Kuffner: It went very well. So I started learning Japanese. And I also, at the time, something like 85% of the world’s robots were in Japan. And they were doing mostly industrial automation tasks. But I could see Honda and many of these other companies building humanoids and the mechatronics. And I think it was harmonic drag gears had really made it possible to have an implementation of zero moment point on a humanoid that is fully self-contained, walking up and down stairs and balancing. And that was just so exciting that the field was really growing at the time and so many new capabilities with these small harmonic drag gears that we could use that had not as much backlash. Before, the backlash for making these legged robots with six degrees of freedom in each leg was causing a lot of issues and problems for people trying to do it. Of course, you had Mark Raybird in the leg lab jumping, but this was really exciting to me. And I particularly wanted to build algorithms to allow humanoid robots to compute their motion on the fly in real time and avoid obstacles and remain balanced. So it’s a really tough problem and it was a perfect time to be exploring new techniques. So I applied this motion generation techniques to not only robots, but also to animated characters, complex 3D animated characters. And that was one of the things that also connected my love of graphics and video games to robots.

Q1: So did you end up spending time in Japan in any of the labs there?

James Kuffner: Yeah, I had my first experience through the National Science Foundation Summer Institute. I spent three months at Professor Shigeo Hirose’s lab at the Tokyo Institute of Technology and was just blown away by the mechatronic and the craftsmanship of the power electronics and the mechanical systems. And I was a trained computer scientist, so I was bringing a lot of software engineering capabilities. But combining that with the hardware, I felt was a great opportunity and complementary skills. And so I was learning about how to design and build better mechatronic systems and at the same time collaborating and trying to build really clean architectures. At that time, C++ was just becoming popular, a way of having 4

object-oriented programs that were much more robust to failures like segmentation faults that would happen all the time on C-based systems.

Q1: And so how does that then influence your work going forward?

James Kuffner: I really enjoyed a lot and I determined that I would, after I graduate, I would go back. I took a year off to start a company with, as many of my friends did at Stanford at the time. There were 30 of us in the PhD program in computer science during my class. And the internet was just taking off. And so many new companies were being formed and we decided to create a company that was called The Motion Factory. And I helped program this middleware for games that was eventually released on Prince of Persia 3D and the company was sold to Avid Entertainment. So it was actually a really wonderful experience also about starting a company and trying to build technology and get customers and how much work that is. But it was also really fun and there was so much energy at the time because many new companies were being born. And what happened was is that when I graduated and got my PhD, I went back to school after I’d worked at tThe Motion Factory and only 14 of those original 30 students actually graduated. And the joke was that those were the dumb ones because the others started companies like Yahoo and Google. Some of my colleagues like Larry Page and Sergey Brin who were PhD students at Stanford started those companies. But I really was grateful that I had a chance to finish my degree and I was very fortunate that Professor Steve LaValle, who at the time was a postdoc in our lab and he went to Iowa State, he really mentored me and we collaborated and formed this wonderful collaboration around motion planning and sampling based planning that resulted in the rapidly exploring random tree work that we did together. But I’m very grateful to Steve for encouraging me to go back and finish my PhD and make sure that I graduated.

​ And then after I did graduate, I took a postdoc for two years through the Japan Society for Promotion of Science, the JSPS. It’s a Gakujutsu Shinkoukai Tokubetsu Kenkiwi in Japanese. That is at the University of Tokyo Laboratory led by Professor Hiroshika Inoue, known as a kind of father of robotics in Japan. And their lab was very unique because it was the Joho System Kogaku Buu or JSK lab. And they had a wonderful teaching method where it was very hands-on. Each student had a robot that they would build, by themself, everything, and program by themself. And they were using Lisp. So they were using higher level kind of symbolic language, functional languages, and they were rebuilding software modules that could be reused, which was nice because that was always a limit for people trying to build their own robot is that you have to build all the software in the infrastructure yourself, which before you can actually do any research or make any progress in the field, you have to do so much infrastructure just to get the robot to move. But they were reusing software and it was a wonderful place. So for those two years, I met some lifelong colleagues and friends and learned so much about the robots. And it just so happened that their lab was building humanoids. So when I showed up on the first day, there was a humanoid that I could program and try to make walk and manipulate things. That was the H5 humanoid in 1999. That was when I had my first postdoc experience. And I stayed there for two years, worked on the H6 and the H7 and then HRP2 robots and got them to do motion planning and balancing at the same time, using lots of different techniques. And then that led to my successful appointment as a faculty member at Carnegie Mellon University at the 5

Robotics Institute. And I got to work and learn from so many amazing robotics professionals at that place. So that is really how my connection to Japan grew after my graduation. And then it continued for the rest of my life.

Q1: So can you tell us a little bit about your time at Carnegie Mellon?

James Kuffner: Yes. So when I came to Carnegie Mellon as a young faculty member, I was fortunate to work with Professor Jessica Hodgins, who was running the computer graphics lab there. And she also had a blend background of graphics and vision and control and robotics that was really similar in some sense to the things that influenced me. And I learned so much from her. She really helped mentor me. I got to work with her students and her lab equipment and participate on grants. It was really wonderful. I also got to work with Professor Takeo Kanade, who was a former director of the lab. And he also mentored me so much, built collaborations with Japanese companies. We had collaborations with Denso. I had collaborations with Honda and Toyota. We were the first lab. I established the planning and autonomy lab. And we were the first lab to get an Asamo outside of Japan. And we made footstep planning and other motion planning work with that robot, but built lots of other little humanoid robots. We had little dog robots with the DARPA, local motion made by Boston Dynamics. We had HRP2 through the AIST, the Sangyutsu Sogyo Kenkyujo Laboratory in Japan. That was Takeo Kanade established the Digital Human Research Center, and that I became affiliated with. So I had a long collaboration of sending students as well as exchanging scholars between Japan and my lab at Carnegie Mellon. I also forged a partnership with Karlsruhe Institute of Technology. Rudiger Dillman and Tamim Asfoor came and we exchanged students and collaborated with their Armar humanoid.

​ And one of the first things I did when I became a young faculty is I created a game programming class. And the reason was I felt that you could teach all of computer science with games. Because you needed to teach memory management, you had to have very good performance in terms of networking and display. So there was human computer interaction. You had non-player characters that are basically AI agents making decisions in the game. You may have also path planning and navigation and operating systems, all of those things. And I had to work in real time and have high frame rate updates. So many systems, things and issues of concurrency and threading you could teach. And so when I started this, I asked to get this new course approved. I said, “I’m going to teach game programming.” And I got pushback from the faculty. Carnegie Mellon was a systems engineering place and many faculty were like, “Well, our students don’t play games.” But of course we had Randy Pausch and the Entertainment Technology Center. And Randy Pausch gave me some great advice. He said, “James, all you have to do is retitle your class, real time graphics.” And I, sure enough, as soon as I did that, it was approved. But unofficially it was known as game programming.

​ And so I was able to take this class together and we were able to teach a lot of things related to robots as well. Because one of the things that I really felt was building a simulation of a robot could allow you to debug the algorithms and the software before trying to embed them on the hardware. Because, if you have buggy software running on prototype hardware, when 6

something goes wrong, you’re not sure is it the software or the hardware. But if you can simulate it or emulate the hardware with a simulation, then you can try out algorithms and test them and then verify the expected behavior in motion virtually. And it doesn’t hurt anyone and it costs a lot less. And then when you’re confident that the software is working, you can try to embed it in the hardware and then when something doesn’t go wrong, most likely it’s a system integration issue. So that was really a wonderful experience for me. Collaborate with Chris Atkinson on Locomotion and we, Takeo Kanate and myself, also co-advised students. So that was my great experience for almost seven years, learning and exploring what we could do with humanoid robots. I also got involved with autonomous helicopter projects and also got involved with advising students who were working on Carnegie Mellon’s automated driving team, which was participating in the DARPA Grand Challenge and the Urban Challenge, particularly the parking lot planner and the other motion planning and issues. And later that led to my next career move, which was leaving Carnegie Mellon and joining Google to work on the then secret self-driving car project in 2009.

Q1: Tell us about that!

James Kuffner: So it was basically Sebastian Thrun who had recruited me. In 2007, Carnegie Mellon won the Urban Challenge and I think that really showed people that, hey, this technology is really starting to mature. I think it was because of the reliable LiDAR sensors that were allowing good localization and mapping. The GPS was getting better, differential GPS and inertial measurement systems. Also the cameras and the speed of computer vision was getting better. And we just had powerful enough computers that we could compute these trajectories and keep the car safe and navigate traffic and recognize traffic lights. So there were so many things that were happening that Google decided that, can we commercialize this? This will completely change transportation, hopefully for the better. And I knew that this was going to be exciting and I had to be a part of it. So I joined a team of about 12 engineers in those days. And the hard part was I couldn’t say what I was working on. People probably had some ideas because of the people that they were gathering. But it was really amazing. I thought I was a good software engineer before I came to Google, but then I realized that I didn’t know anything about how to write real quality code. That means having good code review, having unit testing and regression testing, having a style guide with uniform naming conventions and documentation, just really making a well tested, well architected system that’s readable and collaborative development can scale. That was just amazing. And I think Google did a really smart thing. They hired domain experts from robotics. I was responsible for the planning on the car, but they had also seasoned engineers who had helped build Google’s web search business who were some of the most skilled and gifted software engineers in the world come and work with us and help us create a code base that was well architected and designed.

​ And then eventually, of course, that effort became Waymo. And I don’t know if they are using any of my code, but I’m proud of the code that I wrote and what I learned. And it was such an amazing time. At that time, only big companies like that had data centers or access to data centers. So when I was in my lab at Carnegie Mellon, I would buy giant radar rays and hard disks and try to collect data, but it didn’t scale the same way that a Google Cloud scaled. And so 7

it almost changed the way I thought about robotics problems. And that really led to me deeply going into this idea of cloud robotics. The idea that, instead of having one robot collecting experiences and then trying to learn from its own experiences, having a fleet of robots that could upload their experiences to the cloud, you could train a machine learning system, and now suddenly, the whole fleet becomes more intelligent. And also, you could offload a lot of expensive computation to a cloud with a network connection because at the time, wireless communication improved by a factor of a thousand. Like the bandwidth that we could send wirelessly suddenly became huge, and so many things became possible. So I really got into this concept of cloud robotics, and I’m happy to be able to kind of promote that concept. But in those days, the only way you could get access to a cloud was to work for Microsoft or Google or Amazon and really see the power of having almost unlimited computation and data storage. And then it changes your mind about how to architect an algorithm, how to think about problems when you have that as a computing resource and not a single threaded PC that’s running in your lab.

Q1: So when you think, just like the premature questions, I know you have stuff to talk about also in terms of working with Toyota, but I’m just curious because you have this experience of both academics and university work and then work, as you’re mentioning, with these very different kinds of resources. So when you think about robotics at the academic, on the academic side, and then robotics as it’s developing on the industry side, what do you think are the strengths of each and what do you think each has to contribute going forward?

James Kuffner: I remember when I was deciding whether or not I should leave my tenured job and move from Pennsylvania to California and work on this project that may fail. Should I take that risk or how should I think about it? But it stuck with me late at night. I was working in my lab on a paper and I was very proud of the paper that we had just finished and I was looking at and I was thinking to myself, I really hope somebody reads this and turns it into a product. And then it occurred to me, actually I could just skip that step and work on the product. And so I think you actually need both to make an innovation. You need research and you need to open that research and publish it. So there’s the power of the paper, the dissemination of ideas, expanding the boundary of human knowledge. But there’s also the power of the product, turning those ideas into something that actually changes people’s lives and impacts people in a tangible way. And so I felt hungry to somehow have the output of what I was working on, have a more direct impact on people. And that was really wonderful to have both of those experiences. And I was able to continue to publish at the same time, working on products. And it’s just been a joy to learn lots of different things and have the power of the paper and the power of the product both try to make an impact.

Q1: There’s significant concern, I think, in academia at this point also, just in terms of, in some sense, the disparity of resources, right? So what do you think, I’m kind of curious, like what do you think there is, aside from, in terms of, is it mostly opening up new areas of research that might not be close to the product? And therefore, that’s where academia can make a difference?

8

James Kuffner: I think there’s so much room for innovation of trying out ideas that are high risk, high reward. And I think curiosity-driven research, where you explore new areas or you combine ideas from either bio-inspired robots or psychology and human cognition, all these things can, I think, somehow come together better in a university environment in many ways, because companies tend to be more short-term focused or product focused. But at the same time, in the example of the cloud, eventually these resources become democratized. Now, instead of having to build your own data center, you can just rent cycles and storage. Anyone, in fact. Having cloud computing as a service has changed the industry tremendously and created so many new companies, because now you have an idea, you can leverage the stability, and the reliability and the speed and performance of a modern cloud without having to build one or manage it yourself. And so those resources, I think, have become much more democratized. And now, anyone with an idea can start a company hosted on the cloud and only pay for the cycles and storage they use. So it’s really wonderful, you know, how that has changed.

​ But going back to the question about the role of research, I feel that the closer you can get to making a product, as a researcher, the more successful it’s likely to be. And one of the things that I liked about Google research, as opposed to, I think, Microsoft research had more of a, I think it’s changed now, but back then, it was more of a research group that would build a lot of ideas and publish them. But they would kind of throw the prototype over the wall to a product group who hadn’t actually originally designed it. And somehow, this system, they would have to try to make reliable and low cost. And that doesn’t always work, especially with software. So I think having the original authors of a software who are doing research, but at the same time, they’re actually responsible for launching it and making sure that the customers have a good experience and fixing bugs. It’s actually really healthy. And you can always dial that in a different way. Like some people are really good at doing research and making prototypes, and other people are really good at debugging and making sure that the code works perfect and paying attention to small details. But in a mix of those people, you can make a strong team. So we actually adopted that model when I left Google to establish with Dr. Gil Pratt, the Toyota Research Institute in Silicon Valley. That was in 2016. And I was the Chief Technology Officer there. And I tried to adopt that same approach, having research that would be connected to the development teams and the product teams.

Q1: So can you tell us a little bit about how that came to be?

James Kuffner: Yeah, so after the self-driving car project became public, one of the things that we were able to accomplish was what they called back then the Larry 1K. So Larry Page had outlined a thousand miles of traffic routes all over California, around Lake Tahoe, down the coast in Hollywood, up San Francisco, all the bridges of the Bay Area. And these routes were really challenging. And the hope was that, if we could do these routes autonomously without human intervention, that we would have achieved some milestone that would maybe unlock a gate for the next phase of the project. And after that happened, it was so exciting. The team grew a lot bigger.

9

​ And at the time, I started doing managing of some other research teams that were doing computer vision work. We did some 3D scanning work and modeling work. And then I was getting really interested in how cloud technology could accelerate robotics by accelerating fleet intelligence. And it came from an experience that I had at Carnegie Mellon, where one of my machine learning colleagues told me, “James, I know this machine learning algorithm is going to work. It’s just that my robot breaks down before I have enough data. So instead of having one robot generate 10,000 hours of data, let’s have our 100 robots generate 100 hours of data each and get the same amount of information and maybe even more information because you’ve got variance and it’s more robust to uncertainty and not overfitting to one mechanism.” So I really got excited about this cloud robotics and I worked with a small team at Google that we organized and pitched and got funding for to work on cloud robotics. We open sourced an Android framework to use people’s old Android phones as small robots that people could program and use. And then later, we got a lot of funding to actually really build a formal robotics team at Google.

​ So I pitched it to the then vice president, Andy Rubin, who was the father of Android and he supported us. So we ended up buying several robotics companies, including Boston Dynamics and got some amazing talent. I had some former students who were working there, so I got to work with them again. It was really a great time. And when Andy Rubin left Google, I was in charge of the whole team. And so one of the struggles though was that there were a lot of robot technologists and research, but making a real business, especially for non-factory or non-warehouse applications was still very hard. And machine learning was still in the nascent phases. It was starting to change, but I think we were a little bit ahead of the time. And when Andy Rubin left, we had not as much support for what we were doing.

​ And right around that time, Dr. Gil Pratt reached out to me and said, “Hey, we’re going to start this new research institute in Silicon Valley for Toyota to really try to commercialize some of the automated driving technology and turn it into safety technology.” And that really motivated me personally, because my grandfather was killed in a car accident. And because of that, I never met him. And I always thought that if he had been driving a modern car, that he would have survived. And I’m sure that’s the case. And I really felt that Toyota, when I looked at how they talked about safety technology and their commitment to it, that this company was serious about it. And that instead of just focusing on a driverless taxi business, they were thinking about how we can use this automated driving to expand mobility for everyone. And it was also personal for me because a couple of years ago, I had to take away the car keys from my mom, together with my dad and my siblings, because she was losing her memory and her ability to drive. And when that happens, people lose a lot of freedom in their life. But imagine if you didn’t have to drive, and the car could drive for you. You could go visit your friends, you could go wherever you want, you would still have the freedom of mobility. And that was Toyota’s mission, was freedom of mobility for everyone. And that really resonated with me. So someone who couldn’t drive either they were blind or too old to drive, in fact, all of us at some point become too old to drive.

​ That really resonated. So I made the decision to leave Google and join Toyota. And we established the Research Institute, we hired some of the best people I could find in the industry, a combination of PhDs and industry specialists, had about half and half. And we partnered with 10

MIT, Stanford and Michigan and now many universities to try to push forward this safety technology and embed a lot of the things that we had developed for driverless cars and the techniques that we’re burgeoning now with machine learning into Toyota’s cars. But it became apparent by 2018 that, if we really wanted to make an impact with our work, someone had to go to Japan. And that was me. So I went in 2018 to Tokyo to establish the Toyota Research Institute Advanced Development Company. And it was a TRIAD. And eventually, that company was renamed the Woven Planet Group, because we had created other companies that were helping support its mission. The main mission was to productize this technology into Toyota and Lexus cars. And that culminated in our launch of the advanced drive teammate on the Toyota Mirai and also the Lexus LS 500 product. So that was level two, ramp-to-ramp automated driving hands off with lane changing capabilities. And a lot of innovations that were first for Toyota, including the first deep learning system that was in mass production. So I’m very proud that we were able to actually do that. But this experience was amazing for me because I learned so much about how hard it is to make a car and to sell a car and certify a car and all the requirements around ISO 26262, ASLD safety certification and the regulations of selling the car in different countries. And really just the amount of testing and polish that you have to do because people’s lives are at stake. But I’m very proud of what the team was able to do.

​ And I eventually took on new roles at Toyota beyond just being CEO of the Woven Planet Group, which is now Woven By Toyota. But I had hired, you know, thousands of engineers. And I was appointed to the Board of Directors of Toyota and became their Chief Digital Officer and served for three years, really trying to transform the embracing of digital tools, also pushing the company to be a cloud first company with modern data processing and storage and machine learning and, and really bringing more AI into a lot of the work. And in parallel in 2020, the president, Akio Toyota, promoted an idea to turn the old site of the TMEJ, the Toyota Motor East Japan factory in Susono, Japan at the base of Mount Fuji, into a new test course for mobility called Woven City. So I got to lead that project from the beginning to really build something new for the world that allowed us to test mobility rapidly in a closed course environment with automated driving, with personal mobility, with delivery robots. And that was really fun because I got to go back a little bit more into robots and hire some of my friends again in the robotics community and push that forward. So it’s really exciting to see that project now opening up its first phases in Japan.

Q1: And what do those first phases look like?

James Kuffner: Well, we are still testing internally many of the products, but one of the things that’s really exciting is that it’s meant to be a place like an inventor garage that will allow entrepreneurs to experiment with new kinds of mobility technology, also carbon neutral-related technologies for sustainability. So energy, the environment, mobility and safety, all those things together, we can test in a framework that is supported by Toyota. So I’m very excited about that. It’ll officially be opening next year. But we’re finished with the phase one design. And the president has said it should be ever evolving, a test course for mobility. So, a living laboratory and be human centric, meaning our goal is to make technology that supports happy, healthy human life. And that’s, after all, what I think robotics should never lose sight of. There’s a lot of 11

fear that AI will somehow replace us or dehumanize us. But, I believe if we’re careful and we have the right framework, the robots that we build will help us and enhance our lives and hopefully bring us more happiness. So I hope that that can happen. But, you know, the society will have to decide and wrestle with that.

Q1: How do you, as you develop these kinds of systems, you know, aside from the technical physical safety side, how do you incorporate this, you know, you’re mentioning kind of a care for the human and for human well-being, how does that end up playing a role in the type of work that you do?

James Kuffner: Yeah, I’ve always been continuing to be inspired by the kinds of robots that were portrayed in Star Wars. Friends, and having personalities and interactions and intelligence, and a lot of robustness and autonomy that right now we still haven’t achieved. But I think we will achieve and I hope we achieve in our lifetime. And I do believe that they could coexist with us and help us get things done. There’s always been the robotics, classical 3Ds, the dull, dirty, dangerous jobs that could be automated and have, you know, give us back more time to do things that we love, like art or music or taking care of our children or taking care of our aging parents. These are things that humans should be doing. But robots can help us and I believe that they can also become companions and maybe even help loneliness. So I have a lot of expectation and optimism that if we do it well, that we can be happier as a society with robots in our world.

Q1: Thank you so much. That was a beautiful way of going back to where I was.

James Kuffner: Yeah, yeah!

Q1: Is there anything that we kind of didn’t cover or that you would want to add into the narrative?

James Kuffner: Yeah, I guess. Yeah, when I reflect on 30 years of working on robots and automated driving technology, one of the things that I have realized is that models that we are building, you know, as I was trained as a classical roboticist, include analytical models with physics and dynamics and kinematics and all the things that we use to reason about a system. And recently there’s been a lot of amazing data driven techniques which are just based on training neural networks, which essentially are approximating these functions towards reproducing outputs. And generative AI has produced some amazing results. But, I guess what I’d like to propose for the community is that we don’t necessarily need to choose one or the other. It’s a false choice. We should actually leverage everything we know about physics of motion and kinematics and dynamics, because we can reason about them and we have explainability and we can debug them and improve them. But we can use data-driven techniques, statistical models, to fill in the gaps of the analytical models. And to me, that’s a wonderful potential. And I’m so 12

excited to see how generative AI and data driven techniques are going to help improve the robustness of our systems.

​ The other thing I would like to say is, I feel, going back to computer graphics and the virtualization, you know, creating digital twins, we now have computer graphics capabilities of generating photorealistic images so that we could simulate a camera almost indistinguishable from images taken from a real camera. So we could virtualize a robot embedded in a VR-like world and test its behavior in a safe way. And in fact, you could parallelize testing that robot just as we do now on the cloud when we test a software module, we have a unit test and regression tests that can all run in parallel on the cloud, and then come back within five minutes and give you a report that the system is stable and improved and correct. And this is a wonderful way to validate our systems, not only for automotive and intelligence, but robots. So I think to me, one of the most exciting things is now that all these things are converging, there’s cloud technologies, there’s new sensors, new actuators, new power, powerful computing. And we have these wonderful graphics and modeling techniques that are now truly creating realistic digital twins of our robot systems that I hope will allow us to create safe robot systems. And safety to me is something that I’m really passionate about, because obviously, I told my team when we were programming cars at Toyota, it’s one thing if your web browser crashes or your phone crashes, it’s a completely different thing if a car crashes. So we have a moral responsibility to have bug free code. And we need good hardware and good software to make a good system. And that, I believe, is going to be fundamental to making robots actually help us.

​ And the final thing I would say is, we can use these data driven techniques. We can use AI and embed them, but we can have, at the highest level of logic of a system, programmed and debugged by a human. That means we can bound the behavior of the system and not have it do something so unexpected that’s out of our control. And I think that is a responsible way to try and design, you know, a system with guardrails that has all the innovations of data-driven techniques where it might be harder to explain. But if we can debug and modularize the system in a way that the behavior of the system is bounded, we can create robot systems that will always be understandable and help us as we move forward. So to me, that gets rid of some of the irrational fears that people have about these systems, and we can be responsible about how we develop them.

Q1: A wonderful way forward.

James Kuffner: Thank you. 13