Interview Transcript
Q1: Hi professor. Thank you so much for your time today to start an interview, because you tell me your name and a bit about your background and upbringing.
Ayoung Kim: Yeah. Hi, my name is Ayoung Kim. I’m currently working as a social professor at South Asian University in Mechanical Engineering. But basically what I’m working on is robotics broadly and specifically I’m working on a field called SLAM, simultaneous localization mapping. But that’s basically a combination of perception and navigation. So my background is mostly in mechanical engineering. I did my undergraduate, master, and PhD in engineering. I did my PhD at University of Michigan, Ann Arbor. But during that time period I worked on underwater robots, and I was working in the naval architecture and engineering department, so I think I have quite a varied background, like most robotics people having mechanical engineering. And I had six years of education working experience in naval architecture, working with underwater robots. And then after graduation, I came back to Korea and I joined KAIST, Korea Institute of Science and Technology, in the Department of Civil and Environmental Engineering. So from mechanical engineering to naval architecture and the civil engineering department. And I worked there for seven years as an assistant and associate professor. And I came to Seoul four years ago. My parents live very near to Seoul. Now my parents are living like 20 minutes from where I’m living, so it’s quite close. Yeah. Yeah, I think that’s pretty much my background.
Q2: So you started with underwater robotics. How did you get interested in robotics in the first place, whether it was any particular event or books or people?
Ayoung Kim: So I think how I got into this robotics area is in Seoul National. I did my undergraduate at National University, Mechanical Engineering right here, and there is a first-year freshman student course, creative design class, that’s basically designing a robot. And I think that class really motivated me a lot to work on mechanisms. And at the time there wasn’t much focus on software, but still the robot hardware and mechanical mechanisms really attracted me a lot. And very fortunately my team got quite a good score, and I was selected to go for another level, which was the worldwide RoboCup, a great worldwide competition. And then there, many different countries came together, and there was a lot of diversity within the team. We had to collaborate, communicate, and build a robot within two weeks. That was quite challenging, but it was really interesting and an amazing experience, and that kind of led me to this career path, I think.
Q3: Where was that RoboCup at?
Ayoung Kim: At the time, the RoboCup was held at MIT. The UK, France, Germany, Japan, Brazil, and Korea, I think six different countries came in. Each team had diversity, so one from each country. That consisted of students from all different countries.
Q4: Did you remember the first robots you heard about?
Ayoung Kim: Well, the first robot I heard about is from cartoons because I think many of us watched cartoon animation a lot when we were young. There were famous Japanese cartoon animations like Gundam or Atom. So I think that’s the very first robot that I saw.
But that was a little bit far from what we are doing. It was a robot or something like movie characters, like Superman and Batman. So for me, I had in my mind there was a distance, I think.
And the first robot that I think about is when I started working on robotics, and then when I built it, I started watching TV and ASIMO came out. All the nice walking robot platforms and mechanisms came out. I think those later came into my career path, but I think that’s more impactful.
But if you ask me, the very first robot is probably coming from cartoons or animation.
Q5: What were some of the main research questions or aspects of robotics you were interested in, and how did they change over time?
Ayoung Kim: So my research interest was mainly because my background is on underwater robots. I work on SLAM perception. And if you think about seeing things underwater, it’s quite challenging.
Sometimes if you go to San Diego, Hawaii, nice clean water, you can see through things probably a meter away from you, but it’s not always the case. There are some coastal areas where the water medium is quite turbid and you can’t really see through.
And that’s when I learned there are more sensing capabilities than vision RGB. Humans rely heavily on our eyes. We see things and understand things. But for underwater robots, what we do have is sonar.
You change the sound signal into some form of imaging and then understand things. Later, for autonomous vehicles, we have many different sensors.
So the research question that I have is: okay, for humans there might be one major dominant sensor. But robots have many more varieties, and you can make a lot of choices. They have their own pros and cons.
How can I combine things better to work in a complementary manner? And when something fails, the other sensor can back up the sensing capability, and vice versa. So that’s one major research question that I have.
Q6: Have you encountered any major challenges in your career or research?
Ayoung Kim: Yeah, yeah, I do. My background was on underwater robots, so I was aware of the challenges that I might experience during robot navigation.
But when I was working at KAIST, I was involved in one of the projects, and I had to fly a drone in a firehouse for a search and rescue mission.
I thought at the beginning it should be doable, but when I actually saw the situation, the environment was filled with smoke and fog, with really thick particles in the air.
The camera fails in thick fog or smoke situations. Those two sensors are what most robotics people are mainly focusing on, and then I was so frustrated. I needed to find something else.
Luckily there was a professor in Chile, Martin Adams, and he had a conference and randomly contacted me. He emailed me introducing himself. He was working on this stuff, and one of the items was radar.
It was random email communication, but he came to our department and gave a talk on how we can use radar. He showed us how radar can help the mining industry, where thick particles make cameras and LiDAR have severe limitations.
He showed some promising capability of radar. So I started working on radar ever since I met him. After learning from there, one of my major current ongoing works is based on radar sensors for robot navigation as a backup for cameras and LiDAR.
That was a big breakthrough.
Q7: Continuing from where I left off: Yeah, yeah. And that really taught me a lot of lessons because that kind of showed me not only the research aspect, but it taught me how I should communicate with other researchers.
Ayoung Kim: Ever since that happened, I tried to reach out when I go to a conference in the US, Japan, Europe, or China. I try to reach out to nearby universities. It’s quite random. There is someone I would like to meet with. I might motivate somebody else, or I can also be motivated by somebody else after meeting somebody.
Yeah. Relationships. Yeah. Yeah. That’s very, very important, I think.
Q8: You mentioned these amazing academic relationships. Do you also work with industry?
Ayoung Kim: Oh yeah, yeah, I do. In Korea, Hyundai Motors is one of the major collaboration groups that we are working with. We have a joint lab between them and Seoul National University working on robot manipulation.
It might be confusing. I’ve been saying that I work on navigation and how manipulation works, but for me, the manipulation, finding the motion of the manipulator, the hand, or the end effector, is quite similar to, for example, a drone flying into space with a camera on it.
The robot hand with a camera moving into space has quite similar characteristics. So I’m also working on the manipulation part in that sense, from a perception perspective.
Hyundai is one of the major collaborators. I also work with Samsung Heavy Industries for shipyard monitoring with robots. Also, the defense industry. I work with them, like Hanwha. Those are domestic Korean companies.
For international companies, I worked with Honda, another motor company. They have a robotics lab, so I had a chance to collaborate with them.
And currently I’m working with the Robotics AI Institute, previously Boston Dynamics AI. We are in a long-term project relationship. We started last December and it goes through three years.
I was on sabbatical leave from last September to this August. During that time, I visited the Robotics AI Institute and worked with them, writing papers and doing research together. So yeah, there are quite diverse industry partnerships that I have.
Q9: You manage a lot of big collaborations. That’s not happening at one time. I’m talking about like the last five years. How does it usually start, this collaboration between industry?
Ayoung Kim: Many of them, there is no unique route. But I think I meet people at conferences.
For international partnerships, there was IROS Kyoto two or three years ago. I was giving a keynote there, and after the keynote, people usually come and ask questions and discuss things.
That’s how I met Honda, because that was happening in Japan after my keynote. They came and showed me some problems they wanted to solve, and then asked me if I was interested in solving that together.
So yeah, there was also RSS. One of the robotics conferences happened two or three years ago in Korea. At the time, Boston Dynamics AI, now Robotics AI Institute, Mark and Al came to RSS and had a chance to visit my lab. I showed them our work.
So I think conferences are really a major thing because that’s the major time when people move to a certain location and we all meet together, so we can communicate a lot.
I think conferences are a major route. Or if I publish some papers or upload a nice YouTube video about my research, and then people see it and reach out to me, that’s another major route.
Q10: Here we are at the conference. You’ve accomplished so many different things and projects in your career, but is there anything that you always wanted to pursue but haven’t done yet?
Ayoung Kim: Oh yeah. I don’t know. I would like to—I have kind of yearned for field robotics.
I feel much more motivated when I work in the field rather than sitting at my desk working on papers, documents, and coding. I work a lot at my desk, but still I feel much more motivated when I work outside in the field. I prefer coding outside, sitting on one of the rocks and coding and working out there. I try to go for really challenging environments because I can learn a lot from them, but I haven’t really had a chance to work on that much yet. Space robotics or—I did most of the shallow water underwater robots, something like the Mariana Trench and really deep dive-level underwater exploration, robotic exploration. So I want to go somewhere more challenging, like space or deep underwater, something like that.
Q11: One of the things that roboticists like to talk about, looking at the current state of robotics, is that there are way more opportunities to go out in the field and observe robots in the real world. Have you noticed any major changes over the past 10 to 20 years?
Ayoung Kim: Yeah, robotics changes so fast. Twenty years is a long history. I think I would like to focus more on recent advances, but still it’s quite a big gap for us because AI came in over the last ten years. Ever since AlphaGo showed AI’s potential and ChatGPT came out, now the main challenge that we try to solve is how we can solve something that we couldn’t solve before with models. Can AI fill the gap and let us go through and complete this engineering? I think that’s the major challenge that we all try to achieve. There’s a little controversy. People always worry about losing their jobs because robots take their jobs. I don’t think that’s necessarily the case. There is a possibility, but there are more chances that robots can alleviate the burden for humans and let humans work on the work that people want to work on. I think I see much more opportunity in that direction. I’m an optimistic person, so I see more possibilities.
Q12: I see as an academic in robotics, do you get this kind of more negative or pessimistic feedback from what kind of people? Do you hear it often?
Ayoung Kim: Yeah, I hear this from everywhere. Like when I go to conferences, if I go to Uber and they ask me, “Where are you going?” and I say, “I’m going to a robotics conference,” some people have a very optimistic view, but some people don’t. They are worried about losing their jobs, and then they don’t want to work for, you know. So I hear that a lot. But I try to persuade them that there are much more opportunities that we can get. Like having a robot vacuum cleaner in your house doesn’t really mean you’re losing a job. You can use your time more efficiently because of the robot. So I try to persuade them.
Q13: Related to that, what do you think is your biggest contribution to the field of robotics?
Ayoung Kim: I think I can say two things. One is regarding my research. I’ve been working on some non-conventional sensors a lot over the past five or six years, such as radars or thermal cameras. I tried to show some showcase studies where those sensors can be helpful, and how those sensors can be combined with existing major sensors so that people can understand their potential. Radar is noisy. Thermal cameras don’t really show much, but I try to show their potential and possibilities. So I think that’s what I am mainly working on. If it is not the sensor itself, I try to tackle challenging objects. For example, glass or transparent objects. There are abundant transparent objects in our lives, and sensing capability may be limited for those objects.So either the sensor is challenging or the object is challenging, I try to solve those challenging corner problems.Those are some contributions that I make as a researcher.
Another thing is, together with all the SLAM people in the world, with around 70 authors, we wrote the SLAM Handbook together. The draft is done. We are finalizing the very final step. I was very privileged to work as an editor for that book together with other amazing editors, Luca Carlone, Frank Dellaert, Tim Barfoot, and Daniel Cremers.So I think that’s another contribution that I make for this community.
Q14: Who is the main target for the handbook?
Ayoung Kim: When we were writing it, our main target was first-year graduate students or senior undergraduate students who are trying to learn SLAM. So for beginners, this is a handbook. It starts with the history and all the theoretical parts. Then the second part covers applications. You have LiDAR SLAM, visual SLAM, event-based SLAM, and radar SLAM applications. The third part is how AI and deep learning can boost SLAM and what the future directions are. So this is going to be a quite complete handbook. It took us two full years.
Q15: Oh wow. Congratulations.
Ayoung Kim: Yeah, yeah. It’s going to be available worldwide. It is already available from our GitHub, so you can see the PDF. We are working with a publisher, Cambridge, and we hope to see the actual physical book next year.
Q16: Looking forward to it.
Ayoung Kim: Yeah.
Q17: So there’s a bit of a change in the subject. So we’re at the Humanoids Conference in Korea.
Ayoung Kim: Yes, yes.
Q18: What do you think makes robots humanoid compared to, for instance, other robots that are not humanoids?
Ayoung Kim: Well, as a perception person, for me, humanoid robots and wheeled robots do not have a major difference. Because whenever we have eyes and we can see through, from a perception perspective, it’s similar. But I think from an application perspective, why we develop robots is to lessen the burden that we have in human-level labor. Humanoid robots are needed to access areas where wheeled robots or four-legged robots have limitations. So I can definitely see the possibility in humanoid mobility, and the fact that it can use bimanipulation, I think that really is the key difference.
Q19: So why do you think specific applications that humanoids can be helpful in real life?
Ayoung Kim: I don’t think this is going to happen in the near future because it’s quite challenging. But at the beginning of this interview, I told you that I worked in civil and environmental engineering. During that time, and still today, one of my research fields is construction site monitoring. There are many areas in construction sites and the entire construction process where robotics can be really helpful. Safety and quality management are really critical in civil engineering. I don’t know if you have ever been to a construction site before, but it is quite compact and very complicated and unstructured because we are building something. People are moving through very narrow paths in very high locations. For those situations, robots have limitations. Drones have battery life and duration limitations. Humanoid robots also cannot currently go through indoor spaces very safely and work with humans in that sense. Humanoids, in terms of human-robot interaction and complicated construction site accessibility, have much more potential. But I think it’s really challenging. People try to make humanoids work in factories. It’s quite a clean environment. Everything is regulated, predefined, and nicely cleaned. But construction sites are far from a simple, clean environment, so it’s much more challenging—another level. But I think in the future we should be able to have humanoids helping people at construction sites.
Q20: What are the biggest challenges to get to that point so far, like in humanoid robots?
Ayoung Kim: Many, many things. Scene understanding is also very important because in a construction site everything is changing rapidly. If you want to build a building, you’re building something, so every day you’re going to see new things. There are many things you have to see at the same time. The reasoning behind it is not based on our everyday-life common sense. You need your own domain knowledge for that. So I think how we can establish construction-site-specific knowledge, or any other application-specific domain knowledge, will be one of the challenges. Hardware-wise, it is much more challenging too. It’s not just walking on a flat surface. Sometimes the surface is not flat. It’s quite bumpy, and sometimes it’s not stable. If you are working on concrete or cement, you may not assume the floor is hard and very stable. It may not be stable. So there are situations where assumptions that you can make in daily life or very structured environments may break down. Yeah, it’s very challenging.
Q21: Even for humans, navigation is quite challenging.
Ayoung Kim: Yeah, yeah.
Q22: So to wrap up this interview, when do you think we can see humanoids on construction sites?
Ayoung Kim: I would really love to. One of the fields that I want to cover—like underwater or space—was quite challenging. But in the near future, I am also working on construction-site monitoring and related things.I hope maybe within ten years we should be able to see something going into construction sites, with more machines and more robots helping humans. Humans can be freed from some of the hard labor, and we can work on higher-level things while letting robots do the physical work for us.
Q23: That’s very exciting. Within ten years. And now you’re working on it.
Ayoung Kim: Yeah.
Q24: That’s all my questions. Thank you so much for your time. It was very insightful.
Ayoung Kim: Yeah. Okay. Thank you very much.