Interview with Marcia O’Malley

Marcia O’Malley: Okay, sure. So I’m Marcia O’Malley. I was born in Zanesville, Ohio. Spent most of my childhood in Ohio and then went to Purdue University for my undergraduate degree in mechanical engineering, and then Vanderbilt University for my PhD in mechanical engineering, and then became a faculty member after that.

Q1: Excellent. A faculty member?

Marcia O’Malley: So, I’m a professor of mechanical engineering at Rice University in Houston, Texas.

Q1: Excellent, thank you. Do you remember at what point in life you first got interested in robotics? Was there an early robot that you saw? Things of personal culture or family connection?

Marcia O’Malley: So I can think of two stories from sort of early years. So in high school, so I was coming through high school and the typical thing, I’m good in math and good in science and teachers are saying, you should look at engineering. And biomedical engineering was just becoming a thing. And I was doing a project for an anatomy and physiology class in high school where I learned about this physician, Ilizarov who had created this cage that could be placed around bones. So you screw into bone, and you can apply stresses to the bone, and you can cause bone regrowth. Not actually a robotic device because it wasn’t actuated, but that was sort of my first exposure to an engineered device that could improve an individual’s existence and experience. So for people with shortened limbs or bad breaks.

​ When I went to Purdue as an undergraduate, I explored some undergraduate research opportunities, and I was in a biomechanics lab. They were developing prosthetic knee implants. And I thought that that was really what was interesting to me was the prosthetic device, itself. So artificial knees, but they had a robotic device where they would test their prosthetic knee implants. This is a six degree of freedom knee simulator that you could load the prosthetic knee into, and it would cycle that knee through the gait cycle, applying all of the loads that would be experienced for an individual who might have that prosthetic implant. So they were doing fatigue testing and looking at wear on the materials. And I, again, in retrospect, I can look back and think that that was maybe a formative experience. At the time, I just, again, was sort of interested in this mechanical device, this idea that we could engineer or design systems that interfaced with people, and have some healthcare benefit. 2

Q1: I’m curious about how you see the difference between biomedical engineering and robotics, and where you draw that line. Is there a first project that you remember, that you saw most definitely as being part of the robotics?

Marcia O’Malley: Yeah, I think so. You know, when I had that undergraduate research experience in the lab and sort of looking at this knee simulator, I was definitely interested in doing more research and that’s why I went to graduate school. The lab that I joined in graduate school was, well, maybe I can back up a little bit. So when I was looking at graduate programs, it was a mix of mechanical engineering programs and biomedical engineering programs. And on some of those lab tours, it became clear to me that I didn’t really wanna be a biomedical engineer. I was really a mechanical engineer. I ultimately joined a lab that had more robotics focus to it. That was not what drew me to that institution, but ended up being the lab that I joined and was really interested in the work. And that lab was working on teleoperation, haptic devices, so no medical application at all. But the project that I pursued was, I was working with an early haptic interface. So haptic refers to the sense of touch, right? So it was a stylus-based device and you could move the stylus around and interact with a virtual environment that’s being rendered with the torques being fed back by the motors.

​ And so for me, it was really similar to the kinds of examples that I gave earlier. There’s still a person in the loop, right? There’s still this physical interaction between a human and a robot. And that was really the piece of it that was interesting to me. So I don’t know that I really draw a distinction between biomedical engineering and robotics. I would say robotic devices have their place in biomedical applications. And really what would constitute the difference between a prosthetic knee, the knee implants that I was looking at as an undergraduate, and the robotic devices that I work with now is really the sensing and actuation.

Q1: So that may be your earliest set of prime projects. What came next for you in your career?

Marcia O’Malley: So in my PhD research, I was interested in how we could understand the effect of the fidelity of the haptic display on human perception. So if you think about, you know, this is a thing we’re really familiar with with computer monitors, right? You want a certain refresh rate of the computer monitor to fool the visual system. We were exploring, what is the equivalent in a haptic system? What is the force output that we need or the bandwidth that the system must have to really render this virtual environment that is compelling and realistic to a human user? So that was the topic of my PhD research. And then I went immediately into faculty position at Rice and started my lab. And when I started my lab at Rice, I sort of took, the field of rehabilitation robotics was really active around that time. So this was early 2000s research labs out of Palo Alto, VA and Stanford, and also MIT were developing robotic devices that could be used for rehabilitation for people with stroke and spinal cord injury. And it turns out that the devices, the way that you would design a device for rehabilitation is really similar to the way you would design a haptic device to interact with a virtual environment. So a lot of the experience I 3

had in working with that haptic device in my PhD translated really well to developing rehabilitation robots.

Q1: Wow, now I know these are closely related but, as you think about the relationship between projects and research questions, what were the main research questions guiding you at this point in time?

Marcia O’Malley: I think, I’m gonna have to give that some thought. I think that common themes through research when I was first establishing my lab at Rice was the role of this force feedback, this haptic feedback on human performance. So if you think about the rehabilitation domain, how do we moderate the forces being fed back or the assistive forces that are helping an individual move their limbs to promote some kind of motor recovery following stroke? And in the more sort of haptics and virtual environment domain, what is the role of this haptic feedback in teaching somebody a new skill or providing them some embedded experience in this virtual environment that makes that interaction in VR more compelling?

Q1: You can think about this one.

Marcia O’Malley: Okay.

Q1: But, as you think back to the challenges you faced in kind of answering the questions you want, they could be technical challenges, or institutional challenges, or the challenges of working with interdisciplinary teams, what do you think were the most significant challenges that you faced as a researcher?

Marcia O’Malley: So, my PhD focused on sort of a single question that I was able to explore deeply over the five years or so that I was completing that degree program. When I became a professor and was starting my own lab, now I had to sort of guide multiple projects at the same time to advise students. I didn’t have a lot of experience advising students myself. I felt like I was still a student as a new faculty member. So, I think that was one challenge, was how do I do this in parallel? How do I pursue multiple research questions in parallel while also now teaching courses and serving on committees and doing all the things that academic does? Rice University is a relatively small institution. So in my department, I was the one person doing that kind of work. So I didn’t have, there wasn’t a lot of other robotics faculty around yet. We had one at the time, and that’s Lydia Kavraki who’s in computer science, but our research areas are pretty distinct. So she was a fantastic mentor, but in terms of research collaboration, it really wasn’t a natural fit at the time.

​ And so I really had to sort of establish myself independently without a lot of others to work with closely. An interesting outcome of that is that I started to establish some pretty 4

interdisciplinary collaborations. And one of my longest running collaborations is with Mike Byrne, who’s in the department of psychological sciences at Rice. So his background is in human computer interaction and how we can understand skill learning and human performance in visual feedback environments. And so we began collaborating in the late 2000s. And I mean, well, we’ve, I think we’ve had joint-funded projects now for over 15 years. But that was sort of a result of the fact that I’m at a place that has a lot of individuals, and trying to find those folks that I could work with, I often had to go pretty far outside my own department to find people with similar research interests, but just looking at them from a very different perspective.

Q1: Given the nature of your research, I’m curious, do you work with medical practitioners directly with patients and how, what has that meant for you and your research?

Marcia O’Malley: So one of the reasons that I started to pursue rehabilitation robotics at Rice was because we’re across the street from the largest medical center in the world. So the Texas Medical Center has 40 different institutions that make up the Texas Medical Center. There’s a hundred thousand people work there. So it’s this city really, within a city, right across the street from campus. So there were lots of opportunities to meet physical therapists and physiatrists and doctors of physical medicine and rehabilitation who were interested in how you could bring technology to the clinic for therapy. I also work with surgeons who are using virtual reality simulators for training their residents. I work with some surgeons who use the DaVinci surgical robot, and thinking about their performance on that platform and what might be the role of haptics to enhance that surgical procedure.

Q1: Has that affected at all your sources of funding? Can you say a little bit more about kind of where your funding has come from over your career?

Marcia O’Malley: So I’ve been fortunate to get a lot of funding from National Science Foundation. From my career award that I had early on at Rice, I’ve had support from CISE, the Computer Information Science and Engineering Directorate, and then also the Engineering Directorate through CMMI. And I think this field of sort of physical human-robot interaction has a lot of alliance with, well, I should say that differently, right? So the field of human-robot interaction, you can address these problems from sort of the human-centered perspective or the device-centered perspective. So the human-centered perspective and how humans and robots are interacting tends to be more on the CISE side of things, whereas, if I think about the device itself, its design, the control algorithms, that tends to be more on the engineering side. So the bulk of my funding over the years has been from NSF. I’ve also benefited from our proximity in Houston to Johnson Space Center. So I’ve had, early on, I spent two summers at JSC in a summer faculty fellowship program that exposed us professors to all the different research happening at JSC with lab tours. And that helped me develop some collaborations with the Dextrous Robotics Lab. That’s the lab that developed Robonaut and Valkyrie, so two pretty well-known humanoid robots in our field. And then, through the collaborations with the Texas Medical Center, I’ve also 5

had a small amount of NIH funding as well. Yeah, so the bulk of the federal funding, I’d say NSF, a little NASA, and a little bit of NIH.

Q1: You mentioned the Da Vinci systems. Are there other kind of, technologies or tools? I’m thinking of everything from small tools like Connect to conceptual tools like large language models. As you think, again, about tools coming from outside that have affected your research, have there been really significant moments?

Marcia O’Malley: Yeah, I can think. When I was first establishing my lab, I mentioned having this focus on rehabilitation robotics, but also then haptics in virtual environments. And a particular aspect that I was interested in was training. So how do I assess somebody’s performance in a virtual environment, and then what kinds of haptic feedback can I provide them that might accelerate their learning curves, or make them more proficient at a task? And a key piece of that is, what are the performance measures that really capture expertise? So I was exploring how we might research these questions in the field of robotic surgery, and spent a number of years trying to negotiate IP agreements with multiple institutions with not much success. Intuitive is a very successful company and also very protective of their IP. But through that process of meeting more and more individuals in the Texas Medical Center who were using the DaVinci, who were interested in robotics, I met a number of endovascular surgeons. And that field is a little less developed in terms of robotic systems, although there are some. And so, through those interactions, then I was able to collaborate with Hanson Medical before they were acquired, and work with some imaging companies. So Siemens and Philips, for example, they do a lot of the medical imaging that’s necessary when you couple that with robotic devices. So I think that access, right? First from the collaborations with TMC, and then seeing, from the perspective of the surgeons, the tools that they wanna use and the development process of new robotic systems was something unique to my experience.

Q1: Well, this gives me an opportunity to jump forward in my question, but I’m curious about this relationship with commercial partners, and thinking about, as your research moves into the real world, whether that’s a hospital or a commercial firm, can you talk a little bit about that experience, what it’s been like for you as an academic, are there different challenges? Have there been surprises to you?

Marcia O’Malley: So I think that the most impactful translational work that we’ve done is probably in the rehabilitation domain. We’ve made versions of our exoskeleton robots that are deployed to a hospital. So I work closely with TIRR Memorial Hermann Hospital in Houston, and now closely with Houston Methodist. So they’re recruiting patients. We’re running small clinical trials, evaluating the different control algorithms that we implement on our exoskeleton to see how they affect motor recovery. But I’ve never done that with the intention of commercialization, really just trying to understand how do we best design these systems. But through that relationship, there are companies that are distributing and selling exoskeleton robots to help with gait. So I consulted for a number of years at the rehab hospital as they were getting 6

approached by Indego, Parker Hannifin who has Indego and Ekso Bionics and Rex, all of these exoskeleton systems, the physicians really didn’t have a sense of, how do I compare these? What are the features? What are the target populations? How do I understand this exoskeleton technology compared to this exoskeleton technology? So I was able to come in and sort of help them understand what the different offerings were and what might be best suited to their clinical needs. So that’s been a really valuable aspect of my research and it’s also helped some of my students who wanna move in the direction of rehabilitation robotics and industry find some connections as well and start their careers.

Q1: It sounds like you’ve been very successful in getting funding. I’m curious whether the structures of funding have affected the kinds of questions that you’ve been able to pursue. Are there questions that you have not been able to pursue because there’s no existing mechanism for funding that kind of research?

Marcia O’Malley: Maybe this is not something that you’re gonna want. I think I’ve been really fortunate. As I mentioned earlier, we can think about this physical human robot interaction from different perspectives. So maybe it’s, we call it grantsmanship. There’s scholarship and then there’s grantsmanship, and how do you pitch your idea to align with a funding call or a different NSF program? Over the years, I’ve been funded by different programs. The programs themselves evolve. And so I think it’s really a skill of, I’ve got this research question that I wanna ask, how do I frame that in a way that aligns with this funding mechanism so that I can have the resources to do the research that I wanna do? I haven’t done as much on the medical translation. So I have had an NIH grant and, interestingly, it was through the National Robotics Initiative. In the early years, it was a joint program. So all your proposals go to NSF, but they might get funded by different federal agencies. So I submitted an NSF proposal that was picked up and funded by NIH. I’ve not, myself, pursued NIH funding. And I think it’s just a different flavor of mechanism. Maybe I see myself more on the sort of fundamental research questions about these physical human robot interaction scenarios, as opposed to really thinking about a specific clinical situation where we need a medical device, a robotic device to address that clinical need.

Q1: You’ve answered this next question, or aspects of it in a variety of ways. But if you could think about your most significant contributions to the field, that would be helpful to kind of have those all in one place. And will you have the talk a little bit about your legacy?

Marcia O’Malley: I actually feel like my legacy is the people, and not so much the research. I mean, of course, I’m very proud of my own research and I think it’s important and been worthwhile to pursue. And I think we’ve answered some really important questions about how can we use haptic feedback to train new skills? How can we design devices that provide haptic feedback to users? But really that’s been a mechanism to really bring examples to the classroom with undergraduates and get them excited about research and want to pursue graduate study or, in training, graduate students who might want to go to industry, or have a startup company, or go to 7

academia themselves. So when I look at, through my career and what I’ve done, I would say it’s the people that are the legacy more than maybe the research impact itself.

Q1: Again, I’ll use that to jump forward in my questions. But as you think about students entering the field today, you’ve talked about a wide variety of skills, including grantsmanship. Kind of what do you think are the necessary ingredients of a career and kind of what would you suggest for students thinking of entering the field? What do they need to master?

Marcia O’Malley: I think these days, students coming to robotics have much more exposure prior to arriving on a campus than in my generation. I think of my own kids. So I have 17-year-old twin boys who are applying to colleges right now. And there’s a 3D printer next to their bedroom, and they’re teaching themselves SolidWorks and they can have access. I think in some ways, we are fortunate that they have that access to technology prior to entering their studies. So I think they have maybe students these days might have a better sense of the landscape of robotics. When I started graduate school, I had to define haptics for everybody. Nobody knew what haptics was. Now, you know, haptics is in my smartwatch, haptics in my phone. So maybe there’s a greater sense of understanding the landscape of the field of robotics now. But in terms of the skills, we definitely, you know, still the fundamentals, right? You can’t do anything without the math and physics that underlies everything. As a mechanical engineer, I think about sort of dynamics and controls and mechatronics and these courses.

​ But in terms of like a career-long success, I would say, persistence and resilience are probably the characteristics that are most indicator of success, right? So the experiment fails, the robot isn’t working that day. How do you debug, address the problem, find a workaround and keep moving it forward? Whether it’s the experiment or the robot or the paper that gets rejected or the proposal that gets rejected. Stepping back and saying, “Okay, how do I reformulate my idea or how do I change my strategy so that I can still keep moving towards the goal?” That would be what I, it’s what I look for in students when I’m recruiting, that notion of, you know, I’m not gonna be, I’m not gonna let that setback really derail what I’m trying to do.

Q1: As you think beyond your own career and to the field as a whole, and however you understand that, how has the field changed over your experience?

Marcia O’Malley: The first ICRA conference that I went to was in 2001. It was in Seoul, and I was near the end of my PhD, and I don’t remember the exhibits. I remember the sessions. I remember the keynotes. I remember meeting people, you know, other people who were sort of my cohort, and there was probably an exhibit hall, but now, you walk through the exhibit halls, you know, there’s 50 to 100 companies, there’s robot parades, there’s, I mean, these were things that just didn’t exist. It was a very, maybe academic exercise, the conferences when I was more junior to now we see industry representation, we see funding agencies coming. I think there’s just 8

a lot more coordination and heterogeneity to the population that’s attending conferences and the opportunities for students.

​ If you were coming out with a PhD in 2001 like I did, you were probably going to a government lab or to academia. There were maybe a few companies that were hiring intuitive surgical, but not very many, and I think now the opportunities are much greater for students coming out. It’s not just, you know, it’s gonna be academia or maybe there’s this one or two companies. There’s lots of different startup companies, more established companies, even very large corporations that are thinking about how do we bring robotics in, right? Toyota, GM, just for example, in the US. I’m in Houston. Oil and gas is an industry that’s looking a lot at automation and robotics. I think these opportunities are just much more numerous and varied than they were before.

Q1: So that’s a kind of structural change. It’s an expansion in the diversity of the field and of the opportunities to students. Do you think that, what if anything, do you think that those structural changes have had, how have they affected research questions in the field?

Marcia O’Malley: Yeah, no, I think that there’s certainly much more of a role of industry in robotics research than earlier in my career, right? So when I started out, you go to NSF for DARPA or DOD or NIH to fund your research. But as other companies have come on and they have their own R&D initiatives, they’re at conferences recruiting interns. I’ve had some really interesting collaborations with the company formerly known as Facebook, right? So Facebook to Meta, they have postdoc positions, right? So students coming out of labs are now able to do an internship, maybe bring aspects of that research back to the lab, and bring those research questions that are coming more from an industry perspective as opposed to the traditional academic perspective. So I think there’s certainly more engagement with industry as research collaborations, but also as pathways for students who are coming out.

Q1: As you look forward, then, what kinds of changes do you see coming or would you like to see coming to the field and to the industry?

Marcia O’Malley: I’d say one thing that I have seen and really appreciate is just the community. The diversity of the community is much greater than it used to be. There’s greater, across every dimension, right? So gender, race, country of origin, perspectives on problems. So I think that’s quite impactful for the field. The industry opportunities and industry alignment with the field, while variable, because you see companies hiring a ton of people, but then they’re not hiring a ton of people. I think that keeps us grounded in a way that we wanna make sure that we’re addressing important problems that are facing society. We also wanna have opportunities for translating ideas from the lab to really have that impact with the community, with society.

9

Q1: I can’t not ask this question, so I’m gonna have to. What are you thinking about the role of generative AI within your research or your field?

Marcia O’Malley: That’s probably one I don’t wanna answer. Yeah, I… In terms of my own research, I think the role of generative AI is… Well, I’ll start that differently. I come from the camp of, if we’ve got some physics that can explain the behavior of the system, we should use the physics and then keep the generative AI, the machine learning, the AI algorithms for those cases when modeling is difficult. And those certainly have a place in my own field. Because I come at robotics from the perspective of a mechanical engineer, where I’m thinking about the components and the dynamics of the system, I tend to be more a model-based researcher than a generative AI machine learning approach, but I certainly value those tools for the applications where they make the most sense.

Q1: As you were talking about the future of the field, you talked about the social implications of the research that you do with your students. How do you think about your relationship as a researcher with kind of larger questions? I mean, obviously you work in rehabilitation, so there’s that direct appeal, but how do you think about the kind of larger relationship or responsibilities that robotics has to, I guess, to the world?

Marcia O’Malley: I guess one perspective—

Q1: I’ll reframe that.

(Q1 and Marcia talk over each other, INAUDIBLE)

Marcia O’Malley: Yeah, no, I’m thinking about… (Indistinct) No.

Q1: I didn’t mean it that way.

Marcia O’Malley: So the two fields that I work in is rehabilitation robotics and haptics. And I think we have to make sure make sure that we are educating the public about realistic expectations. You go to the movies and you see robots doing all these amazing things, and then the reality of that is maybe not quite where Hollywood has us. And so I think we need to do a better job of communicating, not just limitations, but still opportunities, right? So the application of rehabilitation robotics to stroke rehab has had an impact, but maybe not the impact that we envisioned, but still benefit, right? We’ve got assistive robotics, we’ve got exoskeletons, we’ve got people using really advanced prosthetic devices, and that’s having an impact on their quality of life. Maybe it’s not the complete recovery. In haptics as well, there’s this notion that haptic 10

feedback is, again, it’s not Ready Player One in the haptic suit, we’re not there. But we can show advances in the field with new devices and new… We can show advances in the field with new devices and new haptic rendering algorithms, new ways of applying haptics and giving those experiences to users in VR and progress is being made. So for me, I think the role that we have is to better communicate so that we can make sure the expectations are aligned with reality.

Q1: Thank you for that. We’re nearing the end of our time, but I wanted to give you a kind of open opportunity. Are there some topics that we haven’t talked about yet, that you would really like to, or some aspects of your experience or career that we really haven’t talked about yet?

Marcia O’Malley: We could talk a little bit about earlier, you mentioned funding. And I think that we have a real challenge to, again, educate the public on the value of robotics research, educate those who are making public policy about robotics, both from an investment in research perspective, but also in a policymaking perspective. So we don’t necessarily wanna always be playing catch up with policy and regulations. We need to be out in front of this. So if we, as a community, were engaged, and we’ve seen successful examples of this over the years. The National Robotics Initiative that I mentioned that funded a good bit of my research over the years was a community-based effort to work with policymakers and funding agencies to secure resources to fund robotics research. At the same time, we’ve got very visible people in our field influencing AI policy and public policy. We could have more of this. We’d have more engagement, more education of those that are really setting the directions, at least in the US.

Q1: Excellent. That was a wonderful way to kind of include it. Darryl, did you get everyone you need? That was great.

Marcia O’Malley: Yeah, thank you. 11