Share Podcast
How Robots and AI Are Changing Job Training
Matt Beane, assistant professor at the University of California, Santa Barbara, finds that robots, machine learning, and AI are changing how we train for our jobs — not just...
- Subscribe:
- Apple Podcasts
- Spotify
- RSS
Matt Beane, assistant professor at the University of California, Santa Barbara, finds that robots, machine learning, and AI are changing how we train for our jobs — not just how we do them. His study shows that robot-assisted surgery is disrupting the traditional learning pathway of younger physicians. He says this trend is emerging in many industries, from finance to law enforcement to education. And he shares lessons from trainees who are successfully working around these new barriers. Beane is the author of the HBR article “Learning to Work with Intelligent Machines.”
CURT NICKISCH: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Curt Nickisch.
Like it or not, people increasingly do their jobs with robots, machine learning, or artificial intelligence. These developing technologies are already destroying some jobs and changing how many others are performed.
But it’s not just work that’s changing. It’s also how people train for their work. New research shows that how people learn to do their jobs is also being disrupted.
Our guest today has studied surgery, one of the first places where robot-assistance is changing the actual job. Basically, he says robots give surgeons more hands to work with. And that means surgeons don’t have to hand off some work to trainees like they used to.
And our guest today says this trend is emerging in all kinds of industries, including finance and education. Matt Beane is an assistant professor at the University of California Santa Barbara. He also wrote the HBR article “Learning to Work with Intelligent Machines.” And he joins me now. Matt, welcome.
MATT BEANE: Delighted to be here Curt.
CURT NICKISCH: If surgery is a bellwether for the way training can change in many, many other workplaces down the road, if that’s where the future is here and now, just bring us up to speed. How was surgery training done up until you had robots in the O.R.?
MATT BEANE: Right. So, the maximum in surgery is see one, do one, teach one. That’s the understood, local phrase in the surgical community for what other folks know as apprenticeship, training, coaching, on-the-job learning, on-the-job training. There’s lots of names.
You watch, say you’re, Curt you’re training to be a urologist and to do urological surgery, you’d watch a given procedure a number of times. And there’s this sort of, not a firm boundary between that and doing one, you start to do at the edges. You start to do things that are useful to the procedure, but not critical, core, extremely risky.
And as you demonstrate confidence and you’re shoulder to shoulder with the person who’s assessing your competence, and who by the way needs your help – it’s not an optional activity. There are certain portions of the procedure where four hands are quite literally required to get a part of that job done.
So, there’s this symbiotic relationship between mentor and trainee. And then over time, you start to take the driver’s seat more and more, and eventually then that senior surgeon backs away and allows you to do parts of this procedure with a little bit of supervision from them, but then you’re in turn supervising the next student coming up the skill ladder. So, that “see one, do one, teach one” means of gaining surgical skill is as old as the surgical profession.
CURT NICKISCH: Essentially kind of an apprentice model of watching somebody do work and then performing it yourself under their supervision.
MATT BEANE: Right. And it’s delightful and easy to go back and find studies of training and learning across a radically different set of contacts. There’s about a good say, 80 years’ worth of careful study of everything from midwifery to brick laying, to architecture. They’re fascinating to read, but this pattern is just taken for granted.
But, trainees slow things down and they make more mistakes just by definition almost. Logically that’s obvious. You’re just not as good and so, involving trainees is costly. It’s about good apprenticeship, good on-the-job learning, just reduces the likelihood of trouble and gives you the best shot at perhaps you’ve never done this thing before, but I’m going to do it now and it’s going to go reasonably well.
CURT NICKISCH: So, now enter robots in the operating room, stage left. What’s happening with this early future that you observed and studied?
MATT BEANE: Using a robot to do surgery and there are many different kinds of robots, by the way. The one that I studied is the one that is by far, most widely deployed and used, is this da Vinci Surgical System that involves instead of a big incision, and two hands holding that open and somebody working inside the patient, you have between four and six small keyholes, and inch incisions made in the belly of a patient and then the robot attached through those incisions, each of the instruments on that robot go inside the patient.
One of those arms has a stereoscopic camera, a 3D camera on it. And so, to operate, a surgeon goes 15 or 20 feet away to sit an immersive, many of them call it a glorified Xbox controller. So, it requires this intimate feet and hands control. But you are no longer at the patient. You’re not touching the patient. You can’t see the patient’s body. You can only see a small view that is presented to you through this camera.
Which is, by the way, presented to everyone else in the O.R. on large screen TV’s. The trainee can go to a second console if they’re lucky and sit in that console and watch and take over if I digitally delegate control to them. The critical thing for training and learning though, is that that resident is just watching. That entire time they are an optional participant. It’s up to me as a senior surgeon to punch them in so to speak, on this little touch pad to give them practice. But it’s not required anymore.
CURT NICKISCH: Is the issue here the fact that they’ve now been given more capabilities and they just don’t have that forced incentive to hand off something that other people can train on? Or is it the fact that they are also learning these new systems and if they are getting trained and using new systems to do surgeries, they don’t have as much time to train other people?
MATT BEANE: Right. I did see a little bit of that latter category you’re talking about. But I did everything I could to focus at the top institutions in the U.S. So the U.S. by far does more of this kind of robotic surgery than any other country. And so I deliberately selected the wunderkind robotic surgeons who essentially grew up with this technology, or demonstrated extraordinary capability with it. It’s absolutely stunning to watch these people work. It’s as if they were born to these machines.
CURT NICKISCH: And they weren’t spending as much time training others essentially.
MATT BEANE: Yeah. So, if you’re an average resident in training, at least back when I was doing this study, you would get say 15 minutes to operate using this robot during a four, four and a half hour procedure. Whereas, on a traditional procedure you get four or four and a half hours of practice. I mean you’re literally required for the entire procedure. Practically, most residents were down to say 15 minutes of practice and oh, by the way, on the simplest, easiest, safest portions of the procedure most typically.
CURT NICKISCH: What worries you about this?
MATT BEANE: Well, so imagine at a given hospital, teaching hospital, you’ve got say, four or five residents who are graduating each year. The bulk of them will have had 15 minutes of practice across perhaps 10 procedures, maybe 20, depends, by the end of their residency, which is wildly insufficient. It’s practically not enough time to develop the skills that they need to do this on their own which is where they are headed.
So, at the end of your residency, you are legally empowered to use this tool to do surgery and you do. And you are practically not capable to what most of us would hope would be a sufficient level. By the end, and in fact you, many residents try to avoid situations where they’re going to have to do robotic surgery. So, what worries me most and I think worried a lot of my informants, as I sort of revealed these patterns back to them, is that the bulk of their residents that they were graduating everyone seemed to just casually assess as really not having what it took to do these procedures efficiently, if not safely.
CURT NICKISCH: I mean it sounds like a systemic problem with some market forces that could help fix it and maybe there’s some hope that this will get solved. The bigger problem I guess is that even if they fix this in surgeries that these are the sorts of problems that we’re going to face in lots of jobs. And institutions have to deal with it, but so do individual workers. Like what did you observe about medical residents? What did they do to make sure that they were getting the training they did, or what kind of power did they have to make sure that they could learn the job that they’re actually trying to train to do?
MATT BEANE: Right. So, most of them did what they thought they should and what everyone thought they should, which is they tried to learn through the old means. So, I’m going to put them to the side because that clearly just didn’t work. There was a small minority, about one in five, one in eight, it depends on the sites that I studied, who essentially, whether they consciously did so – in many cases it wasn’t conscious – they just realized that learning the old way with the new tool was not going to work.
So, they essentially recognized this problem way upstream. Some of them found this in undergrad, but most of them, in fact all of these folks that I called “shadow learners,” who were engaging in shadow learning, they at least by medical school had found ways to get into the OR, in real procedures with this robot and found ways to get involved.
CURT NICKISCH: Like what? And I ask that partly because the term “shadow learning” sounds a little bit like shadowing somebody, but that’s not what you’re talking about.
MATT BEANE: Yeah, no the shadow bit is about essentially cheating to learn. So —
CURT NICKISCH: Kind of hacking work.
MATT BEANE: Yes. Although there’s this, there, we like to talk about sort of people who work around rules to innovate and so on these days. And I suppose that’s technically true in this case. But every one of the practices that these successful learners discovered was in some way strongly counter normative.
So, not just a little outside the rules, but for example, in medical school I am supposed to be getting a generalist education. I should be learning about psychiatry, anatomy, physiology. I should be learning about urology. Surgical techniques should be a small postage stamp in that education.
And the residents who were successful at robotic surgery were carving out large portions of that generalist’s education to make room for robotic surgery. Which in hindsight, none of them thought it was appropriate. And so, that was strongly counter normative. The next one upstream was to practice digitally. I called it “abstract rehearsals.” So, watching many, hundreds of hours of YouTube videos of recorded surgeries, using a simulator, say 100 times more than the required amount. That was common. And in general sort of doing what they could to rehearse portions of this procedure digitally.
Which that ran directly counter to 100 year old norms about what surgery even means, what a surgical professional is and how they learn. I had chiefs of various services – I had one who said, “Watching movies doesn’t make you an actor.” So, all of this digital rehearsal seemed wildly inappropriate and they didn’t broadcast this kind of digital rehearsal at all.
The last of the three that they really critically depended on was what I called “under-supervised struggle.” They all, all learners recognized that you need to struggle to learn. You need to be close-ish to the edge of your capability and trying something you’re not quite sure you can do, and have an expert nearby for support.
And these residents recognized on some level that they were not going to get that if they tried to learn in approved ways. So, they found ways to rotate into procedures and participate in procedures where they got a lot more leeway to operate and with very little, sometimes no direct supervision of their work.
And there is not one medical professional that I’ve spoken with that things this is appropriate. And yet, they all tell stories about how they learned the most when they really had to sort of handle something by themself. I call it “shadow learning” because it’s definitely in the moral shadows for this profession. And in all of, in many other professions and lines of work that I’ve since explored, these are practices that none of us would approve of, and essentially involve risk to you, risk to your customer, risk to your, the person training you and so on. Risks that we really shouldn’t have to take in order to build the skill to learn how to do our job.
CURT NICKISCH: Now, surgery is, seems probably to many people to be just a very specialized skill and profession, and I wonder, how universal, generalizable is this to other professions? What other professions are you seeing similar, you know similar learning curves happening in right now?
MATT BEANE: So, this was the burning question for me as I wrapped up my study and sent it off for publication. The dynamics that I observed seemed pretty generalizable. If you make a trainee’s involvement optional, if a new technology allows you to, as an expert, work without their support that seems like it’s going to compromise their learning whether or not we’re talking about chip design, policing or surgery. But I had no evidence, or limited evidence.
So, in a year and a half that followed that study, I ended up covering over two dozen different kinds of work across very different industries, ranging from policing, to investment banking, to online education, to chip design, to robotics deployment, to the military, to manufacturing, you name it. I have been across many, many industries and then again, many, many roles within each of those industries.
And also involving really radically different kinds of machine intelligence. So, in my case, you have a pretty dumb, very sophisticated, but not artificially intelligent robot, ranging to a platform using machine learning to match workers – local laborers – to people who needed their help in their community. Or, to a predictive policing algorithm, online training and learning. So a lot more digital stuff involving what we traditionally associate with AI, like machine learning and so on.
And the dynamics were the same. It started to get pretty boring actually. I suppose a mix of exciting and boring. It’s exciting because it looks like wow, this really seems to be a very consistent effect across really radically different kinds of work. But also, boring in that you realize well, it’s the same story over and over and over again.
CURT NICKISCH: I mean this works in favor of a lot of the organizations that are implementing these technologies. It also works against them in the sense that if you make these specialized experts who now work with these machines, the sole people doing the work and you have no succession plan essentially, that can create a real talent problem for you too.
So, are the solutions and possible ways forward that you’re thinking about?
MATT BEANE: Essentially my shadow learners, the people relying on shadow learning, showed the way. In much the way that an architect looks at early foot traffic across a piece of grass to decide where the paths should go. These folks across multiple settings took serious risks to learn because learning takes time.
So, it’s not as if you sort of cheat once and hope to get away with it. You have to routinely cheat very well, thank you very much. And so, copying their practices is obviously a bad idea. There’s no organization that can just drag sort of – the surgical profession is not about to say, “OK fine, you should just operate without an attending in the room.” That’s not going to happen.
But shadow learners have clearing and consistently demonstrated a few things. One, is that you need to struggle to learn. You need to find ways in whatever work you’re doing to be challenged. You also however need expert support for two reasons. One is you need for them to backstop against a catastrophe of some sort. There needs to be some, some adult supervision.
But also, you need to be able to go to them for pointers and they need to be able to offer them in return, just not in the ways that sort of our come most naturally given these technologies. So, in those three features of work, we can insist and design them in, insist of them and design them into work, into the technologies we build and sell, the organizations we design. So —
CURT NICKISCH: And possibly even back into the educational systems like medical school.
MATT BEANE: Exactly. And it’s not as if this is not happening anywhere, it’s just not happening enough and it’s very – this productivity temptation is increasingly potent with these technologies. It’s just, the output for one unit of an expert’s time just seems to be skyrocketing. You have all these sort of superstar performers. So, it’s a very tempting target.
But a manager, a technologist, a worker, a trainee, we, all parties can insist on handling these technologies in a way that offers yes, productivity, the enhancements, the amazing opportunities they present, but also to design the work and implement these technologies in a way that ensures skill building as we handle these technologies.
So, I’ll ground this out in surgery. It is possible, right now, and has been for years to share control of that robot across two consoles. So, if you were my mentor you could give me one of the three arms to control and you could take the other two. It’s awkward. It’s almost never done. I asked even representatives from Intuitive Surgical, whether this is done? They said extraordinarily rare. But you could in some way nudge surgeons and trainees in that direction. Now, I’m not a technologist, I don’t design user interfaces. This is not a problem that I sort of have a solution, a ready solution for.
But you can imagine those technologist and their users, surgeons, and their organizations that buy these systems to insist that no matter what the solution looks like, it must extend human capability as it’s deployed. That’s sort of a fundamental requirement. And it seems like there’s a lot of white space. It’s just not – we don’t focus on-the-job learning training, “see one, do one, teach one” problem because we take it for granted. It’s thousands of years old. Why wouldn’t it be working?
CURT NICKISCH: I just wonder if there are other places out there that, in this case surgeons, could learn from?
MATT BEANE: Yes. Absolutely. And we could use AI and these intelligent technologies to help with this problem. So, not just remediating for them, but for example, in the old method if I was an apprentice and you were my master, or the expert I was working with, we had to be physically collocated most cases, or if not, we had to be talking in real time when the problem was going on. We have the internet now. We have the cloud and we have machine learning to help make matches between apprentices and masters.
So, we can reconstitute apprenticeship using these very tools in a way that makes it global, distributed and allows everyone to contribute to and draw from this sort of semi-real time repository of expert support. And it could be across organizations, across time, across space, even across skillsets. It could be for example, if I was trying to learn to write, but my job is, I’m a welder, or I do advanced fabrication, but I need to learn to write. My mentor for that work, my “see one, do one, teach one” path could be with somebody from far away in space and time.
And AI, some algorithm, machine learning system could match us in the same way it matches rideshare riders to drivers, or all kinds of two-sided markets. This is a two-sided market. We’ve just essentially had an old model for doing apprenticeship that is no longer working, given the precision of these technologies; how tempting they are. We just needed a more distributive model. And I think we can use these tools to enable that kind of a future.
CURT NICKISCH: I mean you’ve talked a lot about what we all can do, but I’m wondering what we individually can do? If you are a manager or a leader or a senior doctor, what kinds of things can you be thinking to yourself and telling yourself to do when you see this kind of emerging problem creep into your own workplace?
MATT BEANE: The one thing that seems pretty clear is that whether or not – if you get a sense that traditional means of learning, how to do a job are not working, that’s a very important and early warning sign to pay a lot of attention to. The next step though is really critical, which is against your better judgment or maybe intuition, it is a near certainty that there’re going to be people out there who in spite of these new barrios, are finding ways to learn anyway.
Now, they’re not going to be maybe aware of it. These ways of getting skill are not going to be perhaps appropriate and they shouldn’t be copied. That’s why I called it shadow learning. But in each organization, and with each new technology there are going to be ways of getting skill that people have sort of through the school of hard knocks found that are different from the old way.
And that while this is, it’s sort of in some ways new. We have all these new technologies and these things might be more intense. On some level I think we all understand that if you followed the rules all the time, for how one ought to learn, you wouldn’t learn. We all need to work around the edges and do things that are not quite expected to get challenge in our work.
So, looking towards the people who are struggling and maybe succeeding, in a sensitive way, without trying to make an example of them, or punish them, can offer you early leading indicators of how we might need to reconstitute training. So, in the case of surgery, for example, clearly a lot more digital rehearsal is needed. You need practice with these systems before you set foot in the O.R. So, that’s, it’s as easy to use that robot, roughly, as it is to use your hands to tie a knot. So, once you know that, once you know that successful people do that in spite of requirements that make it hard, then you can either shift your program in that direction or you can keep it, or you can keep things the way that they are.
But it’s those shadow learners I think that offer us really valuable clues about early, small changes we can make to training programs, and to just the way that jobs get done, so that people can get their hands on the practice they need.
CURT NICKISCH: Matt, thanks for coming on the show and talking about this.
MATT BEANE: You’re welcome. Thank you very much.
CURT NICKISCH: That’s Matt Beane, assistant professor at the University of California, Santa Barbara, and the author of the HBR article “Learning to Work with Intelligent Machines.” You can find it in the September-October 2019 issue of HBR or at HBR.org.
This episode was produced by Mary Dooe. We get technical help from Rob Eckhardt. Adam Buchholz is our audio product manager.
Thanks for listening to the HBR IdeaCast. I’m Curt Nickisch.