Knowing the Limits of Expertise

Two best-selling authors—a reporter and a surgeon—explore how medical experts can improve outcomes by learning how expertise itself works.

Commentators

• Atul Gawande, MD; Brigham & Women’s Hospital; Boston, MA
• Malcolm Gladwell; The New Yorker; New York, NY

Transcript

In keeping with a major objective of the conference to bring in outside perspectives—and ideas from outside traditional thinking in medicine—two best-selling authors and contributors to The New Yorker magazine spoke on the nature of imperfection.

Dr. Atul Gawande is Assistant Director for the Center for Surgery and Public Health at Brigham and Women’s Hospital in Boston. He is also a regular columnist for the New England Journal of Medicine and author of the book, “Complications: A Surgeon’s Notes on an Imperfect Science,” about the challenges at the center of modern medicine.

Dr. Gawande was joined by Malcolm Gladwell. Gladwell is author of “The Tipping Point,” about understanding social and business trends, and the more recent book “Blink,” which explores the connection of instinct to decision-making.

The two speakers separately promoted new paradigms of thinking about medical mistakes, using examples, before sitting down to explore error prevention themes together. Gladwell went first, and one anecdote he shared helped explain why even top experts in their field make mistakes in judgment. Mistakes for which experts themselves have almost no explanation because of a fundamental misunderstanding of expertise.

Gladwell said that it is false for experts to believe their expertise is robust. Instead, according to Gladwell, expertise is fragile.

“…and that experts can only behave at an extraordinarily high level under particular circumstances and in the absence of those circumstances, their expertise can vanish.

“I spend a lot of time in my book hanging around with police officers, and in the world of law enforcement, it’s a huge issue. They make errors in judgment all the time, and they don’t have the luxury of sending cops back to the police academy to be retrained or replacing huge slates of police officers with another slate of police officers. They are forced to work with what they’ve got and what they have learned is there are some structural changes you can make in the decision making of doctors that can go a long way to correcting the kind of errors they make. 

“The two obvious examples are banning high-speed chases and single officer squad cars. Officers by themselves make much better decisions than officers in groups. The data says that really, really clearly. They don’t do stupid things.  They don’t feel invulnerable. They slow down, they take their time. They are a lot more cautious and everyone is happier and healthier and better off when they are by themselves. That’s why we have single officer squad cars more and more.
 
“High-speed chases, same thing. Cops do stupid things at the end of high-speed chases. We know that now, right?  Rodney King. The last three race riots in this country were all caused by dumb things cops did at the end of high-speed chases.  Why? You have no judgment when you come out of a car at the end of a high-speed chase. What’s the solution? Ban the high-speed chase. There is a structural change that has the effect of turning a flawed decision maker into a much better decision maker. There are things we can do on the structural level, in other words, that can vastly improve the ordinary everyday performance of experts. 

“I thought of that as well in that film, because part of the anger of the patients was the sense that not everything had been done that could have been done to diminish the possibility of that error. When we make structural changes like that, those are things that we can point to and that we can tell everyone, look, we made a good faith effort to correct the source of error in this institution.”

Another example from Gladwell involved how orchestras dramatically increased the percentage of their musicians who are female. This was done almost overnight decades ago by placing a screen between auditioning musicians and the maestro judging the audition. Gladwell maintains that the maestros’ expertise did not change, but the environment was altered to allow their judgment to prevail.

“Well, that story is a wonderful example of an error, a diagnostic error by experts.  These maestros are like highly skilled doctors in a medical setting. These are people who have enormous experience, who have enormous expertise and yet time and time again over the course of musical history, they made an error.  Somehow they let their feelings about women get in the way of their ability to make an accurate diagnosis of musical ability.  So the question is, what can we learn from that experience?  What does that experience tell us about why errors happen and, more importantly, how we can reduce them and change the situation in which they are made?

“Well, the first sort of obvious lesson from that, is that experts aren’t very good at understanding the sources of their own error. Now that is something that is profoundly true of expert judgment. When you look at experts in a variety of settings, you will see that they all share this trait that so much of their expertise is now taking place on the unconscious level that they are not very good at understanding how they make decisions, at least explaining how they make decisions.”

Dr. Gawande followed; with some suggestions about how to produce better patient outcomes with providers who are already experts. Dr. Gawande emphasized the value of focusing research and interventions on understanding what can be standardized, and the need to pursue innovation in performing better what we already know how to do in medicine. Whether it’s lowering rates of nosocomial infection or surgical complication, or it’s improving the success rates of cystic fibrosis care, Dr. Gawande said that measurements and data collection about outcomes at the level of individual providers will be key to better outcomes.

One way to move everyone from the middle of the quality bell curve to the top-performing end is to re-frame how we view patient safety.

“Safety has not quite bought everybody on. Partly the patients, because it is kind of an assumption that safety is what we already should have in the system, and that’s already there and they kind of take it as an assumption that there is going to be safe care. It is hard to fight for safety. At the same time, the clinicians find it a little insulting. We are safe. I’m not an unsafe surgeon. Why would I not be a safe surgeon? And I’ve begun to think that the way we frame this in a way that brings the patient and the physicians and the nurses, but also the institutions together, is service. 

“I was at the Mayo Clinic last week giving a lecture, and I made it as a condition of the lecture that they let me come and stay for about four days. Just to see how the place works, because I have heard so many rumors about the place. It was fascinating to me, because it is such a fundamentally different place.  For one thing, for example, the patients come and they don’t usually know who their doctors are. They are coming to the Mayo. If you come on a Monday with a colon cancer, you get the Monday surgeon and he will see you and operate on you on Wednesday. If you come on a Tuesday though, you get the Tuesday surgeon, and the results are expected to be similar from one team to another team. The result of that, is that they have broken down barriers in just how service works. Each team wants to be able to deliver care. It was inconceivable that you could have a patient come in, be seen on a Monday and get to surgery by Wednesday. 

“I can’t get a patient to surgery -- unless they’re an absolute emergency; it’s going to take me three weeks. But [at Mayo] they will get a cardiology appointment if they need it. The cardiologist will come down and see the patient in the same room where the surgeons are. Then they will get any imaging they might need.  They will be set up for the following day for some basic preoperative appointments and laboratory tests and they will be ready to go. 

“Well, the focus on service made it so that the physicians really enjoyed what they do. It was possible for them to focus on what they cared about, which is taking care of patients and being very good at it. It made the patients feel well cared for and it made the institution feel responsible for making all of that go well. And so, in many ways I’m also beginning to sense and see in this video, for example, it is the same issue. It is, how do we make the care skillful but also the care caring?  Service is really about doing both of those things.  It may be the frame that lets us move the ball forward and tip us in the right direction.”

During the Question & Answer session, Gawande and Gladwell explored issues together. They were asked how judgment can work subconsciously, and how understanding instinctive judgment can lead to improved decision-making. Malcolm Gladwell responded first:

Q:        “Could you tell the group the story you tell in Blink about the ability to discern long before it comes to the consciousness, whether the particular set of red cards or blue cards are likely to lead to a more positive financial outcome?”

A:         “There has been a lot of work in psychology trying to measure our kind of unconscious early warning signals. In one famous experiment, they had people play these card games.  You have these two decks of cards, and you are asked to bet on the various outcomes of cards. What you don’t realize going in, is that when you bet on one set, you will always lose.  On the other set, because of the configuration of the cards, you can actually make money that way.  The question is, how long does it take you to figure out that you can only make money betting on one set of cards?  The answer is, it takes quite a long time for you to figure out that you are actually getting fleeced by one deck and the other is making you money. 

“But, if you measure people’s things like their heart rate, their skin conductance, all this kind of very subtle physiological measures of stress, what you discover is that very, very, very early on in the process, people’s unconscious is warning them about the bad deck and it’s saying people are actually experiencing stress.  And long before they are conscious of the fact that they are starting to avoid the bad deck and only bet on the good deck, they are starting to avoid it.  Their behavior changes well before their awareness of their behavior change happens.  That’s a tribute to just how sophisticated and powerful our unconscious hunches are after we have generated a certain amount of experience.”

Dr. Gawande picked up that point and placed it squarely on medicine. Reducing errors and improving the performance of clinical experts requires understanding how expertise works and how it becomes vulnerable. Dr. Gawande said that an appreciation for how expertise and judgment is partly reliant on unseen factors, like instinct and experience, can help medical providers improve their performance. He used the example of variation in performance between high-volume surgeons and low-volume surgeons for a given procedure.

“If we are to be innovative about it, then what we are going to have to need to do is understand, you know, how did that high volume surgeon become that high volume surgeon?  When did they develop that unconscious ability that later became a conscious ability to shape and do what they do? And how do we get that information to a surgeon who is doing these operations not for the first time necessarily, but not as frequently as others are doing. 

“Part of our research group’s effort, is to try and figure out what the answer to unlock that is.  We have kind of an initial guess, and I don’t know if it will work or not.  We are in the midst of starting the trial to test this one, and it really came from baseball.  The idea was that when people bring on a rookie pitcher in baseball, they’ve been tried out in the minor leagues for a while, and they have seen them perform. But, they don’t know that they are going to, you know…pitching in Triple A is not the same as pitching in the major leagues. 

“So, when the Red Sox brought up John Pappelbon last year, he was a kid with a great fast ball.  What he didn’t really have, if I remember correctly, was a slider.  He knew it.  He had a decent slider, but he did not want to try it in the major leagues.  He described having a five-minute conversation with Curt Schilling in which Schilling told him, after watching him for a little while, here’s how I use my slider and here’s how you could potentially use your slider. In that five-minute conversation, he came out as a two-pitch pitcher in the major leagues even though he had been a one-pitch pitcher in the minor leagues. 

“I tried to think about how we function in medicine. If you are a surgeon or a physician, working in a particular area where you are faced now with some patients or a single patient who you have to do something for them that you haven’t done very much before, we are very loath to have that five-minute conversation with the person of great expertise.  The reason why we are is because we are afraid that we are going to look dumb and that our questions are going to seem dumb.  We mostly are trying to look like, yeah, yeah, yeah; we know what we’re doing.  I’m becoming convinced just from my own practice. One of the first things I did for the first month was go over every single patient I had with an experienced surgeon, my partner, so that I could get better results.  Lo and behold very quickly, you know, I was tracking my complication rates and they were better than I had hoped they might be in the beginning, and I got confident and I stopped asking questions.  And then my complication rate started to rise.

“So I went back to asking questions and I did it for more than a year about every patient that I had.  It was definitely sometimes annoying to the surgeons that I would go to and ask about things.  I mean, one year after your practice, you’re still asking about a basic hernia repair, are you all right as a surgeon?  Sometimes it was embarrassing for me. But I have to say, I learned something out of every conversation that changed the way that I was practicing.  So, the theory that I would have, is that maybe five-minute conversations with people of greater expertise will allow you to learn more quickly and achieve that unconscious level of performance more quickly.  I don’t know that it’s true.  We are doing the trial and it may yet fail.  It may turn out that you can ask Roger Federer how to hit a backhand all you want and you will never play like Roger Federer.  On the other hand, it could be that physicians are like other pros, that we are all like John Pappelbon, that we are good enough to be really good pitchers for the equivalent in what we do, but that we don’t get our chance to ask the equivalent of our Curt Schillings in our professions what might make us better.”