Surgeon Atul Gawande writes frequently in The New Yorker about how he uses statistics on national norms for various surgeries to monitor his own performance.
A reader wonders whether those statistics are available to the general public. In particular, can you find out whether a surgeon you are considering having slice you open is above or below average? Anybody know?
After eight years, I’ve performed more than two thousand operations. Three-quarters have involved my specialty, endocrine surgery—surgery for endocrine organs such as the thyroid, the parathyroid, and the adrenal glands. The rest have involved everything from simple biopsies to colon cancer. For my specialized cases, I’ve come to know most of the serious difficulties that could arise, and have worked out solutions. For the others, I’ve gained confidence in my ability to handle a wide range of situations, and to improvise when necessary.
As I went along, I compared my results against national data, and I began beating the averages. My rates of complications moved steadily lower and lower. And then, a couple of years ago, they didn’t. It started to seem that the only direction things could go from here was the wrong one.
Maybe this is what happens when you turn forty-five. Surgery is, at least, a relatively late-peaking career. It’s not like mathematics or baseball or pop music, where your best work is often behind you by the time you’re thirty. Jobs that involve the complexities of people or nature seem to take the longest to master: the average age at which S. & P. 500 chief executive officers are hired is fifty-two, and the age of maximum productivity for geologists, one study estimated, is around fifty-four. Surgeons apparently fall somewhere between the extremes, requiring both physical stamina and the judgment that comes with experience. Apparently, I’d arrived at that middle point.
A reader wonders whether those statistics are available to the general public. In particular, can you find out whether a surgeon you are considering having slice you open is above or below average? Anybody know?
A reader wonders whether those statistics are available to the general public.
ReplyDeleteThey're not. And it would probably be a bad thing if they were.
A reader wonders whether those statistics are available to the general public.
ReplyDeleteNo.
There are two classes of people who have access to the statistics:
1) The folks in the hospitals who work in a specialty which generally goes by a name like "Outcome Management", and
2) The actuarials in the insurance companies.
And there are two classes of people who can make an educated guess which is pretty close to being correct:
3) The medical colleagues [fellow surgeons, anesthesiologists, OR nurses, PACU nurses, ICU nurses, etc], and
4) Members of the local bar who specialize in medical torts [either as litigants' or as defendants' attorneys].
Often there will be some intersection in these categories; for instance, a Department Chair or a Division Chief might serve on an Outcome Management oversight committee [and would then have access to the exact statistics which would confirm or refute his suspicions]; similarly, an Outcome Management oversight committee would typically have at least one in-house lawyer [from "Legal Affairs"].
ANYWAY, if you can get the ear of any one of these kinds of people, then they may not be able to tell you who is the absolute best, but they can darned sure tell you who to AVOID [and that is always (far and away) the most important piece of information].
Of course brilliant Surgeon A, who takes on the most challenging cases which colleagues won't touch, has a much worse death rate than mediocre Surgeon B, who sticks to tried and tested, run of the mill stuff.
ReplyDeleteBy sampling, one could compile good statistics on surgery in general without in fact collecting data on most individual surgeons. In any event, if this "batting average" idea were taken seriously, surgeons would have a nasty incentive to avoid operating on the riskier categories of patients.
ReplyDeleteI would not trust any such data, for several reasons. First, most MDs stay within their comfort zones regarding the procedures they undertake. There is very little incentive to try to do something you are not reasonably good at. There are incentives to ask for help or just refer it to someone else. This tends to homogenize outcomes regardless of differing levels of skill. Second,and related, the most skilled Docs are generally the ones with the confidence and the reputation to take on the tough cases - for the challenge of it and the lack of worry about being thought less of when the results are not perfect.
ReplyDeleteOh god yes. And lawyers too, while you are at it.
ReplyDeleteThough I suspect the best according to the stats will be those who only take a swing at the fat pitches versus those who will do their best for all patients.
Can surgeons turn down patients? If so, there's not much point in placing much emphasis on batting averages.
ReplyDeleteExcellent question. My guess is that they won't be available cause the surgeons don't want that. I have heard of Dr's suing review websites cause of bad reviews.
ReplyDeleteWhen I was in college the professors kept saying the school shouldn't create a professor review/ratings doc because they'd be biased from disgruntled student reviews. They eventually came out on the internet. It really did serve the customers too.
How to find lawyers' winning percentages is another good question. Lawyers are as hard to pick as surgeons. (Yes, I'm aware good lawyers may lose many cases).
ReplyDeleteAlso, if surgeons were judged by these sort of rankings, it would make them very reluctant to do risky operations, which might not be a very good thing if there are no other options for the patient.
ReplyDeleteGo to a teaching hospital, let me repeat, go to a teaching hospital. Things don't fall through the cracks when interns, residents and your own physician are all checking on you, and by necessity your doc will be keeping up with state of the art in his own field of medicine or surgery (the best way to learn something, of course, is being required to teach it).
ReplyDeleteWhile the valet parking and tastefully decorated rooms at private (non-teaching) hospitals are surely more aesthetically pleasing, wake the eff up, you're not shopping for a timeshare! You'll get better medical care at a grungy teaching hospital.
Of course, you've all heard about that classic case--the plastic surgeon who hung himself?
ReplyDeleteIn any event, if this "batting average" idea were taken seriously, surgeons would have a nasty incentive to avoid operating on the riskier categories of patients.
ReplyDeleteWell, every time he refused three patients in a row you'd have to count that as a strikeout.
From my observations on certain blogs, CNN and FOX, journalism is another one of those careers in which you can easily start going soggy after age 45...
ReplyDeleteRelated question: why do Indians produce at least some gifted doctor-writers? Other than Gawande, there's Abraham Verghese and Siddhartha Mukherjee.
ReplyDeleteBeowulf's suggestion about going to a teaching hospital sounds like a pretty good bet.
ReplyDeleteHere is another another possible method that might work for those who live in or near cities.
Suppose that you need, oh, a brain tumor removed. Ask doctors in other fields whom they believe your area's best neurosurgeon to be. (The assumption that I make is that MDs will be more likely to be objective about practitioners not involved in their own specialty.)
If the various cardiologists, oncologists, endocrinologists, and so on that you speak with keep on mentioning the same two or three names, you will have narrowed the field considerably.
I don't know about surgeons, but I figure the way to find a good internist or GP is to ask a surgeon and/or oncologist. Ask them: who has consistently referred patients to you when you still had time to do something for them?
ReplyDeleteThere was a book, Best Doctors in America, published about 15 years ago, with a good methodology: the author's interviewed the heads of the various departments in the leading medical schools, asking specialists who they would choose to do their specialty on a loved one. They weren't allowed to name themselves, obviously, nor anyone else on their staff. This initial list was then refined by asking all the people on it the same question. They ended up with several thousand names of the best doctors in a couple of hundred specialties, ten to twenty doctors on average in each. The individual names for each specialty were scattered all around the country, but occasionally one hospital would have 2 or 3 of them in one department, which told you it was strong. Sloan Kettering and MD Anderson were of course well-represented for oncological surgeries, but otherwise it was a pretty diverse list of institutions. (Philadelphia has a lot of good doctors I recall. ) Anyway that book is now out of print, and my copy is getting rapidly out of date. But for a thousand dollars or so you can get the updated information (supposedly) on line, but with limited flexibility. Three names.
ReplyDeleteI've saved or extended or improved the quality of quite a few lives using that book.
@ why so many good Indian doctors? Good question. The team that operated on me in NYC was composed of a Jew, a Muslim, and a Hindu. The Hindu was in charge. (His name, incidentally, was Dr. Shah. Later I met another doctor on the floor who told me his name was Dr. Shah-ha. That's interesting, I said, where's Dr. Shah-ha-ha?)
ReplyDeletere: teaching hospitals, another hint: choose the "chief" of surgery in the department you are looking at if you have no other information (and he's not much over sixty). The chief is usually considered best
ReplyDeleteKudzu Bob: Beowulf's suggestion about going to a teaching hospital sounds like a pretty good bet.
ReplyDeleteThere is a downside to it: If your case seems routine, chances are ~95% that it will be some resident (i.e., the most inexperienced physician possible) who will perform the actual surgery/procedure. And with that, YMMV. On top of that, there are even good chances that during post operative care some clueless medical student will be making the actual decisions (unless things are already unusual, a typical chain is that overworked resident rubber-stamps what student tells him, and clinical professor in charge does not bother to check on you beyond listening to the resident).
Anon: Related question: why do Indians produce at least some gifted doctor-writers?
Maybe related to why Indians excel at spelling bees? The traditional explanation is cultural emphasis on rot memorization but I always felt that there should be more to it.
With very rare exceptions, you don't contract with a surgeon to operate on you. You (or more precisely, your insurance company) pays Peoria General Hospital to give you medical care. They in turn assign a team of people (surgeon(s), anesthesiologist, X-ay technicians, nurses, etc) to treat you.
ReplyDeleteYou don't get to pick and chose the anesthesiologist and so on who works on you.
When we purchase services we usually buy them from corporate entities. You can't go into McDonalds and tell them "I want Jim Smith to make my burger". If your house is on fire you don't call the fire dept and say, "Before you send an engine over, can you fax me the yearly assessments of all your staff so I can pick out the ones I want to deal with my fire?" That's just not the way the world works.
Instead the Acme Corporation sells us a service (let's say health care) with the explicit understanding that all of their staff meet some minimal level of competency. Credentialism gets a bad rap sometimes, but done properly it fills an important role.
Most mathematicians do their best work past age 30. Many baseball players have also, such as Barry Bonds.
ReplyDeletei can't speak to the field of surgery in general, but i have seen that at least a few surgeons list this information directly on their websites now, if they are specialists looking to establish themselves as the best in some particular field.
ReplyDeletea few years ago i had surgery for a somewhat common genetic disorder, about 15% of men have it to various degrees. it's usually not that big of a problem in most cases and most guys don't even get surgery, but if you have a bad case of it, it is tricky and difficult to permanently fix. and it gets worse over time.
so there is incentive not only to get it fixed if you have a bad case, but incentive for surgeons to come up with new and better ways to treat it. more effective, less likely to recur, less invasive. the techniques for surgery on this condition have been improving for 40 years.
there's actually about 5 different procedures you can have for it now, each with a different set of possible side effects, chances of recurrence, and so forth. laparoscopic surgeons do it one way, microsurgeons do it another way, and radiologists treat it in yet other ways. a few of the best guys in each field, publish their results on their websites, mainly, because they're so good, they've reduced the chances of something bad happening down to like 1%. so they show that.
i looked at about 6 or 7 surgeons all across the US and talked to 3 or 4 of them and eventually settled on a guy that was pretty good. so in this case, i did get to pick.
By 40 your best work may be behind you.
ReplyDeleteGood reason to start your own company.
Gilbert P.
1) STAY THE HELL AWAY FROM TEACHING HOSPITALS unless you have something utterly bizarre which requires massive amounts of ICU time.
ReplyDeleteIf you have anything even remotely "normal", then you WILL get dumped into the care of a bunch of nitwit residents.
Never forget that:
1A) Half of all residents graduated in the BOTTOM half of their med school classes [and even the ones who graduated in the top half of their med school classes are still utterly inexperienced nitwits].
1B) As regards the professors - never forget that those who can, DO; those who can't, TEACH.
2) And as regards the prospect of riskier procedures hurting a great surgeon's statistics - please see "Outcome Management" and "Actuarials", as above - they've got massively sophisticated software [and databases] now which account for myriad different anomalies in any individual patient's case history, and correct for them [as regards expected "Outcomes"].
A reader wonders whether those statistics are available to the general public
ReplyDeletecomplication rates for standard surgeries are scattered throughout the specialty journal literature. Thus,it is only read and aggregated in the minds of the practitioners who read this literature.
This is why I prefer East Asian, specifically Korean patient care. Over there, they have very good medicine Oriental and Western. The Asian doctor knows that you will die. It matters little that you die at 50 or 70. It matters not at all that you die at 70 or 100. You get the total package there. Here in the US, you get batting averages.
ReplyDeleteRoger:
ReplyDeleteSpeaking of mathematicians, there were once four constipated mathematicians; they all used different methods on the problem.
The first did it by hand. The next used a pencil and paper. The third used his slide rule and the last guy used logs to work it out.
Having been married to a doctor, I learned a few quick and dirty tricks on how to evaluate docs. One, find out who other doctors go to. As they say, "surgeons bury their mistakes". However, most doctors are in a much better position to judge. Another trick is to find out where the doctor has hospital privlidges. Hospitals do a very thorough background check before they will let some practice there. It is not fool proof but several hospitals doing due diligance is better than none.
ReplyDeleteIn order to properly measure outcomes you would need to be able to properly measure the base line of the patients. A case in point, say you have two infectious disease doctors. One's patient base are gays and the other has heroin users. One group is very compliant, educated, has health insurance, money which buys good food and shelter, can afford medicine and they have a partner who looks after them. The other group is high all the time, does not care about personal hygiene, forgets to take their medicine, lives on the street, and is hated and shunned by everyone (including the doctor in with the wealthy practice). Which group / doctor do you think will have a better outcome?
ReplyDeleteThere is a downside to it: If your case seems routine, chances are ~95% that it will be some resident (i.e., the most inexperienced physician possible) who will perform the actual surgery/procedure.
ReplyDeleteBig deal and you're wrong about the "most inexperienced physician possible", my kid sister had delivered 10 babies by the time she graduated medical school, if there's an experienced doc over her shoulder (and there was), the patients aren't in any danger. If things go south for mother or child, an experienced OB/GYN is already in the room.
The advantage of a teaching hospital isn't so much they're better at the extraordinary cases, but their superiority with routine cases, simply having more eyeballs reading your chart every day means there's less likely to be a "routine screwup" that goes unnoticed (ignored symptoms, ordered tests that were never run, improper drug dosage, etc) than if just one doctor was reading your chart. I saw this happen just last month to a family member, and the kid sister mentioned above had our relative change hospitals.
"Never forget that:
ReplyDelete1A) Half of all residents graduated in the BOTTOM half of their med school classes"
That's actually wrong. One-third of all US residencies (which are funded by Medicare incidentally) are foreign medical graduates, FMGs are almost certainly going to be in the top half of their class to match with a US residency program.
In other words, its more like only one-third of all US medical residents graduated in the bottom half of their class.
Anon: Related question: why do Indians produce at least some gifted doctor-writers?
ReplyDelete--
Gawande and Mukherji are brahmins
Verghese is a Syrian Christian merchant caste
That's actually wrong. One-third of all US residencies (which are funded by Medicare incidentally) are foreign medical graduates, FMGs are almost certainly going to be in the top half of their class to match with a US residency program.
ReplyDeleteIn other words, its more like only one-third of all US medical residents graduated in the bottom half of their class.
Foreign medical schools != US medical schools. The United States is one of the very few countries where medicine attracts truly the cream of the crop, and those other few countries where it does are by and large not the ones from which residency programs get most of their graduates. In countries where doctors make little more than police or low level bureacrats, the quality of doctors is always suspect.
"It’s not like mathematics or baseball or pop music, where your best work is often behind you by the time you’re thirty. Jobs that involve the complexities of people or nature seem to take the longest to master"
ReplyDeleteIs this guy serious? Mathematics and baseball AND pop music? Mathematics is a "job" that does not "invlove complexities"? This passes for high level sophisticated comment?
It just reinforces my observation that physicians are narrowly speialized highly skilled labourers full of themselves.
The other docs know. When I need a giant cell tumor removed from my finger, my PCP gave me three surgeons as options. She recommended one in particular, saying, "He's a prick, but if I had my hand severed, he's the one I'd want doing the surgery."
ReplyDeleteFYI, he was an Indian-American.
Because literarily gifted Indians are more likely than whites to feel compelled to major in something respectable i.e. scientific, and because medicine is the most humanistic of the available scientific fields?
ReplyDeleteCould be that something similar was going on with a previous generation of Jewish doctor-writers i.e. Sherwin Nuland and Jerome Groopman.
* * *
Anon: Related question: why do Indians produce at least some gifted doctor-writers?
As others have mentioned, your best bet is to ask an OR nurse which doc to avoid. My dad's friend was scheduled for open-heart surgery with some butcher. Luckily, the man's daughter was an OR nurse who knew the surgeon's horrible reputation and demanded, and got, the top heart surgeon in the country to operate on her father instead. Squeaky wheel, grease.
ReplyDeleteIt pays to have friends and family members in law and medicine.
Tennessee surgeon Bill Frist is one of the few politicians who retired from politics at the peak of his career (and he was no backbencher, he was Senate majority leader) to go back to his old profession.
ReplyDeleteNow that I think of it, he wouldn't make the worst vice president (he could spend his time in the OR of Bethesda Naval Hospital if he wants).
The United States is one of the very few countries where medicine attracts truly the cream of the crop
ReplyDeleteWhat a load of crap. Every single student that I personally knew (N=8 as I now recall) who ended up going to med school and becoming a clinical doctor was a poor or only so-so science student. Decent but on a dumber side of it. If I had to generalize, their main traits seem to be: careerism, lack of squeamishness in gaming the system on every level possible, and a talent for rot memorization.
This is consistent with the most prominent traits majority of doctors display: semi-robots with poor analytical skills. Which is also well reflected in the sorry state of clinical publications.
Is there data available on the IQs, passage rates of qualifying exams to practice medicine, board certification, etc. for foreign medical grads vs US grads?
ReplyDeleteMost first world countries do as well as or better than we do in terms of observable medical outcomes, as I understand it. This seems like a first-cut argument against the idea that foreigners from those countries make much worse doctors, though that doesn't tell you much about third world doctors.
I've been treated by a number of Indian doctors, all of whom seemed to me to be quite capable. (But I'm not sure I could judge how good a doctor is technically, just in terms of observable stuff like actually taking a history, washing his hands, etc.).
"Because literarily gifted Indians are more likely than whites to feel compelled to major in something respectable i.e. scientific, and because medicine is the most humanistic of the available scientific fields?"
ReplyDeleteThis sounds very plausible.
You have to know more than the surgeons' battering average. You'd have to know how he bats against left-handers vs right-handers. Dome stadium or not? How's his fastball vs his curve-ball or changeup?
ReplyDeleteThe metrics for successful surgery are even more complicated and depend on a host of variable outside the surgeons control. How do we even define successful surgery? The number of years you live after it?
I've worked with plenty of foreign medical grads (the PC term is "International Medical Grads", and on the whole they seem just as competent as American grads. To qualify for an American residency these IMG's have to pass written tests more stringent than those given American grads.
ReplyDeleteKeep in mind that American medical school doesn't necessarily mean Johns Hopkins or UCLA -it may mean East Carolina or Loma Linda. Also, there is a whole other route for Americans who can't get into an M.D. medical school: D.O. schools have much weaker entrance requirements yet still churn out an American doctor after 4 years.
However, it is very tough for an IMG to get a competitive residency (orthopedics, radiology, etc.) in a prestigious American university. They tend to train in non-competitive fields (primary care, general surgery, etc.) and/or Podunk hospitals. But then, so do most American grads.
Per my understanding. IMG have to take an exam called USMLE, the mean score on that is 82, and the standard deviation is 4
ReplyDeleteMeaning, a score of 90, would mean 2 SD above the average
Ask how he / she did in USMLE
A lot of ignorance in the comments. Public reporting of quality information is a big new topic in medicine. In many states, this info is becoming increasingly available. This information has clinically significant implications ie it can be used to predict risk adjusted mortality rates. Interestingly, the public seems reluctant to use this information (so far).
ReplyDeletehttp://jama.ama-assn.org/content/283/14/1884.full
http://www.nejm.org/doi/full/10.1056/NEJMsa064964
Particularly famous is New York where they began publicly reporting bypass surgery data:
"We examined the impact of New York State’s public reporting system for coronary artery bypass surgery fifteen years after its launch. We found that users who picked a top-performing hospital or surgeon from the latest available report had approximately half the chance of dying as did those who picked a hospital or surgeon from the bottom quartile"
http://content.healthaffairs.org/content/25/3/844.full
"A lot of ignorance in the comments. Public reporting of quality information is a big new topic in medicine. In many states, this info is becoming increasingly available....."
ReplyDeleteTrue, but.....
Figures are available only for very few types of surgeries, such as coronary-bypasses.
Some years ago, NEJM had an article showing that except for a few rarefied procedures, the hospital matters a lot more than the particular surgeon. Think about it: a coronary bypass is a technical skill learned by tens of thousands of people. Yes, a wrong suture can kill, but so can a f-up by the anesthesiologist, the ICU nurse, the hospitalist, etc.
Posters have suggested that patients ask OR nurses for surgeon recommendations. That is wrong. OR nurses only see purely technical stuff and have cuddly feelings for surgeons who are fast and don't yell at them (much). They have no idea about pre-op selection or post-op care.
In the case of coronary bypasses, the medical literature indicates that many patients shouldn't have them at all.
Most first world countries do as well as or better than we do in terms of observable medical outcomes, as I understand it. This seems like a first-cut argument against the idea that foreigners from those countries make much worse doctors, though that doesn't tell you much about third world doctors.
ReplyDeleteAnd like I pointed out in my original post, the vast majority of foreign med school graduates applying for US residency programs do NOT come from first world countries. It's mostly India, Africa, and the Middle East that provide the majority of applicants. Well paid doctors from countries with smoothly functioning health care systems have no incentive to come here and suffer through 3-5 years of slavery (residency) for the privilege of a 30% pay raise at the end of it.
What a load of crap. Every single student that I personally knew (N=8as I now recall) who ended up going to med school and becoming a clinical doctor was a poor or only so-so science student. Decent but on a dumber side of it.
Cool story, bro. As a matter of fact, I am currently a pre-medical student and I want you to know that nobody these days (aside from blacks and hispanics) ends up going to med school if they're a "poor or so so science student." Average GPA for med school admits today is 3.67, and the vast majority are science majors. This 3.67, note, is including blacks and Hispanics who can and do get in with atrocious GPA's and MCAT scores. Maybe there was a time (hey, I don't know how old you are) when mediocre students could get into medical school, but those days are looong gone.
Furthermore, the average MCAT score for white admits is 32. Just to put things in perspective, I was a national merit semifinalist in high school (so 99.5+ percentile on SAT) and on my practice MCATs I'm getting in the 33-36 range atm, and the real thing is known to be harder than the crappy practice ones. Point is, 3.67 is a good gpa, and 32 is a good MCAT. And those are average statistics for admits these days.
Felix, relax man.
ReplyDelete'... the vast majority of foreign med school graduates applying for US residency programs [IS] mostly India, Africa, and the Middle East
I think most of them are from India.
"Well paid doctors from countries with smoothly functioning health care systems have no incentive to come here and suffer through 3-5 years of slavery (residency) for the privilege of a 30% pay raise at the end of it."
All real doctors make $100K after residency.Period. Radiologists make 400K. Few doctors anywhere else in the world make that much. Certainly not in India.
<<<>>>
It's not that tough to get into medical school. The overall acceptance rate is 40% or so (same as dental school). You can always go to a Caribbean school or a D.O. school.
You probably should tone down your arrogance when you go for your med school interviews.
Felix: "nobody these days (aside from blacks and hispanics) ends up going to med school if they're a "poor or so so science student." Average GPA for med school admits today is 3.67, and the vast majority are science majors."
ReplyDeleteDon't be a moron. High GPA has very, very little to do with whether a student has good analytical skills or not. I've seen utter morons with GPA 3.9.
"good analytical skills or not. I've seen utter morons with GPA 3.9."
ReplyDeleteYou also know that having great analytical skills has little correlation with being a moron or not. I've seen all kinds of reknowned scientists that I would regard as true morons.