ProPublica Surgeon Scorecard Ad
A major concern surrounding ProPublica’s recently released “Surgeon Scorecard” is that rating physicians in a public database solely on complication rates may discourage surgeons from operating on high-risk patients. The fear, it turns out, is a legitimate one. A study published this month in the Journal of the American College of Cardiology examined the effect that public reporting of procedural outcomes has on heart attack patients. The authors looked at two states, Massachusetts and New York, which mandate the public reporting of hospital outcomes of percutaneous coronary intervention (PCI) for myocardial infarction (MI). PCI, commonly known as an angioplasty, is a widely used procedure to open blood vessels that lead to the heart during or soon after a heart attack.
The investigators used the Nationwide Inpatient Sample to examine whether patients who were hospitalized for MI were more or less likely to go through PCI in states with public reporting compared to regional comparator states. Second, they looked at the impact of public reporting on patients with MI overall. The study found that in Massachusetts and New York, patients presented with MI were much less likely to undergo PCI than in non-reporting states. Further, this was disproportionately concentrated in patients with the highest risk: older patients and those with ST-segment MI (a severe heart attack) or cardiac arrest or cardiogenic shock. Cardiogenic shock, for example, is a severe condition where the heart suddenly cannot pump enough blood to meet the body’s needs—if treated immediately, about half of patients survive.
More importantly, the study noted that patient health suffered in public reporting states. The study found that patients with MI in reporting states had higher risk-adjusted in-hospital mortality rates than those in non reporting states. The study found that this was observed primarily in patients who did not receive PCI in the public reporting states.
The results, while observational, strongly suggest that interventional cardiologists are shying away from difficult cases where a patient has a high risk of complication, including death, despite treatment. Under the PCI public reporting metric, providers aren’t dinged if they choose to operate only on the patients with the greatest likelihood of survival.
C. Michael Gibson, an interventional cardiologist, Harvard Professor, and founder of WikiDocs, posed this question that clearly frames the issue at hand:
Clearly, if a patient wants a better chance at survival, PCI treatment is the way to go. However, the public reporting methodology and other transparency initiatives focus exclusively on complications. ProPublica’s recently released Surgeon Scorecard, for example, defines complications as “readmissions related to the surgery” and deaths during the initial surgical stay. However, it does not show the negative effects of NOT doing a surgery.
Much of the pushback on ProPublica’s search tool is that it positions itself as a comprehensive grading system and that patients will make actual healthcare decisions based on whether their surgeon falls in a green, yellow, or red category (i.e. “Making the Cut: Why Choosing the Right Surgeon Matters Even More Than You Know.”)
This is an example of how a surgeon is ranked–this one performed over one hundred prostate removals.
ProPublica’s Surgeon Scorecard has experienced pushback from the surgery community. An article on Kevin MD, which is worth a read, takes aim at how ProPublica defines “complication rate.” Here is a sample from the article:
So the 84-year-old patient three weeks out from hip replacement who is admitted through the ER with “increasing confusion” due to insomnia and overuse of narcotic pain meds is a red mark against the orthopedic surgeon. Urinary tract infection two weeks after spinal surgery in a patient with known BPH. The anxious 27-year-old lady readmitted at midnight two days after a LC because of refractory nausea. The 49-year-old male who develops chest pains ten days after lumbar fusion surgery. All these are reportable offenses that don’t necessarily have anything to do with the quality of said procedure performed.
—
Public reporting and efforts at increasing the transparency of surgical procedures have a laudable goal in mind of guiding patients to the best care. While it is easier to point out the flaws in a reporting system than come up with a perfect methodology, the public reporting of PCI study shows that true patient harm can result if the reporting metrics continue unchanged. Further, a fundamental criticism of the Surgeon Scorecard is how it is being framed by ProPublica. Their ad campaign seemed like more “click-bait scare tactic” rather than a true patient tool. The scorecard could actually cause people who need surgery to avoid it altogether, and it points to surgeons as the enemy (watch this “teaser” video).
The authors of the public reporting outcomes study conclude that moving forward, “public reporting of outcomes should balance the benefits of transparency and accountability against the potential influence of physician risk aversion.” Dr. Robert Yeh, one of the authors, notes that perhaps one improvement would be to exclude certain high-risk individuals, such as cardiac arrest patients, from public reporting (see his interview here). While it is very difficult, and potentially impossible, to find a metric that accounts for all the internal and external variables that take place when a patient goes in for a surgical procedure, the study hopefully guides reporting policies away from particularly problematic unintended consequences.