A recent article in the New York Times examined how much of President Obama’s praise for health care reform based on a few select hospitals like the Mayo Clinic for delivering high-quality care at low costs, was based on a pointed analysis that may be unwarranted.
Information Mr. Obama often cited was about hospitals from a widely cited analysis called the Dartmouth Atlas of Health Care, produced by the Dartmouth Institute for Health Policy and Clinical Practice. NYT noted that “an article in The New Yorker magazine last year written by Dr. Atul Gawande that used the Dartmouth Atlas as its organizing principle became required reading in the White House last year.”
A recent analysis however, written in The New England Journal of Medicine (NEJM) by Dr. Peter B. Bach, a physician and epidemiologist at Memorial Sloan-Kettering Cancer Center in Manhattan, “suggests that much of the Dartmouth Atlas is flawed and that it should not be used to compare the relative efficiency of hospitals.” This implication is very serious going into the Health Care Summit this week, especially because “proposals in Congress called for using analyses like those found in the Dartmouth Atlas to begin spending less money on regions where medical care is especially costly, including places like New York City.”
One Problem Dr. Bach sees with the Atlas study is that it “assess hospital efficiency overall on the basis of costs incurred for non-representative patients — decedents who were enrolled in fee-for-service Medicare. This group varies among hospitals in terms of severity of illness and is not representative of a given hospital’s overall spending pattern.” According to Dr. Bach’s analysis, Atlas’s methodology “makes no effort to determine if the hospitals are any better at saving people’s lives, and do little to adjust for the relative health of the patients being treated, among other problems.”
In fact, while the Dartmouth Atlas tends to rate poorly hospitals that provide lots of expensive procedures, a different study published last week by researchers at the University of Pittsburgh found that patients tend to live longer in such intensive care hospitals.”
The conceptual problem Dr. Bach sees is that Atlas uses all health care costs that are incurred by patients over the 2 years before their death are attributed to the hospital where they were admitted most frequently during that period. This method assumes that the hospital controls all, or at least most, patient care, even if it occurs outside the hospital or in another hospital. This method is problematic according to Dr. Bach because much of the care from a hospital is beyond their control, something the Atlas authors agree with. Dr. Bach even cites that one third of the patients who were included in an assessment of hospital efficiency had been admitted to the hospital in question only once, not accounting for the other places they received care from.
Another problem Dr. Bach discussed is that the Atlas-based analyses use only costs (i.e., resources consumed), instead of weighing both resources consumed and outcomes. He noted that “since outcomes vary among hospitals and providers, both costs and outcomes must be assessed in evaluating efficiency.”
Responding in a separate article to Dr. Bach’s analysis, Dr. Elliott S. Fisher, director of the Center for Health Policy Research at the Dartmouth Institute, said “agreed that the current Atlas measures should not be use to set hospital payment rates, and that looking at the care of patients at the end of life provides only limited insight into the quality of care provided to those patients.”
It will be interesting to see which studies and research Mr. Obama will use at his summit in discussing ways to reform health care. Sometimes myths are better than the real story, in the case of hospital disparity that may be a greater reality.
I certainly have a vested interest in all of this, as I’m at Dartmouth, and an author of the companion paper in the NEJM — which appears not to have been read! So my comments can be discounted accordingly.
1. Fisher was quoted out of context — check his NYT letter to the editor.
2. Bach apparently did not understand that all our end-of-life measures are risk-adjusted. For example, we are interested in how cancer patients in their last 2 years of life are treated at hospital A versus hospital B, and how much they are bounced around from one hospital to another during this difficult time. We think it’s informative — but it’s only one of many measures we look at.
3. As we show in our New England Journal of Medicine article (http://content.nejm.org/cgi/content/full/362/7/569-a), it doesn’t matter whether one uses heart attack patients (which avoid all of the hypothetical concerns voiced by Bach) or end-of-life measures, that the cost measure yields the same answer: high cost hospitals for end-of-life care are also high cost hospitals for treating heart attack patients. In other words, his hypothetical complaints are, well, hypothetical.
4. That Bach doesn’t like end-of-life measures says nothing about the dozens of different measures of utilization that are downloadable (www.dartmouthatlas.org). The media and the bloggers got this so wrong.
5. I’ve also seen the lazy claim that somehow Amber Barnato’s study “refutes” Dartmouth work. Nothing of the sort — she is a coauthor of ours! Some studies find tiny positive associations between spending and outcomes; her results suggest hundreds of thousands of dollars must be spent to extend the lifespan of a frail elderly person by a few months. A recent study by Lena Chen and colleagues from Harvard found essentially no effect, and our NEJM paper (the one mentioned above) found a negative association between spending and outcomes. The key point is that quality and spending across hospitals is essentially uncorrelated, meaning that there is tremendous opportunity to both save money and save lives.