(Last in a series. Part one can be read here, and part two can be read here.)
When
Trinity Regional Health System's west campus in Rock Island was
named a "Top 100" hospital earlier this year, that distinction
quickly became a staple of the hospital's marketing.
But as a health-care consumer, the honor should lead to questions: Who makes the decision? What are the criteria? Does any money change hands to get the award?
One Trinity official even conceded that consumers should be skeptical of awards. "A lot of times it's even a marketing ploy on their side [the company that's giving the honor] to say that you've won this award, because they've got something now that they want you to do with it," said Kathleen Cunningham, Trinity's vice president of operations.
"It's going to be very difficult for the consumer to know what criteria they are even using," she continued. "As you look at the CMS data [from the federal government], and you look at some of those outcome measures, those are probably more meaningful from a quality perspective than for a consumer to say, 'Oh, well, Trinity's a Top 100 hospital ... .' ... The consumer probably doesn't understand what that is based on."
I'm not trying to get Cunningham fired, and the reality is that a Top 100 Hospital designation is significant. The company that produces it, Thomson (which purchased Solucient, the health-care information company that previously gave the award), is reputable, and hospitals neither apply for the award nor pay money to use it in their marketing. (They can purchase a detailed report designed to help the hospital improve its performance.)
But the Top 100 Hospital designation takes into account much more than medical care, for example looking at expense per discharge, operating margin, cash-to-debt ratio, and growth in patient volume - things that are far less important to consumers than the quality of health care. (More information on the award is available at http://www.100tophospitals.com.)
The critical point is that without the proper context, a list of awards or accreditations doesn't mean much.
Over the past two weeks, the River Cities' Reader has described the financial aspects of hospital competition in the Quad Cities and explored the challenges associated with evaluating hospital performance. For the final part of this series, we're looking at head-to-head measures of quality. We'll also discuss the ways in which awards and accreditations can be valuable to consumers.
Quality Measures
Accepted measures of quality are the best comparison because hospitals are expected to collect and report the information in the same way.
The federal government currently tracks 22 process measures in four areas of care: heart attack, heart failure, pneumonia, and surgical care. That information is presented on numerous Web sites, but for this article the data was drawn from the Department of Health & Human Services' Hospital Compare (http://www.hospitalcompare.hhs.gov).
To give a sense of what the statistics look like, Hospital Compare measures the percentage of heart-failure patients given discharge instructions. Top hospitals score 94 percent on this measure. Nationwide, hospitals averaged 63 percent. In Illinois, the number was 68 percent, and in Iowa, 62.
Genesis' Davenport facilities scored 65 and its Illini facility 87. The DeWitt hospital had a small sample size of five. (Ratings are not given for hospitals with fewer than 25 instances.)
Trinity's Illinois facilities scored 87 percent, while Terrace Park scored 81 percent.
The data can be broken down in several ways.
You can compare individual hospitals. In the 19 measures in which there was any difference between the performance of Genesis' Davenport facilities and Trinity's Illinois hospitals, Trinity had the edge, 11 measures to eight. (In many cases, though, the difference was not appreciable. The percentage of pneumonia patients given oxygenation assessment was 100 for Genesis, and 99 for Trinity, for example.)
You can also compare performance to the overall performance of hospitals in their states.
But that mirrors the trends you'd notice if you compared local hospitals' performance to national averages. And put simply, Quad Cities hospitals met or exceeded national averages in these measures a vast majority of the time. Even the worst performer among area hospitals - Genesis' Illini campus - significantly lagged behind the national average in only 30 percent of applicable measures.
The
River Cities' Reader
compiled a comparison of local hospitals on all 22 quality measures
that's available here. In
summary, Genesis hospitals were within 2 percentage points of
national performance or better 82 percent of the time in 2006, while
Trinity met that criterion 100 percent of the time. (See accompanying
charts.) The analysis excludes situations with small sample sizes, as
defined by Hospital Compare.
As we explained in last week's article, these measures are evolving and are imperfect, and they don't necessarily mean that Trinity is a better health system than Genesis. It means that in this narrow way of looking at health care provided - specific, evidence-based courses of action in four core areas - Trinity performed better last year.
The
frustration with Hospital Compare's process measures is that they
doesn't give any sense of outcomes - how the patients fared.
That issue is addressed by, among other organizations, Health Grades (http://www.healthgrades.com), which compares actual outcomes to expected outcomes and covers a far greater range of care than Hospital Compare.
Still, head-to-head comparisons are sometimes difficult because of small sample sizes. Furthermore, comparisons across state lines can be impossible. For example, maternity care is rated in Iowa but not in Illinois for Health Grades, so that measure is largely meaningless for the Quad Cities market.
Health Grades has been criticized for inconsistently applying its star ratings - five stars for better-than-expected performance, three stars for expected performance, and one star for worse-than-expected performance.
The University HealthSystem Consortium, in a 2005 "Market Watch" publication, noted that these star ratings can be deeply flawed: "When hospitals from different rating categories were compared individually, risk-adjusted mortality rates were either comparable or even better in the lower-rated hospitals in more than 90 percent of comparisons."
But the risk-adjusted rates themselves weren't questioned.
Those rates were what the River Cities' Reader compared in a sampling of 2007 ratings from Health Grades. We selected six areas intended to illustrate outcomes across a range of medical procedures: heart attack, heart failure, total knee replacement, gastrointestinal procedures and surgeries, back and neck surgery (except spinal fusion), and community-acquired pneumonia.
Sometimes the outcome measured was "major complications," and other times it was mortality (often in several time frames). For the six areas, there were 11 total outcome measures. (The comparisons are available here.)
For gastrointestinal surgeries and procedures, for instance, Health Grades looks at in-hospital mortality.
Genesis' Davenport facilities had an expected mortality rate of 5.10 percent, while their actual mortality rate was 7.24 percent. The system's Illini hospital had an expected rate of 6.73 percent and an actual rate of 6.04 percent. The DeWitt hospital was not rated in this area.
Trinity's Illinois facilities had an expected mortality rate of 5.42 percent and an actual rate of 4.83 percent. The health system's Terrace Park had an expected rate of 3.33 percent and an actual rate of 3.12 percent.
Overall, the results are less encouraging than the process analysis. Genesis hospitals were more than 2 percentage points behind the expected outcomes in 33 percent of the measures, while Trinity was also more than 2 points behind expected outcomes 33 percent of the time.
In 13 percent of measures, Genesis was more than 2 percentage points better than expected outcomes, while Trinity had no measure in which it exceeded outcomes by 2 points.
Health-System Responses
Numerous officials at both Genesis and Trinity were shown the comparisons and given the opportunity to provide a response. (Their full responses are available at the end of this article.)
Genesis' response started by noting a high level of care in this geographic area: "After reviewing this data, all Quad Citians should take great comfort in the knowledge that the quality of health care in this community is clearly among the best in the nation. This realization has been confirmed by many national studies, including the most recent study by the Commonwealth Fund that reports the overall quality of health care in Iowa to be among the best in the nation."
Among other questions, the two health systems were asked: "In situations in which performance was below state/national averages (with Hospital Compare) or the expected (with Health Grades), what factors might help explain the deficiency?"
Cunningham, who provided Trinity's response, wrote: "The data accurately reflects our hospital statistics for the time period in which it was captured; however, statistics alone don't always tell the whole story. In some measures when the sample size is small, applying that statistic to overall performance is not as meaningful. Yet, Trinity supports providing information through sites like Hospital Compare that make the public better consumers; it gets patients asking questions and therefore becoming more involved in their care."
Genesis' response was more specific: "As for the specific measures considered by the Hospital Compare Web site, the insignificant differences reported in this region are negligible and can be explained by sample sizes, reporting techniques, and accuracy and other data issues. But, again, the public should celebrate the fact that in virtually all measures health care in the Quad Cities ranks near the top of the state and nation."
Genesis noted some of the deficiencies in these quality measures that were discussed in last week's article: "Consumers should also bear in mind is that the Hospital Compare Web site is relatively new. And, the quality measures it focuses on so far are fairly basic."
Genesis declined to comment on its Health Grades performance: "Since we do not believe the methodology employed by Health Grades accurately reflects the current level of quality performance, we feel its value to the consumer is limited. More importantly, Genesis does not put a great deal of stock in vendor awards, that is, awards from organizations whose primary business interest is the sale of products or services. For all of these reasons, we do not comment on Health Grades data."
Health Grades offers research for and consulting to hospitals. As Tod Ibrahim, executive vice president of the Alliance for Academic Internal Medicine, noted in 2005: "The cottage-industry entities - like Health Grades, the Leapfrog Group, and NQF - are accountable to neither the public nor official sponsors. At the very least, these evaluators should describe their methodologies and justify their conclusions. ... Health Grades is a for-profit enterprise, which makes it even easier for some to disparage."
Cunningham emphasized in her response that "Web-site rankings should merely be taken in context. They are simply a tool health-care consumers can use to research their care, not the final authority on whether a hospital can take care of you."
Are They Set Apart?
Beyond those head-to-head data comparisons, awards and accreditations can be useful tools when looking at a specific area of care.
Having an accreditation in an area "means that you've met the standard that's put out by that independent body of what is deemed to be good care, if not high-quality care," said Dr. Morris Seligman, Trinity's vice president of medical affairs. "You're doing it the way that it should be done."
But Trinity's Cunningham cautioned that applying for accreditation costs money, and noted that an unaccredited program can provide the same level of care as an accredited one.
Still, distinguishing oneself from competitors through accreditations is a strategy with benefits beyond a specific area of care. Cunningham explained that the premise is to "pull in those patients who are seeking that specialized care, and deliver the best quality, the best patient satisfaction, so that you begin to build some of that loyalty. ... 'That's where I'm going to get top quality no matter what kind of care I need,' whether it's that differentiated service or not ... ."
In its response, Genesis enthusiastically endorsed accreditations as a good line of inquiry for health-care consumers: "One excellent source of consumer guidance are the independent, specialized accreditation boards. For instance, among other accreditations system-wide, the Genesis Center for Bariatric Surgery is the only program in the region or the state of Iowa certified by the American Society for Metabolic & Bariatric Surgery. The Genesis Regional Rehabilitation Center is the only program in the region certified by the Commission on Accreditation of Rehabilitation Facilities."
"I think it's an organization's commitment to getting those extra accreditations, going above and beyond the [basic] Joint Commission accreditation, that can help consumers ... decide: Are they really the same, or are they set apart ... ?" said Dr. James A. Lehman, Genesis' vice president of quality.
Genesis' response also noted: "Heart patients should seek out the advice of the Society of Thoracic Surgeons and the American College of Cardiology. For example, based on recent outcomes, Genesis just received the highest rating from the Society of Thoracic Surgeons for open-heart surgery outcomes."
In an interview, Cunningham cited Trinity's accreditation with the American College of Surgeons Commission on Cancer and the American Association of Cardiovascular & Pulmonary Rehabilitation. Her written response did not cite any accreditations in a specific area of care.
She also in her interview noted the affiliation of Trinity's open-heart program with Cardiac Surgery Associates, a group based in the Chicago suburbs that staffs 19 cardiac-surgery programs in Illinois, Iowa, and Indiana. According to Mike Dessert, director of cardiology services at Trinity, the health system has one full-time heart surgeon who lives and works exclusively in the Quad Cities. Surgeons from Cardiac Surgery Associates rotate weekly at Trinity on an eight- or nine-week schedule, he said.
A Culture of Quality
Certifications can be useful for somebody seeking care, said Genesis Vice President of Corporate Communications Ken Croken, but collectively "they're just countless, and I don't know that making a list is helpful to any given consumer."
That's where more general awards and accreditations can assist in evaluating hospitals overall.
For both Quad Cities area hospitals, the list is impressive.
"Genesis Medical Center, Davenport, has been recognized as Magnet facility by the American Nurse Credentialing Center, a distinction received by fewer than 4 percent of hospitals in the nation," Genesis wrote in its response.
The health system further wrote, "Genesis Health System is the only Quad City-based health care organization this year recognized for its investment in state-of-the-art medical technology with a designation among the nation's '100 Most Wired' [awarded by Hospitals & Health Networks magazine]. And next month, Genesis will be bringing digital mammography for the first time to women in the QCA. ...
"Genesis is the only health system in the nation to have won the Magnet, the Nova [an award that hospitals apply for that's given by the American Hospital Association and Hospitals & Health Networks magazine], multiple Consumer Choice Awards [given by vendor National Research Corporation], and Most Wired designations."
Cunningham, in Trinity's response, cited the Top 100 Hospital award and the CareScience Select Practice National Quality Leader designation. "Neither award is applied for, and both focus on excellence in clinical outcomes, safety, financial performance, and efficiency of care," she wrote. "Plus, both are only given to a small group of recipients (one of 100 hospitals and one of 45 hospitals), from the thousands of health-care systems nationally." CareScience is a vendor, but it does not charge hospitals to market the award.
While it might be difficult or impossible to say with any confidence whether Trinity or Genesis is a better hospital, it's clear that in today's health-care environment, there's a push to improve the quality of care and outcomes.
As Genesis said in its response, "The organization is now committed to the Malcolm Baldrige [National] Quality [Award] journey and is aggressively pursuing this next level of quality achievement. At Genesis Health System, we consider the pursuit of quality a race without a finish line."
Trinity's Seligman also cited that award from the president as the pinnacle for a health-care organization, and said that his organization wants to be in the top 10 percent in the nation for all quality measures.
"If you can eventually say that ... across the board, we are in the top 10 percent in the nation in these areas, the public should be pretty confident when they walk in the door," Seligman said. "The standards out there, they're national."
It's also apparent that competition is in part driving this quality movement in health care.
"We [as a community] do very, very well in [health-care] quality," said Bob Travis, Genesis' vice president of strategic development. "I think competition improves quality. I'm quite confident of that."
"We get better every day because of what Genesis does," said Bill Leaver, president and CEO of Trinity Regional Health System. "I believe Genesis gets better every day because of what we do."
The first two parts of this series were published in Reader issue #650, and Reader issue #651.
Hospital responses
Both local hospital systems were given the opportunity to review and respond to the process and outcome data that was used in this article. They were asked four questions:
1) Are there measures that don't accurately reflect your hospital's performance in that area? Please explain.
2) In
situations in which performance was below state/national averages
(with Hospital Compare) or the expected (with Health Grades), what
factors might help explain the deficiency?
3) These measures are meant to reflect a mix of process and outcomes based on patient data. What other publicly available, comparative process or outcome statistics would you point to as a reflection of your health-care organization's quality?
4) What awards and/or accreditations do you think are most reflective of your health-care organization's overall quality? (Please choose no more than five.)
Genesis response:
After reviewing this data, all Quad Citians should take great comfort in the knowledge that the quality of health care in this community is clearly among the best in the nation. This realization has been confirmed by many national studies, including the most recent study by the Commonwealth Fund that reports the overall quality of health care in Iowa to be among the best in the nation.
As for the specific measures considered by the Hospital Compare Web site, the insignificant differences reported in this region are negligible and can be explained by sample sizes, reporting techniques and accuracy and other data issues. But, again, the public should celebrate the fact that in virtually all measures health care in the Quad Cities ranks near the top of the state and nation.
However, what consumers should also bear in mind is that the Hospital Compare Web site is relatively new. And, the quality measures it focuses on so far are fairly basic. For example, while Genesis is certainly proud of our "Top Hospital" performance rating for pneumonia care, we feel this is an area of basic performance and we would rather be noted for the more sophisticated measures of quality, such as our national rankings for heart attack or stroke response.
As for the Health Grades data, Genesis has a slightly different perspective. Since we do not believe the methodology employed by Health Grades accurately reflects the current level of quality performance, we feel its value to the consumer is limited. More importantly, Genesis does not put a great deal of stock in vendor awards, that is, awards from organizations whose primary business interest is the sale of products or services. For all of these reasons, we do not comment on Health Grades data.
One excellent source of consumer guidance are the independent, specialized accreditation boards. For instance, among other accreditations system-wide, the Genesis Center for Bariatric Surgery is the only program in the region or the state of Iowa certified by the American Society for Metabolic and Bariatric Surgery. The Genesis Regional Rehabilitation Center is the only program in the region certified by the Commission on Accreditation of Rehabilitation Facilities. And, Genesis Medical Center, Davenport has been recognized as Magnet facility by the American Nurse Credentialing Center, a distinction received by fewer than 4% of hospitals in the nation.
Consumers seeking to evaluate a healthcare organization's quality also should focus on the specific area of service the consumer needs. Rather than taking comfort in a non-specific "Top Hospitals" claim, heart patients should seek out the advice of the Society of Thoracic Surgeons and the American College of Cardiology. For example, based on recent outcomes, Genesis just received the highest rating from the Society of Thoracic Surgeons for open-heart surgery outcomes.
Though it may seem counter-intuitive, even unexpected medical needs can be anticipated. Going to the "nearest emergency room" may not always be wise. Not all emergency rooms are alike. Consumers should know the capabilities of each emergency room in the area to ensure that the staff at the nearest location is prepared to address your health issues. Otherwise, you may find yourself on a time-consuming and life-threatening transfer to another ER. Based on independent, credentialing organizations, the Genesis ER in Davenport is the area's most sophisticated trauma center and is also best equipped to respond to heart attack patients.
Also, consumers might wish to consider a health system's commitment to quality in general. Does the organization demonstrate a commitment to quality and patient safety through its investments? For instance, Genesis Health System is the only Quad City-based health care organization this year recognized for its investment in state-of-the-art medical technology with a designation among the nation's "100 Most Wired". And next month, Genesis will be bringing digital mammography for the first time to women in the QCA. Also, consumers may want to inquire about other aspects of the organizational commitment to quality. Is there a local executive overseeing quality issues or is direction coming from outside the immediate organization, like a corporate headquarters several hundred miles away from the patient, possibly in another state.
Finally, consumers also can ask family and friends about their health care experiences. Although not a "fool-proof " system, chances are if your neighbors have been satisfied with the quality of the care they have received; you will be, too. For this reason, we at Genesis point with pride to the National Research Corporation's annual health care market research. For the past six years, Genesis has been chosen as the preferred Consumer Choice by Quad Citians. This is particularly satisfying since the research is tied to the U.S. Census Bureau's MSA designation, which places nearly two-thirds of those polled in Illinois.
As for accreditations and awards, again, Genesis will point with pride to the independent accreditations and awards, that is, those awarded based on performance and not premised on contractual relationships. While we at Genesis are pleased to have been named to the list of "Top Hospitals," we are quick to acknowledge a vendor relationship with the naming organization. Rather, I would refer back to the independent awards noted earlier, such as the Magnet designation for quality nursing care awarded by the American Nurses Credentialing Center and the Society for Thoracic Surgeons' three-star rating for open-heart surgery.
Also, I would focus less on past awards and more on future goals. For example, Genesis is the only health system in the nation to have won the Magnet, the Nova, multiple Consumer Choice Awards and Most Wired designations. But, the organization is now committed to the Malcolm Baldrige Quality journey and is aggressively pursuing this, next level of quality achievement. At Genesis Health System, we consider the pursuit of quality a race without a finish line.
Trinity response:
1) The data accurately reflects our hospital statistics for the time period in which it was captured ; however, statistics alone don't always tell the whole story. In some measures when the sample size is small , applying that statistic to overall performance is not as meaningful. Yet, Trinity supports providing information through sites like Hospital Compare that make the public better consumers; it gets patients asking questions and therefore becoming more involved in their care.
2) Trinity is committed to continuous quality improvement whether it relates to clinical quality indicators, safety, patient satisfaction, or efficiency and was one of 4,000 providers voluntarily submitting their data for this project when it first became available. When we find opportunities to improve, we use those measures to help us set priorities for improvement. From a provider standpoint, these reportable data sets give us a measuring stick by which we can gauge our own performance improvement efforts. Previously, we could only gauge against ourselves. New reporting mechanisms allow us to gauge against other health systems and strive toward leading practice.
3)
The CMS data is the most widely known public data available, and
while consumers have a responsibility to be informed about their
health care, Web site rankings should merely be taken in context.
They are simply a tool health-care consumers can use to research
their care, not the final authority on whether a hospital can take
care of you.
4) I would select the Solucient Top 100 Hospitals and the CareScience Select Practice National Quality Leader designations. Neither award is applied for, and both focus on excellence in clinical outcomes, safety, financial performance and efficiency of care. Plus, both are only given to a small group of recipients (one of 100 hospitals and one of 45 hospitals), from the thousands of health-care systems nationally.
