Medical Staff

Quality Data Transparency: A Two-edged Sword

Medical Staff Affairs Monthly, March 15, 2006

Dear Colleague,

I opened the newspaper this morning to headlines noting serious errors recently made by the company that scores SAT tests for college bound high school students. The news was full of speculation on how these mistakes might affect the future prospects of students in a highly competitive market for college placement. Less well publicized, but closer to home for those working in the healthcare community, was an admission last week by US News and World Reports that it had used inaccurate data to rank health care plans in its now familiar annual report to the public. The resulting rankings were seriously off base as a result. The publication was quick to point out that the fault lay elsewhere since it relied for its ratings on data provided by the "watch-dog" agency for health plans. "We recently discovered an error in the complex calculations used to develop the rankings for the Best Health Plans project," the National Committee for Quality Assurance said in a statement. "We want to emphasize that this error originated with NCQA and not U.S. News." NCQA is a private non-profit health care rating agency, which apparently failed to adequately consider some preventive clinical measures in its data collection.

In another news item this month, the Government Accountability Office (GAO) recommended that The Centers for Medicare and Medicaid Services (CMS) should take steps to improve the accuracy and completeness of hospital quality data used to calculate their performance on ten quality measures to ensure the maximum benefit of making this information available to the public ("Hospital Quality Data: CMS Needs More Rigorous Methods to Ensure Reliability of Publicly Released Data" (GAO-06-54)). It seems that many hospitals are providing the government something less than rigorously vetted quality data. No wonder since under the Medicare Modernization Act (MMA), hospitals that fail to report the quality data on both Medicare and non-Medicare patients lose 0.4% of their Medicare payment update. In fiscal year 2006 this reporting needed to occur with 80% accuracy to avoid the 0.4% reduction. Certainly no six sigma goal there.

It would appear that as we go about making health care performance data more transparent to the public, what we are also revealing is the weak data infrastructure that exists in the industry. Of course, the adverse consequences of reporting inaccurate data are more significant then ever as we see increased migration to 'pay for performance', consumer driven health plans, the "tiering and steering" of providers and patients by health plans, and more stringent criteria for physician privileging based on aggregated performance data. It is probably too late to put up a caution flag and slow the reporting process down - the curtain is up on this show and the audience wants to see what the players are up to. So it becomes increasingly important to ramp up our efforts in performance measurement and get it right. For hospital management, this means putting the necessary resources behind the effort. For practitioners, this means using the organized medical staff to improve the accuracy of hospital data collection activities. In recent years, a number of medical staffs have established a 'data trends council' to weigh in on the choices in data for collection and to vet the utility of different measures of performance. This medical staff committee consists of individuals interested in this work and who are given additional training so the group becomes a real locus of expertise. It becomes a source of rigor at a time when it is sorely needed. Every staff should consider establishing this kind of work group.

The JCAHO announced this month that it was going to make the data it collects for its Quality Compare program available to the public. The organization's president noted that part of the JCAHO's mission is to disseminate clinical-performance information and its database of hospital performance information should be viewed as an open, public utility "free to whomever wants to access it." The rush to disseminate information is well-intentioned, but when I think of the state of the art I can't help think of a quote by Robert Cringely in the pages of InfoWorld Magazine: If the automobile had followed the same development cycle as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside. We are making great strides in the measurement of clinical performance, but as more and more healthcare data circulates, let's hope that the new transparency doesn't inadvertently "blow up" the reputations of competent providers and the business viability of quality health care enterprises.

Best Regards,

Todd Sagin, MD, JD
Vice President and Medical Director
The Greeley Company

Most Popular

Related Articles