• E-mail
• Print
• PDF

Experts say quality surveys flawed

Hospitalist Management Advisor, November 1, 2008

Consumer-driven healthcare

Experts say quality surveys flawed

The case for redesigning patient satisfaction surveys

With the advent of Hospital Compare and the dawn of user-aggregated Web sites, people aren’t simply choosing hospitals and physicians out of convenience anymore. Instead, they’re price shopping and researching quality online—organization versus organization and physician versus physician.

In an effort to catch up with consumer-driven healthcare, many hospitals have taken the baton from the government and passed it to contracted vendors such as Press Ganey Associates, Inc., and other companies that compare healthcare facilities.

These agencies conduct patient quality surveys and use the data to design customer service initiatives. But in many cases, hospital administrators also use these data for individual performance reviews and even hospitalist compensation.

Most surveys are not specific enough to gauge the performance of physicians in newer fields such as hospital medicine, according to The Phoenix Group’s August white paper, “Hospitalists Meeting the Challenge of Patient Satisfaction.”

“These surveys don’t accurately show what patients are saying about their doctors. The current methodology is not accurate in terms of a patient’s satisfaction with a specific physician,” says Ron Greeno, MD, cofounder and chief medical officer at Cogent Healthcare in Brentwood, TN, and a founding member of The Phoenix Group.

The search for accuracy

But finding accurate data isn’t easy. In fact, it requires fundamentally changing how the government and vendors design surveys to uncover quality information, Greeno says. The Phoenix Group began as a think tank in 2007 with a mission to provide leadership to thousands of hospitalists in private practices. Today, it acts as a watchdog for healthcare quality and the increasing role of surveys in illuminating that quality.

Unfair comparisons

“More and more, quality and satisfaction will be used to determine payment,” Greeno says.

As of March, the Centers for Medicare & Medicaid Services (CMS) requires hospitals that receive money from Medicare—except for critical access hospitals—to survey patients each year and submit their findings. In turn, CMS will publish those data online. The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) on CMS’ Web site are used by all hospitals in compliance with Medicare requirements.

The goal is to collect standard data so consumers can compare hospitals on a local or regional basis, says Adam Singer, MD, CEO of IPC The Hospitalist Company, Inc., in North Hollywood, CA, and founding member of The Phoenix Group.

“I believe the industry is still trying to figure out what to do with this data,” Singer says. “It appears that more and more hospital CEOs are going to be incentivized and bonused in some formulaic manner according to how they do on these surveys.”

However, The Phoenix Group found that the HCAHPS is flawed for the following reasons:

• Surveys are tailored for the hospital and not the individual. The survey questions were designed to compare hospital to hospital, not group to group or physician to physician. The questions are generic, nonphysician-specific, and the answers can be misleading.

“They are meant to look at a patient’s overall satisfaction with the experience,” Greeno says. But many hospitals are taking those data and assigning a value to them when measuring hospitalist performance. “It wasn’t designed to do that,” he says.

• Small sample size. Hospitals are required to survey a certain number of patients, depending on the hospital size—usually only a few hundred patients per year, Greeno says. This is not a fair population size: Out of all respondents, only one or two may have been cared for by a specific hospitalist, he says.

It’s a case of simple math. “The smallest degree of patient dissatisfaction will lead to a poor score. How is that significant or actionable?” says Singer.

• Misplaced blame. Another flaw of the HCAHPS survey is that the administration may see a poor score as a reflection of the work performed by an entire hospitalist group.

“In an effort to shift responsibility, hospitals are the ones that are taking the data and focusing on the doctors when they should really be focusing on the entire patient experience,” Singer says. In that scenario, a hospitalist group’s contract may be put at risk because of the survey data. It’s wrong to “blame an entire hospitalist program for a low score, as that group may be managing the bulk of the facility’s inpatients. The doctors are only one part of the patients’ experience,” he says.

• Comparing apples and oranges (hospitalists and primary care physicians [PCP]). Another area of misunderstanding identified by The Phoenix Group occurs when a hospital measures hospitalist groups against PCPs. For example, a hospitalist may score a 94% satisfaction and a PCP 95%.

“Does it surprise anyone that a patient might be happy to see a familiar face? That is to be expected,” Singer says.

Vendor surveys part of the problem

Hospitals typically contract out to vendors who survey a cross section of the patient population. Those surveys generally include HCAHPS questions and the standard patient satisfaction questions the vendor has previously designed, says Singer.

As in the HCAHPS questions, pitfalls appear when using the vendor survey questions to measure quality. The first problem is that the questions haven’t evolved with the changing healthcare landscape and are not hospitalist-specific. The second problem with vendor surveys is hospitals scramble to use internal identification systems to discover which hospitalist a patient is complaining about in his or her survey responses. As Greeno notes, hospital administrators are often wrong.

According to internal data from one vendor of patient satisfaction in the white paper, patients weren’t coded correctly roughly 20% of the time, misrepresenting whether any hospitalist had been involved in their care.

“But hospitals are still using this inaccurate data,” Greeno says, adding that not only can it be inaccurate, it can be misdirecting.

For example, if a hospital looked at patient satisfaction surveys from the hospitalist group versus the cardiology group, the scores would naturally be different. Due to the nature of their role, hospitalists see groups of patients who don’t have insurance or a doctor, says Greeno.

“Everyone knows that patients that get admitted to hospitalists are in a population that historically have lower satisfaction scores,” he adds. “The surveys are statistically stacked against whoever admits these patients.”

Further, The Phoenix Group identified patients’ understanding of their case as another challenge related to the surveys. Some older or sicker patients may not be able to distinguish a cardiologist from a surgeon from a hospitalist, Greeno says. Hospitalists can help their cause by keeping their patients better informed about who they are and what their role entails, he says.

At Cogent, hospitalists work with clinical care coordinators to help educate patients about the role of the hospitalist team. For now, the data remain incomplete, and the surveys need redesigning, Greeno says.

Practical tips

In the meantime, hospitalist programs can do the following:

• Seek vendors who want accurate survey results. The Phoenix Group states that hospitals should first be aware of the flawed methodology used by vendors and shop for vendors whose methodology is demonstrably valid.

“If what the hospital wants is to measure patient satisfaction to the group or the individual hospitalist, then they have to find the vendor who can do that. They shouldn’t settle for anything less,” Greeno says.

• Pressure your vendor to change. Remember, you are the vendor’s customer and you call the shots. Call your vendor and insist that it consider the hospital’s specific needs when tailoring new questions.

“[Vendors] understand the risk in utilizing their surveys in this manner, and I would hope that the company takes steps to curb this activity,” Singer says. “I would like to see them demonstrate more leadership in this field and acknowledge that new, more useful survey tools would be helpful.”

• Engage hospitalists in the process. Hospitalists should have a say in the process to improve satisfaction since they are the component of the hospital experience that patients are measuring, adds Singer.
• Advocate for regular committee meetings. You can also seek suggestions about quality from within the hospital, without vendor input. Meet with staff members at multiple levels throughout the hospital’s organizational structure; these types of meetings are not difficult to organize.

“This is the correct venue for these issues, as well as many others, to be vetted, discussed, and plans made and monitored,” Singer says.

These meetings will require participation from more than just hospital executives. Ideally, participants will be those on the front lines of improving satisfaction in the clinical and administrative arms of the hospital.

• E-mail to a Colleague
• Print