A study reviewed the Hospital Safety Grade system, and found that performance measures had little correlation with independent measurements of hospital-acquired infections.
The Hospital Safety Grade system from the Leapfrog Group is based on hospital self-reports of safety protocol adoption, but not whether the hospital adheres to protocols.
Excerpt: “These days, several organizations give hospitals report cards, combining measurements of how well hospitals serve their patients and keep them safe. Some use only objective patient outcomes, such as infection rates, while others allow hospitals to report on their efforts to improve safety. But a new study finds that a well-known hospital grading system may put too much weight on the wrong things.
“The study, published in the journal Medical Care by a University of Michigan team, takes an in-depth look at the Hospital Safety Grade system, run by The Leapfrog Group, a nonprofit organization that focuses on patient safety.
“The researchers find that most hospitals that participate in Leapfrog’s hopsital survey self-report that they’re in perfect compliance with most, if not all, of the Safe Practices in the survey. But researchers show that how a hospital did on these measures had little in common with independent measurements of hospital-acquired infections, or with whether the government penalized the hospital for high infection or readmission rates.
“’The Safe Practices part of the Hospital Safety Grade is based on whether a hospital self-reports that it has adopted certain protocols, and not whether those protocols are actually followed,’ says Jennifer Meddings, M.D.,M.Sc the study’s senior author and an assistant professor of internal medicine at the U-M Medical School.
“For example, she says, one Safe Practices measure involves hand hygiene. ‘It may be news to consumers that this measure involves hospital self-reports of having specific hand-washing protocols in place, not how often health care workers are washing their hands.’
“She also notes that any one hospital’s Safe Practices score counts toward its overall Hospital Safety Grade based on a comparison with the average from all participating hospitals. So hospitals receive little grade advantage by reporting perfect scores — but are at risk of much lower grades if they report less-than-perfect scores.
“The researchers note that Leapfrog will issue a new grading formula next month, but not in time to calculate the new grades that will come out soon after.”
Source: Michigan Health Lab
WBB Take: In many healthcare situations, outcomes measures are impracticable due to data availability issues or cost of collection. In many cases, process measures are used as a proxy for outcomes, and perform adequately as unbiased predictors of achievement. However, once ingrained, the habit of using process measures can become less a matter of practicality, and more a matter of convenience. In this way, quality measures may become ever more divorced from the outcome they were meant to predict, and may become self-sealing fictions that obscure any deviation and variance from outcomes goals. When using process measures as a proxy for outcomes, periodic outcomes sampling must be carried out to detect any loss of fidelity, and intervention should be triggered when selected measures are no longer performing as accurate and unbiased predictors.
Cited by Matthew Loxton