Although VA has “implemented clinical productivity metrics as well as statistical models to track clinical efficiency,” the Government Accountability Office (GAO) is questioning its accuracy.
Not having accurate and complete data is detrimental to the validity of VA reports on effectiveness of facility resource utilization. GAO has made several recommendations to VHA to rectify discrepancies.
Excerpt: “Specifically, GAO identified four limitations with the agency’s metrics and models that limit its ability to assess whether resources are being used effectively:
• Productivity metrics are not complete because they do not account for all providers or clinical services. In particular, the metrics do not capture all types of providers who deliver care at VAMCs, including contract physicians and advanced practice providers, such as nurse practitioners, serving as sole-providers. Further, the metrics do not capture providers’ workload evaluating and managing hospitalized patients.
• These metrics may not accurately reflect the intensity—the amount of effort needed to perform—clinical procedures or services. Consequently, the VA’s productivity metrics may not accurately reflect provider productivity, as differences between providers may represent coding inaccuracies rather than true productivity differences.
• The metrics may not accurately reflect providers’ clinical staffing levels. According to the GAO, 5 of 6 selected VAMCs auditors visited reported that providers do not always accurately record the amount of time they spend performing clinical duties, as distinct from other duties.
• Efficiency models may also be adversely affected by inaccurate workload and staffing data. To the extent that the intensity and amount of providers’ clinical workload are inaccurately recorded, some of VA’s efficiency models examining VAMC utilization and expenditures may also be inaccurate.”
“In a written response to the GAO report, the agency concurred or concurred in principle with the GAO’s recommendations.
“’VHA has developed a detailed action plan in response to all four GAO recommendations; the action plan is included in the published GAO report,” said an agency spokesperson in a written statement. “The issues identified in the report are complicated and will involve changing many of our existing business practices. VHA has initiated work on the recommendations and is committed to seeing the recommendations through to completion.’”
Source: Health Data Management
WBB Take: VA is not alone in being less than perfect regarding having inaccurate or unreliable clinical productivity metrics and statistical models. In the healthcare industry, data collection and creation of metrics are often more a function of convenience than necessity. Organizations often collect data on the basis of ease and low cost of collection, rather than whether they are required for a specific metric. Likewise, organizations frequently develop metrics because of availability of data, rather than to monitor and evaluate key points in a value chain. To run a healthcare system safely, efficiently, and effectively, a value chain should be developed to articulate the processes that will be individually necessary and collectively sufficient to achieve organizational business goals. Outcomes, process, and balancing metrics should be developed in order to monitor and evaluate performance of key parts of the organizational value chain, and data should be collected and curated that serves those metrics.
Cited by Shannen Irwin