Researchers from George Washington University have uncovered significant measurement issues in patient experience data collected from U.S. emergency departments. These errors include high variability and limited construct validity.
The reason why the new research is important is because patient experience data, in a digital format, is incorporated into the U.S. Centers for Medicare and Medicaid Services public reporting. Not only does this inform those tasked with government spending, such data is used to construct value-based purchasing models for in-patient hospital care. Moreover, the data is employed in the implementation of the Medicare Access and CHIP Reauthorization Act, driving insurance policies.
There are other uses for the data too; the responses are used to assess physician performance and to establish hospital rankings. These not only signal to patients about the competency of the medical facility, they influence hospital managerial decisions like compensation and employment.
The research was led by Professor Jesse Pines, who examined commercially-generated patient experience data from 2012-15 collected from a large sample of U.S. emergency departments. The analysis revealed the data varied considerably and, in doing so, did not provide a meaningful story. One factor with this was the very low response rate (as low as 3 percent and no higher than 16 percent) and the tendency for only people inclined to score low or high to complete a survey. The researchers call on new measures of analysis to be used in order to generate more reliable data.
The new research has been published in the journal Annals of Emergency Medicine, contained within the research paper “Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores.”
