The Measure of Work
Beware: Incoherent measurement drives irrational behavior
In the radiology value chain, measurement in itself is a non-value-added activity. That is to say, the measurement of operational performance, quality, or safety does not directly contribute to the output of the radiology workflow, nor is this the goal of the activities of the radiologist with regard to patients. Perhaps most importantly, measurement is also not the patients' goal. The purpose of the radiologist is not to measure their performance, but to provide specialized services to the patient that improve health outcomes.
However, some degree of operational performance measurement is necessary to achieve the goals and to serve patients in a safe, timely, effective, efficient, equitable, patient-centered, and accessible way. We measure in order to improve quality and safety, and to support evidence-based decisions on resource allocation, equipment purchases, and staffing in order to match throughput and capacity to patient demand. For example, measuring the flow of patients through the radiology department is necessary to gauge throughput, efficiency, and capacity, but measurement does not cause throughput, efficiency, or capacity.
In order to know if Radiology is achieving patient goals, we need to measure output. Output measures may include the number of patients seen, the kinds of imaging performed, or the findings of scans. Since output measures can only be taken at the culmination of each patient flow instance, we may therefore wish to institute several process measures. Process measures are unbiased predictors and leading indicators of future output. A good process measure informs us about progress, trend directions and magnitude, and whether at current progress we are likely to meet our care objectives.
Since many success factors influence each other, we may also wish to implement balancing measures. Balancing measures warn us in a timely way if a part of a process is increasing risk, causing issues, or resulting in missed opportunities. These metrics can help prevent adverse effects to the patient, to the staff, or to the radiology workflow. A process measure may allow us to reallocate resources when staff and equipment are idling at one point in the process while patients are waiting at another.
Unfortunately, measurements tend to accrete and bloat over time. We may accumulate metrics that no longer serve one of these three purposes. For example, once a process, equipment, or goal is changed, we frequently do not revise or eliminate the previous measurements and simply layer new measurements atop old ones. We may also have measurements forced down on the process from external bodies, sometimes with good cause, but often to satisfy a curiosity or need that has little to do with "doing good radiology." Finally, there is a strong and persistent temptation to measure simply because we can. Many metrics are created just because the data are available or the tool is at hand, rather than because it would serve a direct purpose in quantifying the achievement of goals, predicting achievement, or warning us of variation or deviation from the path to achievement.
While it may not sound too burdensome to measure a few extra things for reasons other than achieving goals, incoherent measurement tends to drive irrational behavior. For example, Radiology teams that are overly focused on Relative Value Units (RVUs) may see quality drop, errors increase, and burnout manifest in staff.
Even when focus on misaligned metrics is less obvious, there are usually overt or subtle incentives - rewards or punishments tied to achievement or failure to meet metrics - that influence human behavior. If the incentives are not directly aligned to the goals of radiology, the resulting behavior is likely to reduce ability to achieve core goals. At the very least, incoherent measurement can become a source of frustration, confusion, and dissatisfaction. At worst, it can be a direct cause of safety and quality issues, and staff burnout.
By all indications, this is the current state of radiology – some things, such as patient outcomes and experience, are not being adequately measured, but the typical radiology department is drowning in measurements that have little bearing on reality. It is perhaps a good time for prudent housekeeping and to prune out the metrics that do not serve as an output, process, or balancing measurement of the practice of radiology.