Abstract
The success or failure of a clinical documentation integrity (CDI) program is often evaluated using a designated set of metrics. However, these metrics change over time, and an understanding of these changes is critical to properly judge the efficacy of the CDI effort. The authors propose a model of the natural history of a CDI program based on commonly used CDI metrics. The authors believe that this model can assist CDI leaders in anticipating and understanding the course of CDI performance over time.
The Natural History of CDI Programs: Metrics and Models
Clinical documentation integrity (CDI) programs assist clinicians in proving complete documentation of patient status within the medical record. Complete documentation enhances the accuracy of measures of illness severity, the patient’s needs for care, and the provider’s effort on their behalf. The efficacy of a CDI program is often judged by reviewing a designated set of metrics. Common metrics used in the evaluation process of traditional inpatient-based CDI efforts include Case Mix Index (CMI), query rate, and provider query response and agreement rates. The growth of ambulatory CDI programs into the ambulatory care space brings newer measures such as Hierarchical Categorical Condition (HCC) content and Risk Adjustment Factor (RAF) scores into the mix.
While CDI metrics are often assessed in isolation, in practice they are intimately linked and often follow a predictable course as a CDI program grows from erratic infancy to high-performing maturity. However, healthcare executives not attuned to the intricacies of CDI work may have an incomplete understanding of these metrics. A key role for CDI leadership is to anticipate these perceptions and “set the table” to for appropriate administrative expectations, assuring that CDI program performance is assessed on a rational basis.
The authors of this paper propose a metric-based model of the natural history of a CDI program illustrating the expected course over time of specific inpatient and ambulatory care CDI metrics. The authors believe this model can assist CDI leaders to assess the maturity of their identify their CDI program, and to provide healthcare executives with reasonable expectations for assessing the impact of CDI work.
The Metrics Model
The authors propose a metric-based model of the natural history of a CDI program. While many of the general concepts for inpatient and ambulatory CDI metrics are held in common, the authors will use separate illustrations for each setting in order to avoid the confusion of multiple overlapping trendlines within a single graphic. It should also be understood that these models are built on qualitative measures, and not on absolute ratios, percentages, volumes, or values.
The authors also acknowledge two additional considerations underlying the model. First, the model presumes that the patient, provider, and CDI staff populations driving these metrics are static. In reality, factors such as changes in service line offerings and provider turnover may undoubtedly impact individual metrics, especially in the quantitative realm. But as qualitative projection, the authors hold that the model remains valid.
Finally, the authors recognize that specific elements of the model (e.g., CMI) are based on the United States Center for Medicare and Medicaid Services (CMS) Medical Severity – Diagnosis Related Group (MS-DRG) and HCC/RAF systems. However, the authors believe that the model is applicable to any non-fee-for-service payer scheme. Other elements such as query rates and provider response/agreements rates are payer-agnostic.
Inpatient CDI Metric Model
The model begins with “Perfect CMI” (Figure 1). CMI is the calculated average of the Relative Weights (a numerical measure of illness severity) associated with the DRG assignments of each patient within a specified clinical population. The “Perfect CMI” is the value achievable with full and complete provider documentation of every clinically recognized Principal and Secondary Diagnosis within each inpatient record. It is a flat line, as the DRG system itself has an inherent ceiling; no matter how many Secondary Diagnoses appear on the record, only one Complication/Comorbidity (CC) or Major Complication/Comorbidity (MCC) is needed to maximize the DRG assignment and the associated Relative Weight.
Two other lines within Figure 1 correspond with the CMI. The first is the “CMI Before Queries.” (Written or verbal queries are the primary means by which CDI staff ask providers to document for clarity and specificity within the medical record). This metric serves as a measure of provider education within the CDI effort. At the start of CDI work, there will be a significant gap between the “Perfect CMI” and the “CMI before queries.” Over time, as physician educational efforts result in enhanced documentation habits, this gap should close. Persistent failure to narrow the gap should prompt early reassessment of the provider-focused CDI educational program.
By way of contrast, the “CMI After Queries” is a measure of CDI program performance describing the end product of CDI-generated provider queries, provider response rates, and provider agreement rates. This metric will begin life closer to the “Perfect CMI” than the “CMI Before Queries” line, as the “ask-and-answer” queries often have a more immediate impact upon CMI than the longer-term educational work. This metric should also gradually trend upwards with the growth of overall CDI efforts. Initially, a wide gap is expected between the CMI pre-and-post query metrics; over time, as targeted clinician educational efforts take root, the gap between the pre-and-post query CMI should narrow. The authors propose that tracking this progression, using CMI values in association with CMS Base Rates to estimate additional reimbursement, is key to giving a tangible value to otherwise intangible CDI processes.
The authors also introduce in Figure 1 the “Executive Expectation Line,” or the EEL. Healthcare executives unfamiliar with the built-in “ceilings” within the DRG-based systems may not understand that, even in a perfect world, CMI is limited given a static patient population. The ceiling may rise from time to time with changes in services offered by the facility (for example, the addition of cardiovascular surgical care), but even then the concept of the “ceiling” still remains. Over time, as the actual CMI approaches the “Perfect CMI” metric, rises in CMI will continually decrease in magnitude and eventually plateau. Administrators with only a superficial understanding of DRG-based systems may view CMI as an infinite pathway, especially following dramatic improvements in this metric, as seen in the youth of a CDI effort. It is important for CDI leadership to assist other executives in limiting unreasonable expectations.
“Query Rate” is another key metric within our model (Figure 2). At the onset of a CDI program, one anticipates a great many opportunities for query. However, as the program matures (and especially with effective clinician education), the query rate should fall. Where the query rate does not fall, or even rises, it should prompt the CDI leader to evaluate the efficacy of the query process.
It is also of import to note that not all queries produce the same result. Queries that simply add clinical specificity or address quality measures may result in no changes to CMI or other “hard” measures. Accordingly, the authors introduce the metric of “CMI Impact Query Rate,” which serves as a more specific means to trend queries by type. Even if overall query rate rises over time due to the expansion of CDI work into new areas of opportunity, the evolution of the CDI program and its accompanying educational efforts should decrease the proportion of queries focused strictly on reimbursement.
Ambulatory CDI Metric Model
The ambulatory CDI metric model is similar to that on the inpatient side, although the RAF score takes the place of the CMI as the index measurement. In value-based healthcare purchasing plans (such as Medicare Advantage), providers are reimbursed based on the risk profile of their patient population. The risk profile for any given patient is a mathematical amalgamation of the patient’s age, sex, and current health conditions; the calculation is known as a RAF score. The health conditions are documented as HCCs, which are clinical issues considered to contribute to the patient’s current health status. These conditions are established through clinician documentation within the medical record. If the RAF score may be considered analogous to the inpatient CMI, HCCs can be thought of as the clinical diagnoses within the inpatient record that drive DRG assignment.
In this aspect of the model, the authors once again see an ideal in the “Perfect RAF” (Figure 3). This is the RAF score if all HCCs were completely and correctly documented in all patients served by an ambulatory care provider, practice, or network. The authors also find trend lines similar to the inpatient model for “RAF Before Queries” and “RAF after Queries” in Figure 3.
The ambulatory model also features an Executive Expectation Line (EEL, Figure 3). While the “Perfect RAF” ceiling is undoubtedly more flexible within the RAF/HCC scheme than in the MS-DRG system (the more medical issues documented per patient within the population, the higher the RAF score can go), this flexibility may lead to the conclusion that RAF scores may infinitely rise. However, there is a natural limit to how many medical conditions any one individual might have, and as more patients have complete documentation of their HCCs, HCC opportunities will approach organic limits and begin to taper off. The effect is likely not as pronounced as one might see in CMI-based metrics, but it is nonetheless incumbent upon CDI leaders to prepare upper-level executives for this transition.
Figure 4 illustrates the anticipated trends over time for “Query Rate” and “Queries with RAF Impact.” These metrics have similar implications for provider educational efforts and overall ambulatory CDI program performance as their inpatient counterparts. However, in the ambulatory space, provider education and query efforts are focused on the documentation of HCCs rather than inpatient diagnoses. Depending upon the scope of the ambulatory care project, queries may be directed toward quality measures and care gap closures; however, the authors believe that this form of query is quantitatively much less common in the ambulatory space.
Provider CDI Metric Model
Finally, the authors would propose a final metric-based model for tracking the effectiveness of the CDI program with clinicians (Figure 5). The authors would establish a flat line “Perfect Response/Agreement Rate,” where every clinician always responded to the CDI query and always agreed with the optimal CDI suggestion. The “Provider Response Rate” and “Provider Agreement Rate” would be anticipated to start low and rise over time. The authors would not expect the “Provider Response Rate” to reach the level of perfection, as even within the most reliable medical staff there are vacations and turnover; similarly, the authors would not expect the “Provider Agreement Rate” to reach perfection, as clinical care is rife with judgment calls that result in unanticipated query answers or responses indicating clinical uncertainly. However, as clinician acceptance and adherence to a CDI program matures, trendlines for both Provider Response and Agreement Rates should rise, and the gap between response and agreement rates should narrow. The authors believe this model of provider-focused metrics is equally applicable to inpatient and ambulatory settings.
(Please note, the authors use the term “provider” in this document simply to reflect that, in some jurisdictions, nurse practitioners or physician assistants may practice independently and assume primary responsibility for patient-focused clinical documentation, including CDI query response. In practice, the authors feel strongly that physicians, by virtue of experience, training, and expertise, should be distinguished from other healthcare providers.)
CDI Metrics and Business Cycle
In “The Age of Paradox,” Charles Handy proposes that an S-shaped “Sigmoid Curve” underlies every type of human activity or system (Figure 6). The curve encompasses experimentation and learning, growth and development, and an inevitable downward turn. In the context of a business, the Sigmoid Curve graphically represents the life cycle of an enterprise. The business or project starts with enthusiasm but quickly learns what it doesn’t know; this is followed by a period of exponential expansion as the effort eventually finds its way. However, if things do not change and innovation stops, eventually the work plateaus and then declines. Ideally, leaders aware of this cycle will intervene while things are going well, before any peak or decline, developing new products or services to begin another cycle of learning and growth. (This period of “re-orientation” is illustrated by the segmented line in Figure 6.)
The authors see a distinct parallel between Handy’s model of the business cycle and the natural history of a CDI program. In the early stages of a CDI program, as the initial enthusiasm of the startup is challenged by the recognition of the potential scope of work, the depth of the issues encountered, and the breadth of knowledge required to meet these challenges, productivity transiently falls. Once a focused work plan is developed and implemented, growth and productivity occur at an accelerated rate. However, over time, the current effort will have yielded its maximum benefit, and further work at this high level does not add to a positive outcome, but leads to staff exhaustion and burnout as intensive work no longer shows the same results.
The authors believe that the use of metrics as a model for the natural history of a CDI program can assist CDI leaders in anticipating the apex of the sigmoid curve, and in doing so preventing likely downturns in program efficacy. As CDI metrics within the model become static, or gaps between measures become fixed, CDI leaders can use these trends as indicators of the need to establish a new origin point along the sigmoid curve and reorient the CDI program into new areas of work such as documentation for patient safety indicators (PSI), hospital readmission rates, or inpatient quality measures. Using the metric model to identify times of transition insures the constant upward momentum of the CDI program and helps the CDI team manifest a growing footprint within the organization.
Utility of Other Metrics in the Model
A model is only as good as it reflects reality, and only as it can account for the multiple influences upon the metrics used to assess its validity. As the authors look at other metrics that may be applied (in whole or in part) to CDI efforts, the authors find them more problematic for inclusion in this model. As the specific metric is less dependent upon clinical documentation alone and reflects variances in provider care patterns, patient needs, or community resources, the number of variables contributing to the metric grows in an exponential fashion.
An easy way to think about this difference is to consider inpatient provider query rate versus inpatient length of stay. The variables underlying query rate are fairly straightforward, focused on physician documentation habits and the efforts of CDI staff. While there are clearly variances in both of the practices (some clinicians document better than others, and individual CDI staff may have different skill sets), both are amenable to CDI efforts.
If you look at length of stay (LOS), however, it’s a different scenario. It’s true that excellence in clinical documentation can promote optimal DRG classification; each DRG is associated with a specific geometric mean length of stay (GMLOS). But CDI efforts do not actually impact real-time length of stay, merely the anticipated length of stay associated with the patient’s documentation-based DRG assignment. Actual inpatient length of stay is dependent upon the care provided, socioeconomic status of the patient, placement needs, and a host of other factors outside the control of CDI work. It seems unreasonable to use LOS as a metric specific to CDI.
The same is true (to varying degrees) for measures such as readmission rates, observed to expected mortality ratios, patient safety indicators (PSI), hospital-acquired conditions (HAC), and other quality measures. While CDI efforts may reinforce that the documentation within the medical record properly includes or excludes patients for categorization in these measures, or insure the chart best reflects the patient’s severity of illness, CDI work does not impact any of the multitude of other facility, provider, or patient-centered factors which contribute to these indices. Accordingly, the authors find it difficult to chart these parameters in any simple, concise, and consistent fashion within the model. CDI programs that use these metrics as part of their reporting process must be careful to develop a mechanism that distinguishes the influence of the CDI effort from the remainder of other clinical factors that impact upon the metric, and must set reasonable expectations for the magnitude of change in these parameters attributable to CDI efforts alone.
Conclusion
The authors believe their metric-based model provides a theoretical basis for CDI leaders to evaluate the evolution and efficacy of a CDI program over time. When combined with the concept of the business cycle sigmoid curve, the authors hope this model may serve as a roadmap to the continued expansion, relevance, and impact of CDI efforts within healthcare institutions and systems.
Acknowledgements
The authors gratefully thank Janine Landowski for creating the graphics for this work.
Author Biographies
Howard Rodenberg is a physician advisor for clinical documentation integrity at Baptist Health in Jacksonville Florida.
James D. Campbell is a physician advisor for utilization management and clinical documentation integrity at Wolfson Children’s Hospital, Baptist Medical Center, in Jacksonville Florida.