by George A. Gellert, MD, MPH, MPA; Linda Catzoela, BSN, RN; Lajja Patel, MD; Kylynn Bruner, BS, MBA; Felix Friedman, MS; Ricardo Ramirez, BSIT, LVN; Lilliana Saucedo, RN, MSN; S. Luke Webster, MD; and John A. Gillean, MD
Background: One strategy to foster adoption of computerized provider order entry (CPOE) by physicians is the monthly distribution of a list identifying the number and use rate percentage of orders entered electronically versus on paper by each physician in the facility. Physicians care about CPOE use rate reports because they support the patient safety and quality improvement objectives of CPOE implementation. Certain physician groups are also motivated because they participate in contracted financial and performance arrangements that include incentive payments or financial penalties for meeting (or failing to meet) a specified CPOE use rate target. Misattribution of order sources can hinder accurate measurement of individual physician CPOE use and can thereby undermine providers’ confidence in their reported performance, as well as their motivation to utilize CPOE. Misattribution of order sources also has significant patient safety, quality, and medicolegal implications.
Objective: This analysis sought to evaluate the magnitude and sources of misattribution among hospitalists with high CPOE use and, if misattribution was found, to formulate strategies to prevent and reduce its recurrence, thereby ensuring the integrity and credibility of individual and facility CPOE use rate reporting.
Methods: A detailed manual order source review and validation of all orders issued by one hospitalist group at a midsize community hospital was conducted for a one-month study period.
Results: We found that a small but not dismissible percentage of orders issued by hospitalists—up to 4.18 percent (95 percent confidence interval, 3.84–4.56 percent) per month—were attributed inaccurately. Sources of misattribution by department or function were as follows: nursing, 42 percent; pharmacy, 38 percent; laboratory, 15 percent; unit clerk, 3 percent; and radiology, 2 percent. Order management and protocol were the most common correct order sources that were incorrectly attributed.
Conclusion: Order source misattribution can negatively affect reported provider CPOE use rates and should be investigated if providers perceive discrepancies between reported rates and their actual performance. Preventive education and communication efforts across departments can help prevent and reduce misattribution.
Keywords: computerized provider order entry (CPOE); CPOE misattribution; CPOE performance metrics; CPOE attribution error
Computerized provider order entry (CPOE) is a technology used by clinicians to directly and digitally enter pharmacy, laboratory, radiology, and other orders into a computer system or mobile device, from which the orders are transmitted electronically to the respective department or service for execution. CPOE supports standardized, evidence-based, and legible orders and, through clinical decision support (CDS), can improve quality and safety by reducing medication and other errors at multiple stages of the order management process; it can also reduce redundant testing. CPOE accelerates the ordering process and delivery of care, improves efficiency, and reduces the number of individuals required to participate in the clinical workflow. It thereby decreases care delays, adverse events, and errors due to miscommunication and handwriting illegibility. The widespread introduction of electronic health records (EHRs), digital physician documentation, and CPOE have demonstrably reduced healthcare-related errors, eliminated problems related to handwriting illegibility, and improved patient safety and clinical care outcomes.1–20
Order source misattribution in CPOE involves the inadvertent misuse of order source selection and attribution by nursing and ancillary departments. Misattribution of order sources can deflate physician/provider CPOE use rates, a key metric in physician EHR performance evaluation. This report describes analyses that were conducted when a group of hospitalists at one medical center reported their perceptions that the individual and group use rate reports provided to monitor CPOE adoption were inaccurate, overstated their use of paper orders, and understated their use of electronic order entry.
With the advent of MACRA and MIPS, the Centers for Medicare and Medicaid Services (CMS) has de-emphasized reporting of CPOE use metrics, and therefore the issue of misattribution may appear to be of diminished importance. However, the fact that CPOE use reporting is no longer central to MACRA/MIPS does not mean that CPOE adoption is no longer an issue for many hospitals across the nation or that the goal of having physicians in the United States issue 90 to 95 percent or more of patient care orders through CPOE with decision support has been reached. Indeed, one in four US physicians still have not adopted an EHR. Furthermore, many hospitals and physicians that have adopted EHRs still have significant opportunity for growth in individual and facilitywide CPOE use rates.
The decreased emphasis on reporting of CPOE use by CMS by no means indicates that the US healthcare system is at desired levels of CPOE use. Ensuring the highest possible accuracy of CPOE attribution and reported use rates should not be lessened by MACRA/MIPS. US healthcare and our patients’ interest in quality and safety should not reduce the clinical care and patient safety objectives of national CPOE adoption to a government agency reporting function. Recent Meaningful Use reporting has identified CPOE acceptable use levels as low as 60 percent, well below that required to achieve the objectives of CPOE.
The nation’s objectives for CPOE adoption remain unfulfilled and undiminished: to standardize and distribute evidence-based healthcare practice via EHR adoption, and to deploy clinical decision support to reduce an epidemic of national mortality due to preventable healthcare errors.21, 22 Thus, while the nation as a whole has progressed substantially over the last five years with respect to CPOE adoption, we are not yet achieving the 90 to 95 percent use rates required to realize these care objectives. In addition, even if CPOE use were ubiquitous, any level of ongoing misattribution of patient care orders has serious potential implications for patient safety and quality, as well as medicolegal implications should that care subsequently be called into question clinically and/or legally.
Additionally, some fraction of provider orders may continue to be issued on paper. For example, paper may be used to record verbal orders issued when the physician is engaged in a procedure or emergency care and unable to utilize a computer, and telephone orders issued when a physician is driving or otherwise similarly unable to enter orders electronically. In our system, we estimate this level of likely long-term paper order use to be, conservatively, 5 percent and possibly as high as 7 to 8 percent depending on physician specialty.
Physicians care about their monthly CPOE use rate reports because they understand and value the patient safety, care effectiveness, and quality improvement objectives of CPOE implementation.23–42 In addition, certain groups of physicians are motivated in part because they may participate in clinical performance/financial partnership arrangements with providers. Within these partnership agreements, our medical center utilizes incentive payments for meeting or exceeding—or financial penalties for failing to meet—specified provider performance objectives, including CPOE use targets for a group or specialty.
Misattribution complicates the accurate measurement of physician CPOE adoption and compliance with CPOE use requirements, and has additional implications where performance targets have been established in physician contracts. Misattribution can also have medicolegal implications because it means that the actual physician of record for a set of orders remains unknown. Further, while most US hospitals and health systems have already adopted CPOE, many facilities have achieved use compliance rates below 90 percent. Therefore, correct order source attribution remains important in ensuring the credibility of reported use rates as facilities strive to exceed 95 percent use rates and to eliminate the use of paper orders.
The hospital in this study is an urban community hospital with 110 beds and a typical daily census of between 80 and 110 patients. The hospital began implementation of CPOE in mid-2012. Meaningful use of CPOE was defined as direct order entry of laboratory, radiology, and pharmacy orders by physicians, nurse practitioners, physician assistants, and residents. A critical component of our strategy to drive physician adoption was the monthly distribution of a CPOE use rate listing each physician in the facility and identifying the percentage of orders that were entered electronically in CPOE by each individual versus those issued on paper. The list also included the potential positive effect on the overall facility CPOE use rate that would occur if each physician fully adopted CPOE and issued nearly all orders electronically.
We measured and reported actual CPOE use rates of individual physicians directly from our enabling software, Horizon Business Insight from McKesson Performance Analytics. All CPOE use rates reported are actual and not inferred. Individual provider CPOE use rates were derived by dividing the total number of orders that each provider issued electronically by the total number of orders that the provider issued electronically and by written, verbal and telephonic means.
In this facility, one hospitalist group admitted and managed a majority of the patients, and this group issued the largest volume of patient care orders. This hospitalist group contracted with the facility for their services. Their contract included both a minimum required group CPOE performance metric (85 percent), and a bonus payment incentive for achieving 90 percent or greater total CPOE use by all group members.
Before the implementation of CPOE, orders were most commonly issued in written form. The implementation of CPOE required the creation of additional order source options in our EHR system (MEDITECH Client Server 5.66). The creation of additional order source options generated opportunities for order source misattribution. Neutral order sources can be used to generate orders that do not contribute to a provider’s CPOE use rates. Figure 1 outlines the best-practice usage of neutral order sources and the correct attribution of orders to the appropriate source.
Figure 2 illustrates how neutral order sources should be utilized when orders are attributed to the provider and source, and presents an example of how misattribution can occur. In this example, a nurse can correctly attribute a protocol order to a provider, or can misattribute the order source in a manner that would negatively affect the reported CPOE use rate for that provider.
A detailed analysis of the magnitude and source of misattribution was precipitated by the hospitalists’ perception that the monthly reports of the use of non-CPOE paper orders was overstating the use of paper orders and understating the use of CPOE. As a result, we needed to evaluate these claims and to quantify any negative impact of order source misattribution on individual hospitalists’ and the group’s CPOE metrics, which were used for performance evaluation and incentive payment. We regarded the imperative to identify and quantify the causes of order source misattribution, and to convey preventive education to prevent recurrence, as integral to the provision of safe, effective clinical care. Therefore, it was essential that the health informatics department ensure the accuracy and credibility of CPOE use rate reporting.
Table 1 shows the CPOE use rate for the hospitalist group by month for the first four months of 2014. The mean group use rate for the period, which was the metric used to determine opportunity for incentive payment, was 89.0 percent, or 1 percent below the 90 percent required for the group to achieve a bonus. These monthly use rates were immediately suspect to the hospitalists, who were convinced as a group that they were performing at 90 percent or higher. This report describes the methods utilized to investigate and quantify sources of order misattribution and presents the findings of the investigation. In addition, we review the strategies and tactics employed to reduce or prevent misattribution at its source, and the effect of those efforts on reported misattribution rates.
Root cause analyses were completed for each member of the inpatient hospitalist group, consisting of 12 physicians, and focused on those having the highest CPOE utilization rates and patient/order volume in the facility. We assessed incorrect order sources over a one-month period, focusing on written orders exclusively because providers are unable to reject or detect written orders that have been incorrectly attributed to them.
Patient care orders issued in the month selected for a detailed examination were processed through McKesson Performance Analytics and a customized MEDITECH Client Server 5.66 Data Warehouse tool. All orders (not a sample of orders) issued by the hospitalist group in the month of analysis were reviewed, and all potentially misattributed orders were identified, reviewed, and classified. Each order was evaluated with respect to its source attribution and was validated or corrected if it was incorrectly attributed. Order validation was achieved by reviewing medical records or examining an electronic audit trail. We had no a priori reason to suspect that this particular month of orders analyzed differed systematically from any other month. The month analyzed was, to our knowledge, typical and representative of any month of CPOE use and use rate reporting.
By using these tools and investigating each order individually for each provider, we were able to determine when each misattribution occurred, as well as the party or source responsible for each misattribution. The hospitalists helped expedite this process by identifying which specific orders they suspected were incorrectly attributed to them. Nonetheless, the process was very labor intensive for the clinical informaticist completing the review.
We estimated the misattribution proportion as the ratio of the number of orders with misattribution to the total number of orders issued. The number of incorrectly attributed orders out of all orders issued followed a binomial distribution. The resulting proportion was an estimate of the population proportion of the misattribution rate. We calculated a 95 percent confidence interval (CI) for the estimated proportion using the Wilson score method.43
Early in our deployment of CPOE we utilized facilitywide individual provider/physician use rate reports to quantify and help facilitate adoption. We discovered that a small but not dismissible percentage of orders (4.18 percent; 95 percent CI, 3.84–4.56 percent) were attributed inaccurately to physicians who had not issued those orders. Initial analysis identified 525 written orders with potential misattribution to the hospitalist group during the one-month period examined (see Table 2). Further analysis determined that 29 of these 525 orders were legitimate written orders issued by the hospitalist group, leaving 496 misattributions over the monthlong evaluation period.
In our CPOE implementation, certain areas of high clinical or pharmacotherapeutic complexity (e.g., total parenteral nutrition, chemotherapy) were left on paper and are not yet electronic. In addition, physicians are permitted to issue verbal orders when they are in the midst of a procedure or the delivery of emergency care, in which the use of a computer for electronic order entry is not possible. These orders are subsequently signed on paper. Telephone orders are permitted when a physician is driving or otherwise occupied in a manner that prohibits direct access to and use of a computer for CPOE. In this month, 348 verbal and/or telephone orders were issued by the hospitalist group (see Table 2).
When we removed the 496 misattributed paper orders, the hospitalists’ monthly CPOE group use rate increased by 4.1 percent, from 92.6 percent with misattributions included to 96.7 percent after misattributions were eliminated (see Table 2). Although a 4.1 percent use rate correction for misattribution may appear modest, for providers performing at a CPOE use level greater than 85 percent—and having incentives to achieve greater than 90 percent—it is a substantial difference that was regarded by the hospitalists as a significant improvement in use rate reporting accuracy. Without this corrective adjustment for misattribution, hospitalists were unable to achieve incentive/bonus payment for what was actually a superior CPOE performance that exceeded the 90 percent use threshold (see Table 1).
We learned that the tendency to incorrectly attribute orders to hospitalists was not random. These providers have the highest patient volume, provide the most care on many units, and issue the highest volume of orders in the facility. When another member of the care team (such as a nurse, unit clerk, or pharmacist) could not determine who wrote a particular set of orders, often because the handwriting was illegible, they tended to attribute the order to one of the hospitalists. This attribution is a logical, if flawed, determination and action. Because the hospitalists are the most common attending physicians in the facility, they are often selected as the default provider for orders when the actual author or source is unclear or illegible.
Multiple departments and actors, including nursing, radiology, laboratory, and other ancillary team members, can be the source of misattribution. The clinical departments that were sources of misattribution, and the magnitude of each department’s contribution to the order source misattribution rates, are identified in Figure 3. The nursing and pharmacy departments were identified as the source of 80 percent of misattributions in this evaluation. This finding may reflect the high volume of orders that each service routinely executes as issued by the providers, and it does not per se indicate a higher rate of error in order source attribution.
The causes of order source misattribution were evaluated. Figure 4 presents misattributions by cause.
Each occurrence of an incorrect order source was investigated individually, and the order source that should have been selected was identified. Figure 5 shows the distribution of order sources that would have occurred if the correct source had been used.
Misattribution of order sources can deflate individual and group CPOE utilization metrics and can result in physician dissatisfaction and distrust of reported CPOE use rates. As noted, order source misattribution can also have medicolegal implications if the wrong provider is identified as delivering a particular component of care that is later called into question. A challenge in monitoring misattribution is that attribution is not typically automated in many CPOE software programs. It may be useful for facilities to conduct misattribution assessments periodically (perhaps once or twice a year) among physicians with a high frequency of CPOE use, or when physicians assert that their reported use rates are not accurate.
Because misattribution often involves assignment of paper orders to the wrong physician, it deflates the CPOE use rate of the physician to whom an order is inaccurately attributed. Correction of misattribution ensures that reported use rates are accurate and thus credible to physicians as they aspire to the highest possible CPOE use. Failure to correct attribution errors can undermine successful CPOE adoption if misattribution is high, and can reinforce the prejudices against CPOE of the most resistant physicians.
This evaluation of the impact of order source misattribution on a critical group of hospitalists illustrates the punitive effects that misattribution can have by inappropriately disqualifying providers from performance rewards for achieving targets that they had actually attained. We regarded this rate of misattribution among a group of critically important, high-volume providers (the main hospitalist group of the facility) as meaningful. After the misattributions were corrected, we recalculated the hospitalists’ CPOE utilization rates. As a result of this quantification of misattribution, facility leadership determined that a 4 percent adjustment in reported individual and group performance metrics would be allowed henceforth. Moving forward, the group could achieve payment incentives for strong CPOE performance and not be penalized as a result of order source misattribution. Leadership regarded this adjustment as important in order to maintain an effective relationship with this critical physician group.
When suspected misattribution was first reported by physicians, we instituted a policy requiring physicians to print their full name below their signature so that the illegibility of physicians’ signatures would not cause misattribution. We subsequently learned that physicians who issued paper orders and had not adopted CPOE frequently dropped the paper orders off at the nursing station. Often, a nurse would not be present to ensure that the physicians had printed their names clearly on their orders in addition to signing the orders. We engaged in a communication effort to advise physicians against this practice and to ensure that they conveyed their paper orders directly to a nurse. However, this effort was unproductive.
Several strategies were subsequently identified to help mitigate the effects of order source misattribution and to prevent dilution of physicians’ CPOE use rates. Recurrent multidisciplinary education is needed to create and maintain awareness of the need to avoid misattribution among each department that contributes to order source misattribution, including the nursing, pharmacy, and laboratory departments. In departments that have significant staff turnover, such as nursing, it is important to educate new employees about order source misattribution as part of their orientation, and to periodically reinforce the message to all personnel in departments where misattribution was documented and remains a risk. This education can be done by direct communication between the facility’s clinical informaticist and the respective department heads. We also added order source education to our MEDITECH nursing education portal.
The effort to prevent recurrence of order source misattribution through multidepartment education was effective and valuable. Table 3 identifies the interventions employed to prevent or reduce order source misattribution. Job aids were used to educate and message about the prevention of order source misattribution in the facility (see Figure 6). In some departments, team members who require education about past order source misattribution and its future prevention may react initially with defensiveness. Communicating in a collegial, nonpunitive manner helps to foster an open and productive dialogue about order source misattribution between health informatics professionals and the departments that are the source of misattribution.
The ultimate solution to the problem of misattribution is the complete elimination of paper orders, a goal many hospitals are still aspiring to achieve. Misattribution would be greatly reduced if the overall facility had a CPOE use rate of 95 percent or higher. In hospitals that are able to successfully mandate CPOE use to achieve this rate, misattribution is likely to be infrequent. However, our hospital system does not employ physicians; rather, physicians align with our hospitals at will, and thus far physicians have not lost admitting privileges as a result of having a CPOE use rate below 95 percent. Although system leadership communicates that the expected standard of care in our facilities is the exclusive use of CPOE, most of our hospitals have facility use rates between approximately 75 percent and 90 percent.
We suspect that many hospitals across the nation operate within a similar situation, and thus misattribution remains an issue and can undermine efforts to drive individual and facility CPOE use rates to 90 percent or higher. We have demonstrated positive clinical outcomes associated with high CPOE use to our physicians and have educated them about the risk to patients when paper and electronic orders coexist. However, in large metropolitan and highly competitive markets, many physicians—particularly consultants—often work in multiple systems with different EHRs. Among consultants, only a minority seem willing to engage in learning and committing to memory the CPOE navigation processes for three or four different EHRs.
Although misattribution tends to occur with paper orders as a result of physician signature illegibility, misattribution can also occur electronically within CPOE when an incorrect order source is utilized by members of the multidisciplinary care team. The best method for mitigation of electronic misattribution in CPOE may lie in the creation of a unique physician identifier in our new enterprise data warehouse, to which we will migrate CPOE use rate tracking and reporting.
This evaluation demonstrates that CPOE order source misattribution may occur in a typical community hospital setting, particularly among high-volume physicians with high CPOE use, such as hospitalists. Identifying and addressing misattribution is important from a patient safety and quality standpoint and when physician performance metrics and payment incentives are focused on achieving high CPOE use rates. Although a 4 percent misattribution rate among hospitalists may appear quantitatively minor, for providers performing at the 75 to 85 percent level of CPOE use and striving to achieve 90 to 95 percent, retroactive corrective adjustment was viewed as a significant improvement in the validity of reported rates. For this group of hospitalists, the elevation in performance enabled them to receive incentive payment. Had misattribution continued to obscure their actual use rates, their failure to achieve incentive payment despite strong CPOE adoption efforts could have undermined the physicians’ morale and damaged their relationship with facility leadership.
Misattribution complicates the measurement of physicians’ CPOE performance. This kind of evaluative process, any resulting correction of use rates, and preventive education across departments in which order source misattribution originates should be undertaken periodically, especially if physicians report large differences between their perceived CPOE use and the rate reported from the EHR. It must be supported by the facility’s administrative and clinical leadership, with assistance provided by the health informatics department. The political support of facility leadership helps ensure engagement by the departments and service lines where order source misattribution originates. Identifying sources of misattribution involves a labor-intensive manual review by a clinical informaticist, an activity that temporarily diverts this individual’s focus from supporting physicians at the elbow.
Facility (or hospital system) administrative and clinical leadership should regard this ongoing effort to identify and reduce misattribution as important to maintaining the credibility of CPOE use rate reporting. The ability to identify the source of specific care orders is equally valuable for patient safety and for medicolegal documentation. Correction of order source misattribution fosters providers’ trust in facility leadership and health informatics staff. Failure to correct misattributions can potentially undermine efforts to advance CPOE adoption toward the 95 percent use level if a physician community, or an important specialty group, believes that their reported use rates do not accurately reflect their actual CPOE use. In order to confirm the validity of CPOE use metrics and to ensure physician confidence in the reporting of CPOE use rates, suspected order source misattribution should not be ignored but investigated, and if misattribution is found, it should be addressed systematically through education at the source.
George A. Gellert, MD, MPH, MPA, is an associate chief medical information officer in the Department of Health Informatics at CHRISTUS Health in San Antonio, TX.
Linda Catzoela, BSN, RN, is a clinical informaticist at CHRISTUS Health in San Antonio, TX.
Lajja Patel, MD, is a consulting hospitalist at MedCede Physician Services in San Antonio, TX.
Kylynn Bruner, BS, MBA, is a clinical informaticist at CHRISTUS Health in San Antonio, TX.
Felix Friedman, MS, is a biostatistics consultant in Eden Prairie, MN.
Ricardo Ramirez, BSIT, LVN, is a clinical informaticist at CHRISTUS Health in San Antonio, TX.
Lilliana Saucedo, RN, MSN, is the director of health informatics at CHRISTUS Health in San Antonio, TX.
Luke Webster, MD, is the chief medical officer, Jvion Inc., in Atlanta GA.
John A. Gillean, MD, is the executive vice president and chief clinical officer at CHRISTUS Health in Irving, TX.