In 1963 Bob Dylan sang, “the times they are a-changin.” This can be the anthem for health information management professionals for this decade and beyond. Perspectives in Health Information Management has selected a variety of articles for this issue which exemplify the current tsunami of health information change washing over the healthcare industry.

One article describes the challenges as organizations implement electronic health records (EHRs) for Meaningful Use (MU) and the impact upon patient expectations, another the concerns encountered when preparing the industry to effectively adopt ICD-10-CM/PCS, while a third discusses the use of new methods such as telemedicine to deliver high-quality care. Many of the initiatives using health information technology and standards have an impact across the world, attested to by the study of physicians and classification systems from Greece. The use of EHRs to train students is reported on, as is the validation of new competencies for AHIMA’s new clinical documentation improvement credential. The common thread binding all of these articles together is that they are relevant to effective, up-to-date health information management practice and education.

This can be a daunting time to be a health information management professional.  The technologies (EHRs, smartphones, tablets, and telemedicine) to handle the information continue to evolve at lightning speed. The standards (classification systems, messaging, and credentials) for the information content must be developed to ensure the quality of the information as well as convey semantic interoperability and meaning adequate for reimbursement and policy decisions. Education, both the content and the modes, are changing. Educators must grow their own skills and knowledge to train a workforce capable of helping the entire healthcare industry effectively assimilate the massive changes occurring and pending.

Lifelong learning is now essential and HIM professionals must have a plan and methods for staying current. It cannot be left to chance. An excellent place to begin is with the evidence and data included in this and other issues of Perspectives in Health Information Management.

Susan H. Fenton, PhD, RHIA, FAHIMA, is an assistant professor of Health Information Management at Texas State University in San Marcos, TX.

Abstract

Tele-ICU is the use of an off-site command center in which a critical care team (intensivists and critical care nurses) is connected with patients in distant ICUs to exchange health information through real-time audio, visual, and electronic means. The aim of this study is to review the available literature related to the efficacy and cost-effectiveness of tele-ICU applications and to study the possible barriers to broader adoption. While the available studies draw conclusions on cost based on mortality and length of stay, actual costs were not reported. Another problem with the studies is the lack of consistent measurement, reporting, and adjustment for patient severity. From the data available, tele-ICU seems to be a promising path, especially in the United States, where there is a limited number of board-certified intensivists.

Keywords: cost-effectiveness, critical care, telehealth

Introduction

There is a shortage of intensivists in the United States, and the demand for them is only going to increase with the aging population.¹ As of 2010, less than 15 percent of intensive care units (ICUs) are able to provide intensivist care.² There are 6,000 ICUs but only 5,500 board-certified intensivists.³ Studies have shown that hospitals with a dedicated intensivist on staff had a significant reduction in ICU mortality and average length of stay (LOS).^{4, 5} The complexity of today’s ICU services entails the need for sharing health information through off-site ICU centers.⁶ Tele-ICU is the use of health information exchanged from a hospital critical care unit to another site via electronic communications.⁷ Tele-ICU intensivists provide real-time services to multiple care centers regardless of their locations. Tele-ICU uses an off-site command center in which a critical care team (intensivists and critical care nurses) is connected with patients in distant ICUs through real-time audio, visual, and electronic means. Similar to a bedside team, offsite tele-ICU intensivists require full access to patient data. Tele-ICU is capable of providing real-time monitoring of patient instability or any abnormality in laboratory results, ordering diagnostic tests, making diagnoses and ordering treatment, and implementing interventions through the control of life-support devices. As a result, tele-ICU holds great promise in improving the quality of critical care given to patients and increasing the productivity of intensivists. This article explores the available studies related to efficacy and cost-effectiveness of tele-ICU applications and outlines possible barriers to broader adoption.

Methods

Electronic databases were searched to identify relevant articles. Searches were limited to the English language and the most recent publication date of March 2012 for each database. PubMed/MEDLINE, Embase, CINAHL (Cumulative Index to Nursing and Allied Health) with full text, PsychINFO, Evidence-Based Medicine Reviews (e.g., Cochrane Database of Systematic Reviews, ACP Journal Club, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Health Technology Assessment, and NHS Economic Evaluation Database), Scopus, Education Resources Information Center (ERIC), and Turning Research into Practice (TRIP) were used to conduct the literature searches. Searches used subject headings and subheadings if available and were combined with keywords. Search terms used included telehealth, benefits of tele-ICU, tele-ICU outcomes, telemedicine in the ICU, and tele-ICU cost.

Selection Criteria

The article was included if did any one of the following:

1. Pertained to uses of telemedicine in the ICU;
2. Assessed the outcome of implementing tele-ICU by measuring its effect on mortality rate and LOS;
3. Explored staff attitudes toward implemented tele-ICU systems.

Articles not relevant to the topic were excluded. Potential eligibility of the articles was first determined from the title and abstracts identified from the searches. Full-text articles were then retrieved and evaluated for relevance. Articles were excluded if they were not found to meet the above criteria once the full text was examined (see Figure 1 for a flowchart of article retrieval). A second researcher confirmed the relevance and findings from the selected articles. 

Data Extraction and Outcome Measures

The articles were reviewed, and a data extraction form was used to record details pertaining to the study quality such as study design, number of subjects, and study population, as well as a description of the program. The following types of outcomes that were of interest for this review were recorded:

1. Clinical process: outcomes related to service delivery, such as attendance and adherence to programs and recommendations, as well as healthcare provider and staff satisfaction with the program;
2. Healthcare utilization: events that occur outside the program’s scope and that the program may aim to reduce or increase, such as hospitalizations, ICU admissions, and average LOS; and/or
3. Costs: from the patient’s, provider’s, or organization’s perspective, all costs (savings and/or expenses) associated with the use of tele-ICU.

Result

As Figure 1 indicates, 25 studies were retained after the initial screening of titles and abstracts and the full-text retrieval of pertinent articles. The clinical process outcomes, healthcare utilization, and costs reported in the studies are presented in the following sections.

Clinical Adoption of Tele-ICU

The concept of tele-ICU has evolved over time; the approach used in the 1970s and later involved a video connection between the bedside care providers and outside consultants without any access to patient monitoring data. The most frequent adopted approach today is continuous access and monitoring care that focuses on providing supplemental critical care expertise.^8­–13 In 2000, Sentara Healthcare was the first hospital to implement the new tele-ICU approach. As of 2011, 41 ICU command centers had been installed, with a total of 5,789 ICU beds covered throughout 249 hospitals.¹⁴ Even with the early positive impacts of tele-ICU, only 5 to 7 percent of adult ICU beds are covered by this technology in the United States.¹⁵ Adoption of tele-ICU is greatly obstructed by the lack of documented outcomes and unproven return on investment (ROI).¹⁶ Moreover, some tele-ICU centers have been deactivated for reasons such as physicians’ resistance to change in both patient management and the requirement of sharing control over patient care with other, off-site physicians. Technical difficulties and lack of training could also be other impediments.¹⁷

Barriers to Tele-ICU

Tele-ICU is relatively new; many bedside doctors and nurses do not understand how the system works. They believe that the nurses and intensivists at the tele-ICU command center are watching them and trying to take over.¹⁸ In reality, “the purpose of the system is to provide improved safety through redundancy and enhance outcomes through standardization.”¹⁹ The tele-ICU team has a supportive role; they have an overview of all the patients in the unit and can alert the bedside staff if any problems occur.²⁰ One study noted that “the hospital admitting physician continued to be the attending of record and was responsible for establishing the care plan,” while the tele-ICU staff were the primary contact for the on-site nurses.²¹ Studies show that the more proactive the tele-ICU physicians are, “the more improved are the outcomes.”²²

Another barrier to ICU telemedicine is the clinician’s acceptance of the technology. This could be one reason why some studies did not show improvement in LOS and mortality in tele-ICU patients. In a study done by Thomas et al., “two-thirds of the patients in our study had physicians who chose minimal delegation to the tele-ICU.”²³ Other clinicians feel that everything is running perfectly and nothing needs to be fixed. Showing these physicians comparative data and the benefits of tele-ICU may change their mind.²⁴

The lack of integration was a problem at some hospitals, especially those that did not have electronic records. Thomas et al. observed that although the tele-ICU team had real-time access to most of the patient’s information, the monitored unit did not share clinical notes or computerized provider order entry; instead, these notes were faxed daily.²⁵ Berenson et al. also noted the limitations related to the lack of interoperability.²⁶

Outcome Assessments

The acute nature of ICU patients’ healthcare needs and the high cost associated with critically ill patients makes survival rates and cost savings among the most desirable outcomes measured. Consequently, integration of distance monitoring and intensivists’ services into bedside care were significantly associated with a decrease in the mortality rate and LOS in hospitals that were early adopters of tele-ICU. By optimizing telemedicine applications in the ICU, both the mortality rate and LOS could be influenced positively. A review of available published articles is presented in Table 1.

The results from the articles were mixed regarding the mortality rate and LOS in ICUs after the adoption of tele-ICU. For example, according to Thomas et al., “remote monitoring of ICU patients was not associated with an overall improvement in mortality or LOS.”²⁷ On the other hand, Lilly et al. found that “tele-ICU intervention was associated with reduced adjusted odds of mortality and reduced hospital length of stay.”²⁸ Young et al. concluded that tele-ICU was associated with a decrease in mortality and LOS in the ICU but not in the hospital.²⁹ A study done by Morrison et al. concluded that a difference in mortality could not be determined because the hospital’s ICU mortality rate was already low.³⁰ Lilly et al. found that after the implementation of tele-ICU, tools were developed for real-time auditing and reconciliation, which increased the adherence to best practices and also led to a decrease in the rates of complications in the ICU.³¹

Telemedicine in the ICU may also prevent intensivist and nurse “burn-outs and posttraumatic stress.”³² Physicians who are tired because of long hours or stress are more prone to making mistakes. “The tele-ICU is that ‘second set of eyes’ that provides additional clinical surveillance and support.”³³ It has also helped residents who are new to the field.³⁴

Financial Impact of Tele-ICU

The adoption of tele-ICU requires a substantial up-front capital investment with ongoing costs of operation and maintenance. These costs may impede the adoption of this technology, especially with the lack of reimbursement for tele-ICU services and uncertainties about ROI calculations. Moreover, the ROI is merely calculated using indirect clinical effects and the expected LOS reduction.

Payback period or net present value (NPV) are the indictors used for ROI. More specifically, the financial equation related to tele-ICU is desired to be the following.³⁵

[Capital Cost + Operating Cost] ≤ [Revenue from Reimbursement + Cost Savings Attained]

The cost of tele-ICU varies depending on the setting, hardware, software, training, and compatibility with other systems. One study reported a cost of more than $2 million to set up a command center and its components.³⁶ In general, $2 million to $5 million is the estimated cost to set up a command center and install a tele-ICU system, with operating costs ranging from $600,000 to $1.5 million per year, according to costs reported by various adopters.³⁷

On the revenue side, one study found a 10 percent reduction in ICU length of stay, creating the ability to care for one new ICU patient per day, which could result in a positive $2.5 million NPV.³⁸

Most studies reviewed used LOS and mortality to determine cost savings. For example, according to Rosenfeld et al., ICU costs decreased between 25 percent and 31 percent during the intervention period, and hospital costs decreased by 12 percent to 19 percent.³⁹ Breslow et al. hired an independent consulting firm to determine the financial outcome of a tele-ICU program.⁴⁰ They determined the cost of care per day of service and also included equipment costs, staff costs, and other costs associated with having a tele-ICU system. The report showed a 24.6 percent decrease in variable costs per patient. This decrease is probably due to a shorter LOS in the ICU and improved clinical outcomes.^41–43

Staff Acceptance of Tele-ICU

Implementation of tele-ICU requires a change in the practices of many health workers. Most studies that measured the acceptance of tele-ICU showed high acceptance of the increased ICU coverage. Moreover, tele-ICU has a favorable impact both on patient care and on organizations. Thomas et al.⁴⁴ conducted a pretest-posttest attitude survey for physicians and found that their attitudes regarding safety significantly increased after implementation. Tele-ICU also increased the confidence that patients were adequately covered. Another study, conducted by Kowitlawakul, measured nurses’ attitudes through a survey; it revealed that tele-ICU would be beneficial in units without adequate physician coverage.⁴⁵ Chu-Weininger et al. measured the teamwork and safety environment of three ICUs before and after implementation.⁴⁶ Their results showed that implementation of a tele-ICU system improved teamwork and the safety climate in some units, especially among nurses. As a result, the collaboration needed to enhance the value of the tele-ICU system is acquired through effective implementation of a continuous change management plan. Through implementation of best-practice protocols and other quality assurance measures, the scope of tele-ICU is expected to evolve and extend to other microsystems such as emergency departments, high-risk delivery units, long-term acute care hospitals, and other departments that are designated to provide an immediate response to patients.

Discussion

The current studies in the articles reviewed are early steps, and more research needs to be done before tele-ICU will become more widely adopted. Some studies did not show any difference before and after adoption of tele-ICU because the institutions already had good outcomes (see Table 1). Other studies showed a large decrease in LOS and mortality, which could be attributed to the fact that a hospital is an open system (Table 1). Similarly, Yoo and Dudley also found heterogeneity in tele-ICU systems and believe that “it is unlikely that any single study can definitely address the benefits of telemedicine for the critically ill.”⁴⁷ They also mentioned that there is a “lack of consistent reference in the literature to a unifying conceptual framework of what ICU care is and how tele-ICU could improve it.”⁴⁸

Another problem in the studies was the lack of consistent measurement, reporting, and adjustment for patient severity.⁴⁹ This problem could have led to inflated results relating to mortality and LOS. One hospital may be a Level 1 trauma center and experience many deaths, while another facility could be a smaller hospital that does not typically see that type of patients. Cost-effectiveness is another area in which more research is needed. While many studies draw conclusions on cost based on mortality and LOS, actual costs were not reported. This is an important consideration, especially for smaller facilities that want to ensure a return on their investment.

Limitations of this Review

One of the limitations of this systematic review is that it uses studies published in peer-reviewed journals. A publication bias toward studies that have positive findings has been well documented.⁵⁰ Therefore, studies that do not demonstrate any effect or report a negative effect of tele-ICU implementation may not carry as much weight in the synthesis of the data because they were not identified through the search. Moreover, this review did not include studies that looked at patient assessment because the focus of this review was on tele-ICU intervention programs. This review was a first attempt to identify scientifically sound evidence on telemedicine intervention programs and synthesize and critically appraise the published literature in this area. In part, this review also helps identify possible directions for future studies.

Conclusion

This systematic review identified a substantial amount of scientific literature in the relatively new area of tele-ICU. The review showed that although the published studies differ in terms of study designs, settings, and outcomes measured, there is a consistent trend in the literature supporting the efficacy and effectiveness of tele-ICU. In conclusion, from the data available, tele-ICU seems to be a promising path, especially in the United States, where there is a limited number of board-certified intensivists.

Sajeesh Kumar, PhD, is an associate professor in the Department of Health Informatics and Information Management at the University of Tennessee Health Science Center in Memphis, TN.

Shezana Merchant, MD, is a graduate student in the Department of Health Informatics and Information Management at the University of Tennessee Health Science Center in Memphis, TN.

Rebecca Reynolds, EdD, RHIA, is an associate professor and chair of the Department of Health Informatics and Information Management at the University of Tennessee Health Science Center in Memphis, TN.

Notes

1. Mullen-Fortino, M., J. DiMartino, L. Entrikin, S. Muliner, C. W. Hanson, and J. M. Kahn. “Bedside Nurses’ Perceptions of Intensive Care Unit Telemedicine.” American Journal of Critical Care 21, no. 1 (2012): 24–32.
2. Goran, S. “A Second Set of Eyes: An Introduction to Tele-ICU.” Critical Care Nurse 30, no. 4 (2010): 46–54.
3. Ries, M. “Tele-ICU: A New Paradigm in Critical Care.” International Anesthesiology Clinics 47, no. 1 (2009): 153–70.
4. Celi, L. A., E. Hassan, C. Marquardt, M. Breslow, and B. Rosenfeld. “The eICU: It’s Not Just Telemedicine.” Critical Care Medicine 29, no 8 (2001): N183–N189.
5. Morrison, J. L., Q. Cai, N. Davis, Y. Yan, M. L. Berbaum, M. Ries, and G. Solomon. “Clinical and Economic Outcomes of the Electronic Intensive Care Unit: Results from Two Community Hospitals.” Critical Care Medicine 38, no. 1 (2010): 2–8.
6. Halpern, N. A., S. M. Pastores, and R. J. Greenstein. “Critical Care Medicine in the United States 1985–2000: An Analysis of Bed Numbers, Use, and Costs.” Critical Care Medicine 32, no. 6 (2004): 1254–59.
7. Grundy, B. L., P. K. Jones, and A. Lovitt. “Telemedicine in Critical Care: Problems in Design, Implementation, and Assessment.” Critical Care Medicine 10, no. 7 (1982): 471–75.
8. Goran, S. F., and T. Van der Kloot. “Savings in RN Staffing Costs Pre and Post eICU Implementation.” In eICU Program Success Stories. Baltimore, MD: Philips-VISICU, 2008, 34-38
9. Kohl, B., J. T. Gutsche, P. Kim, F. D. Sites, and E. A. Ochroch. “Effect of Telemedicine on Mortality and Length of Stay in a University ICU.” Critical Care Medicine 35, no. 12 (2007): A22.
10. Thomas, E. J., J. F. Lucke, L. Wueste, L. Weavind, and B. Patel. “Association of Telemedicine for Remote Monitoring of Intensive Care Patients with Mortality, Complications, and Length of Stay.” JAMA 302, no. 24 (2009): 2671–78.
11. Zawada, E., and P. Herr. “ICU Telemedicine Improves Care to Rural Hospitals Reducing Costly Transports.” Critical Care Medicine 36, no. 12 (2008): A172.
12. Gracias, V., et al. “Outcomes of SICU Patients after Implementation of an Electronic ICU (‘eICU’) System and Off-Site Intensivist.” Presented at the IATSIC-AAST Conference, Montreal, Quebec, Canada, August, 2007.
13. New England Healthcare Institute and Massachusetts Technology Collaborative. Critical Care, Critical Choices: The Case for Tele-ICUs in Intensive Care. December 2010. Available at http://www.masstech.org/sites/mtc/files/documents/2010%20TeleICU%20Report.pdf.
14. Ibid.
15. Ibid.
16. Ibid.
17. Ibid.
18. Mullen-Fortino, M., J. DiMartino, L. Entrikin, S. Muliner, C. W. Hanson, and J. M. Kahn. “Bedside Nurses’ Perceptions of Intensive Care Unit Telemedicine.”
19. Goran, S. “A Second Set of Eyes: An Introduction to Tele-ICU.”
20. Stafford, T. B., M. A. Myers, A. Young, J. G. Foster, and J. T. Huber. “Working in an eICU Unit: Life in the Box.” Critical Care Nursing Clinics of North America 20, no. 4 (2008): 441–50.
21. Breslow, M. J., B. A. Rosenfeld, M. Doerfler, G. Burke, G. Yates, D. J. Stone, P. Tomaszewicz, R. Hochman, and D. W. Plocher. “Effect of a Multiple-Site Intensive Care Unit Telemedicine Program on Clinical and Economic Outcomes: An Alternative Paradigm for Intensivist Staffing.” Critical Care Medicine 32, no. 1 (2004): 31–38.
22. Willmitch, B., S. Golembeski, S. S. Kim, L. D. Nelson, and L. Gidel. “Clinical Outcomes after Telemedicine Intensive Care Unit Implementation.” Critical Care Medicine 40, no. 2 (2012): 450–54.
23. Thomas, E. J., J. F. Lucke, L. Wueste, L. Weavind, and B. Patel. “Association of Telemedicine for Remote Monitoring of Intensive Care Patients with Mortality, Complications and Length of Stay.”
24. Celi, L. A., E. Hassan, C. Marquardt, M. Breslow, and B. Rosenfeld. “The eICU: It’s Not Just Telemedicine.”
25. Thomas, E. J., J. F. Lucke, L. Wueste, L. Weavind, and B. Patel. “Association of Telemedicine for Remote Monitoring of Intensive Care Patients with Mortality, Complications and Length of Stay.”
26. Berenson, R. A., J. M. Grossman, and E. A. November. “Does Telemonitoring of Patients—the eICU—Improve Intensive Care?” Health Affairs 28, no. 5 (2009): w937–w947.
27. Thomas, E. J., J. F. Lucke, L. Wueste, L. Weavind, and B. Patel. “Association of Telemedicine for Remote Monitoring of Intensive Care Patients with Mortality, Complications and Length of Stay.”
28. Lilly, C. M., S. Cody, H. Zhao, K. Landry, S. P. Baker, J. McIlwaine, M. W. Chandler, R. S. Irwin, and University of Massachusetts Memorial Critical Care Operations Group. “Hospital Mortality, Length of Stay and Preventable Complications among Critically Ill Patients Before and After Tele-ICU Reengineering of Critical Care Processes.” JAMA 305, no. 21 (2011): 2175–83.
29. Young, L. B., P. S. Chan, X. Lu, B. K. Nallamothu, C. Sasson, and P. M. Cram. “Impact of Telemedicine Intensive Care Unit Coverage on Patient Outcomes.” Archives of Internal Medicine 171, no. 6 (2011): 498–506.
30. Morrison, J. L., Q. Cai, N. Davis, Y. Yan, M. L. Berbaum, M. Ries, and G. Solomon. “Clinical and Economic Outcomes of the Electronic Intensive Care Unit: Results from Two Community Hospitals.”
31. Lilly, C. M., S. Cody, H. Zhao, K. Landry, S. P. Baker, J. McIlwaine, M. W. Chandler, R. S. Irwin, and University of Massachusetts Memorial Critical Care Operations Group. “Hospital Mortality, Length of Stay and Preventable Complications among Critically Ill Patients Before and After Tele-ICU Reengineering of Critical Care Processes.”
32. Ries, M. “Tele-ICU: A New Paradigm in Critical Care.”
33. Goran, S. “A Second Set of Eyes: An Introduction to Tele-ICU.”
34. Ries, M. “Tele-ICU: A New Paradigm in Critical Care.”
35. Breslow, M. J., B. A. Rosenfeld, M. Doerfler, G. Burke, G. Yates, D. J. Stone, P. Tomaszewicz, R. Hochman, and D. W. Plocher. “Effect of a Multiple-Site Intensive Care Unit Telemedicine Program on Clinical and Economic Outcomes: An Alternative Paradigm for Intensivist Staffing.”
36. Ibid.
37. Ibid.
38. Ibid.
39. Rosenfeld, B. A., T. Dorman, M. J. Breslow, P. Pronovost, M. Jenckes, N. Zhang, et al. “Intensive Care Unit Telemedicine: Alternate Paradigm for Providing Continuous Intensivist Care.” Critical Care Medicine 28, no. 12 (2000): 3925–31.
40. Breslow, M. J., B. A. Rosenfeld, M. Doerfler, G. Burke, G. Yates, D. J. Stone, P. Tomaszewicz, R. Hochman, and D. W. Plocher. “Effect of a Multiple-Site Intensive Care Unit Telemedicine Program on Clinical and Economic Outcomes: An Alternative Paradigm for Intensivist Staffing.”
41. Ibid.
42. Celi, L. A., E. Hassan, C. Marquardt, M. Breslow, and B. Rosenfeld. “The eICU: It’s Not Just Telemedicine.”
43. Rosenfeld, B. A., T. Dorman, M. J. Breslow, P. Pronovost, M. Jenckes, N. Zhang, et al. “Intensive Care Unit Telemedicine: Alternate Paradigm for Providing Continuous Intensivist Care.”
44. Chu-Weininger, M. Y., L. Wueste, J. F. Lucke, L. Weavind, J. Mazabob, and E. J. Thomas. “The Impact of a Tele-ICU on Provider Attitudes about Teamwork and Safety Climate.” Quality and Safety in Health Care 19, no. 6 (2010): e39.
45. Kowitlawakul, Y. “Technology Acceptance Model: Predicting Nurses’ Acceptance of Telemedicine Technology (eICU).” PhD dissertation, George Mason University, 2008.
46. Chu-Weininger, M. Y., L. Wueste, J. F. Lucke, L. Weavind, J. Mazabob, and E. J. Thomas. “The Impact of a Tele-ICU on Provider Attitudes about Teamwork and Safety Climate.” Quality and Safety in Health Care 19, no. 6 (2010): e39.
47. Yoo, E. J., and R. A. Dudley. “Evaluating Telemedicine in the ICU.” JAMA 302, no. 24 (2009): 2705–6.
48. Ibid.
49. Young, L. B., P. S. Chan, X. Lu, B. K. Nallamothu, C. Sasson, and P. M. Cram. “Impact of Telemedicine Intensive Care Unit Coverage on Patient Outcomes.”
50. Egger, M., G. D. Smith, and J. A. Sterne. “Uses and Abuses of Meta-analysis.” Clinical Medicine 1, no. 6 (2001): 478–84.

Printer friendly version of this article.

Sajeesh Kumar, PhD; Shezana Merchant, MD; and Rebecca Reynolds, EdD, RHIA. “Tele-ICU: Efficacy and Cost-Effectiveness of Remotely Managing Critical Care.” Perspectives in Health Information Management (Spring 2013): 1-13.

Abstract

Health information management (HIM) professionals play a significant role in transitioning from ICD-9-CM to ICD-10-CM/PCS. ICD-10-CM/PCS coding will impact many operational aspects of healthcare facilities, such as physicians’ documentation in health records, coders’ process for review of clinical information, the billing process, and the payers’ reimbursement to the healthcare facilities. This article examines the level of readiness and planning for ICD-10-CM/PCS implementation among hospitals in Alabama, identifies training methods/approaches to be used by the hospitals, and discusses the challenges to the ICD-10-CM/PCS coding transition. A 16-question survey was distributed to 116 Alabama hospital HIM directors in December 2011 with follow-up through February 2012. Fifty-three percent of respondent hospitals began the planning process in 2011, and most facilities were halfway or less than halfway to completion of specific implementation tasks. Hospital coders will be or are being trained using in-house training, through seminars/webinars, or by consultants. The impact of ICD-10-CM/PCS implementation can be minimized by training coders in advance, hiring new coders, and adjusting coders’ productivity measures. Three major challenges to the transition were identified: the need to interact with physicians and other providers more often to obtain information needed to code in ICD-10-CM/PCS systems, education and training of coders and other ICD-10-CM/PCS users, and dependence on vendors for major technology upgrades for ICD-10-CM/PCS systems. Survey results provide beneficial information for HIM professionals and other users of coded data to assist in establishing sound practice standards for ICD-10-CM/PCS coding implementation. Adequate planning and preparation will be essential to the successful implementation of ICD-10-CM/PCS.

Keywords: ICD-10-CM/PCS (ICD-10) coding, health information management

Introduction

Introduced in the United States in 1979, the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) established a coding system to document inpatient diagnostic and procedural codes.¹ However, ICD-9-CM is now over 30 years old. The new age of technology has brought numerous improvements in medical procedures and applications impacting the effectiveness of this coding system.^2–4 The federally mandated transition to the International Classification of Diseases, Tenth Revision, Clinical Modification, and the International Classification of Diseases, Tenth Revision, Procedure Coding System (ICD-10-CM/PCS) brings an expansion of the existing ICD-9-CM code set, adding 54,000 diagnosis codes and 83,000 procedure codes.^5–6 Implementation of the ICD-10-CM/PCS coding systems’ increased granularity will bring challenges to providers and healthcare organizations throughout the healthcare delivery system, particularly as healthcare entities attempt to comply with mandates for health information exchange (HIE).^7–13

As many healthcare providers and organizations continued to plan for the transition to ICD-10-CM/PCS,^14–15 the Department of Health and Human Services considered the possibility of yet another extension to a new compliance date of October 1, 2014.^16–17 On September 5, 2012, a final rule was published in the Federal Register announcing the delay of ICD-10-CM/PCS to October 1, 2014.^18–19 Some healthcare organizations ready for the transition may feel the extra time is unwarranted; however, for those providers who delayed ICD-10-CM/PCS implementation because of challenges associated with planning,^20–24 costs of implementation,^25–26 and training,^27–31 the extension offers time to successfully complete the implementation process.³²

According to data from a 2012 collaborative study by the Georgetown University Student Consulting Team and Booz Allen Hamilton, “training was regarded as the most significant and costly component of the transition.”³³ This transition impacts the delivery of care within the overall healthcare system; therefore, the incorporation of appropriate roles and job functions into training programs is critical to the success of an implementation plan.³⁴ Moreover, understanding an organization’s level of readiness for implementation provides a foundation for the development of a successful ICD-10-CM/PCS transition plan.

The purpose of this study was to assess Alabama hospitals’ readiness for ICD-10-CM/PCS by the mandated implementation date. Prior literature has highlighted methods to prepare for ICD-10/PCS implementation and has identified potential challenges;^35–44 however, no study was found that assessed readiness among Alabama hospitals for the implementation of ICD-10-CM/PCS by the scheduled implementation date. This research focused on three specific objectives:

• To assess the level of readiness among Alabama hospitals for the upcoming implementation date for ICD-10-CM/PCS;
• To assess training methods and approaches by the hospitals in preparing to transition from ICD-9-CM to ICD-10-CM/PCS coding; and
• To assess the challenges and barriers of ICD-10-CM/PCS coding transitions.

Methods

Survey Participants and Procedure

The study participants were volunteers drawn from the Alabama Association of Health Information Management directory. Data were collected from a self-designed survey by researchers. A total of 116 surveys were sent to hospital health information management (HIM) directors in Alabama hospitals during December 2011. The 116 surveys were e-mailed through Surveymonkey.com to HIM directors who provided valid e-mail addresses. For seven HIM directors without valid e-mail addresses or who preferred to receive a hard copy of the survey, a survey was mailed to the address provided. Three follow-up surveys were sent to nonrespondents in two-week intervals ending the first week of February 2012. A total of 43 valid surveys were received, yielding a response rate of 37 percent. This study was approved by the University of Alabama at Birmingham (UAB) Institutional Review Board.

Survey Development and Measures

The 16-question self-completed survey was based on AHIMA’s “ICD-10-CM/PCS Transition: Planning and Preparation Checklist,”⁴⁵ “AHIMA Survey on ICD-10 and 5010 Compliance,”⁴⁶ and AHIMA’s “5010 and ICD-10 Compliance Q2 Survey Results.”⁴⁷ The survey questions included the respondent’s individual information (position, credentials, and education level); hospital characteristics (type, location, bed size, number of HIM employees); HIM roles in the transition to ICD-10-CM/PCS coding; training plans and approach; the status of ICD-10-CM/PCS transition and preparation; and the respondent’s perception of challenges and barriers to ICD-10-CM/PCS transition. The responses were maintained in Surveymonkey.com. Results were analyzed via Surveymonkey.com and Microsoft Excel. The survey questions were pretested for validity and clarity by HIM professionals in the principal author’s department, and revisions were made based on the pretest feedback.

Results

Characteristics of Respondents

Of the 43 survey respondents, the vast majority (93 percent) were either HIM directors or managers, and the rest of the respondents (7 percent) were hospital administrative personnel. Of the valid 41 responses from the facilities, 41 percent were federal or nonfederal government hospitals, 37 percent were not-for-profit, and 22 percent were investor-owned for-profit hospitals. About 39 percent of hospitals had 100 or fewer beds, and slightly more than half of the hospitals (51 percent) were located in rural areas of Alabama. (See Table 1.)

Planning for ICD-10-CM/PCS

The respondents who hold HIM leadership positions within their respective facilities played different roles on the ICD-10-CM/PCS implementation planning team. Data indicated that these roles included team leader (31 percent) and team member (31 percent). The remaining 38 percent of respondents were either in the process of forming a team or had not begun the implementation planning process.

Respondents’ comments indicated that only 15 percent of the facilities began planning for ICD-10-CM/PCS implementation before 2011, 53 percent began the planning process in 2011, 25 percent would start planning in 2012, 3 percent would start planning after 2012, and 5 percent were not sure when they would begin planning.

Regarding a timeline for specific implementation tasks, data showed that a large portion of facilities were halfway or less than halfway to completion (50 percent). The implementation tasks (see Table 2) that were less than halfway completed or at the halfway point of completion include conducting clinician and code set user education (94 percent), identifying and budgeting for required information system changes (93 percent), creating awareness of the impact of the ICD-10-CM/PCS code set throughout the organization (91 percent), assessing the status of payers’ and other business associates’ progress toward ICD-10-CM/PCS preparedness (86 percent), and determining organizational structure and responsibilities for the transition (83 percent).

Respondents were allowed to select one or more choices to the question of “Who will be responsible for leading the planning for ICD-10-CM/PCS?” Responses included the HIM director (65 percent), a coding manager/supervisor (25 percent), and a consultant or coding vendor (15 percent); 15 percent were unsure, and the remaining 20 percent listed various individuals from hospital administration and other departments, such as the chief financial officer (CFO) and the information technology (IT) department.

Training Plans and Approaches for ICD-10-CM/PCS

ICD-10-CM/PCS implementation training is not only focused on coders but also involves other staff, such as senior executives, HIM leadership, financial management, and IT personnel. According to the Centers for Medicare and Medicaid Services (CMS) and the American Health Information Management Association (AHIMA), the ICD-10-CM/PCS implementation planning and preparation process includes four phases: phase 1, implementation plan development and impact assessment; phase 2, implementation preparation (including coder training); phase 3, “go-live” preparation; and phase 4, postimplementation follow-up.^48–49 Depending on the facility type, size, and complexity, the phases may vary in length or may overlap.⁵⁰

The training plans reported by survey respondents varied for each phase and are shown in Table 3. Phase 1 training activities were focused on senior executives (96 percent), HIM leadership (90 percent), and IT personnel (81 percent); phase 2 training was focused on coding staff (90 percent), financial management, including accounting and billing personnel (72 percent), and HIM leadership (70 percent); phase 3 training was focused on coding staff (69 percent), IT personnel (69 percent), and business associates (65 percent); while phase 4 training was on for coding staff (66 percent), HIM leadership (60 percent), and financial management staff (60 percent).

Figure 1 shows that the coder training approaches identified by the respondents included in-house training (58 percent), facility-paid seminars or webinars for employees to attend (58 percent), facility-paid courses to be provided by a consultant or consultants (33 percent), seminars or webinars (given by AHIMA, the state HIM association, or other organizations) that coders attend on their own (18 percent), and courses that coders take on their own (such as an academic course from a university or college; 9 percent). Three percent of respondents indicated that no training would be provided by the healthcare facility.

Challenges and Barriers to ICD-10-CM/PCS Implementation

Respondents were asked how they would deal with the anticipated loss of efficiency and productivity that the implementation of ICD-10-CM/PCS would bring. As shown in Figure 2, most of the respondents (77 percent) said that they would train coders in advance to minimize the impact; others would adjust coders’ productivity measures (47 percent), hire new coders (35 percent), increase coders’ hours to compensate for the loss of productivity (18 percent), take other approaches (15 percent), or do nothing (12 percent).

Data shown in Figure 3 indicated that the top three most important perceived challenges and barriers to the ICD-10-CM/PCS transition were the need to interact with physicians and other providers more often to obtain information needed to code in ICD-10-CM/PCS systems (100 percent), the need for education and training of HIM department coders (94 percent), and dependence on vendors for major technology upgrades for ICD-10-CM/PCS systems (94 percent). Other challenges and barriers included the loss of individual productivity for coders (91 percent); the need for education of other ICD-10-CM/PCS users (revision of charge masters, encounter forms for ancillary/emergency room areas, etc.; 91 percent); and managing two databases for codes for a period of time (84 percent). A few additional challenges that were not included in the figure included:  a decrease in the number of experienced coders if they retire and leave the position (78 percent); limited resources for training of HIM coders (78 percent); and the need to hire new coders to compensate for decreased productivity of current staff at the beginning of the implementation process (56 percent).

Discussion

The majority of the survey respondents represented the HIM field as directors or managers, and the majority of respondents felt that HIM directors would be responsible for leading the planning for ICD-10-CM/PCS implementation. Despite the low percentage of respondents indicating that their facilities had not formed teams prior to 2011 (15 percent), 53 percent indicated entering the planning process in 2011. Whether this delayed timeline for planning was a result of anticipated changes to the start date^51–52 is unknown as the survey did not address this issue.

Data showed that a large majority of the facilities were halfway or less than halfway toward completion of the ICD-10 CM/PCS implementation, indicating that facilities that entered the planning stage in 2011 moved quickly to address specific implementation tasks identified in Table 2. All respondents indicated plans that fell within the four implementation phases proposed by CMS and AHIMA. Interestingly, the majority of phase 1 activities were associated with senior executives, HIM leadership, and IT personnel with limited involvement of other facility staff. However, development of the implementation plan and impact assessment must reach beyond the higher level tier of administrative authority to address issues of staff ownership of the change process. Respondents indicated that phase 2 involvement did include training for coders, medical staff, and other data users within the facility. Involvement of a variety of staff (administrative and clinical) at this point in the process may facilitate a sense of ownership for the needed change; however, it does not speak to the challenge of interacting with providers specifically to obtain input and buy-in regarding the planned implementation. The study results suggest that the surveyed hospitals in Alabama are meeting their target implementation tasks and, if they continue through phase 4, will be ready to transition to the ICD-10-CM/PCS coding systems. Having the implementation date of October 1, 2014, published in a final rule allows facilities to benefit from the additional time to prepare.

Indeed, data showed that 100 percent of the respondents felt that the need to interact with physicians and other providers more often to obtain information needed to code in this new system was the top challenge to ICD-10-CM/PCS implementation. Respondents also consistently noted other challenges, including the need to train coders and the dependence on vendors for technology upgrades. Respondents agreed that a variety of options were available to address an anticipated loss of efficiency and productivity. The majority of respondents stated that they would train coders in advance of start-up to minimize this loss; more than half of the respondents stated that they would hire new coders to compensate for decreased productivity of the current staff at the beginning of the implementation process.

This study has several limitations: First, despite a 37 percent response rate, the survey does not provide a complete picture of the healthcare facilities that did not respond to the survey. Second, since ICD-10-CM/PCS coding has not been implemented, the results are restricted to the current facilities’ perception of their challenges and the anticipated planning. The proposed plans and training approaches may be changed during actual implementation phase. Finally, the study was conducted among hospitals in Alabama. It cannot be assumed that Alabama hospitals are more or less prepared than hospitals elsewhere in the nation, so the results may not necessarily be generalizable.

Implications for the HIM Field

Our study has the following implications for the HIM practitioner’s role in this transition. The implementation of ICD-10-CM/PCS represents the most significant change in the coding and billing of healthcare services in the last 30 years. It is essential that HIM professionals prepare for this and lead their healthcare facilities to a successful and smooth transition to the ICD-10 coding systems.

Better Assessment

Each facility should continue to monitor the current level of readiness, identify any gaps, and be prepared with trained staff and new processes in place for the transition to the new coding systems. HIM professionals must remain engaged in the organization’s ICD-10-CM/PCS implementation team and continually assess the organization’s state of readiness. Some facilities may have the benefit of an outside vendor that can lead the effort with minimal impact on the leadership. Using an assessment tool allows a facility to identify its strengths and weaknesses so that the areas that need to be addressed can be identified as early as possible. The delay of the implementation date offers HIM professionals and other leaders the opportunity to identify the multiple uses of coded data in the facility in order to plan for the revision of billing and/or encounter forms with ICD-10-CM diagnosis codes, to revise program codes in billing and clinical applications, and to communicate with vendors and payers to be aware of their state of readiness. As each year passes, changes that occur in the ICD-10-CM/PCS codes have to be updated in any ICD-10-CM/PCS systems in place for testing in organizations. The implementation team is the leadership’s key to monitoring the organization’s level of readiness for the implementation of ICD-10-CM/PCS systems.

Better Planning

Each facility should select the training approach or approaches that best meet the facility’s needs and characteristics. Hospitals in Alabama range in size from small rural facilities to tertiary academic medical centers. Training methods appropriate for one facility may not work for another facility. With smaller facilities, it may be more cost effective to send coders to a training seminar. Larger organizations may opt to bring the trainers into the organization and plan workshops to reduce the amount of time and travel involved. Online courses through academic institutions offer potential training opportunities and eliminate the need for traveling. Professional organizations (AHIMA, state HIM associations), vendors, and consultants may offer a combination of online training and face-to-face lecture presentations. Facilities should examine their own resources for ICD-10-CM/PCS training and develop a training plan to meet their own needs.

Conclusion

This study examined the readiness of Alabama hospitals for implementation of ICD-10-CM/PCS by the mandated date. Results confirmed that while some facilities were slow to start the implementation process prior to 2011, the majority have now started planning process. With the delayed date for the implementation of ICD-10-CM/PCS systems, hospitals have additional time to complete their preparation and training. Training approaches addressed in this study targeted a full range of facility personnel and included methods designed to enhance employee skills while limiting facility costs. HIM directors were associated with primary leadership roles for implementation, a significant responsibility that is in line with their professional knowledge and skill sets. The survey results confirmed that Alabama hospitals are following recommended guidelines to assess, prepare, and plan for the implementation of ICD-10-CM/PCS coding systems. Adequate planning and better preparation will be essential to the successful implementation of ICD-10-CM/PCS by October 1, 2014.

Shannon H. Houser, PhD, MPH, RHIA, is an associate professor in the Department of Health Services Administration at the University of Alabama at Birmingham in Birmingham, AL.

Darius Morgan, RHIA, is a compliance intern at the Baptist Health System corporate office in Birmingham, AL.

Kay Clements, MA, RHIA, CCS, CPC, is a health information management program director at the University of Alabama at Birmingham in Birmingham, AL.

Susan Hart-Hester, PhD, is a professor of family medicine and director of the Health Professional Shortage Core in the Delta Regional Institute at the University of Mississippi Medical Center in Jackson, MS.

Notes

1. Centers for Medicare and Medicaid Services. ICD-10 Overview. Available at https://www.cms.gov/ContractorLearningResources/Downloads/ICD-10_Overview_Presentation.pdf.
2. American Hospital Association (AHA) and American Health Information Management Association (AHIMA). ICD-10-CM Field Testing Project. September 23, 2003. Available at http:www.ahima.org/downloads/pdfs/resources/FinalStudy_000.pdf.
3. Jackson, V., and A. Muckerman. “Navigating Regulatory Change: Preliminary Lessons Learned During the Healthcare Provider Transition to ICD-10-CM/PCS.” Perspectives in Health Information Management (Winter 2012).
4. Centers for Medicare and Medicaid Services. ICD-10-CM/PCS: An Introduction. Available at http://www.cms.gov/Medicare/Coding/ICD10/downloads/ICD-10Overview.pdf.
5. American Medical Association. Preparing for the Conversion from ICD-9 to ICD-10: What You Need to Be Doing Today. April 13, 2010. Available at http://www.ama-assn.org/ama1/pub/upload/mm/399/icd9-icd10-conversion.pdf.
6. Hughes, C. 5010 & ICD-10 Overviews. American Academy of Family Physicians. 2011. http://www.aafp.org/online/etc/medialib/aafp_org/documents/prac_mgt/codingresources/overview5010icd10.Par.0001.File.dat/5010ICD10Overview.pdf.
7. American Medical Association. Preparing for the Conversion from ICD-9 to ICD-10: What You Need to Be Doing Today.
8. Hughes, C. 5010 & ICD-10 Overviews.
9. Department of Health and Human Services. “Health Insurance Reform; Modifications to the Health Insurance Portability and Accountability Act (HIPAA); Final Rules.” 45 CFR Part 162. Federal Register 74, no. 11 (January 16, 2009): 3296–328. Available at http://edocket.access.gpo.gov/2009/pdf/E9-740.pdf.
10. AHIMA Coding Products and Services Team. “Destination 10: Healthcare Organization Preparation for ICD-10-CM and ICD-10-PCS.” Journal of AHIMA 75, no. 3 (2004): 56A–D.
11. Hazlewood, A. “ICD-9-CM to ICD-10-CM: Implementation Issues and Challenges.” AHIMA’s 75th Anniversary National Convention and Exhibit Proceedings, October 2003. Available at http://library.ahima.org/xpedio/groups/public/documents/ahima/bok3_005426.hcsp?dDocName=bok3_005426.
12. Dimick, Chris. “ICD-10 Delay Impacts All Sectors of Healthcare: Industry Attempts to Answer the Question ‘What Now?’” Journal of AHIMA 83, no. 6 (June 2012): 32–37.
13. Sanders, T., F. Bowens, W. Pierce, B. Stasher-Booker, E. Thompson, and W. Jones. “The Road to ICD-10-CM/PCS Implementation: Forecasting the Transition for Providers, Payers, and Other Healthcare Organizations.” Perspectives in Health Information Management (Winter 2012).
14. AHIMA. 5010 and ICD-10 Compliance: Q2 Survey Results. July 2010. Available at http:www.ahima.org/downloads/pdfs/busdev/5010_and_ICD-10_Survey_Results.pdf.
15. AHIMA. Tracking the Industry’s Progress: AHIMA Survey on ICD-10 and 5010 Compliance. September 2011. Available at www.ahima.org/downloads/pdfs/busdev/ICD10SurveySept2011.pdf.
16. Department of Health and Human Services. “HHS Announces Intent to Delay ICD-10 Compliance Date.” News Release, February 16, 2012. Available at http://www.hhs.gov/news/press/2012pres/02/20120216a.html.
17. American Academy of Family Physicians. “HHS Proposes Delaying ICD-10 Compliance to October 2014.” April 10, 2012. Available at http://www.aafp.org/online/en/home/publications/news/news-now/practice-professional-issues/20120410icd-10date.html.
18. Department of Health and Human Services. “Administrative Simplification: Adoption of a Standard for a Unique Health Plan Identifier; Addition to the National Provider Identifier Requirements; and a Change to the Compliance Date for the International Classification of Diseases, 10th Edition (ICD-10-CM and ICD-10-PCS) Medical Data Code Sets.” 45 CFR Part 162. Federal Register 77, no. 172 (September 5, 2012): 54664.
19. Dimick, C. “HHS Announces: ICD-10 Delayed One Year.” Journal of AHIMA. August 24, 2012. Available at http://journal.ahima.org/2012/08/24/hhs-announces-icd-10-delayed-one-year/.
20. Piselli, C., K. Wall, and A. Boucher. “A New Language for Health Care?” Healthcare Financial Management, January 2010. Available at http://www.gcchimss.net/enews/PiselliWallBoucher.pdf.
21. AHIMA e-HIM Workgroup on the Transition to ICD-10-CM/PCS. “Planning Organizational Transition to ICD-10-CM/PCS.” Journal of AHIMA 80, no. 10 (October 2009): 72–77.
22. Schwend, G. “Expanding the Code: The Methodical Switch from ICD-9-CM to ICD-10-CM Will Bring Both Challenges and Rewards to Healthcare.” Health Management Technology 28, no. 6 (June 2007): 12, 14.
23. Rivers, P., J. Frimpong, and M. Rivers. “Transitioning from ICD-9-CM to ICD-10-CM: The Examination of Potential Barriers.” Journal of Health Care Finance 35, no. 2 (2008): 76–83.
24. “An ICD-10 Roadmap: Seven Experts Provide Ideas for Implementation, Including Upfront Planning and Training, Considered Important Factors in a Successful Transition to This Coding System.” Health Management Technology 31, no. 5 (2010): 20, 22–23.
25. Baldwin, G. “ICD: Intensive Cash Flow Disruption?” Health Data Management 18, no. 9 (2010): 27–28, 30, 32.
26. Law, L., and M. A. Porucznik. “Switching to ICD-10: The Impact on Physicians.” AAOS Now, February 2009. Available at http://www.aaos.org/news/aaosnow/feb09/reimbursement1.asp.
27. Jackson, V., and A. Muckerman. “Navigating Regulatory Change: Preliminary Lessons Learned During the Healthcare Provider Transition to ICD-10-CM/PCS.”
28. American Medical Association. Preparing for the Conversion from ICD-9 to ICD-10: What You Need to Be Doing Today.
29. Schwieters, J. “Strategies for Dealing with the National Coding Shortage.” Healthcare Financial Management 64, no. 4 (April 2010): 36–38.
30. Majerowicz, A. “Developing an ICD-10-CM/PCS Coder Training Strategy.” Journal of AHIMA 82, no. 4 (2011): 58–60.
31. AHIMA. “Transitioning to ICD-10-CM/PCS in the Classroom: Countdown to 2014.” Journal of AHIMA 83, no. 6 (June 2012): 68–73. Available at http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_049632.hcsp?dDoc.
32. AHIMA ICD-10-CM/PCS Academic Transition Workgroup. “Transitioning to ICD-10-CM/PCS—An Academic Timeline.” Journal of AHIMA 80, no. 4 (April 2009): 59–64.
33. Jackson, V., and A. Muckerman. “Navigating Regulatory Change: Preliminary Lessons Learned During the Healthcare Provider Transition to ICD-10-CM/PCS.”
34. American Medical Association. Preparing for the Conversion from ICD-9 to ICD-10: What You Need to Be Doing Today.
35. Piselli, C., K. Wall, and A. Boucher. “A New Language for Health Care?” Healthcare Financial Management, January 2010.
36. AHIMA e-HIM Workgroup on the Transition to ICD-10-CM/PCS. “Planning Organizational Transition to ICD-10-CM/PCS.”
37. Schwend, G. “Expanding the Code: The Methodical Switch from ICD-9-CM to ICD-10-CM Will Bring Both Challenges and Rewards to Healthcare.”
38. Rivers, P., J. Frimpong, and M. Rivers. “Transitioning from ICD-9-CM to ICD-10-CM: The Examination of Potential Barriers.”
39. “An ICD-10 Roadmap: Seven Experts Provide Ideas for Implementation, Including Upfront Planning and Training, Considered Important Factors in a Successful Transition to This Coding System.”
40. Baldwin, G. “ICD: Intensive Cash Flow Disruption?”
41. Law, L., and M. A. Porucznik. “Switching to ICD-10: The Impact on Physicians.”
42. Schwieters, J. “Strategies for Dealing with the National Coding Shortage.”
43. Majerowicz, A. “Developing an ICD-10-CM/PCS Coder Training Strategy.”
44. AHIMA. “Transitioning to ICD-10-CM/PCS in the Classroom: Countdown to 2014.”
45. Bowman, Sue, and Ann Zeisset. “ICD-10-CM/PCS Transition: Planning and Preparation Checklist.” AHIMA Body of Knowledge, March 2011. http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_034622.hcsp?dDocName=bok1_034622.
46. AHIMA. Tracking the Industry’s Progress: AHIMA Survey on ICD-10 and 5010 Compliance.
47. AHIMA. 5010 and ICD-10 Compliance: Q2 Survey Results.
48. Centers for Medicare and Medicaid Services. Preparing for ICD-10 Implementation in 2011 National Provider Teleconference. January 12, 2011. Available at https://www.cms.gov/Medicare/Coding/ICD10/downloads/Jan122011_ICD10_Call.pdf.
49. Bowman, Sue, and Ann Zeisset. “ICD-10-CM/PCS Transition: Planning and Preparation Checklist.”
50. Centers for Medicare and Medicaid Services. Preparing for ICD-10 Implementation in 2011 National Provider Teleconference.
51. Department of Health and Human Services. “HHS Announces Intent to Delay ICD-10 Compliance Date.”
52. American Academy of Family Physicians. “HHS Proposes Delaying ICD-10 Compliance to October 2014.”

Printer friendly version of this article.

Shannon H. Houser, PhD, MPH, RHIA; Darius Morgan, RHIA; Kay Clements, MA, RHIA, CCS, CPC; and Susan Hart-Hester, PhD. “Assessing the Planning and Implementation Strategies for the ICD-10-CM/PCS Coding Transition in Alabama Hospitals.” Perspectives in Health Information Management (Spring 2013): 1-15.

Abstract

As the health information management (HIM) profession continues to expand and become more specialized, there is an ever-increasing need to identify emerging HIM workforce roles that require a codified level of proficiency and professional standards. The Commission on Certification for Health Informatics and Information Management (CCHIIM) explored one such role—clinical documentation improvement (CDI) practitioner—to define the tasks and responsibilities of the job as well as the knowledge required to perform them effectively. Subject-matter experts (SMEs) defined the CDI specialty by following best practices for job analysis methodology. A random sample of 4,923 CDI-related professionals was surveyed regarding the tasks and knowledge required for the job. The survey data were used to create a weighted blueprint of the six major domains that make up the CDI practitioner role, which later formed the foundation for the clinical documentation improvement practitioner (CDIP) credential. As a result, healthcare organizations can be assured that their certified documentation improvement practitioners have demonstrated excellence in clinical care, treatment, coding guidelines, and reimbursement methodologies.

Keywords: job analysis, survey, clinical documentation improvement (CDI), documentation, Commission on Certification for Health Informatics and Information Management (CCHIIM), credential, exam, health information management (HIM) job roles

Introduction

As the health information management (HIM) profession continues to expand and become more specialized, there is an ever-increasing need to identify emerging HIM workforce roles that require a codified level of proficiency and professional standards. These evolving roles often advance into specialty areas or concentrations within the larger HIM industry and morph into in-demand positions with specialized competencies. The Commission on Certification for Health Informatics and Information Management (CCHIIM) explored one such role—clinical documentation improvement (CDI) practitioner—to define the tasks and responsibilities that the job comprises as well as the knowledge required to perform them effectively. An in-depth job analysis was conducted to codify the role, which later formed the foundation for developing the clinical documentation improvement practitioner (CDIP) credential. As a result, healthcare organizations can now have the confidence that their certified documentation improvement practitioners have demonstrated excellence in clinical care, treatment, coding guidelines, and reimbursement methodologies.

Background

Emerging professions or job roles bring an exciting air of possibility and uncertainty. Professional regulation, standards, and universal competency levels for these new roles are often ambiguous at best, leaving employers and job incumbents alike searching for a legitimate measure of job competence. A job analysis is the best tool to fully study and delineate these new workforce roles. The job analysis can later be used to form the foundation for a certification examination designed to assess the competency level of those interested in pursuing this role.

A job analysis (also known as a practice analysis, job/task analysis, or role delineation study) is conducted to determine the relevant tasks and knowledge, skills, and abilities (KSAs) needed to competently perform those tasks for a particular role. The main goal of a job analysis is to clearly and concisely define, through subject-matter expert (SME) validation, what professionals in that role do on the job.^{1, 2} The job analysis is an essential method for demonstrating the job relatedness of certification examination content, as the empirical study of a workforce role provides a linkage between job-related data and exam content.³ The importance of job analyses is further outlined through National Commission for Certifying Agencies (NCCA) and American National Standards Institute (ANSI) standards and guidelines. NCCA Standard 11 states: “The certification program must employ assessment instruments that are derived from the job/practice analysis and that are consistent with generally accepted psychometric principles.”⁴ The ANSI standard ANSI/ISO/IEC 17024:2003 further notes that a properly executed job analysis forms the basis of a valid, reliable, and fair assessment that reflects the KSAs required for competent job performance.⁵

A sound, comprehensive job analysis is integral to the legal defensibility of a credentialing exam, as the content domains and knowledge topics tested must be clearly linked to job-related performance criteria, resulting in content validity.⁶ Job analyses are often used as evidence of content validation during high-stakes examination legal challenges. Standard 14.14 of the Standards for Educational and Psychological Testing notes: “The content domain to be covered by a credentialing test should be defined clearly and justified in terms of the importance of the content for credential-worthy performance in an occupation or profession. A rationale should be provided to support a claim that the knowledge or skills being assessed are required for credential-worthy performance in an occupation and are consistent with the purpose for which the licensing or certification program was instituted.”⁷ In addition, the following criteria must be met in order for a job analysis to produce a content-valid examination:

• The exam domains, or main subject matter areas, must be accurately weighted to reflect their relative importance on the job;
• The difficulty level should match minimal competence for the credential; and
• The job analysis should cover the full range of tasks performed in that role.⁸

CCHIIM conducts routine environmental scans to monitor any changes or growth opportunities in the health information and informatics workforce that affect the profession, and as a result, the commission decided to conduct a CDI practitioner job analysis. Numerous industry trends, such as the increased adoption of electronic health records (EHRs), an increase in health insurance fraud, and the need for complete and accurate documentation to support the requirements of the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM), all suggest the need for a highly qualified, specialized set of documentation improvement specialists who meet stringent professional guidelines.⁹ Additionally, general emphasis on revenue cycle processes, regulatory requirements, and continuous quality improvement converge to necessitate this type of credential. Because clinical documentation specialists have expertise in clinical care, coding guidelines, and reimbursement methodologies, a nationally recognized CDI-related credential would distinguish those practitioners as competent to provide direction relative to clinical documentation in the patient’s health record, thus promoting the HIM profession overall.

To explore the business need for and feasibility of developing a new CDI credential, CCHIIM conducted a thorough needs analysis and idea brief outlining the business impact, strategic context (including industry trends and member/customer needs), value proposition, and sustainability of this exam. The commission concluded that the exam would be a natural extension of the American Health Information Management Association (AHIMA) offerings that support clinical documentation improvement, including CDI practice briefs, a CDI tool kit for healthcare organizations and professionals, a practice community, related educational resources, and in 2007, through its House of Delegates, an approved resolution on quality data and documentation in EHRs.^10–12 Additionally, creating a salient credential to validate the clinical documentation role was found to be both reactionary and forward-thinking because it would be a response to market demand from clinical documentation specialists already working in the HIM continuum, but also an opportunity to further expand and welcome complementary healthcare professionals to the HIM arena. This research served to solidify the general scope of a CDI-related credential and justify further exploration of developing this exam.

Methods

A task force composed of 19 CDI SMEs met for two days in May 2011 to create a job analysis survey to be sent to CDI industry practitioners. The SMEs on the task force were selected based on their clinical documentation expertise, as all were currently working in roles focused on clinical documentation improvement, education, and/or medical coding quality. A mix of SMEs, as reflected in Table 1, was chosen to reflect diversity in work setting, geographical location, supervisory level, and gender in order to obtain a representative sample of the specialty as a whole.

The job analysis task force was charged with developing a comprehensive list of knowledge and task statements required of the CDI practitioner role. Additionally, the group had to define the major domains (also known as topics or content areas) that represent the primary job responsibilities or facets of the job. The group determined that the knowledge and task statements would each be mapped to one of the six domains represented in Table 2.

To help define the scope of the related credential, the task force used an initial list of knowledge and task topics prepared in advance by AHIMA staff together with a small team of experienced CDI specialists. The task force then refined this task and knowledge list and supplemented it with their own insights based on their shared experience on the job. Additionally, the group developed “future topics” to identify potential developmental areas and predicted future job requirements for the CDI field as it continues to evolve. These included tasks that CDI practitioners may not be presently engaged in but will likely be asked to perform in the future, and knowledge areas that CDI practitioners will likely need to learn for the future.

These knowledge areas, tasks, and future topics were used to create the job analysis validation survey. In addition to defining the role in terms of the required knowledge and tasks performed on the job both currently and in the future, the task force also created survey scales regarding frequency and importance (listed in Table 3 and Table 4) to be used in the job analysis survey. A discrete, five-point Likert scale was selected to evaluate frequency, with possible response choices of “Never” (1), “Quarterly” (2), “Monthly” (3), “Weekly” (4), and “Daily” (5). A discrete, three-point Likert scale was used for the importance ratings, with the possible responses of “Not Important” (1), “Somewhat Important” (2), and “Very Important” (3). The task force members selected these rating scales because they felt that they best approximated the rate of occurrence and general importance levels relative to the job.

In order to find the appropriate group of practitioners to survey, random sampling of a targeted sector of the AHIMA membership database was conducted. To meet the criteria for inclusion in the survey, individuals had to be in one of four roles, practice in one of three clinical settings, and have at least one of three credentials, as shown in Table 5. At the time, 12,914 individuals in the AHIMA membership database met those requirements. A random number generator randomly assigned each member a number from 1 to 12,914, with replacement. In the sample, 4,923 candidates received numbers below 5,000 and were included in the survey. Based on the criteria of clinical setting, supervisory level, RHIA and RHIT certification, CCS certification, and RN registration, the sample selected was within 1.5 percent of the distribution of members for each criterion.

Survey invitations were e-mailed to the 4,923 potential respondents on Friday, June 24, 2011, and the survey closed at midnight on Tuesday, July 12, 2011. The response rate was 14.7 percent, with 733 respondents completing the survey and demographic questions. The sampling error was +/- 1.1 percent at the 95 percent confidence level.

In July, the job analysis task force reconvened to review the survey results. The original weightings given in their preliminary exam blueprint were compared to the weights resulting from the job analysis validation survey. To reconcile the two, the task force voted for the target weights for each content area within the knowledge and task domains. The percentage weighting of each domain was determined based on the aggregate importance and frequency ratings given to each domain. The domains that contained tasks and knowledge statements rated as more important or more frequently performed received higher percentage weights.

For each of the target weights, a range of +/- 2 percent was calculated to create the maximum and minimum percentages for each domain. These maximum and minimum percentage weightings became the weightings for the final exam blueprint and determined the total number of test items included in each domain. A percentage range, as opposed to an absolute percentage, was created to allow for variance between preliminary blueprint expectations and survey responses, serving as a buffer for the margin of error. Additionally, the maximum and minimum domain percentages allowed for some leeway to slightly adjust weightings by topic area as necessary based on industry changes.

Results

The final domain weightings, including the maximum and minimum percentage ranges, preliminary weightings, survey weightings, and target weightings, are shown in Table 6. The target weighting was determined by the task force after comparing the survey data with the original preliminary blueprint.

Table 7 and Table 8 reflect the frequency and importance survey ratings for each task and knowledge statement ranked from highest to lowest in each domain. The weighted average of each task and knowledge rating was calculated from the aggregate survey responses. Because the frequency ratings used a scale of 1 to 5 and the importance ratings used a scale of 1 to 3, a scaling factor of 1.667 was used to multiply the importance rating so that its weight would be equal to that of the frequency rating. These corrected mean frequency and importance ratings were used to rank the tasks and knowledge statements within their domains and were also used to calculate the weight for each domain.

The Record Review & Document Clarification and Clinical & Coding Practice domains received the highest target weightings on the exam blueprint (26 percent and 24 percent respectively) because they had the greatest number of task or knowledge items that also had the highest frequency and importance weightings based on the survey responses. Because these areas make up the greatest proportion of the work done on the job and the knowledge required to complete those tasks, they form the largest proportion of the exam. Conversely, the Compliance domain has the smallest overall target weighting on the exam blueprint (6 percent) because it had fewer task or knowledge items, which also had the lowest frequency and importance ratings.

Table 9 and Table 10 depict the survey ratings for the “future” task and knowledge topics included in the survey. The data show that the majority of survey respondents felt that all of the future knowledge topics would be needed in the short term (within six months to one year), with the knowledge areas related to electronic health records (EHRs) being the most highly rated. The future task topic data show how many respondents were already performing each task, how frequently they perform it, and how important they rate it. Those who indicated that they do not currently perform a task were asked when they expect themselves or their organization to perform the task. Respondents were also asked to rate under which domain they felt the future task belonged. The data show that 10 to 40 percent of respondents were already performing one or more of the future tasks, while the majority of those who were not performing the tasks indicated they would either begin in the next six months to one year or would never perform that task.

Finally, the survey respondent demographics are represented in Tables 11, 12, 13, 14, 15, 16, 17, 18, 19, and 20. Respondents’ geographic area, work setting, practice setting, facility size, type of health record system, employee status, department, job title, and age were all captured to ascertain the representativeness of the sample. All demographic characteristics were appropriately distributed, as they closely match the population’s demographic profile.

Discussion

The opinions and experience of a representative sample of CDI specialists was obtained through the job analysis process to build a solid, legally defensible foundation for the CDIP credential based on job-related competency. This foundation takes shape in the exam blueprint, which outlines the main content domains tested on the exam. The weighting for each domain proportionately reflects the major components of the CDI practitioner job role. By following job analysis and test development best-practice methodology, CCHIIM was able to codify the clinical documentation improvement specialty by defining the critical factors of the job role and developing a standardized tool used to evaluate CDI practice competency. This credential will strengthen the CDI role by instilling employer confidence in CDIP-credentialed individuals who have met measured, defined, and validated professional standards.

Additionally, the job analysis will help provide direction for the specialty as it continues to grow. The job analysis included measurement of both current and future task and knowledge statements to track how the CDI practitioner role may evolve and what knowledge and abilities will be required of these workers as they grow in their roles. These “future” topics will be monitored and reevaluated in the next job analysis (typically conducted every three to five years, or sooner if the specialty undergoes an extreme transformation) to determine what adjustments should be made to the CDIP exam blueprint to best represent the profession.¹³

Numerous steps to minimize job analysis survey bias were taken. Survey incentives (such as the award of one continuing education unit [CEU] and an entry into an American Express gift card drawing) were offered to limit nonresponse bias and increase the response rate. Additionally, e-mailed survey reminders were sent in order to reach as many respondents as possible. Undercoverage bias was also avoided by ensuring that the demographic composition of the sample mirrored that of the population. The distribution of respondents meeting the parameters of the population (credentials, work setting, and job role) showed no significant difference in demographics when compared to the sample cohort as a whole. Therefore, neither undercoverage nor nonresponse bias was found to be a significant problem in the sample.

As Watzlaf, Rudman, Hart-Hester, and Ren noted in their 2009 article, the roles and job functions of HIM professionals are continuously changing and becoming more specialized.¹⁴ New specializations continue to emerge because of a variety of regulatory and environmental factors, and the new specializations in turn increase the need to certify individuals working in these nontraditional roles to ensure the integrity and quality of their work. HIM certification bodies must stay on top of these trends in order to provide meaningful professional guidelines and standards of excellence for these growing fields. As the CDI role and the entire HIM industry evolve, CCHIIM will continue to routinely examine job roles and functions and update the requisite body of knowledge and competency required for HIM excellence through job analyses and exam blueprint updates.

Limitations

While care was taken to ensure representativeness of the sample and obtain a satisfactory response rate, the study has some limitations. Because the population and resulting sample were drawn from the AHIMA membership database because of financial constraints and other factors, the survey results could have possibly been strengthened by casting a wider net and surveying individuals who do CDI work but are not AHIMA members.

Additionally, there is some debate about the use of five-point and three-point scales (as used for frequency and importance in this survey) versus four-point, forced-choice scales in survey research. Some argue for the use of four-point rating scales because they eliminate the tendency toward the middle and force respondents to pick a side, as opposed to a three- or five-point scale that has a “neutral” midpoint. However, four-point scales can force respondents to answer in a way that does not truly reflect their opinions, in cases when respondents may truly be neutral or middle-of-the-road in their opinion of a certain topic.¹⁵ Forcing respondents to give an untrue answer will unnecessarily skew results. These reasons led to the decision to use three- and five-point survey scales. Respondents were also given the opportunity to write in any comments they had about their ratings or the survey questions for each domain.

Conclusion

To fill an industry need for a validated professional standard of CDI excellence, CCHIIM explored the possibility of creating a new CDI credential for this growing field. To do so, a job analysis was conducted to thoroughly yet concisely define the requisite tasks and knowledge areas for the CDI practitioner role. The job analysis data were used to develop the CDIP exam blueprint in accordance with test development best-practice methodology, in that the domain weightings were determined based on SME rankings of task or knowledge criticality and frequency. Because validated, job-specific content is the crux of the CDIP exam, those who list the CDIP credential after their name have proven their competency and expertise related to the codified CDI body of knowledge. As a result, the HIM field in its entirety is strengthened by having a defined, measurable, and future-thinking measure of proficiency related to ensuring the quality of patient health information.

Jessica Ryan, MA, is a learning specialist at the University of Chicago Medical Center in Chicago, IL.

Karen Patena, MBA, RHIA, FAHIMA, is a clinical associate professor and director of health information management programs at the University of Illinois at Chicago in Chicago, IL.

Wallace Judd, PhD, is a psychometrician at Authentic Testing, Inc., in Gaithersburg, MD.

Mike Niederpruem, MS, MA, CAE, is the director of education and research at the Dental Auxiliary Learning and Education Foundation in Chicago, IL.

Notes

1. Wang, N., D. Schnipke, and E. A. Witt. “Use of Knowledge, Skill, and Ability Statements in Developing Licensure and Certification Examinations.” Educational Measurement: Issues and Practice 24 (2005): 15–22. Available at doi:10.1111/j.1745-3992.2005.00003.x.
2. Mehrens, William A., and W. James Popham. “How to Evaluate the Legal Defensibility of High-Stakes Tests.” Applied Measurement in Education 5, no. 3 (1992): 265–83.
3. Chinn, Roberta N., and Norman R. Hertz. Job Analysis: A Guide for Credentialing Organizations. Lexington, KY: Council on Licensure, Enforcement, and Regulation (CLEAR), 2010, p. 14.
4. National Commission for Certifying Agencies. Standards for the Accreditation of Certification Programs. Washington, DC: Institute for Credentialing Excellence, 2004.
5. American National Standards Institute. Guidance on Psychometric Requirements for ANSI Accreditation (Public Guidance No. PCAC-GI-502). 2009. https://www.ansica.org/wwwversion2/outside/ALLviewDoc.asp?dorID=62&menuID=2.
6. Mehrens, William A., and W. James Popham. “How to Evaluate the Legal Defensibility of High-Stakes Tests.”
7. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association, 1999, p. 161.
8. Chinn, Roberta N., and Norman R. Hertz. Job Analysis: A Guide for Credentialing Organizations, p. 14.
9. Rudman,William J., John S. Eberhardt III, William Pierce, and Susan Hart-Hester. “Healthcare Fraud and Abuse.” Perspectives in Health Information Management 6 (Fall 2009).
10. AHIMA. “Guidance for Clinical Documentation Improvement Programs.” Journal of AHIMA 81, no. 5 (May 2010): expanded web version. Available at http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_047343.hcsp?dDocName=bok1_047343.
11. AHIMA. “Clinical Documentation Improvement Toolkit.” 2010. Available at http://library.ahima.org/xpedio/groups/ public/documents/ahima/bok1_047236.pdf.
12. AHIMA Physician Practice Council. “Resolution on Quality Data and Documentation in the EHR.” FORE Library: HIM Body of Knowledge (2007).  http://library.ahima.org/xpedio/idcplg?IdcService=GET_HIGHLIGHT_INFO&QueryText=%28Resolution+on+Quality+Data+and+Documentation+in+the+EHR%29++%3CAND%3E++%28xPublishSite%3Csubstring%3E%60BoK%60%29&SortField=xPubDate&SortOrder=Desc&dDocName=bok1_035781&HighlightType=HtmlHighlight&dWebExtension=hcsp
13. Raymond, Mark R. “Job Analysis and the Specification of Content for Licensure and Certification Examinations.” Applied Measurement in Education 14, no. 4 (2001): 369–415. Available at doi:10.1207/S15324818AME1404_4.
14. Watzlaf, Valerie J. M., William J. Rudman, Susan Hart-Hester, and Ping Ren. “The Progression of the Roles and Functions of HIM Professionals: A Look into the Past, Present, and Future.” Perspectives in Health Information Management 6 (Summer 2009). Available at http://perspectives.ahima.org/the-progression-of-the-roles-and-functions-of-him-professionals-a-look-into-the-past-present-and-future/.
15. Young, Scott A., and Karen M. Barbera. “Content Essentials: A Primer in Survey Development.” Valtera, 2010. Available at http://info.valtera.com/bid/107384/Content-Essentials-A-Primer-in-Survey-Development.

Printer friendly version of this article.

Jessica Ryan, MA; Karen Patena, MBA, RHIA, FAHIMA; Wallace Judd, PhD; and Mike Niederpruem, MS, MA, CAE. “Validating Competence: A New Credential for Clinical Documentation Improvement Practitioners.” Perspectives in Health Information Management (Spring 2013): 1-38.

Abstract

This study investigated the main factors affecting physicians’ attitudes toward the implementation of international classification systems of diseases. A cross-sectional study was carried out during September 2010. The sample consisted of 158 physicians older than 24 years who were working in a public hospital and a private hospital in central Greece. A questionnaire was drawn up based on the relevant literature. Results indicated that younger physicians and those who worked in the public hospital were most familiar with classification systems. Female physicians and specialists with more than 10 years of experience (since qualifying as a specialist) were not particularly familiar with these systems (58 percent and 56 percent, respectively). Both having a master’s degree and attending conferences or seminars had a remarkable impact on knowledge of these systems. Almost all physicians (98 percent) holding a master’s degree or a PhD believed that these systems contribute to the compilation of valid statistical data. The majority of physicians would like to use these systems in the future, as long as they are provided with the appropriate training.

Keywords: International Classification of Diseases, Clinical Modification, knowledge, health statistics, classification system

Introduction

The development of computer technology has substantially increased the need for communication among the various healthcare professions, and as a result, it has become essential to use a common language. This need was the point of departure for the process of classifying the individual terms that make up medical terminologies. As William Farr stated in 1856, “Classification is a method of generalization. Several classifications may, therefore, be used with advantage; and the physician, the pathologist, or the jurist, each from his own point of view, may legitimately classify the diseases and the causes of death in the way that he thinks best, adapted to facilitate his inquiries and to yield general results.”¹

Classification denotes the process of arranging concepts or objects in groups in a standardized fashion, with clear and specific criteria for the use (or absence) of a code for each specific unit. In most healthcare classifications, codes are arranged in groups so that selecting an individual code will predetermine prognosis, survival, or some other factor.²

A classification system of diseases must consist of a restricted number of categories that are mutually exclusive and capable of covering the entire range of ailments. Any specific pathological entity that occurs frequently and is of special significance for public health must have its own separate category. Each disease or ailment must have its own clearly predetermined place in a series of categories.³

The most widely disseminated classification systems are the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10), the International Classification of Primary Care, Second Edition (ICPC-2), and the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM), followed by the European Diagnostic Manufacturers Association (EDMA) classification, the Global Medical Device Nomenclature (GMDN), and the International Classification of Health Interventions (ICHI).

Each classification serves different purposes. ICD-10 was developed to allow for systematic analysis of records, as well as interpretation and comparison of mortality and morbidity data.⁴ It provides a codification of illnesses, symptoms, and abnormal findings and of social circumstances and external causes leading to injury and illness, as classified by the World Health Organization, using an alphanumeric code consisting of a letter at the beginning and numbers in the second, third, and fourth places.⁵ The main goal of the ICPC-2 is the collection and analysis of patient data and clinical activity in the areas of general and family medicine and primary healthcare.⁶ ICPC-2 is a biaxial classification system consisting of 17 chapters within one axis and seven components in the other. The chapters correspond to capital letters of the Latin alphabet, and the components are the same for each chapter, comprising terms, or the classification rubrics, which are represented by two-digit numbers from 01 through 99.⁷ The ICD-10-CM classification is a clinical modification of ICD-10, and it is used to classify diagnoses, the reasons for visits in all areas of healthcare, and ailments leading to surgical intervention.⁸ Its code consists of three to seven characters, wherein the first is alpha, the second is numeric, and the third, fourth, fifth, sixth, and seventh can be alpha or numeric.⁹

The ΕDMA classification was introduced in order to classify in vitro diagnostic products and correlates a code to each type of product or category of products. According to the EDMA codification, each term is characterized by a code configured as “AA.BB.CC.DD”, in which the combination “AA” stands for the category, “BB” for the group, “CC” for the subgroup, and “DD” for the serial number within the hierarchical structure.¹⁰ The GMDN classification system is a unified taxonomy for medical devices. Today it is the most modern and widely accepted nomenclature of its kind. Its codification system consists of five digits, and its classification structure consists of three basic levels within the nomenclature structure.¹¹ Finally, the aim of the International Classification of Health Interventions (ICHI)is to provide a clearer and more complete record of medical practice no matter where it is carried out, whether in or out of a hospital setting and whether in the public or private sector. The International Classifications of Health Interventions (ICHI) consists of a catalog of medical terms corresponding to specific codes describing medical practices. The medical practice catalog is divided into categories and subcategories according to predetermined rules.¹²

The applications of an integral classification and codification system for medical data cover a wide area. Classifications and codifications are applied in clinical practice, epidemiology, and research, where they are linked to the patient’s electronic health record. These systems can also be applied to the codification and classification of primary healthcare services as well as to specialized medical practice. Furthermore, codification and classification systems are applied in the area of healthcare in general, with respect not only to the care provided by physicians, but also to that provided by nurses and other healthcare providers, to which the appropriate classification can and must be applied.¹³

Physicians are called upon to choose from among many different classification systems, each of which are used for different reasons, and our study examines the factors influencing physicians’ attitudes in this respect. Some studies have been carried out in the past with respect to the various choices made by physicians. It is important to note that only a few similar studies were found in the literature, and moreover these studies concerned the codifications for psychiatric diseases.

As for the reasons why physicians choose to use specific systems, studies can be cited. For example, in research carried out in 1994 on a sample of 653 French psychiatrists, it was observed, in a comparison of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R; not investigated in our study) with ICD-10, that the former was used primarily in scientific studies while the latter was applied more often to patient caregiving.¹⁴ Another study, undertaken from 1993 to 1995 in 10 different countries (six in Europe, two in Asia, one in North Africa, and one in Latin America) on a sample of physicians from 19 psychiatric wards, with the aim of defining the factors influencing the use of the ICD-10 system, pointed to 10 categories that represented 40 percent of all major psychiatric diagnoses, while 32 other categories were never used at all. Because “undefined” categories were also found to have been used, this study showed that the degree of difficulty inherent in a system has an influence on whether it is likely to be used by physicians. This finding emphasized the need for enhanced training as well as for a revision of the system with a view to making it more user friendly.¹⁵

Finally, in another study carried out in the year 2000 in 66 countries, the factors likely to influence the choice of various classification systems were examined through the use of a questionnaire. The sample consisted of 205 individuals (51 psychiatrists from the United States, 89 from Europe, 26 from Africa and the Middle East, and 39 from Asia and the South Pacific). It was observed that the diagnostic system used most often were ICD-10 and its modifications (86 percent of the psychiatrists applied it to clinical treatment, 72 percent applied it to training, and 63 percent applied it to research). The International Classification of Diseases, Ninth Revision (ICD-9) was applied above all to clinical treatment, according to 27 percent of respondents. Other diagnostic systems were used less often and were used primarily for clinical treatment (51 percent), research (78 percent), and training (60 percent) purposes.¹⁶

In Greece, a new system was implemented in 2011 concerning the reimbursement of hospitals based on diagnosis-related groups (DRGs). The reimbursement per patient is determined according to the ICD-10 category of the disease and the type of medical intervention. According to an encyclical letter edited by the General Secretary of the Ministry of Health, in 2011 each hospital was obliged to record, both in the discharge note and on the invoice sent to the social security funds, the discharge diagnosis according to the ICD-10 system. Therefore, it is essential for physicians to be familiar with the use of the ICD-10 system in order to classify (in collaboration with administrative personnel) each patient encounter to the appropriate DRG.

Moreover, these classification systems play a significant role in evidence-based health policy because they make it possible to map the demand side of healthcare in a consistent way. By implementing these systems, a country gains the ability to report reliable statistics about discharges in each disease category and also about the prevalence of diseases in the entire country. If each patient’s diagnosis is classified into an ICD-10 category, then it is easy for a national statistical authority to create a reliable epidemiological database that can be used for national and international diachronic comparisons. Keeping these data in mind, policy makers may set the priorities of the healthcare sector in a systematic way.

Methods

In order to examine the attitudes of physicians working in central Greece with respect to the use of international classification systems for diseases and the clinical modification ICD-10-CM, a questionnaire was drawn up. (ICD-10-CM was included in the study because the study was carried out in September 2010. In 2011, the Ministry of Health mandated the use of ICD-10.) The sample selection procedure chosen was convenience sampling, a type of nonprobability sampling according to which each member of a population is not equally likely to be included in the sample and is included in the study when he or she happens to be “in the right place at the right time.”¹⁷ The questionnaire was addressed to physicians over 24 years of age working at a public or private hospital in central Greece. This area was chosen because it includes the country’s only university department of biomedical informatics. Because of this, it was supposed that physicians in this area would be much more familiar with these classification systems than physicians in other areas of the country. The content validity of the questionnaire was tested on the basis of a pilot study in which physicians at a general hospital (Ν = 28) responded to the questionnaire. The internal consistency of the questionnaire was assessed by using the Cronbach reliability coefficient, which was found to be 0.7. After the pilot study was completed, the main study was carried out at the aforementioned healthcare units during September 2010, with a response rate of 76 percent (158 out of 207). The questionnaire included two parts. The first part was related to the international classification systems (ICD-10, ICD-10-CM, ICPC-2, and ICD-9, which was included because, as the former version of ICD-10, it was expected to be widely known) and examined the respondents’ knowledge about the classification systems, information sources, knowledge of existing Greek translations of the systems, and the use (or non-use) of classification systems, as well as the reasons for using them or not using them. The second part dealt with the demographic and social aspects of the study population.

The questionnaire consisted mainly of closed-ended questions, in which the respondents chose from a list of possible answer choices. However, it also included open-ended questions in which respondents provided their own answer (e.g., their area of specialization). Continuous variables are presented as mean (± standard deviation), while categorical variables are given as absolute and relative frequencies. The normality of continuous variables was checked with the Kolmogorov-Smirnov test and histograms. Statistical analysis included the chi-square test, the t-test for independent samples, the Mann-Whitney test, and multivariate logistic regression analysis. In the case of logistic regression, the dependent variables were knowledge of classification systems and future use of these systems, while the model applied was that of stepwise regression with backward elimination of variables. In logistic regression models, we calculated odds ratios, 95 percent confidence intervals for odds ratios, and p-values. Statistical significance was determined to be equal to 0.05. Data analysis was performed with SPSS version 18.0.

Results

Table 1 shows the demographic characteristics of the sample. The mean age was 43 years (±10.7). The mean numbers of Greek and international conferences attended were 8 (±5.7) and 4 (±3.7), respectively.

Table 2 and Table 3 show data with respect to the physicians’ knowledge of the classification systems as well as their use thereof, while Table 4 cites the statistically significant factors influencing knowledge of classification systems in correlation with their demographic and social characteristics. Table 2 also includes the results concerning the knowledge of physicians about the translations into Greek of the ICD-10 and ICPC-2 systems conducted by the Ministry of Health.

The majority of physicians stated that they believe that classification systems of diseases contribute to drawing up valid statistical healthcare data on a national level, and that provided they were given appropriate training, they would like to use one of the existing classification systems in the future. Furthermore, most of these physicians were not using these classification systems because they believed they were difficult to apply.

In bivariate analysis, we found that physicians who worked in the public hospital (χ² = 7, p = 0.008), younger physicians (t = 4.9, p < 0.001), male physicians (χ² = 2.856, p = 0.091), specialists (χ² = 18.2, p < 0.001), physicians with an additional postgraduate degree (χ² = 9, p = 0.003), and physicians who had attended a statistically significant higher number of Greek and international conferences (U = 1,700, p < 0.001, and U = 2,204, p = 0.001, respectively) had greater knowledge of classification systems (Table 4).

Table 5 shows that physicians in public hospitals (χ² = 5.823, p = 0.016), younger physicians (t = 3.7, p < 0.001), specialized physicians (χ² = 10.6, p = 0.005), physicians with an additional postgraduate degree (χ² = 11.3, p = 0.001), and physicians having attended a statistically significant higher number of Greek conferences (U = 972, p = 0.002) believed that classification systems for diseases help them draw up valid statistical healthcare data.

Table 6 shows that younger physicians (t = 4.6, p < 0.001), physicians with an additional postgraduate degree (χ² = 2.9, p = 0.09), and physicians having attended a statistically significant higher number of Greek conferences (U = 1,270, p = 0.006) were those who would most like to apply classification systems in the future.

Table 7 and Table 8 present the results of multivariate logistic regression. In Table 7 and Table 8, the dependent variable was the knowledge of classification systems (yes = 1, no = 0) and the future use of these systems (yes = 1, no = 0), respectively. The independent variable found to be statistically significant was, in both cases, age, which explained 44 percent and 33.1 percent of the variability of the dependent variable (R² = 0.44 and R² = 0.33), respectively. Younger physicians had greater knowledge of classification systems and would most like to apply classification systems in the future.

Discussion

The present study confirms that of the population studied, 48 percent of physicians know nothing whatsoever about any of the classification systems. Of those who are aware of at least one, most physicians know of ICD-10 (42 percent). Of the physicians who are familiar with classification systems, most of them were informed about these during the course of their academic training (33 percent), followed by those who informed themselves at conferences, seminars, and workshops (28 percent).

Furthermore, of all physicians responding to the questionnaire, 85 percent answered that they believed that classification systems for diseases contribute to the drawing up of valid statistical data in the healthcare sector on a national level. Concerning the demographic characteristics of physicians who declared this attitude, it is interesting that they are mainly physicians working in public hospitals, younger physicians, specialized physicians, physicians with an additional postgraduate degree (master’s degree and/or PhD), and physicians having attended a statistically significant higher number of Greek conferences.

Finally, we observed that the overwhelming majority of physicians (88 percent) are not aware of the fact that the Greek Ministry of Health has translated ICD-10 and ICPC-2 into Greek.

Most of the sample physicians (93 percent) do not use a classification system, and in most cases (49 percent) this is because they believe the systems are difficult to apply. This belief may be related to the fact that 48.1 percent of the questioned physicians declared that they had no information about these systems. On the other hand, only a small percentage (5 percent) believed that the classifications serve no purpose. Of the physicians using no classification system, 78.4 percent replied that they would like to use a classification system in the future if they were provided with appropriate training.

Furthermore, of the 10 physicians who stated that they used one of these systems, most of them used it for research purposes (90 percent), followed by patient treatment and training (40 percent each). These results correspond to those of similar studies. One study, which was carried out in 1994 in France with a sample of 653 psychiatrists, reports that classification systems were used mainly for the purpose of research studies and treatment of patients.¹⁸ In another study, carried out in the year 2000 on a sample of psychiatrists from a total of 66 countries, it was found that the majority of physicians used these systems for research and training purposes, followed by clinical healthcare with a much smaller percentage.¹⁹

Finally, a remarkable percentage of physicians questioned who actually use a classification system stated that they know the codification rules of the ICD-10 classification very well (40 percent), while the same percentage of this group stated that they know very little about ICPC-2 classification. Finally, 70 percent of the physicians who use classification systems would like the translations of classification terms to be improved in the future in order to make them easier to use.

In general, the determining factor most likely connected to the attitude of healthcare professionals with respect to international classification models for diseases and surgical interventions seem to be directly linked to age. Increased age of physicians was associated with decreased knowledge of these systems and decreased intention to use them in the future.

The fact, that at present, approximately half of the sample physicians know nothing about these classifications systems can be considered an obstacle both for the appropriate implementation of the DRG reimbursement system and for the development of a reliable epidemiological database. Consequently, these facts present problems for health policymaking.

On the other hand, we found that most of the sample physicians would like to use one of the existing classification systems in the future, provided that they are granted access to appropriate training. In addition, we observed that older physicians would prefer not to use classification systems, whereas physicians who had attended a fair number of conferences are interested in doing so. These observations point toward the need for additional training and education of physicians on the benefits and use of disease classification systems.

Vasiliki Tsikna is agraduate of the Department of Computer Science and Biomedical Informatics of the University of Central Greece in Lamia, Greece.

Olga Siskou, RN, MSc, PhD, is a senior researcher in health economics and nursing management at the Center for Health Services Management and Evaluation in the Department of Nursing at the National and Kapodistrian University of Athens in Athens, Greece.

Petros Galanis, RN, MSc, PhD, is a senior researcher in public health at the Center for Health Services Management and Evaluation in the Nursing Department at the National and Kapodistrian University of Athens in Athens, Greece.

Panagiotis Prezerakos, RN, MSc, PhD, is an assistant professor of nursing administration in the Department of Nursing at the University of Peloponnese in Sparta, Greece.

Daphne Kaitelidou, RN, MSc, PhD, is an assistant professor of nursing administration in the Department of Nursing at the National and Kapodistrian University of Athens.

Notes

1. Quoted in Bowman, Sue. “ICD-10: All in the Family.” Journal of AHIMA 75, no. 10 (2004): 62–63. Available at http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_025109.hcsp?dDocName=bok1_025109.

2. Lionis, C. “Introduction to Classification of Medical Data, Sub-project: Classification of Primary Health Care According to ICPC-2” [in Greek]. University of Crete Department of Medicine, 2009.

3. ICD-10: International Statistical Classification of Diseases and Related Health Problems. 2nd ed. Geneva: World Health Organization, 2004. Source: Greece’s Ministry of Health and Social Solidarity, http://www.yyka.gov.gr (accessed June 2010).

4. Bowman, Sue. “ICD-10: All in the Family.”

5. ICD-10: International Statistical Classification of Diseases and Related Health Problems. 2nd ed.

6. University of Sidney, Family Medicine Research Centre. “ICPC-2: International Classification for Primary Care.” Available at http://www.fmrc.org.au/icpc2/.

7. Lionis, C. “Introduction to Classification of Medical Data, Sub-project: Classification of Primary Health Care According to ICPC-2” [in Greek].

8. Delmar Cengage Learning. ICD-10-CM: Diagnostic Coding for the Future. Available at http://www.delmarlearning.com/companions/content/1435448243/student_resources/ICD-10-CM_OLC.pdf.

9. Centers for Medicare and Medicaid Services. Quick Reference Information: ICD-10-CM Classification Enhancements. January 2010. Available at https://www.cms.gov/ICD10/Downloads/ICD-10QuickRefer.pdf.

10. Classification of In Vitro Diagnostics, European Diagnostic Manufacturers Association (EDMA) Product Classification Usage Guide. Source: Greece’s Ministry of Health and Social Solidarity, http://www.yyka.gov.gr/ (accessed December 2010).

11. International Nomenclature of Biomedical Devices, Global Medical Device Nomenclature (GMDN) Usage Guide. Source: Greece’s Ministry of Health and Social Solidarity, http://www.yyka.gov.gr/ (accessed January 2011).

12. International Classification of Health Interventions Usage Guide. Source: Greece’s Ministry of Health and Social Solidarity, http://www.yyka.gov.gr/ (accessed January 2011).

13. Lionis, C. “Introduction to Classification of Medical Data, Sub-project: Classification of Primary Health Care According to ICPC-2” [in Greek].

14. Sechter, D. “Survey of the Use of International Classification (DSM III-R—ICD-10) in France, in Private and Public Psychiatry” [in French]. L’Encéphale 21, spec. no. 5 (1995): 35–38.

15. Müssigbrodt, H., R. Michels, C. P. Malchow, H. Dilling, P. Munk-Jørgensen, and A. Bertelsen. “Use of the ICD-10 Classification in Psychiatry: An International Survey.” Psychopathology 33, no. 2 (2000): 94–99.

16. Mezzich, J. E. “International Surveys on the Use of ICD-10 and Related Diagnostic Systems.” Psychopathology 35, nos. 2–3 (2002): 72–75.

17. Merkouris, A. “Phase of Planning: Programming” [in Greek]. In Methodology of Nursing Research. Athens: Hellin, 2008, 79–138.

18. Sechter, D. “Survey of the Use of International Classification (DSM III-R—ICD-10) in France, in Private and Public Psychiatry” [in French].

19. Müssigbrodt, H., R. Michels, C. P. Malchow, H. Dilling, P. Munk-Jørgensen, and A. Bertelsen. “Use of the ICD-10 Classification in Psychiatry: An International Survey.”

Printer friendly version of this article.

Vasiliki Tsikna; Olga Siskou, RN, MSc, PhD; Petros Galanis, RN, MSc, PhD; Panagiotis Prezerakos, RN, MSc, PhD; Daphne Kaitelidou, RN, MSc, PhD. “Investigation of Physicians’ Attitudes Concerning the Implementation of International Classification Systems of Diseases as a Precondition for Evidence-based Policy Making.” Perspectives in Health Information Management (Spring 2013): 1-15.

Abstract

The International Statistical Classification of Diseases and Related Health Problems (ICD) has undergone a long evolution from its initial inception in the late 18th century. Today, ICD is the internationally recognized classification that helps clinicians, policy makers, and patients to navigate, understand, and compare healthcare systems and services. Currently in the United States, hot debates surround the transition from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) to the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). This article presents an analysis of the views of the proponents and opponents of the upcoming change. We also briefly present and analyze the quality of the most frequently cited scientific evidence that underpins the recent debates focusing on two major issues: ICD-10-CM implementation costs and revenue gains and the projected clinical data quality improvement. We conclude with policy and research suggestions for healthcare stakeholders.

Background

The International Statistical Classification of Diseases and Related Health Problems (ICD) has a long history of development and refinement that can be traced back to the French physician J. Bertillon, who introduced the Bertillon Classification of Causes of Death in 1893.¹ In 1946, the United Nations delegated the responsibility for the ICD to the World Health Organization (WHO), which conducts and issues periodical revisions of the ICD.

The International Classification of Diseases, Ninth Revision (ICD-9) was designed in the late 1970s and was adopted by many countries around the world during the 1980s. Although this version was more detailed and interprofessional than the previous versions of the ICD, it did not meet the clinical needs of providers and payers in the United States. To make the application of ICD-9 appropriate to the American healthcare settings, the National Center for Health Statistics (NCHS) and the Council on Clinical Classifications jointly created the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). Since late 1970s, ICD-9-CM has been the required standard for billing and clinical purposes by the most payers (such as the Centers for Medicare and Medicaid Services [CMS]) in the United States.^{2, 3}

During the creation of ICD-9, WHO leaders realized that even bigger classification changes would need to be implemented in the near future. To address this issue, development of the 10th revision of ICD was initiated even before the ninth version was completed.^{4, 5} The WHO Collaborating Centers for the Classification of Diseases experimented with different models and structures for ICD-10. Multiple international users’ and developers’ appeals and requests postponed the publication of ICD-10 from 1985 to 1989. During this extended time, the WHO implemented changes and further developed ICD-10. As a result of this work, ICD-10, published in 1990, included significantly more codes and categories: while ICD-9 had only about 17,000 codes, ICD-10 included more than 155,000 codes tracking a significant number of new diagnoses. ICD-10 was swiftly adopted and implemented by many international healthcare users.

In the United States, the National Center for Health Statistics (NCHS) is charged with developing and updating the ICD.⁶ After the WHO authorization in the middle of 1990s, the NCHS went through a long, multistep process of adapting ICD-10 to American healthcare needs and settings. First, the NCHS released the revised version of ICD-10 for the public comments in 1998. Then, in summer 2003, ICD-10 was field tested by the American Hospital Association and the American Health Information Management Association (AHIMA).^{7, 8} Finally, public suggestions and the field-test results were implemented to create an updated version, known as the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). According to the NCHS, the current “clinical modification represents a significant improvement over ICD-9-CM and ICD-10.”⁹ The specific improvements include the creation of the diagnosis/symptom combinations that reduce the number of codes required to describe a certain medical condition, additional information relevant to the description of managed care and ambulatory encounters, and greater specificity in codes. The NCHS conducts an annual update of ICD-10-CM, and the last update was released in December 2012.¹⁰

After long discussions and debates, the US Department of Health and Human Services (HHS) published a final rule requiring the use of ICD-10-CM to replace ICD-9-CM in the Health Insurance Portability and Accountability Act (HIPAA) electronic transaction standards.¹¹ Initially, the transfer to ICD-10-CM was scheduled to happen in 2011,¹² but it was postponed by CMS to October 1, 2013.¹³ Most recently, HHS proposed that this date be set back one year from the postponed date, to October 1, 2014.¹⁴ Debates about adoption continue. The following sections provide an analysis of the views of the proponents and opponents of the upcoming change.

It is important to note that in the United States, ICD-10 consists of two parts: ICD-10-CM for diagnosis coding and the International Classification of Diseases, Tenth Revision, Procedure Coding System (ICD-10-PCS) for inpatient procedure coding. According to the HHS requirements, ICD-10-CM will become a standard for all US healthcare settings, whereas ICD-10-PCS will be required in inpatient settings only. This article mainly focuses on ICD-10-CM to enable a broader overview of the recent debates.

ICD-10-CM: Implementation Costs and Revenue Gains

It is challenging to estimate the possible costs of the transition to ICD-10-CM. These costs will probably vary widely for different organizations, settings, and providers. However, several rough estimates were recently made. For example, according to general estimates by CMS in 2009,¹⁵ the costs of the ICD-10-CM implementation will be about 0.03 percent of revenue for inpatient and outpatient healthcare settings. Other anecdotal estimates suggest that these costs will be much higher, according to some reports up to approximately $5 million for a large institution (400 or more beds), up to $1.5 million for a medium-size institution (100 to 400 beds) and up to $250,000 for a small institution (100 beds or fewer).¹⁶ Moreover, revenue loss is expected during the transition to the new classification system because of the increased claim denial rates, delays in submission of bills, and increases in claim error rates ranging from 6 to 10 percent.^{17, 18}

According to HHS, the implementation and error costs should subside within a few years.¹⁹ Two years after the implementation, providers’ revenues are expected to increase because of more accurate payments and fewer rejected and improper claims.²⁰ Moreover, the improved disease management that is expected with the help of ICD-10-CM should lead to higher-quality care and improved outcomes.²¹ This trend is especially relevant in light of the possible transition to new, outcome-oriented models of payment such as those used by accountable care organizations and patient-centered medical homes.

In sum, the claims of both the opponents and proponents are based on estimations and predictions, while there is scarce empirical evidence to support either argument. To make financial estimation even more complicated, the US healthcare reimbursement and payment systems are unique, and it is hard to compare them to healthcare systems that have adopted and successfully transitioned to ICD-10 in other countries.

In addition, most of the estimates were made before the creation of requirements for meaningful use that provide financial incentives for healthcare providers to adopt electronic health records (EHRs) and penalize the lack of adoption through an upcoming adjustment of payments by CMS and other payers.²² These recent changes are directly related to the possible costs and revenues estimated for the transition to ICD-10-CM. For example, it is estimated that most healthcare providers will have functional EHR systems in place by 2015,²³ and the costs of transitioning these systems to the new classification might be lower or higher than those expected without EHRs. The suggested revenue estimates might still be relevant but are less straightforward with the widespread adoption of EHRs. For example, it will be challenging to estimate whether the adoption of EHRs or the transition to ICD-10-CM is responsible for increased revenues due to more accurate payments. To conclude, it is difficult to approximate the effect of the ICD-10-CM transition on revenues and costs due to the lack of empirical evidence and scarcity of updated high-quality estimates.

Clinical Data Quality Improvement

Supporters of ICD-10-CM praise its ability to provide a more detailed description of clinical situations and its greater specificity in describing healthcare problems. Similarly to the international version, the overall number of codes and diagnoses has increased significantly from ICD-9-CM (about 17,000 codes) to ICD-10-CM (more than 155,000 codes).^{24, 25} For example, ICD-10-CM includes previously unavailable codes for distinguishing between different types of diabetes and requires providers to document additional important information, such as any underlying condition that caused the diabetes or whether drugs induced the diabetes.^{26, 27} It also enables a more detailed description of the location on the patient’s body, for example left or right limb. This improved level of detail should decrease medical fraud and abuse, for instance by reducing the ability to repeatedly report the same procedure on the same side of the body.²⁸ ICD-10-CM also improves the information coverage for other healthcare disciplines, which should lead to better understanding of healthcare outcomes and processes through improved coding.²⁹ For example, ICD-10-CM codes are focused on human responses to disease, which are more appropriate for nurses, while the previous versions leaned toward disease- or organ-level, physician-oriented content.³⁰

On the other hand, opponents of ICD-10-CM suggest that the claims about its advantages over ICD-9-CM are exaggerated. First, although the number of ICD-10-CM codes and diagnoses increased, the two systems are still very similar.³¹ Indeed, the major increase in ICD-10-CM codes might be attributed to the increased levels of detail about traumatic injuries (about 60 percent of all the codes), while ICD-9-CM devoted only a small percentage (about 15 percent) of codes to these types of injuries. Also, the overall proportion of disease codes decreased from 65 percent (or roughly 8,500 codes) in ICD-9-CM to about 30 percent (about 20,000 codes) in ICD-10-CM.³²

While in the long run ICD-10-CM is expected to decrease medical fraud and abuse, there are concerns that in the transition period, healthcare providers will intentionally or mistakenly misreport codes. This might happen because only a limited number (around 5 percent) of the existing ICD-9-CM codes have a one-to-one match with the ICD-10-CM codes.³³ To avoid severe problems and enable easy mapping between the classifications, CMS has developed special cross-mapping pathways called General Equivalence Mappings (GEMs).³⁴ However, most ICD-9-CM codes are still matched with multiple terms in ICD-10-CM, and there is still room for double billing during the period when the two systems will be activated simultaneously.³⁵

In addition, the opponents of ICD-10-CM claim that throughout the years, the classification has become obsolete and no longer fits the clinical needs of modern healthcare providers; indeed, the classification was designed in the 1980s and therefore lacks codes for more recent medical information. For example, when bilateral prophylactic mastectomy is performed because of the presence of the BRCA2 gene (a genomic variant that significantly increases the risk of breast cancer), there is no option for coding this genomic variant as an indication for surgery.³⁶ Genomics and family history are actually being incorporated into the newest revision (ICD-11), which is expected to be published in 2016. Also, ICD-11 will be somewhat broader in its clinical definitions but will include a straightforward linkage to the Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT), a systematically organized, computer-processable collection of medical terms that is required under the current regulations for the second stage of meaningful use.^{37, 38}

Several scientific studies have compared currently existing classifications. One example often cited when comparing the last two ICD versions is a Canadian study conducted in 2008 by Quan et al.³⁹ The study aimed to assess the validity of administrative and clinical data coded with ICD-10 and to determine whether it offers significant improvements compared to ICD-9-CM. The researchers extracted medical chart data on 32 medical conditions from four teaching hospitals in Canada. The original data were coded using the recently adopted ICD-10 standard, and the researchers hired four coding experts to code the charts with ICD-9-CM. These codes were compared, and the study concluded that the “dually coded database demonstrated that ICD-9-CM and ICD-10 administrative data . . . had similar validity in recording clinical condition information.”⁴⁰

However, several methodological concerns might limit the generalizability of the findings of the study by Quan et al.⁴¹ First, at the time of the study data collection, ICD-10 had only recently been implemented (it had been used for only nine months prior to the study), and it is possible that the coders were not experienced with the new classification. The data were then recoded into ICD-9-CM by experienced experts hired by the study team. Therefore, it is possible that the lack of difference between the two classifications stems from comparing expert coders (using ICD-9-CM) and novice coders (using ICD-10). Moreover, the comparisons were made using data from teaching hospitals in one province in Canada and did not analyze ICD-10-CM. All of these factors weaken the generalizability of the study findings to US settings.

Only one study comparing the most updated versions of ICD-9-CM and ICD-10-CM was identified.⁴² The researchers used expert coders to code 50 clinical notes sampled from four academic medical centers. Similar to the findings of the previous study, this study concluded that the “practical ability of ICD-10-CM to capture content typically contained in clinical records is not measurably better or worse than that of ICD-9-CM.”⁴³ However, the small sample size (50 clinical notes) limits the generalizability of this study’s findings.

To conclude, researchers did not identify significant improvements or deficits between the recent versions (9th and 10th) of the ICD. However, the existing evidence is limited in its generalizability and validity.

Conclusion

Disease classifications have undergone a long evolution from their inception in the late 19th century. Nowadays, ICDs are internationally recognized classifications that help clinicians, policy makers, and patients to navigate, understand, and compare healthcare systems and services. In the United States, ardent debates arose recently around the requirement to transition from ICD-9-CM to ICD-10-CM. Three general sectors are involved in the ICD discussions in the United States: healthcare providers (mostly interested in postponing the implementation of ICD-10-CM because of the financial uncertainties and questionable quality gains), vendors (which will probably gain monetary profits from the implementation of the new classification), and the government (which holds a neutral position and has twice postponed the requirement for the transition to ICD-10-CM for billing purposes).

The first common category of arguments relates to the financial side of the classification change: it is challenging to estimate the possible costs and revenue generated from the transition to ICD-10-CM. The evidence that exists about the estimated costs and revenues also seems to be either anecdotal or somewhat outdated, especially in the light of the recent legislative changes related to meaningful use of EHRs. In order to make strong and valid financial predictions, high-quality evidence with a national scope is desperately needed. This evidence might be obtained by a governmental agency, such as CMS or HHS, or with their funding, as was provided previously.

The second category of arguments relates to the ability of the suggested ICD-10-CM transition to generate higher-quality clinical data that will drive better healthcare management and improve outcomes. Although this argument sounds reasonable, evidence about the actual improvement of healthcare data that is likely to occur with the transition to ICD-10-CM is scarce. Several recent studies comparing the classifications did not identify significant improvements. However, these studies are not without methodological concerns that limit the generalizability of their findings. Therefore, more generalizable research that thoroughly compares and examines the quality improvements between ICD-9-CM and ICD-10-CM is needed. Also, CMS and the NCHS might consider allocating additional resources to facilitate more robust research comparing the quality improvements between the classifications.

ICD-11 is expected to be released by 2016, and at least another year will probably be needed to create an appropriate clinical modification for use in the United States. Until then, it is likely that the switch to ICD-10-CM will continue to be required, but more robust research evidence is needed to facilitate and substantiate the recent debates.

Maxim Topaz, MA, RN, is a PhD student at the University of Pennsylvania’s School of Nursing in Philadelphia, PA.

Leah Shafran-Topaz, PT, is physical therapist at Good Shepherd Penn Partners in Philadelphia, PA.

Kathryn H. Bowles, PhD, RN, FAAN, FACMI, is a professor at the University of Pennsylvania’s School of Nursing in Philadelphia, PA.

Notes

1. World Health Organization. “History of the Development of the ICD.” 2012. Available at http://www.who.int/classifications/icd/en/HistoryOfICD.pdf (accessed August 20, 2012).

2. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext. Clifton Park, NY: Delmar Cengage Learning, 2011.

3. Centers for Disease Control. “International Classification of Diseases-9-CM.” 2007. Available at http://www.cdc.gov/nchs/icd.htm (accessed August 20, 2012).

4. World Health Organization. “History of the Development of the ICD.”

5. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext.

6. Centers for Disease Control. “International Classification of Diseases-9-CM.”

7. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext.

8. World Health Organization. International Statistical Classification of Diseases and Related Health Problems, 10th Revision: Instruction Manual, vol. 2. 2011. Available at http://www.who.int/classifications/icd/ICD10Volume2_en_2010.pdf (accessed August 10, 2012).

9. National Center for Health Statistics. “International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM).” 2012. Available at http://www.cdc.gov/nchs/icd/icd10cm.htm (accessed August 10, 2012).

10. Ibid.

11. US Department of Health and Human Services. “HHS Proposes Adoption of ICD-10 Code Sets and Updated Electronic Transaction Standards.” August 15, 2008. Available at http://www.hhs.gov/news/press/2008pres/08/20080815a.html (accessed August 10, 2012).

12. US Department of Health and Human Services. “HHS Issues Final ICD-10 Code Sets and Updated Electronic Transaction Standards Rules.” January 15, 2009. Available at http://www.hhs.gov/news/press/2009pres/01/20090115f.html (accessed August 10, 2012).

13. US Department of Health and Human Services. “HIPAA Administrative Simplification: Modifications to Medical Data Code Set Standards to Adopt ICD-10-CM and ICD-10-PCS.” 45 CFR Part 162. Federal Register 74, no. 11 (January 16, 2009): 3328.

14. US Department of Health and Human Services. “New Health Care Law Provisions Cut Red Tape, Save up to $4.6 Billion.” April 9, 2012. Available at http://www.hhs.gov/news/press/2012pres/04/20120409a.html (accessed August 20, 2012).

15. US Department of Health and Human Services. “HIPAA Administrative Simplification: Modifications to Medical Data Code Set Standards to Adopt ICD-10-CM and ICD-10-PCS.”

16. Healthcare Information and Management Systems Society. “ICD-10 Transformation: Five Critical Risk-Mitigation Strategies.” 2011. Available at http://www.himss.org/content/files/icd10/G7AdvisoryReport_ICD10%20Version12.pdf (accessed August 20, 2012).

17. Ibid.

18. Minich-Pourshadi, K. “ICD-10 Puts Revenue at Risk.” HealthLeaders Media Intelligence. July 2011. Available at http://content.hcpro.com/pdf/content/268585.pdf (accessed August 20, 2012).

19. US Department of Health and Human Services. “New Health Care Law Provisions Cut Red Tape, Save up to $4.6 Billion.”

20. Ibid.

21. Clark, J. S. “The Facts about ICD-10-CM/PCS Implementation: Implementation Will Improve the Quality of Patient Care.” Journal of AHIMA 83, no. 3 (2012): 42–43.

22. Office of the National Coordinator for Health Information Technology. “Meaningful Use.” 2011. Available at http://healthit.hhs.gov/portal/server.pt/community/healthit_hhs_gov__meaningful_use_announcement/2996 (accessed August 20, 2012).

23. Office of the National Coordinator for Health Information Technology. “Federal Health Information Technology Strategic Plan 2011–2015.” 2011. Available at http://www.healthit.gov/sites/default/files/utility/final-federal-health-it-strategic-plan-0911.pdf (accessed August 20, 2012).

24. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext.

25. US Department of Health and Human Services. “HHS Issues Final ICD-10 Code Sets and Updated Electronic Transaction Standards Rules.”

26. Leppert, M. A. “ICD-9-CM vs. ICD-10-CM: Examine the Differences in Diabetes Coding.” JustCoding News: Outpatient. July 25, 2012. Available at http://www.hcpro.com/HIM-282660-8160/ICD9CM-vs-ICD10CM-Examine-the-differences-in-diabetes-coding.html (accessed August 10, 2012).

27. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext.

28. Fox, B., and J. Sheehan. “Openness and Exactness—Mitigating Fraud Vulnerabilities in the Age of EHRs and ICD-10.” 2012. Available at http://69.59.162.218/HIMSS2012/Venetian%20Sands%20Expo%20Center/2.21.12_Tue/Casanova%20502/Tue_1530/SHIFT5_Bill_Fox_Casanova%20502/SHIFT5_Fox.pdf (accessed August 20, 2012).

29. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext.

30. Warren, J. J. “NCVHS Hearing on Clinical Coding and Classification Issues.” North American Nursing Diagnosis Association. 1997. Available at http://www.ncvhs.hhs.gov/97041617.htm (accessed August 20, 2012).

31. Chute, C., G. S. Huff, J. A. Ferguson, J. M. Walker, and J. D. Halamka. “There Are Important Reasons for Delaying Implementation of the New ICD-10 Coding System.” Health Affairs 31 (2012): 4836–42.

32. Steindel, S. J. “International Classification of Diseases, 10th Edition, Clinical Modification and Procedure Coding System: Descriptive Overview of the Next Generation HIPAA Code Sets.” Journal of the American Medical Informatics Association 17, no. 3 (2010): 274–82.

33. Healthcare Information and Management Systems Society. “ICD-10 Transformation: Five Critical Risk-Mitigation Strategies.”

34. Centers for Medicare and Medicaid Services. “General Equivalence Mappings: ICD-9-CM to and from ICD-10-CM and ICD-10-PCS.” 2010. Available at https://www.cms.gov/Medicare/Coding/ICD10/downloads/GEMs-CrosswalksBasicFAQ.pdf (accessed August 20, 2012).

35. Pershing Yoakley and Associates. “ICD-10 Transition: What Health Lawyers Need to Know.” Presented at the American Bar Association Emerging Issues in Healthcare Law Conference, February 23, 2012. Available at http://www.slideshare.net/PYAPC/icd10-transition-what-health-lawyers-need-to-know (accessed August 20, 2012).

36. Bowie, M. J., and R. M. Schaffer. Understanding ICD-10-CM and ICD-10-PCS: A Worktext.

37. Chute, C., G. S. Huff, J. A. Ferguson, J. M. Walker, and J. D. Halamka. “There Are Important Reasons for Delaying Implementation of the New ICD-10 Coding System.”

38. National Institutes of Health. “SNOMED Clinical Terms.” 2012. Available at http://www.nlm.nih.gov/research/umls/Snomed/snomed_main.html (accessed August 20, 2012).

39. Quan, H., B. Li, L. D. Saunders, G. A. Parsons, C. Nilsson, A. Alibhai, and W. A. Ghali. “Assessing Validity of ICD-9-CM and ICD-10 Administrative Data in Recording Clinical Conditions in a Unique Dually Coded Database.” Health Services Research 43, no. 4 (2008): 1424–41.

40. Ibid., p. 1438.

41. Steindel, S. J. “International Classification of Diseases, 10th Edition, Clinical Modification and Procedure Coding System: Descriptive Overview of the Next Generation HIPAA Code Sets.”

42. Chute, C., G. S. Huff, J. A. Ferguson, J. M. Walker, and J. D. Halamka. “There Are Important Reasons for Delaying Implementation of the New ICD-10 Coding System.”Health Affairs 31, no. 4 (2012): 836-842.

43. Ibid., p. 838.

Printer friendly version of this article.

Maxim Topaz, MA, RN; Leah Shafran-Topaz, PT; and Kathryn H. Bowles, PhD, RN, FAAN, FACMI. “ICD-9 to ICD-10: Evolution, Revolution, and Current Debates in the United States.” Perspectives in Health Information Management (Spring 2013): 1-18.

Abstract

Patient satisfaction has gained the focal position in well-planned healthcare delivery systems. The objective of this study was to investigate patient satisfaction with the quality of services provided before and after the implementation of electronic health records (EHRs) at Primary Health Care Centers (PHCCs) in Kuwait. A self-developed questionnaire was used. A random sampling was used to select 700 subjects. The response rate was 74 percent. The majority of participants (67 percent) were 19 to 34 years of age. Of the participants, 63 percent were female and 92 percent were Kuwaiti nationals. Before EHR implementation, respondents’ disagreement regarding the doctor’s carefulness in conducting the examination, uses of medical terminology, explanations for medication given, and time given for a patient was more than 30 percent. Disagreement regarding the rest of the questions related to the patient/physician relationship after EHR implementation was also higher (25 percent to 39 percent).

Keywords: electronic health records, patient satisfaction, primary healthcare

Background

“Empirical studies of the [electronic health record (EHR)] have increased”¹ recently, but few studies have explored the impact of EHRs on patients’ satisfaction and the physician-patient relationship.

Patient satisfaction has gained the focal position in modern-day, well-planned healthcare delivery systems. Much attention within the healthcare industry is focused on patients’ satisfaction with the quality of healthcare services. Several lines of research have converged on the finding that care providers’ interactions with patients and their families have remarkably strong effects on clinical outcomes, functional status, and even physiologic measures of health.^{2, 3} Measurement of such interactions is used as a key indicator of healthcare quality by many physicians and consumer groups. It gives useful feedback to clinicians and managers on perceived performance and satisfaction with care that may not be apparent through more traditional audit measures.^4–7 However, patient satisfaction has not been widely studied with respect to implementation of EHRs.⁸

A study conducted by Menachemi and Collum indicated that the potential benefits of EHRs include improved clinical, organizational, and societal outcomes. Clinical outcomes include improving the quality of care provided to patients and reducing medical errors. Organizational outcomes include financial and operational benefits. Societal outcomes include the improved ability to conduct research, improved population health, and reduced costs.⁹

The Ministry of Health in Kuwait implemented various projects aimed at improving the quality of healthcare services. These comprehensive projects targeted primary, secondary, and tertiary levels of care. One project was the implementation of EHRs at the Primary Health Care Centers (PHCCs).¹⁰ The move toward implementing EHRs was a result of many factors and problems faced by the Ministry of Health. The Ministry of Health recognized the need for accurate, complete, and comprehensive patient information and data for providing quality healthcare, delivering accurate statistics, helping in the planning process, evaluating treatment effectiveness, and facilitating the decision-making process. In addition, EHRs have the potential to advance the quality of healthcare.^11–17

Patients who get healthcare services from Kuwait’s PHCCs are often observed to complain about the quality of the services provided in general. A study was conducted in 2006 to investigate patients’ satisfaction with primary healthcare in Kuwait after the implementation of EHRs without considering patients’ satisfaction before the implementation.¹⁸ Although EHR implementation has several advantages, barriers to EHR adoption remain.¹⁹ Some of the barriers may include the effects on eye contact and time spent with patients.^{20, 21} Therefore, this study sought to identify patients’ perception of and satisfaction with the quality of the services provided at the PHCCs before and after the implementation of EHRs. In particular, the level of satisfaction with physicians, administrative staff (receptionists), routine procedures and paperwork, waiting time before seeing the doctor, time spent at the doctor’s office, working hours, and appointment availability were analyzed.

The significance of this study lies in assessing the level of healthcare quality as perceived by patients, identifying gaps between patient expectations and actual process, providing an empirical base for changes to be made by policy makers, offering feedback to care providers, and, in the long run, indicating changes that should be made in the medical curriculum.

Method

Sample

The Kuwait Health Care Delivery System includes more than 78 PHCCs. For this study, the population consisted of all patients who visited the various PHCCs in Kuwait. The data were collected using a self-developed questionnaire. The questionnaires were distributed among patients at the time of their visits at PHCCs during morning and evening shifts from September 2004 to December 2004. A total of 700 questionnaires were distributed at various clinics. At each clinic, patients were selected randomly. The patient’s agreement to participate in the study was obtained before each questionnaire was completed.

Instrument

The questionnaire aimed to collect data on participants’ demographic characteristics. In addition, general questions on the clinics’ location, cleanliness, and decoration were included. Moreover, three separate sections addressed the patient/physician relationship before and after implementation of EHRs and the attitudes of other professionals at the clinics. At the end of the questionnaire, three open-ended questions were included. Before the data collection, the questionnaire was pretested for construct validity and reliability. The questionnaires were piloted with 20 patients, and a few adjustments were made regarding the wording of some questions.

Statistical Analysis

Data were analyzed using SPSS software. For reporting, descriptive statistics were used. In addition, factor analysis was used to identify the common significant factors regarding the patients’ satisfaction with the services of the clinics.

Results

The response rate was 74 percent, with 518 patients participating out of 700 surveys distributed. The patients who visited the PHCCs included 344 (67 percent) who were 19 to 34 years of age, followed by 81 (16 percent) between 35 to 49 years and 80 (15 percent) who were 18 years of age or below. Only 12 (2 percent) of the patients were 50 years or older. Among the patients, 325 (63 percent) were female and 189 (37 percent) were male. Regarding nationalities of the patients, 465 (92 percent) were Kuwaiti and 42 (8 percent) were non-Kuwaiti. (See Table 1.)

Regarding visits to PHCCs, 475 (93 percent) had previous visits, while only 38 (7 percent) of the patients had never visited the PHCCs previously. Among the patients, 335 (65 percent) had a visit during the last 3 months, 99 (19 percent) during the last 6 months, and 81 (16 percent) during the last year or earlier. Among the patients, 83 percent had a waiting time of 20 minutes or less to see a doctor, while 17 percent had to wait more than 20 minutes. Among patients who had to wait more than 15 minutes, 18 percent were given an explanation for the delay.

General characteristics related to the clinics include the greeting on arrival, which 25 percent of the patients reported as poor, followed by the waiting room decor (reported as poor by 23 percent), waiting time (reported as poor by 19 percent), promptness of attention (18 percent poor), waiting room comfort (16 percent poor), and courtesy of the receptionist (14 percent poor). Regarding the convenience of the clinic’s location, only 6 percent reported it as poor and 7 percent reported it as fair; the rest of the respondents reported it as good, very good, or excellent. (See Table 2.)

The distribution of characteristics of the patient/physician relationship before the implementation of EHRs is presented in Table 3. More than 30 percent of respondents disagreed or strongly disagreed with items regarding the doctor’s carefulness about the examination, use of medical terminology, explanations provided for prescribed medication, and time allotted for the patient. The percentage of respondents disagreeing or strongly disagreeing with the other items related to the patient/physician relationship varied from 23 percent to 29 percent.

Table 4 shows the data on the patient/physician relationship after EHR implementation. After implementation of EHRs, the percentage of patients agreeing or strongly agreeing with questions related to the patient/physician relationship after implementation of EHRs, (e.g., the doctor focuses on EHR screen rather than on patient, I believe EHR took doctor’s attention from me, at visit, doctor pays more attention to typing, and EHRs increase trust in physicians) varied from 36 to 50 percent.

Data on the behavior of the other professionals at the PHCCs shows the following: except for effective treatment and professionalism of staff at PHCCs, patients’ agreement on other questions were 50 percent or more. Slightly more than 50 percent of the patients did not agree regarding the effectiveness of treatment and other staff members’ professionalism at the PHCCs. This part of the survey is excluded from this study but will be addressed in a future study.

Discussion

The majority of Kuwait’s population use the PHCCs as the first step to access healthcare services. The initial implementation of EHRs by the Ministry of Health in Kuwait took place at the PHCCs in 2002. This study aimed to investigate patient satisfaction with the quality of services provided before and after the implementation of EHRs at the PHCCs in Kuwait.

Factor Analysis

The results of exploratory factor analysis are presented in Table 5 and Table 6. Factor analysis is used to identify the common factors that explain the patients’ relationship with the physician before and after implementation of EHRs and the relationships with other professionals working at the PHCCs. Using the factor loadings from factor analysis also helps to identify the construct validity of our questionnaire. Table 5 presents the factor analysis of items related to the patient/physician relationship before EHR implementation. Factor analysis identified two common factors on the basis of eigenvalues greater than 1, which explain 56 percent of total variations. The loadings of the two factors vary from 0.62 to 0.82, which showed that the construct validity of these items is very high. The first factor explains whether the physician is taking enough time to address patients’ questions, such as by explaining the patient’s medical problem, test, procedure, or prescribed medication, which makes a patient feel confident about the doctor. The second factor explains items that reduce the patient’s confidence in the doctor.

The factor analysis of items related to the patient/physician relationship after EHR implementation identified three common factors on the basis of eigenvalues greater than 1, which explain 66 percent of total variations. The factor loadings vary from 0.57 to 0.90, which again showed that the construct validity of these items is very high. It was easy to name the three factors identified here. The first factor explained the patients’ perception regarding improvement in the quality of care due to the implementation of EHRs. For example, some patients indicated that using EHRs improved relations with the physician and increased trust in the physician’s performance. The second factor explained the patients’ feeling about the shift of the doctor’s attention from the patient to the computer screen, such that less eye contact was received. Finally, the third factor explained the patients’ perception of the negative impact due to the computerization. For example, some patients did not see improvement in the clinic’s system after EHR implementation. (See Table 6.)

One limitation of this study is the low participation of non-Kuwaitis in the study (8 percent) although they form the majority of Kuwait’s population.

As noted above, data on the behavior of other professionals on the healthcare team working at the PHCCs were excluded from this study but will be used in a future study. Another exclusion was the results of the three open-ended questions because of difficulty in categorizing participants’ answers into limited themes.

Conclusion

EHRs in healthcare settings pose challenges to medical practice.²² Therefore, more studies of EHR implementation and its effects on the medical practice in general and on physician-patient relationships are needed.

This study’s results show decreased physician attention toward patients during patient visits due to the use of EHRs. EHR implementation should support positive relationships with patients and improve the quality of care.²³ Implementation of EHR systems in gradual phases in healthcare facilities will help the healthcare professionals adapt to the system as well as maintain good physician-patient relationships.²⁴

The data were gathered during 2004, when technology use was not as common among Kuwait’s population as it is today. Hence, a similar study will be conducted in the near future to investigate further the impact of EHR systems on patients’ satisfaction. The new study will include a larger sample size to better represent the population in Kuwait.

More studies should be conducted related to EHRs and the improvement of patient care. EHR training should be introduced for the various health care professionals and in medical schools’ curricula.²⁵ Other studies should be conducted after more years have passed since EHR implementation to investigate the cost-benefit factors.

Support

This research was supported by Kuwait University, grant no. ZN 03/04.

Eiman Al-Jafar, PhD, is a faculty member in the Department of Health Information Administration, Faculty of Allied Health Sciences at Kuwait University, Kuwait.

Notes

1. Shield, R. R., R. E. Goldman, D. A. Anthony, N. Wang, R. J. Doyle, and J. Borkan. “Gradual Electronic Health Record Implementation: New Insights on Physician and Patient Adaptation.” Annals of Family Medicine 8, no. 4 (2010): 316–26.
2. Kenagy, J. W., D. M. Berwick, and M. F. Shore. “Service Quality in Health Care.” Journal of the American Medical Association 281, no. 7 (1999): 661–65.
3. Bolus, R., and J. Pitts. “Patient Satisfaction: The Indispensable Outcome.” Managed Care 8, no. 4 (1999): 24–28.
4. Thorne, L., and N. Kitchen. “Auditing Patient Experience and Satisfaction.” British Journal of Neurosurgery 16, no. 3 (2002): 243–55.
5. Garrison, G. M., M. E. Bernard, and N. H. Rasmussen. “21st-Century Health Care: The Effect of Computer Use by Physicians on Patient Satisfaction at a Family Medicine Clinic.” Family Medicine 34, no. 5 (2002): 362–68.
6. Goetz Goldberg, D., A. J. Kuzel, L. B. Feng, J. P. DeShazo, and L. E. Love. “EHRs in Primary Care Practices: Benefits, Challenges, and Successful Strategies.” American Journal of Managed Care 18, no. 2 (2012): e48–e54.
7. Holanda, A. A., H. L. do Carmo E Sá, A. P. Vieira, and A. M. Catrib. “Use and Satisfaction with Electronic Health Record by Primary Care Physicians in a Health District in Brazil.” Journal of Medical Systems 36, no. 5 (2012): 3141–49.
8. Rosen, P., S. J. Spalding, M. J. Hannon, R. M. Boudreau, and C. K. Kwoh. “Parent Satisfaction with the Electronic Medical Record in an Academic Pediatric Rheumatology Practice.” Journal of Medical Internet Research 13, no. 2 (2011): e40.
9. Menachemi, N., and T. Collum. “Benefits and Drawbacks of Electronic Health Record Systems.” Journal of Risk Management and Healthcare Policy 4 (2011): 47–55.
10. Al-Azmi, S. F., A. M. Mohammed, and M. I. Hanafi. “Patients’ Satisfaction with Primary Health Care in Kuwait after Electronic Medical Record Implementation.” Journal of the Egyptian Public Health Association 81, nos. 5–6 (2006): 277–300.
11. Zhou, L., C. S. Soran, C. A. Jenter, L. A. Volk, E. J. Orav, D. W. Bates, and S. R. Simon. “The Relationship between Electronic Health Record Use and Quality of Care over Time.” Journal of the American Medical Informatics Association 16 (2009): 457–64.
12. Zastowny, T. R., W. C. Stratmann, E. H. Adams, and M. L. Fox. “Patient Satisfaction and Experience with Health Services and Quality of Care.” Quality Management in Health Care 3, no. 3 (1995): 60–61.
13. O’Connell, R. T., C. Cho, N. Shah, K. Brown, and R. N. Schiffman. “Take Note(s): Differential EHR Satisfaction with Two Implementations under One Roof.” Journal of the American Medical Informatics Association 11, no. 1 (2004): 43–49.
14. Weir, C. R., R. Crockett, S. Gohlinghorst, and C. McCarthy. “Does User Satisfaction Relate to Adoption Behavior? An Exploratory Analysis Using CPRS Implementation.” AMIA Annual Symposium Proceedings (2000): 913–17.
15. Robertson, A., K. Cresswell, A. Takian, D. Petrakaki, S. Crowe, T. Cornford, et al. “Implementation and Adoption of Nationwide Electronic Health Records in Secondary Care in England: Qualitative Analysis of Interim Results from a Prospective National Evaluation.” BMJ 341 (2010): c4564.
16. Jha, A. K., C. M. DesRoches, E. G. Campbell, K. Donelan, S. R. Rao, T. G. Ferris, et al. “Use of Electronic Health Records in U.S. Hospitals.” New England Journal of Medicine 360, no. 16 (2009): 1628–38.
17. El-Kareh, R., T. K. Gandhi, E. G. Poon, L. P. Newmark, J. Ungar, S. Lipsitz, and T. D. Sequist. “Trends in Primary Care Clinician Perceptions of a New Electronic Health Record.” Journal of General Internal Medicine 24, no. 4 (2009): 464–68.
18. Al-Azmi, S. F., A. M. Mohammed, and M. I. Hanafi. “Patients’ Satisfaction with Primary Health Care in Kuwait after Electronic Medical Record Implementation.”
19. Ash, J. S., and D. W. Bates. “Factors and Forces Affecting EHR System Adoption: Report of a 2004 ACMI Discussion.” Journal of the American Medical Informatics Association 12 (2005): 8–12.
20. Linder, J. A., J. L. Schnipper, R. Tsurikova, A. J. Melnikas, L. A. Volk, and B. Middleton. “Barriers to Electronic Health Record Use During Patient Visits.” AMIA Annual Symposium Proceedings (2006): 499–503.
21. Chaudhry, B., J. Wang, S. Wu, M. Maglione, W. Mojica, E. Roth, et al. “Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care.” Annals of Internal Medicine 144 (2006): E12–E22.
22. Shield, R. R., R. E. Goldman, D. A. Anthony, N. Wang, R. J. Doyle, and J. Borkan. “Gradual Electronic Health Record Implementation: New Insights on Physician and Patient Adaptation.”
23. Goetz Goldberg, D., A. J. Kuzel, L. B. Feng, J. P. DeShazo, and L. E. Love. “EHRs in Primary Care Practices: Benefits, Challenges, and Successful Strategies.”
24. Shield, R. R., R. E. Goldman, D. A. Anthony, N. Wang, R. J. Doyle, and J. Borkan. “Gradual Electronic Health Record Implementation: New Insights on Physician and Patient Adaptation.”
25. Ibid.

Printer friendly version of this article.

Eiman Al-Jafar, PhD. “Exploring Patient Satisfaction Before and After Electronic Health Record (EHR) Implementation: The Kuwait Experience.” Perspectives in Health Information Management (Spring 2013): 1-12.

Abstract

The United States will need to train a large workforce of skilled health information technology (HIT) professionals in order to meet the US government’s goal of universal electronic health records (EHRs) for all patients and widespread health information exchange. The Health Information Technology for Economic and Clinical Health (HITECH) Act established several HIT workforce educational programs to accomplish this goal. Recent studies have shown that EHR usability is a significant concern of physicians and is a potential obstacle to EHR adoption. It is important to have a highly usable EHR to train both clinicians and students. In this article, we report a qualitative-quantitative usability analysis of a web-based EHR for training health informatics and health information management students.

Keywords: health informatics, medical informatics, clinical informatics, health information management, electronic health records, usability, students, education

Introduction

Electronic health records (EHRs) have been adopted by multiple countries in the past two decades.¹ In the United States, the progress has been slow in spite of the 2004 executive order that set the goal of universal EHRs by 2014.² EHR adoption in the United States has been accelerated by the Health Information Technology for Economic and Clinical Health (HITECH) Act, which is title XIII of the American Recovery and Reinvestment Act of 2009 (ARRA). This act created multiple programs to promote the adoption of health information technology (HIT), including financial incentives for the adoption of certified EHRs that meet the criteria for meaningful use and the creation of HIT regional extension centers to assist practices with EHR adoption.³ As of June 2012, more than 110,000 eligible clinicians and 2,400 hospitals have received reimbursement.⁴

To support provider adoption of HIT, the HITECH Act also funded workforce training programs to prepare skilled HIT workers to meet anticipated demand. The Office of the National Coordinator for Health Information Technology (ONC) estimated that the United States will need approximately 51,000 skilled workers over the next five years. The HITECH Act established community college and university workforce programs to fast-track the education required by new workers to support EHR adoption, workflow redesign, and technical support. Federal funding has also created the Curriculum Development Centers Program and Community College Consortia to Educate Health IT Professionals in Health Care Program, in addition to the Program of Assistance for University-based Training.⁵

In addition to training a new HIT workforce, current and prospective healthcare professionals (e.g., medical and nursing students, physicians, nurses, and pharmacists in outpatient and inpatient settings) also require training. Unfortunately, many professional education programs have not kept up with the rapid advances in technology.⁶ For hands-on training purposes, an EHR should be readily available in all clinical and HIT educational programs. Training clinicians and HIT professionals in the usage of EHRs faces numerous obstacles such as the lack of a uniform training strategy, constrained training times, and the availability and cost of an EHR suitable for educational purposes. Furthermore, in order to maximize training, an EHR should have good usability characteristics. Uniform usability standards for commercial EHRs, however, are lacking. Usability is defined by the US National Institute of Standards and Technology (NIST) as “the effectiveness, efficiency and satisfaction with which the intended users can achieve their tasks in the intended context of product use.”⁷ Effectiveness is usually measured as the completion of tasks without errors and the potential of the system to cause errors. Efficiency is measured as the time it takes to complete a task. Satisfaction is generally considered a subjective evaluation using tools such as surveys. Others have included ease of learning (learnability) and retention (memorability) as important additional aspects of usability.⁸ The Healthcare Information and Management Systems Society (HIMSS) EHR Usability Task Force listed these usability principles: simplicity (uncluttered application), naturalness (workflow matches practice), consistency (application parts have the same look and feel), forgiveness and feedback (mistakes don’t result in lost time or data), effective use of language (the application uses the same words a clinician uses), efficient interactions (minimizes steps), effective information presentation (easily readable), preservation of context (the application keeps screen changes to a minimum), and minimized cognitive load (information for tasks is on one screen).⁹

Few studies have reported usability comparison ratings among different EHRs. In a study by Murff and Kannry, a large commercial EHR and the EHR used by the Veterans Health Administration (VHA) were rated using the Questionnaire for User Interaction Satisfaction (QUIS). They demonstrated that the VHA EHR was rated in the range of 7.08–7.68 for the five main survey categories, compared to a range of 3.27–4.41 for the commercial system.¹⁰ A 2011 survey of 2,719 family medicine physicians rated 30 of the more common EHR systems on 17 dimensions. Participants were asked to reply to multiple statements, such as “overall this EHR is easy and intuitive to use,” with a scale from “strongly agree” to “strongly disagree.” Twenty-five percent of respondents disagreed or strongly disagreed that their EHR was easy and intuitive to use. Approximately 30 percent of respondents disagreed or strongly disagreed with the statement “I am highly satisfied with this EHR.” Only 38 percent of survey participants agreed or strongly agreed that they would purchase the same system again.¹¹

Two of the coauthors have used a free web-based EHR (Practice Fusion) to teach informatics courses for four years. The informal subjective feedback on this EHR by students was very positive, prompting further study of this EHR with qualitative and quantitative tools. The research questions were as follows:

1. Would a validated survey instrument confirm high satisfaction and usability?
2. Would a system audit log that is part of the EHR provide a valuable, objective time-motion tool to evaluate EHR effectiveness?
3. Does this free web-based EHR provide an overall satisfactory experience for students?

Materials and Methods

Materials

The University of West Florida has used a free web-based EHR since 2008 to teach online Introduction to Medical Informatics and Electronic Health Record courses at the undergraduate and graduate level.¹² Lake-Sumter Community College has used the same EHR since 2008 as a hands-on training model for health information management (HIM) students.¹³ With simple instructions, assisted by YouTube tutorials, students have been able to access and use multiple EHR features. Using fictitious patients, all students are asked to create new appointments, new patient encounters, new diagnoses associated with the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes, and new prescriptions. Undergraduates follow a set script to create these items and then take screen shots and upload them to a learning management system to confirm competency.

The study EHR is a free web-based ambulatory EHR that launched in 2008. The user interface is based on Adobe Flex 2, and the data are archived in a central data repository with bank-level security. The target audience is primarily small primary care practices. The application began with basic functionality but has added multiple features since its inception, largely from user input. The EHR was certified as meeting the criteria for meaningful use in mid-2011. As of 2012, approximately 20 academic information science programs use this EHR for training. Enrollment is straightforward, and the EHR provides multiple test patients. EHR features are listed in Appendix 1.¹⁴

Participants

Students from the University of West Florida and Lake-Sumter Community College were recruited by a research assistant to volunteer to take a usability survey and perform a time-motion study for the 2011 summer semester and the 2012 spring and summer semesters. In 2011, they were not given any incentives and were told their participation would not affect their grades. All contact related to the research study was by a graduate student and not the instructors. Student participation in the study overall was approximately 20 percent. In order to increase participation in 2012, students entered a drawing for a free informatics-related textbook, but participation rates remained the same. Students had to read an informed consent statement and sign it electronically prior to participating. The research proposal was approved by the institutional review boards of both institutions.

Survey Instrument

To measure EHR satisfaction, students were first asked by a research assistant to volunteer to take the Questionnaire for User Interaction Satisfaction (QUIS), version 7.0. QUIS is a licensed, validated survey tool created at the University of Maryland¹⁵ and used to evaluate EHRs^16–17 and other technologies.^18–19 QUIS version 7.0 has 11 sections that measure user demographics (six questions), overall reaction ratings (six questions), screen design and layout (four questions), terminology and system information (six questions), learning (four questions), and system capabilities (five questions). Sections 8–11, which are optional, deal with evaluating technical manuals, online help, and so forth and were not included in this study. The following questions were not included because they were not felt to be pertinent: overall reaction (dull–stimulating); overall reaction (inadequate power–adequate power); highlighting on screen helpful (not at all–very much); supplemental reference materials (confusing–clear); and system tendency (noisy–quiet). We modified the demographics section to add questions about undergraduate/graduate status, gender, age group (30 and under, 31–50, and 51 and older), whether this was the participant’s first experience with an EHR, approximate time spent with this EHR, and prior technology experience (with eight common technologies to choose from). We added questions about the ease of creating a SOAP (subjective, objective, assessment, plan) note, creating new appointments, and using YouTube videos as a means of training. Each question was accompanied by a free-text box enabling the participant to provide comments. Satisfaction ratings were recorded using a Likert scale ranging from 1 (lowest) to 9 (highest). Reliability of this survey tool is excellent (Cronbach’s alpha of .95).²⁰

Time-Motion Study

In order to test performance (time on task and error rates), we evaluated the students’ ability to complete routine EHR tasks. Student volunteers were given instructions via an e-mail message that outlined a series of routine EHR tasks usually accomplished by an office nurse or doctor. They were told to complete the tasks without interruption. After a test patient had been created, students assumed the role of the office nurse and updated the past medical history. Time-stamp times are in parentheses.

1. Under surgery add “appendectomy 2008”
2. Under family history add “sister with breast cancer”
3. Under preventative medicine add “colonoscopy 2010”
4. Under social history add “smoking cessation 1 year prior”
5. Under diagnosis add “asthma (ICD-9-CM code 493)” and “backache, unspecified (ICD-9-CM code 724.5)” with start dates of “today” (time stamp)
6. Under medications enter “Motrin 600 mg” (time stamp)
7. Under allergies enter “sulfadiazine oral tablet, location skin, reaction rash localized, severity moderate” with today’s date (time stamp)
8. Under immunization enter “pneumococcal vaccine, injected right deltoid” (time stamp)
9. Under patient dashboard, patient actions start a new chart note, select SOAP note, enter vital signs: height 66 inches, weight 150 lbs, blood pressure 130/80, temperature 98 degrees, pulse 80, and respirations 16
10. Enter chief complaint of “cough” (time stamp)

Students were then told to assume the role of office physicianandperform the following:

1. With a SOAP note created, under subjective (S) find the template for “URI, bronchitis, sinusitis, otitis media and pharyngitis”; enter the following symptoms by selecting existing text “associated with mildly productive cough, acute onset”
2. Under objective (O) select “pharynx clear” and “lungs: scattered inspiratory wheezes, no rales, rhonchi”
3. Under assessment (A) add acute bronchitis with start date of one week ago and add comment “patient has symptoms consistent with acute bronchitis” (time stamp)
4. Under plan (P) create a new electronic prescription and enter albuterol sulfate inhaler, inhale QID as needed (time stamp)
5. Schedule patient for return appointment in two weeks at 8 a.m. (time stamp)
6. Log off EHR

Times were recorded by the minute and not by the second because of the design of the EHR system audit log. Tasks that could be completed in less than one minute were combined so that task completion times would not be listed as zero.

Error Rates

Errors were determined by an experienced graduate research assistant who accessed the EHR to review every test patient created for the time-motion study to look for errors of omission and commission. The time-motion study included 18 tasks requested of the nurse and 15 tasks requested of the physician, for a total of 33 tasks. With 23 participants each performing these 33 tasks, the total number of tasks performed was 759.

In order to measure error rates, we accessed each test patient’s chart in the EHR. Data were retrieved from the summary section to find the participant’s entries on the date of the study for sections such as past medical history, diagnosis history, medication history, allergies, and immunizations. Also, SOAP notes were reviewed in the events and appointments sections. If any entry did not match instructions in the script, an error was recorded.

The two types of errors that we considered were errors of omission and errors of commission. When the information retrieved from the EHR did not match the information instructed to be entered for each task, a commission error was recorded. For example, if the student was instructed to set up a follow-up appointment at the end of the visit “two weeks from today 8 a.m.,” and the appointment was scheduled in the wrong week and/or at a different time, an error of commission was recorded. If the EHR did not show any follow-up appointment, an error of omission was recorded. Mean errors on the nurse instructions and on the physician instructions were calculated, as well as mean total errors. The error rate was calculated by dividing the number of errors noted by the number of tasks completed as a nurse or as a physician, as well as by the total number of errors. Confidence intervals were included with each calculation.

Statistical Analysis

Descriptive statistics were used to evaluate the QUIS survey data. Mean scores and 95 percent confidence intervals were calculated for the five categories, and a mean for all categories was calculated as the “overall satisfaction” score. Demographic data were converted to dummy codes and analyzed as categorical data. Descriptive statistics were used to evaluate the time-motion study, the mean and confidence intervals for nurse and physician task completion times, and a total time score for both. Time-motion data were normally distributed, but the data violated the assumption of equal standard deviations, so the data were analyzed with nonparametric tests. Mann-Whitney tests were utilized to analyze data between any two demographic groups, and Kruskal-Wallis tests were utilized to analyze data between more than two groups. A Spearman rank test was used to study the correlation between total time-motion scores and total satisfaction scores. Effect size for nonparametric data was calculated using a probabilistic index.²¹ All calculations were made using GraphPad InStat (version 3.10).

Results

Out of a potential student pool of 152 students from the two schools, 44 students completed the QUIS survey, and 23 completed the survey and the time-motion study, reflecting a study participation rate of 27 percent for the survey and 15 percent for the time-motion study. Two time-motion studies were not included in the analysis because they were started but not finished. Sixty-eight percent of study completers were from the University of West Florida, and approximately half of the participants took the study in each year. Of the students completing the survey, 65 percent were female, 77 percent were undergraduate students, 87 percent were over the age of 31, 53 percent were medical informatics certificate students, 50 percent had not had prior EHR experience, 75 percent claimed experience with five or more out of the eight common software packages, devices, or systems listed, and 85 percent had less than 10 hours of hands-on experience with the EHR before becoming a study participant. The time-motion study included one physician and five nurses.

Table 1 summarizes the mean scores and confidence intervals for the five sections of the QUIS survey instrument and the four non-QUIS questions. The scores were high in all categories with slightly lower scores noted in the QUIS question section. There was no statistical difference between overall satisfaction scores in 2011 and 2012 when an incentive was offered (not displayed in the table). Survey free-text comments were positive and primarily related to the intuitive nature of the EHR software.

Table 2 summarizes mean times and confidence intervals to complete time-motion tasks in the roles of nurse and physician. The total time to complete all tasks varied from a low of 11 minutes to a high of 36 minutes. Most students completed all tasks in about 19 minutes. Not surprisingly, clinicians (physicians and nurses) were significantly faster than nonclinicians, p < .0025 (large effect size of .93), calculated by a probabilistic index (not displayed in the table).

Table 3 summarizes the observed errors for the role of nurse and physician and the total errors recorded. Two students performed all tasks with no errors; most students made fewer than three errors. The average number of errors per student was 0.83 for nurse tasks, 1.13 for physician tasks, and 1.88 for total average errors. The error rate (errors divided by tasks) was 4.6 percent for nurse tasks, 7.5 percent for physician tasks, and 5.9 percent for total tasks. The majority of errors made were those of commission and not omission. Because of the small sample size and the inability to record task times by the second, we chose not to categorize errors in more detail.

Table 4 summarizes the multiple comparisons between overall satisfaction scores and subject demographics, total time-motion scores and subject demographics, and total time motion scores and overall satisfaction scores. No subject demographics were statistically significantly related to overall satisfaction with using the study EHR. Prior use of an EHR and male gender were associated with significantly shorter times to complete the time-motion tasks (large effect sizes of .92 and .83), calculated using the probabilistic index.

The total time to complete the time-motion study had no significant correlation with overall satisfaction scores.

Discussion

Our study of an ambulatory web-based EHR used for training demonstrated high usability based on satisfaction scores (survey results), efficiency (time-motion results), and effectiveness (low error rates). Satisfaction scores did not correlate with common demographics such as age, gender, graduate status, technology familiarity, and prior EHR experience, suggesting that the study EHR was easy for a diverse group of students to use. The results suggest that students found the computer interface to be intuitive and user friendly, even with minimal training. The graphical user interface was familiar and associated with a logical workflow. The satisfaction ratings by the students were consistent with the scores reported by clinicians in the study by Murff and Kannry.²² Additionally, the EHR evaluated in our study (Practice Fusion) was ranked fourth in a large survey of family medicine physicians, with about 95 percent agreeing that it was easy and intuitive to use.²³ EHR surveys are limited by a variety of biases, but they do provide an overall perception by actual users.

Our unobserved time-motion study recorded the times to complete multiple common EHR tasks. As anticipated, prior EHR experience or status as a clinician predicted shorter completion times. There was no significant correlation between EHR satisfaction and the time to complete tasks. Our time-motion study was unique in that it used the EHR system audit log to automatically time routine tasks, rather than a trained observer. To our knowledge, this is the first study to report time-motion data using an intrinsic ambulatory EHR feature. Most time-motion studies of EHRs in the literature used human observers to compare the total time taken by a physician to complete a patient encounter using paper compared to an EHR.^24–26 The goal of these published studies was not to time standard EHR tasks such as electronic prescribing or documentation.

The overall error rates for task completion were low, with an average of 1.9 total errors per student or 5.6 percent errors for the 33 tasks completed. Most errors noted were those of commission and not omission. Our low error rates also suggest that routine tasks were easily and accurately completed by our students, regardless of prior use of EHRs, technology expertise, age, or graduate student status. These error rates are low for students, but they did have the advantage of following a written script during task completion.

Currently, there is no standardized usability rating for EHRs, but limited usability testing will be part of the EHR certification for the second stage of meaningful use in 2014.²⁷ The Certification Commission for Health Information Technology is the only EHR-certifying organization that currently offers additional usability testing as part of the certification process. The inspection process consists of a team of jurors rating usability based on observations through a series of questionnaires. The juror panel consists of three clinical jurors, including at least one practicing physician and an information technology security evaluator. The questionnaires used are the After Scenario Questionnaire, Perceived Usability Questionnaire, and System Usability Survey. However, these methods have been reported to provide subjective usability measures based on perceived satisfaction.²⁸ Of the 75 certified EHRs rated using these measures, 98 percent were given four or five stars (the highest rating). This seems optimistic based on other surveys of the same EHRs by actual clinicians.²⁹

EHR vendors are beginning to address usability issues, but historically, because of the competitive nature of the field, there has been little sharing of best practices. EHR models are selected on the number of features, not usability, and there are no enforceable usability standards at this time.³⁰ Multiple organizations such as NIST are contributing to our knowledge of EHR usability and testing. In 2012, NIST released an EHR Usability Protocol (EUP) that outlined the formal procedures for evaluating EHR usability for developers and evaluators. The EUP is a three-step process consisting of EHR Application Analysis, EHR User Interface Expert Review, and EHR User Interface Validation Testing.³¹

Multiple usability questions remain, such as which tests should be used to ascertain usability. Common usability approaches include heuristic evaluation, cognitive task analysis, usability tests, surveys, and focus groups. Current evidence suggests that usability testing should include multiple evaluation methods.³² Schumacher and others have suggested that usability testing be a routine part of the request-for-proposal process commonly used to purchase an EHR.³³

Our study has several limitations that should be noted and could limit the generalizability of our results. Our participation rates were low, and the lack of an association between student demographics and satisfaction scores may have been due to a type II error or recruitment bias. We evaluated students using an EHR for training purposes, so our results may not pertain to clinical staff. The time-motion study was easy to conduct with online students due to internal EHR time stamps but was limited by the fact that tasks could be recorded only by minutes and not by seconds. We are uncertain as to whether EHR system audit logs are routinely used and available to nontechnical staff. Because the time-motion portion of the study was unobserved, we do not know if the variation in performance times could have been due to inattentiveness and not usability issues. According to Zheng et al., observed time-motion studies are the standard, but they are not without problems, so alternatives should be investigated.³⁴

Conclusions

The adoption of EHRs, particularly by small or rural primary care practices, is a key component of healthcare reform in the United States. In order to accomplish this immense task, EHR training will need to occur at multiple levels in our healthcare system. EHR usability is important for all users, particularly clinical staff who are most interested in efficiency (productivity) and effectiveness (patient safety). Healthcare workers and HIM students should ideally train on EHRs that have high usability ratings to prevent frustration and facilitate education. Our usability study suggests that the EHR evaluated is efficient, effective, and associated with high satisfaction levels for informatics and HIM student training. The study also suggests that educational programs do not have to make a substantial investment to have a highly usable EHR model for training purposes. User-centered EHR design and comprehensive usability testing are likely to become the norm in the next few years in order to improve EHR adoption, clinician productivity, and patient safety.

Acknowledgments

We would like to thank Steven Linnville, PhD, and Nora Bailey, MSP, for their review of the manuscript.

Robert Hoyt, MD, FACP, is the director of the Medical Informatics Program at the University of West Florida’s School of Allied Health and Life Sciences in Pensacola, FL.

Kenneth Adler, MD, MMM, is the medical director of information technology at Arizona Community Physicians in Tucson, AZ.

Brandy Ziesemer, RHIA, is a health information manager and associate professor at Lake-Sumter Community College in Leesburg, FL.

Georgina Palombo, MBA, is a graduate research assistant at the University of West Florida in Pensacola, FL.

Notes

1. Schoen, Cathy, Robin Osborn, Michelle Doty, David Squires, Jordan Peugh, and Sandra Applebaum. “A Survey of Primary Care Physicians in Eleven Countries, 2009: Perspectives on Care, Cost and Experiences.” Health Affairs, 28, no. 6 (2009): 1171. Available at doi:10.1377/hlthaff.28.6.w1171 (accessed March 3, 2011).
2. White House Office of Chief Information Officer. “Incentives for the Use of Health Information Technology and Establishing the Position of the National Health Information Technology Coordinator” (Executive Order 13335). April 27, 2004. Available at http://nodis3.gsfc.nasa.gov/displayEO.cfm?id=EO_13335_ (accessed January 5, 2011).
3. “American Recovery and Reinvestment Act of 2009.” Public Law 111-5. February 17, 2009. Available at http://www.gpo.gov/fdsys/pkg/PLAW-111publ5/content-detail.html (accessed January 3, 2011).
4. US Department of Health and Human Services. “More Than 100,000 Health Care Providers Paid for Using Electronic Health Records.” News Release, June 19, 2012. Available at http://www.hhs.gov/news/press/2012pres/06/20120619a.html (accessed June 19, 2012).
5. Office of the National Coordinator for Health Information Technology. http://www.healthit.hhs.gov (accessed January 8, 2011).
6. Hammond, M. M., K. Margo, J. G. Christner, J. Fisher, S. H. Fischer, and L. N. Pangaro. “Opportunities and Challenges in Integrating Electronic Health Records into Undergraduate Medical Education: A National Survey of Clerkship Directors.” Teaching and Learning in Medicine 24, no. 3 (2012): 219–24.
7. Usability Net. “Usability Definitions: International Standards Organization (ISO) 9241-11: Guidance on Usability (1998).” Available at http://www.usabilitynet.org/tools/r_international.htm (accessed August 4, 2011).
8. Nielsen, Jakob. “Usability 101: Introduction to Usability.” Nielsen Norman Group. January 4, 2012. Available at http://www.useit.com/alertbox/20030825.html (accessed July 25, 2012).
9. HIMSS EHR Usability Task Force. “Selecting an EMR for Your Practice: Evaluating Usability.” August 2010. Available at http://www.himss.org/content/files/Selecting_EMR_Eval_Usability.pdf (accessed August 24, 2011).
10. Murff, Harvey J., and Joseph Kannry. “Physician Satisfaction with Two Order Entry Systems.” Journal of the American Medical Informatics Association 8, no. 5 (2001): 499–509.
11. Edsall, Robert L., and Kenneth G. Adler. “The 2011 EHR User Satisfaction Survey.” Family Practice Management 18, no. 4 (July–August 2011): 23–30.
12. Certificate in Medical Informatics courses. University of West Florida. Available at http://uwf.edu/sahls/certificate-informatics/ (accessed February 21, 2011).
13. “Health Information Management and Technology.” Lake-Sumter Community College. Available at http://www.lscc.edu/academics/him/Pages/Home.aspx accessed July 6, 2011).
14. Practice Fusion electronic health record. Available at http://www.practicefusion.com/ (accessed January 19, 2011).
15. University of Maryland Human-Computer Interaction Lab. “Questionnaire for User Interaction Satisfaction.” Available at http://lap.umd.edu/quis/ (accessed February 12, 2011).
16. Sittig, Dean F., Gilad Kuperman, and Julie Fiskio. “Evaluating Physician Satisfaction Regarding User Interactions with an Electronic Medical Record System.” AMIA Annual Symposium Proceedings (1999): 400–404.
17. Jaspers, Monique W. M., Linda W. P. Peute, Arnaud Lauteslager, and Piet J. M. Bakker. “Pre-Post Evaluation of Physicians’ Satisfaction with a Redesigned Electronic Medical Record System.” In S. K. Andersen et al. (Editors), eHealth beyond the Horizon: Get IT There. Amsterdam: IOS Press, 2008, 303–8.
18. Johnson, Todd R., Jhang Zhang, Zhihua Tang, Constance Johnson, and James P. Turley. “Assessing Informatics Students’ Satisfaction With a Web-based Courseware System.” International Journal of Medical Informatics 73 (2004): 181–87.
19. Hortman, Patricia A., and Cheryl B. Thompson. “Evaluation of User Interface Satisfaction of a Clinical Outcomes Database.” CIN: Computers, Informatics, Nursing 23, no. 6 (2005): 301–7.
20. University of Maryland Human-Computer Interaction Lab. “Questionnaire for User Interaction Satisfaction.”
21. Acion, L., J. J. Peterson, S. Temple, and S. Arndt. “Probabilistic Index: An Intuitive Non-parametric Approach to Measuring the Size of Treatment Effects.” Statistics in Medicine 25 (2006): 591–602.
22. Murff, Harvey J., and Joseph Kannry. “Physician Satisfaction with Two Order Entry Systems.”
23. Edsall, Robert L., and Kenneth G. Adler. “The 2011 EHR User Satisfaction Survey.”
24. Lo, Helen G., Lisa P. Newmark, Catherine Yoon, Lynn A. Volk, Virginia L. Carlson, Anne F. Kittler, Margaret Lippincott, Tiffany Wang, and David W. Bates. “Electronic Health Records in Specialty Care: A Time-Motion Study.” Journal of the American Medical Informatics Association 14, no. 5 (2007): 609–15.
25. Pizziferri, Lisa, Anne F. Kittler, Lynn A. Volk, Melissa M. Honour, Sameer Gupta, Samuel Wang, Tiffany Wang, Margaret Lippincott, Li Qi, and David W. Bates. “Primary Care Physician Time Utilization Before and After Implementation of an Electronic Health Record: A Time-Motion Study.” Journal of Biomedical Informatics 38 (2005): 176–188.
26. J. Marc Overhage, Susan Perkins, William M. Tierney, and Clement J. McDonald. “Controlled Trial of Direct Physician Order Entry: Effects on Physician’s Time Utilization in Ambulatory Primary Care Internal Medicine Practices.” Journal of the American Medical Informatics Association 8, no. 4 (2001): 361–71.
27. US Department of Health and Human Services. “Health Information Technology: Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology, 2014 Edition; Revisions to the Permanent Certification Program for Health Information Technology.” 45 CFR Part 170. Federal Register 77, no. 171 (September 4, 2012): 54163. Available at http://www.gpo.gov/fdsys/pkg/FR-2012-09-04/pdf/2012-20982.pdf (accessed October 20, 2012).
28. Certification Commission for Health Information Technology. CCHIT 2011 Usability Testing Guide for Ambulatory EHRs. Available at http://www.himss.org/content/files/cchit_usability.pdf (accessed August 12, 2011).
29. Edsall, Robert L., and Kenneth G. Adler. “The 2011 EHR User Satisfaction Survey.”
30. McDonnell, Cheryl, Kristen Werner, and Lauren Wendel. Electronic Health Records Usability: Vendor Practices and Perspectives (AHRQ Publication No. 09(10)-0091-3-EF). Rockville, MD: Agency for Healthcare Research and Quality, May 2010.
31. Lowry, Svetlana Z., Matthew T. Quinn, Mala Ramaiah, Robert M. Schumacher, Emily S. Patterson, Robert North, Jiajie Zhang, Michael C. Gibbons, and Patricia Abbott. Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records (NISTIR 7804). National Institute of Standards and Technology, February 2012. Available at http://www.nist.gov/healthcare/usability/upload/EUP_WERB_Version_2_23_12-Final-2.pdf(accessed June 2, 2012).
32. Horsky, Jan, Kerry McColgan, Justin E. Pang, Andreas J. Melnikas, Jeffrey A. Linder, Jeffrey L. Schnipper, and Blackford Middleton. “Complementary Methods of System Usability Evaluation: Surveys and Observations During Software Design and Development Cycles.” Journal of Biomedical Informatics 43 (2010): 782–90.
33.  R.M. Schumacher, J.M. Webb, K.R. Johnson. “How To Select An Electronic Health Record System That Healthcare Professionals Can Use.” User Centric, Inc. February 2009. http://www.usercentric.com/sites/usercentric.com/files/usercentric-ehr-white-paper.pdf (accessed March 3, 2011)
34. Zheng, Kai, Michael H. Guo, and David A. Hanauer. “Using the Time and Motion Method to Study Clinical Work Processes and Workflow: Methodological Inconsistencies and a Call for Standardized Research.” Journal of the American Medical Informatics Association 18 (2011): 704–10.

Printer friendly version of this article.

Robert Hoyt, MD, FACP; Kenneth Adler, MD, MMM; Brandy Ziesemer, RHIA; and Georgina Palombo, MBA. “Evaluating the Usability of a Free Electronic Health Record for Training.” Perspectives in Health Information Management (Spring 2013): 1-14.

by Carol A. Campbell, DBA, RHIA, FAHIMA

In this special edition of Perspectives in Health Information Management, the central theme is the obligations of leadership, i.e., individuals in positions of leadership have an obligation to be ethically centered, authentic, and blind to differences. In other words, organizations will suffer (albeit differently) whether a leader views himself as a new-age Don Quixote or behaves like Joseph Conrad’s Kurtz in the disturbing novel “Heart of Darkness.” Good intentions, no matter how zealous the leader, are insufficient in meeting organizational needs and Kurtz-like behaviors, no matter the amount of charisma, will eventually have disastrous consequences.

In his book, Parkinson’s Law, C. Northcote Parkinson, notes that “work expands so as to fill the time available for its completion.” Parkinson diagnosed why certain organizations suddenly deteriorate by explaining that failure occurs when individuals with unusually high combinations of incompetence and jealousy (injelitance) are promoted to positions of authority.If Don Quixote and Kurtz were cloned into a single person, the dominate characteristic would likely be a version of Parkinson’s injelitance.

Fortunately, aspiring leaders are not left with only Don Quixote or Kurtz as examples. Rather, leaders can base their approach on professional codes of ethics; an approach that is described by the Harman et al paper entitled “Code of Ethics: Past and Future.” In their papers, Barefield and Meyer and Layman expand on this ethical cornerstone perspective. Barefield and Meyer describe best-practice methods for supporting major project initiatives in “Leadership’s Role in Support of Online Academic Programs.” Layman suggests that job redesign is a means by which leadership skills can be developed in “Leading by Design.” Campbell explores why some bosses are bad and  suggests six characteristics of appropriate leadership in “Reflections on Leadership”.  Johns reminds us that organizational glass ceilings are not yet fully dismantled in “Breaking the Glass Ceiling,” suggesting that there are fewer women than men with keys to the C-suite.

Much work remains for those who have their eyes set on leadership roles, even more so for those who aspire to fill those roles as ethically-centered, authentic, and blind to differences leaders. Ranging from an historical view of ethical principles to applications of fundamental management practices, this edition of Perspectives in Health Information Management provides a forum of viewpoints on the mandates for leaders.

Carol A. Campbell, DBA, RHIA, FAHIMA, is a Professor at Georgia Health Sciences University, Department of Health Management and Informatics.