by Molly Stillwell, MA; Krisstina Gowin, DO; and Eiko Klimant, MD
Objective: This case report describes an integrative oncology center’s selection of an electronic health record system. The goal is to provide a process framework for community integrative cancer care centers engaging in the selection of an oncology-focused system.
Methods: A core team of experts from the practice assessed needs, held formative meetings, identified a set of candidate vendors to present, and held summative assessment meetings to select the product.
Results and Discussion: Of 13 vendors identified, four were excluded because of insufficient interoperability, program design, lack of training support, and minimal patient involvement. Six companies were eliminated because of usability standards. Three finalists were evaluated utilizing a bubble-sorting algorithm, and one vendor was selected.
Conclusion: Electronic health record selection requires thoughtful integration of clinic and patient needs, service population, and the changing healthcare system. Future studies are needed to establish formal yet adaptable electronic health selection processes.
Keywords: electronic health record (EHR); electronic medical record (EMR); oncology; medical informatics; integrative medicine
An electronic health record (EHR) system needs to be accessible by multiple end users with different levels of technological expertise. Thus, the selection of the product is critical. Because each healthcare community is unique, the EHR selection process is not generic; the process needs to begin with an exploration of the ecology of the particular service community. The literature provides substantial information about implementation of EHRs within traditional medical institutions and does so with a focus on the outcomes of their use after implementation. This focus includes the broad range of new data collection abilities, greater patient involvement, and the availability of new time-saving practices.1–12 However, the literature includes little information about the initial selection process and how that process needs to differ across varying clinical environments, including the emerging field of integrative medicine. What the existing literature does address are the considerations for selecting a new EHR system in light of ongoing challenges faced in the clinical setting. These challenges include time management; collaboration between physicians, pharmaceutical teams, and patients; and storage and access to data including physician notes and test results.13–15 Furthermore, few studies16–18 review the overall steps to be expected in the selection process, ranging from identification of needs to user preferences; therefore, additional studies of the details involved in these stages are needed. Therefore, this article focuses on elucidating the step-by-step process employed by Salish Cancer Center as each consideration and challenge was addressed.
As an integrative cancer clinic bringing together practices such as acupuncture, naturopathic care, Chinese medicine, and chemotherapy, Salish Cancer Center maintains unique cultural considerations. Its patient-centered focus within an integrative oncology paradigm introduces new challenges in collecting, understanding, and utilizing information. Traditional EHR systems are designed to accommodate conventional practices that primarily receive and store structured quantitative data, such as physiologic test results and treatments. The integrative paradigm expands on this understanding by recognizing the contribution of the patients’ and practitioners’ subjective experiences in the process of treatment and recovery as equally valuable sources of knowledge. Thus, the paradigm highlighted here reflects the importance of the collection of narrative data, which presents distinctive considerations for EHR use and selection because of the qualitative nature of the data. Commitment to this multifaceted understanding of healing has informed the study methodology employed here, which emphasizes the importance of maintaining a high level of evidence-based oncology care, engaging patients and families in meaningful ways, and providing the foundation for growth of dynamic and effective services. Therefore, we aimed to identify an EHR system with the capacity to receive, store, and process both structured and unstructured data to provide a holistic picture of each patient; to capture metrics such as meaningful use and outcomes in relation to the requirements of the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA); and to support future alternative payment systems.
This case report describes the process employed by our community cancer center to select a high-performing oncology-specific EHR system. Because the center emphasizes integrative care, our approach is founded within an equally multifaceted understanding of care. This paper outlines a framework intended to support other community care centers in selecting an effective and needs-based oncology-specific EHR system as well as to support oncology centers focused on expanding into integrative practices and supportive care. By following the approach described in this article to prioritize needs, healthcare centers of all kinds can be empowered to make informed decisions in the selection of an EHR system.
Because of the relatively small size and integrative nature of the center, efficiency and attention to detail were determined to be of paramount importance in our approach to the selection of a new EHR system. As a result, the first stage of the process was a rigorous and systematic identification of needs.
To accomplish this step, a core team of experts was assembled from within the practice. These individuals were chosen on the basis of their direct experience with the functioning of the center and their potential contribution to its improvement. Those selected were responsible for assisting in the selection process by assessing needs, holding formative meetings, identifying a set of candidate vendors, attending the vendors’ presentations, and participating in summative assessment meetings to select the final product.
The core team consisted of two oncologists, a pharmacist, an information technology manager, the oncology nurse practitioner, and the chief financial officer. This group conducted internet searches and held formal and informal conversations with members of the EHR community. Terms utilized in the online search included oncology medical record, electronic health records, oncology, electronic medical record, and electronic medical record system. The results of this inquiry contributed to a list of potential vendors for consideration, as shown in Table 1. In total, the six core team members held seven meetings over a period of twelve weeks.
After completion of this list, we conducted an initial review of available vendors. The criteria utilized in this review consisted of the primary tools required for the functioning of an oncology-specific care team, including oncology-focused programs and functions such as chemotherapy templates, chemotherapy ordering capabilities, and common toxicity criteria for adverse events. As a result of this review, all vendors that were unable to provide these features were removed from more rigorous consideration, leaving only those best suited to oncology-specific practice.
The remaining vendors then provided a web-based demonstration of their product to the team. The team was blinded to cost information in order to minimize financial considerations in the early phase of the selection process. After each meeting, the team discussed the product, reviewed notes, and used the vendor interaction both from web-based presentations as well as subsequent meetings to continue refining the scope of needs and priorities. During this stage, usability standards became the primary focus as issues arose regarding system designs that hindered use of the program with patients, made the process of recalling information laborious, and made the system incompatible with existing in-office programs and external sources such as Epic systems. This evaluation of issues regarding usability standards led us to drop additional companies from consideration.
Next, the team held summative meetings, and a final criteria worksheet (see Table 2) was developed to display the most important criteria chosen by the team to be used in comparing the remaining EHR systems.
Before the systems were compared, each team member’s understanding of the criteria worksheet was assessed until agreement across team members was achieved. To accomplish this, two meetings were held. In the first meeting, the team engaged in an open discussion in which each member contributed to the identification of criteria most important for meeting the center’s needs. On the basis of this discussion, the initial worksheet was developed. The following questions and concerns were discussed: (a) cloud-based versus in-house server-based data storage, (b) collection and maintenance of both structured and unstructured data, (c) flexibility that would allow the system to remain relevant over time as the practice evolves, (d) total cost (including initial and long-term service, maintenance, and fees), and (e) user-friendliness. Expansion and improvement of services in the future was also an important theme considered during this phase of the process. For the full list of factors chosen as guidelines for selection, see Table 2. The initial worksheet was then emailed to all team members, and they were encouraged to engage with it over the following week by considering the scope of this worksheet and recording comments and feedback to be discussed with the team as a whole. In the second meeting, the team reviewed the worksheet again and considered the additional comments, suggestions, and changes that had been made or proposed during the week. The worksheet was then updated with appropriate alterations, which the team reviewed line by line to confirm that every member had a firm understanding of each component and its influence on the selection.
The final criteria worksheet was then edited to distinguish between negotiable and nonnegotiable items to reflect the importance of certain features relative to others (see Table 2). This organization was determined by team consensus with the intent of reflecting the varying levels of importance of the criteria. Factors such as data storage and user-friendliness were selected as requirements that could not be compromised in selection, whereas elements such as record transfers and research features were determined to be less critical. Items in the worksheet were then organized in descending order with the most important elements at the top. The worksheet was then used, item by item, to further assess the details of each vendor and examine the products’ form and function within the context of our specific community. On the basis of this examination, the list of potential vendors was reduced to a final few that showed the most potential for meeting the needs of our center.
In the last and most rigorous review, a heuristic bubble-sorting algorithm19 was employed to systematically compare the remaining vendors. This method is a simple sorting algorithm often applied in computer programs when values must be sorted into an appropriate order for best use. In this process, values are compared to one another to determine their most effective order. We applied this method to compare the different features of the worksheet according to their relative importance and assign a numeric value between 1 (least) and 13 (most). For example, criteria such as downtime and data storage were given higher values of 11 and 13, whereas others such as research focus and analytics were given 2 and 3. In this process, all elements from the final worksheet were independently compared to one another to identify which item of the pair had the highest importance to our community practice. When one item was selected over another, it received a greater numeric assignment and moved higher in the ranking. This process was repeated until every element had been compared to every other element on the list and each had amassed a set of points. The total for each option was then calculated to reveal revealed which option most effectively met the desired criteria. The option with the highest total score was identified as representing the values and goals of the center most effectively. This process allowed the team a certain amount of protection against bias toward one vendor over another because the process consisted of a systematic classification of needs and focused decision-making.
Compared with other sorting algorithms, such as the bucket sort20 and insertion sort,21 the bubble-sorting technique offers a unique approach that is inclusive and efficient. While the bucket-sorting technique is effective in making comparisons because of its ability to distribute elements into a series of buckets which are then sorted in relation to others, it can often overgeneralize the information. Similarly, while an insertion sort is adaptive and stable because of its focus on individually identifying elements of data and placing them in order, the results are often very linear. Thus, the bubble-sorting method provided a more organic process of discovery that better suited the amount of data under review, more effectively guarded against bias, and successfully led to the final selection of an EHR vendor that suited the center most appropriately.
In the first stage (see Figure 1), 10 vendors were initially identified; four were excluded because of concerns with overall program design including interface configurations and limited interoperability with other systems used within the practice, as well as the lack of oncology-specific tools such as chemotherapy templates. This step left six remaining vendors for consideration.
In the second stage of assessment, the remaining six vendors were further evaluated for usability standards such as interoperability, training support, and user-friendliness, resulting in the elimination of three vendors. The remaining three vendors were then selected as the most favorable options for more in-depth consideration and assessment.
In the third stage, the final criteria worksheet was developed to prioritize needs and more rigorously assess vendors. When the final three vendors were evaluated, one was eliminated because of the cost and challenges involved in the design of its in-house, server-reliant data storage requirements.
In the fourth stage, two vendors remained for comparison. The heuristic bubble-sorting algorithm was utilized to make the final assessment. In this review, one of the vendors proved to be more desirable than the other in a number of key categories listed on the criteria worksheet (see Table 2); therefore, it received the highest total score and was selected to provide the new EHR system at our center.
Discussion and Conclusion
Our goal in this report is to describe the process of EHR selection at a community cancer clinic and provide a framework for other cancer centers embarking on the EHR selection process. Most notably, our clinical process included an emphasis on recording less structured data through free-text narrative responses, which requires the use of systems offering greater flexibility in design and enabling a more authentic way of engaging with the patient. Our clinicians have the responsibility of providing appropriate education regarding integrative practices as well as managing the impact of different approaches on patients with diverse cultural backgrounds. Although these elements mean that our center’s experience is unique, the EHR selection process itself provides a framework for selection that can be adapted to and applied in other clinics and situations. This generalizability is possible because the needs-based orientation of the process allows for an early focus on identifying the specific elements needed in each practice. Therefore, this framework provides an individualized process that can be tailored to the center’s needs to enable the most effective EHR selection process.
With regard to other selection approaches, we found that while EHR implementation experiences are well documented, information about the process of selecting a new EHR system and the challenges resulting from differences in medical environments is limited.22–26 Thus, individual practices or centers must initiate the process with little up-front direction. To be successful, they must first assemble a team of individuals dedicated to constructing the parameters of the selection process. Second, the criteria for assessment must be discussed and ranked in importance. Third, the team must hold formative meetings to evaluate selected criteria. Fourth, each potential EHR vendor under consideration must be tested according to the criteria. As options are narrowed to the final two companies, the fifth step is to employ a sorting algorithm, such as a bubble sort, to systematically assess both options according to their ability to meet the most important needs of the practice. Our description of this process is important because, to our knowledge, no algorithm currently exists to assist in the selection of an EHR system. Therefore, by defining the process or method identified here (see Figure 2), this account can provide a framework for other centers seeking new EHR platforms as well as a reproducible method that may be evaluated as a general selection process for use in other scenarios in the future.
As the healthcare system moves toward value-based care and reimbursement becomes more dependent on the demonstrated value of care, selection of an appropriate EHR system becomes critical to the future success of any healthcare organization. This case study provides a framework that community cancer centers can follow to guide them in the process of selecting a high-functioning oncology-focused EHR system.
EHRs have become a requirement of modern-day healthcare and play a significant role in both the patients’ and practitioners’ experience of care. Despite the importance of EHRs, there is no systematic way to effectively choose which type of EHR system each center requires. Furthermore, as the medical field continues to diversify, the importance of identifying and fulfilling increasingly complex needs to guide quality care is more important than ever. As a result, continued research on effective EHR selection will offer important foundational contributions to the future of healthcare.
In the process of selecting an EHR system for our center, we identified critical gaps in information that could be addressed by the medical and research community. Areas for investigation include the following:
- The need for a formal yet individualized EHR selection system. Although the process we employed demonstrates one approach to EHR selection, the most effective method for this type of decision-making remains to be determined. Identifying a validated algorithm to form the basis of the selection process is important for streamlining and improving the quality of the work that a team must engage in when selecting a new EHR system. Widely shared needs and concerns can be represented in a systematic approach that serves to guide individuals in making the best choices for their workplace. More research is needed to identify how EHR selections can be made most effectively in order to understand what components this algorithm should include.
- Definition of the scope of assessment. When looking to compare options, a team must take time to determine the measures that will guide the evaluation. Questions here include how to assemble a core team for the EHR selection process and how to ensure all relevant vendors are included the selection process. Additional questions include the best ways to prioritize criteria, how to define the selection criteria, and what sorting algorithm to use in the end. Currently, no existing guidelines address these crucial elements. While each group must examine their own needs in order to identify the system that is best suited for their practice, researching the process and defining a general template from which to begin are also important tasks.
- Implementation of technology. This paper does not address the outcomes of implementation, specifically in regard to its application in this integrative environment; therefore, this area requires additional study. After a center selects an EHR, the team should continue to assess and document the experience of implementation and the system’s function and utility. The challenge within the integrative paradigm of this center is the successful integration of unstructured (subjective) information with structured (objective) data, with the goal of providing a more complete picture of individual patient health. Assessing and documenting this integration is important for our center and is valuable throughout healthcare because a systematic and comprehensive narrative of our EHR experience could provide a template for future technological advances and could contribute to the literature about how care is provided.
Our center is in a unique position to conduct and contribute strong research addressing the concerns identified above. Paying heed to approaches presented by Abernathy et al.27 and Sledge et al.,28 the community could structure itself as a learning healthcare system, in which individual patient data, including demographics, history, physiologic data, treatments, outcomes, and unstructured narrative reports from patients, practitioners, and family, are collected. These data are then delivered into an informatics repository capable of aggregating and analyzing the information. If research at this phase is conducted in accordance with rigorous evidence-based standards, the results can be used to develop evidence-based guidelines for treatment. This approach is valuable because guidelines rarely provide evidence-based information for all treatments at all phases of the disease process. Thus, individual centers must fill in the gaps using clinical expertise and consensus to specify treatments in the absence of evidence. Factors such as these are important to include when considering the long-term development of medical care as well as the adoption of a new EHR system. Elements that constitute a learning healthcare system, such as interoperability and diversified patient records, are therefore vital in selecting the most effective EHR platform and must be present throughout the selection process.
Conflict of Interest
There are no competing interests to identify. The authors declare that they have no conflicts of interest in the research.
Protection of Human and Animal Subjects
Human and/or animal subjects were not included in this project.
This paper was funded by Salish Cancer Center.
Molly Stillwell, MA, is a professional writer at Salish Cancer Center in Fife, WA.
Krisstina Gowin, DO, is an assistant professor of Medicine in the Department of Hematology Oncology at the University of Arizona Cancer Center in Tuscon, AZ.
Eiko Klimant, MD, is a medical oncologist at the Salish Cancer Center in Fife, WA.
- Bedrosian, D. “EMR for Hospital-based Oncology Programs: Practical Tips and Strategies for Hospitals.” Oncology Issues (January–February 2006): 26–29.
- DeVore, S., and G. Figlioli. “Lessons Premier Hospitals Learned about Implementing Electronic Health Records.” Health Affairs 29, no. 4 (2010): 664.
- Kohane, I. “What Minecraft Can Teach the Health Care System.” STAT, April 21, 2016. Available at https://www.statnews.com/2016/04/21/minecraft-health-care/.
- Krauss, J., J. Warner, S. Maddux, et al. “Data Sharing to Support the Cancer Journey in the Digital Era.” Journal of Oncology Practice 12, no. 3 (2016): 201–7.
- Kuperman, G. J. “Health-Information Exchange: Why Are We Doing It, and What Are We Doing?” Journal of the American Medical Informatics Association 18, no. 5 (2011): 678–82.
- Institute of Medicine; Laura A. Levit, Erin P. Balogh, Sharyl J. Nass, and Patricia A. Ganz (Editors). Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: National Academies Press, 2013.
- Lorenzi, A., A. Kouroubali, D. Detmer, and M. Bloomrosen. “How to Successfully Select and Implement Electronic Health Records (EHR) in Small Ambulatory Practice Settings.” BMC Medical Informatics and Decision Making 9 (2009): 15.
- Mandel, J., D. Kreda, K. Mandl, et al. “SMART on FHIR: A Standards-based, Interoperable Apps Platform for Electronic Health Records.” Journal of the American Medical Informatics Association 23, no. 5 (2016): 899–908.
- McGinn, C., M. Gagnon, N. Shaw, et al. “Users’ Perspectives of Key Factors to Implementing Electronic Health Records in Canada: A Delphi Study.” BMC Medical Informatics and Decision Making 12 (2013): 105.
- Miriovsky, B. J., L. N. Shulman, and A. P. Abernethy. “Importance of Health Information Technology, Electronic Health Records, and Continuously Aggregating Data to Comparative Effectiveness Research and Learning Health Care.” Journal of Clinical Oncology 30, no. 34 (2012): 4243–48.
- Warner, J., and E. Hochberg. “Where Is the EHR in Oncology?”
Journal of the National Comprehensive Cancer Network 10, no. 5 (2012): 584–88.
- Warner, J., S. Maddux, K. Hughes, K., et al. “Development, Implementation, and Initial Evaluation of a Foundational Open Interoperability Standard for Oncology Treatment Planning and Summarization.” Journal of the American Medical Informatics Association 22, no. 3 (2015): 577–86.
- Krauss, J., J. Warner, S. Maddux, et al. “Data Sharing to Support the Cancer Journey in the Digital Era.”
- Kuperman, G. J. “Health-Information Exchange: Why Are We Doing It, and What Are We Doing?”
- Institute of Medicine; Laura A. Levit, Erin P. Balogh, Sharyl J. Nass, and Patricia A. Ganz (Editors). Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis.
- Bedrosian, D. “EMR for Hospital-based Oncology Programs: Practical Tips and Strategies for Hospitals.”
- DeVore, S., and G. Figlioli. “Lessons Premier Hospitals Learned about Implementing Electronic Health Records.”
- McGinn, C., M. Gagnon, N. Shaw, et al. “Users’ Perspectives of Key Factors to Implementing Electronic Health Records in Canada: A Delphi Study.”
- Shivasankaran, N., P. Kumar, G. Nallakumarasamy, and K. Raja. “A Hybrid Bubble Sorting Simulated Annealing Algorithm for Job Shop Scheduling.” In 2012 Third International Conference on Computing, Communication and Networking Technologies. Available at http://ieeexplore.ieee.org/document/6395981/.
- Cormen, Thomas H., Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. “Bucket Sort.” Section 8.4 in Introduction to Algorithms. 2nd ed. Cambridge, MA: MIT Press and McGraw-Hill, 2001, 174–77.
- Bender, Michael A., Martin Farach-Colton, and Miguel A. Mosteiro. “Insertion Sort Is O(n log n).” Theory of Computing Systems 39, no. 3 (2006): 391–97.
- Adler, K. “How to Select an Electronic Health Record System.” Family Practice Management 12, no. 2 (2006): 5–62.
- Askari, S., and G. Bacigalupe. “E-health Innovations, Collaboration, and Healthcare Disparities: Developing Criteria for Culturally Competent Evaluation.” Families, Systems, & Health 31, no. 3 (2013): 248–63.
- Columbus, S. “Small Practice, Big Decision: Selecting an EHR System for Small Physician Practices.” Journal of AHIMA 77, no. 5 (2006): 42–46. Available at http://library.ahima.org/doc?oid=64036#.V8tCnZMrK35.
- Liebovitz, D. “Meaningful EHR Attributes for an Era of Accountability, Transparency, Shared Decision Making, and Value Assessment.” Journal of Legal Medicine 34 (2013): 43–53.
- Schumacher, R., J. Webb, and K. Johnson. How to Select an Electronic Health Record System That Healthcare Professionals Can Use. User Centric Inc. 2009. Available at http://www.usercentric.com/sites/usercentric.com/files/usercentric-ehr-white-paper.pdf.
- Abernethy, A. P., L. M. Etheredge, P. A. Ganz, et al. “Rapid-Learning System for Cancer Care.” Journal of Clinical Oncology 28, no. 27 (2010): 4268–74.
- Sledge, G. W., C. A. Hudis, S. M. Swain, et al. “ASCO’s Approach to a Learning Health Care System in Oncology.” Journal of Oncology Practice 9, no. 3 (2013): 145–48.
Molly Stillwell, MA; Krisstina Gowin, DO; and Eiko Klimant, MD. “Selection of an Electronic Health Record System for a Community-based Integrative Oncology Center.” Perspectives in Health Information Management (Fall 2018): 1-12.