by Anjum Khurshid, PhD, MPAff, MBBS; Mark L. Diana, PhD, MSIS, MBA; and Rahul Jain, MPH, CPHIMS
Objective—To study the extent to which community health information exchanges (HIEs) deliver and measure return on investment (ROI) and improvements in the quality of care.
Materials and Methods—We surveyed operational HIEs for their characteristics, information domains, impact on quality of care, and ROI.
Results—A 60 percent response rate was achieved. Two-thirds of respondents agreed that community HIEs demonstrated a positive ROI, while one-third had no opinion or disagreed. One-fourth or fewer respondents reported using various metrics to calculate ROI. Most respondents agreed that HIEs improve the quality of care, though several were not sure and were awaiting further evidence. Most respondents indicated that they did not deliver reports on quality measures (76 percent) and that data were not being used to measure quality performance of participating providers (73 percent).
Discussion—Respondents from most HIEs believe that the HIEs are demonstrating a positive ROI; however, a minority of them indicated they had used or will use specific metrics to calculate ROI. HIE representatives overwhelmingly reported that they believe the HIE activities improve the quality of healthcare delivered, but only a few are using data to evaluate provider performance or generate reports on quality measures.
Conclusion—This study demonstrates the challenge faced by policy makers and healthcare organizations that are investing millions of dollars in HIEs that are believed to improve health outcomes and increase efficiency, but still need more time to develop the evidence to confirm that belief. Our study shows that calculating ROI for HIEs or their impact on quality of care remains a secondary priority for most HIEs. This finding raises serious questions for the sustained support of HIEs, both financially and as a policy lever, given the end of Health Information Technology for Economic and Clinical Health (HITECH) Act funding.
Keywords: Health information exchange; HITECH; return on investment; quality measures; New Orleans
The number of health information exchanges (HIEs) in the United States has grown appreciably in the last few years. A major part of this increase may be attributed to the Health Information Technology for Economic and Clinical Health (HITECH) Act (2009), which supported the development of many new HIEs, particularly as state-designated entities,1, 2 and to an increased emphasis on health information technology in the Affordable Care Act. Significant progress has been made in establishing standards and building platforms for operationalizing these HIEs, yet many HIEs still face daunting challenges to their survival and sustainability.3, 4 While the value HIEs bring to a fragmented healthcare system is generally recognized, questions related to how to estimate, operationalize, and sustain such value and who should pay for this value remain topics of much debate. At the same time, participation in HIEs continues to be an integral part of several federal and state-level regulations, including meaningful use requirements and those related to accountable care organizations.5, 6
In addition to public or community HIEs that were developed through government funding, most large healthcare delivery systems are engaged in setting up private HIEs to connect their various internal information systems and external partners.7–10 As the HITECH funds come close to running out and as healthcare entities continue to pay for or plan for operating new HIEs, the question of the value of, and the return on, such investments remains unclear.11–15 Simultaneously, the need to address the fragmentation of the healthcare system, which leads to silos of patient data in healthcare organizations that do not want to share data because of business, legal, or ethical reasons, remains a key driver of costs, inefficiencies, and poor outcomes.16–18 An increased, albeit limited, demand by motivated patients and their families for greater access to their health data and improved transparency around quality and performance of the health system is driving the need for more community-wide or regional exchange of health information in a secure and confidential manner.19–25
For the continued survival and expansion of HIEs in states and regions, the business models are still evolving because many of the HIEs in operation today were initiated as a result of a grant or one-time investment, either by local partners or from the HITECH Act.26–30 Most HIEs struggle with the question of who pays for what and why. However, a prerequisite to answering these questions is the capacity of HIEs to measure and demonstrate the benefits to patients, service providers, payers, and the community at large. The purpose of this study is to provide a snapshot of the extent to which community HIEs are delivering or employing metrics to demonstrate return on investment (ROI) and improvements in the quality of care.
A survey instrument was developed to capture data for analysis. The survey instrument used in a prior study provided a foundation for iterative analyses.31 Additional questions were developed by the authors on the basis of their primary experience with the operational Greater New Orleans Health Information Exchange (GNOHIE), a comprehensive literature review, and consultation with fellow academics engaged in HIE research. The survey instrument consisted of 47 questions. The first portion of the survey instrument focused on organizational demographics and characteristics, and the second portion focused on sustainability, quality of care, and the organizations’ use of metrics related to ROI.
The survey instrument was built into the Qualtrics online survey platform at the Louisiana Public Health Institute. Four doctoral students studying health information technology in the Department of Global Health Management and Policy at the Tulane University School of Public Health and Tropical Medicine validated the questions selected for inclusion in the survey instrument. Adjustments were made to the questions to ensure appropriateness of terminology, and to maximize clarity to produce measurable responses.
Prospective respondents were identified for inclusion in the study through a screening process. Three published lists of HIEs were selected from a literature review: EHR Intelligence by Xtelligent Media, eHealth Initiative’s directory of HIEs, and the Office of the National Coordinator for Health Information Technology (ONC) State Health Information Exchange Cooperative Agreement Program.32 These lists were consolidated into a single list of HIEs eligible for screening. Contact phone numbers or website addresses were extracted from the published lists, if available. If no contacts or website addresses were documented in the lists, the Google search engine was used to locate the HIE contacts. In instances where a listed HIE was not located, the words “Health Information Exchange” and the city or state of the listed HIE were queried in Google to identify contact information.
Prospective respondents were contacted by telephone and e-mail. If the identified primary contact was unavailable, an attempt was made to gather the contact information of a substitute respondent. We used a scripted set of questions to ask if the respondents identified the HIE as operational and if they were willing to complete the survey instrument. If the HIE was described as operational, it was included in the denominator of respondents. Fifty-eight potential respondents identified their organization’s HIE as being operational and agreed to complete the survey instrument. One organization was excluded because it only provided services to other HIEs. Fifty-seven contacts at operational HIEs verified their active e-mail addresses and were notified that they would receive the survey instrument.
These 57 participants were contacted in November of 2013. Follow up e-mails were sent to all participants as a reminder of the screening and to indicate that they should have received a survey instrument. For participants who had not received the survey instrument, a general link to the survey instrument was re-sent to mitigate challenges associated with e-mail filters. Follow-up e-mails and phone calls were made periodically until January 2014 to generate a larger response rate. Forty-three respondents answered one question on the survey, 35 answered the majority of questions, and 34 answered all of the questions.
Overall, 34 respondents completed the full survey, but 35 respondents completed significant portions of the survey, so we calculate the response rate as 60 percent. Most respondents (94 percent) had 50 or fewer full-time equivalent staff (see Figure 1), with annual operating budgets of $1,000,000 or less (32 percent), between $1,000,001 and $5,000,000 (46 percent), or greater than $5,000,000 (23 percent), as shown in Figure 2. Most HIEs served 1,001–100,000 (21 percent), 100,001–1,000,000 (38 percent), or more than 1,000,000 (38 percent) patients or customers annually (see Figure 3). The most common legal entity model, as shown in Figure 4, was a nonprofit corporation (69 percent), followed by government owned (11 percent) and virtual with no legal entity (11 percent); limited liability corporation, trust, or partnership (6 percent); other models (6 percent); and for-profit corporation (3 percent). The respondents were geographically spread throughout the country. A map of the locations of the HIEs is shown in Figure 5.
The information domains exchanged by HIEs are presented in Figure 6. The majority of respondents exchange patient demographics (94 percent), problems and conditions (85 percent), allergy and drug sensitivities (85 percent), or encounter (82 percent) information. Most respondents exchanged information on health care providers (65 percent), medications (76 percent), immunizations (74 percent), vital signs (68 percent), results (76 percent), or procedures (72 percent). Few respondents reported exchanging information regarding supporting relationships (29 percent), language spoken (26 percent), health insurance providers (41 percent), advance directives (35 percent), or social history (21 percent).
The results of responses to questions about HIE participation and ROI are presented in Table 1. Two-thirds (66 percent) of respondents agreed that based on their HIE’s own performance, community HIEs demonstrated a positive ROI, while one-third (33 percent) had no opinion or disagreed. Generally one-fourth or fewer respondents reported using metrics to calculate ROI. The most common metrics that were used for calculating ROI were labor savings (27 percent), improved communication and better coordination (27 percent), reduced duplication of tests or procedures (24 percent), and improved health outcomes (21 percent). Other metrics indicated included reduced transaction costs (15 percent), reduced medical errors (12 percent), and improved medical adherence (6 percent).
The results of responses to questions about HIE participation and quality of care are presented in Table 2. The majority of respondents (64 percent) agreed that based on the experience of their organization, community HIEs improve the quality of care. No respondents disagreed that HIEs improve quality of healthcare, but 36 percent of respondents were not sure and were awaiting further evidence. Most respondents (73 percent) indicated that data were not being used to measure quality performance of participating providers, while 27 percent of respondents indicated that data were being used for this purpose. Lastly, most respondents (76 percent) indicated that they did not deliver reports on quality measures.
The purpose of this study is to determine whether community HIEs are employing metrics to demonstrate ROI and improvements in the quality of care. We did so by surveying community HIEs that were identified as operational, and then surveying the domains of information these operational HIEs are exchanging.
Our major finding is that while respondents from most community HIEs believe their HIEs are demonstrating a positive ROI, a minority of them indicated they had used or will use specific metrics to calculate ROI. Generally, less than a quarter of respondents indicated that they had used or would use metrics to calculate ROI. This finding suggests that even with the proliferation of HIEs supported by both the HITECH Act and the increased use of HIEs by health systems, evidence about the ability of HIEs to achieve a positive ROI is still lacking.
Similarly, community HIE respondents overwhelmingly report that they believe their HIEs’ activities improve the quality of healthcare delivered, but only a small minority are actually using data to evaluate provider performance or generate reports on quality measures. This finding most likely reflects the difficulties of generating agreement among providers about appropriate quality metrics, and competitive concerns around the transparent sharing of quality performance measures.
These findings, taken in combination with the ending of HITECH funding, raise serious questions about the future sustainability of HIEs, and what form they will take. If community HIEs cannot demonstrate a positive ROI or improvements in quality of care, they may find community support eroding and be unable to generate sufficient revenue to sustain operations. Under such a scenario, health systems would need to find other alternatives for exchanging health information to meet meaningful use requirements, and to achieve other strategic objectives, including improved performance and greater physician alignment.
A majority of the HIEs in our study serve more than a million patients, which may explain the policy to support establishing of community HIEs, including statewide HIEs. The ability to connect disparate health information systems seamlessly and securely, through a community HIE infrastructure, remains an important function that HIEs can perform. Our results also show that the cost of maintaining these HIEs is significant, with most ranging between $1 million and $5 million. The question that remains is who should foot the bill for a community resource of which the ROI and impact on quality are more a belief, based on logic and anecdotes, rather than a fact based on measurable metrics.
Our study has several notable limitations. While we achieved a response rate of 60 percent, the possibility of nonresponse bias remains. Other limitations include the potential variation in knowledge of the individuals completing the survey and the possibility that the questions about ROI and impact on quality are being asked too early in the lifecycle of such projects.
We repeated this study after almost three years, from the early days of the distribution of HITECH funds to the end of this funding stream. 33 While many more HIEs have been started or have become operational during this time, the progress on the measurement of ROI and improved quality of care has been rather slow. This finding may lead to the logical result that the HIEs are an investment that takes too long to achieve optimal functionality. Further, the concept of a shared community infrastructure, which allows for exchange of information among competing and collaborating providers, may be an outdated and impractical solution to a problem for which we need to find new solutions. Whatever the answer, it is still important and worthwhile to determine whether the millions of dollars invested in community HIEs should be considered a sunk cost or should be followed with renewed support.
In conclusion, despite the growth in operational community HIEs, they have not made significant progress on the ability to calculate ROI or to demonstrate improvements in the quality of care. The need to find affordable, effective, and sustainable solutions to the lack of connectivity and exchange of health information remains a major policy challenge in our fragmented and siloed healthcare delivery system. Given that significant federal support for HIEs through the HITECH Act has largely ended, this finding raises concerns about the future sustainability of these organizations, and the extent and form of HIEs in the future. It also creates a policy conundrum for federal decision makers as to whether to invest further in supporting these community infrastructures or to alter the course of action.
Anjum Khurshid, PhD, MPAff, MBBS, is a senior health systems strategist at the Louisiana Public Health Institute in New Orleans, LA.
Mark L. Diana, PhD, MSIS, MBA, is an associate professor in the School of Public Health and Tropical Medicine at Tulane University in New Orleans, LA.
Rahul Jain, MPH, CPHIMS, is a digital health analytics manager at the Louisiana Public Health Institute in New Orleans, LA.
Support: A portion of this project was funded by the Crescent City Beacon Community, sponsored by the Office of the National Coordinator for Health Information Technology, US Department of Health and Human Services.
Acknowledgments: The authors would like to acknowledge Julia Adler-Milstein for her help in survey development.
Anjum Khurshid, PhD, MPAff, MBBS; Mark L. Diana, PhD, MSIS, MBA; and Rahul Jain, MPH, CPHIMS. “Health Information Exchange Readiness for Demonstrating Return on Investment and Quality of Care.” Perspectives in Health Information Management (Fall 2015): 1-15.