Abstract
Assessing for positive deviance is one method of identifying individuals, teams, or organizations that perform substantially better than their peers. This approach has been used to support quality-of-care improvement processes in healthcare settings by identifying healthcare team members who perform comparatively well within a given environment and sharing their opinions, actions, and practices with others. This case study presents an adaptable, straightforward framework for identifying positive deviance, or strong performers, within the healthcare setting and is intended for any primary care health system tracking quality measures and aiming to understand the performance of their providers, clinic sites, or organization. Moreover, this protocol does not require the use of more time-consuming methods, such as interviews, and is instead based on repurposing data already being documented in the electronic health record.
Keywords: Electronic Health Record Data, Positive Deviance, Primary Care, Health System, Healthcare Provider, Quality Improvement
Introduction
Positive deviance is a strengths-based approach to identifying individuals within an organization who excel or perform substantially better than their peers, and then understanding the reasons for their better than average performance.1–4 This field of study has been used as a support to the quality of care improvement process in healthcare settings by identifying healthcare team members who perform comparatively well within a given environment, understanding the reasons for their high performance including their attitudes and actions, promoting those positive practices to others within the organization, and determining ways in which positive changes can be sustained.5–9
Background
Several methods exist to identify strong performers, as presented in methodological reviews of peer reviewed publications and/or staff and physician interviews. Regardless of the methods used, all assessments are performed with the goal of identifying individuals or organizations that are performing above average and that serve as good role models for strategies to improve patient care. Finding ways in which the information gathering process can be supported in primary care is important, given the time constraints of busy clinicians and administrators.
This case study examines whether the identification of strong performers can be supported in a rural safety-net health system by retrospective analysis of national standardized quality metrics housed in an electronic health record (EHR). This study took place in Cabin Creek Health Systems, a West Virginia-based federally qualified health center with the mission to promote the health and well-being of all people in their community, especially the most vulnerable, through healthcare guided by science, compassion, and respect, and to contribute to the education of skilled and caring health professionals. The goal of this study was to examine the feasibility and utility of using existing data to identify strong performers among healthcare providers and generate a standardized data analysis methodology that is applicable to any EHR capable of providing quality metrics by provider. For context, this study contributed to the evaluation of a self-measured blood pressure monitoring initiative implemented at Cabin Creek Health Systems. The aim was to gain insights into healthcare providers who excel not only in blood pressure quality measures but also in other potential quality indicators. The concept of positive deviance was central to efforts in reducing hypertension via understanding quality measures in addition to exploring patient/provider dynamics, delivery of care, and lifestyle coaching. Concurrently, this study supports the recent charge to help standardize positive deviance methodologies across research settings.10
Methods
Data Collection
The concept of “positive deviance” can be used to identify providers who perform above average, or exceptionally well, based on a list of pre-determined quality measures that are tracked within an EHR. For this project, we utilized athenahealth. In this example, a .csv file was collected from athenahealth containing selected quality measures for all available providers. Quality measures stem from Health Resources and Services Administration Uniform Data System clinical measures reporting and other national standard adult preventative metrics, including:
- Blood pressure control among patients with diabetes (systolic <140, diastolic, <90)
- Breast cancer screening
- Cervical cancer screening
- Colorectal cancer screening
- Comprehensive diabetic foot exam
- Controlling high blood pressure
- Counseling medication adherence for patients on a statin
- Diabetes: HbA1C poor control (>9 percent)
- HIV (Human Immunodeficiency Virus) screening
- Ischemic vascular disease (IVD): Use of aspirin or another antiplatelet
- Lipid monitoring for patients with atherosclerotic cardiovascular disease (ASCVD)
- Preventative care and screening: body mass index (BMI) screening and follow-up plan
- Preventative care and screening: screening for depression and follow-up plan
- Statin therapy for the prevention and treatment of cardiovascular disease
- Tobacco use: screening and cessation intervention
Data Cleaning and Quality Checks
Data cleaning and quality checks may be necessary. For example, the data files used in this project included key pieces of data, such as primary care provider and measure name, but also included a variety of unnecessary fields that were omitted as they were not pertinent to identifying strong performers. In this example, the data file was connected to Tableau, and satisfied rates for performance measures by provider were distributed along box and whisker plots to determine quartiles for each quality measure. Any software with the ability to create box and whisker plots visualizing data quartiles could be used in place of Tableau. Physicians with little or no data across quality measures were removed, including providers who were no longer with the health system or providers who did not engage in care delivery related to the quality measures of interest. Data cleaning led to the removal of one provider with no available data. In Tableau, the exclusion of a provider removes them from all quality measures, but the data cleaning process is adaptable and subjective depending on the methods and/or software used.
Data Organization and Visualization
Based on the box and whisker plots for each quality measure created in Tableau, a “Top Quartile Score” column was added to the data file scoring providers who appeared within the upper quartile with a score of “1” and providers outside of the upper quartile with a score of “0” for each quality measure. Each provider could get a maximum score of equal to the number of quality measures, in this case 15, or a minimum score of 0. This file, and the quartile scores within, was used to color code each provider according to the number of quality measures they were marked within the “top quartile,” or as a top performer. In the case of this study, strong performers were identified as those providers scoring in the top quartile in 12 or more (80 percent) of the 15 quality measures examined.
Results
Figure 1 represents output from Tableau scoring providers by their top quartile score. Each dot within the box and whisker plot represents a single provider and is colored according to the number of quality measures for which that provider placed within the top quartile. Darker green indicates higher scores for quality measures and becomes more yellow with fewer top quartile satisfied rates. Provider 16 consistently achieved above-average rankings on 12 of 15 quality measures, demonstrating a strong performance compared to other providers within the same health system.
Discussion
A broad range of publications exist surrounding and evaluating the concept of positive deviance, but three primary steps for identifying strong performers exist: review of methodology; performer identification; and the determination of commonalities. A scoping review of 1,140 studies determined that most studies used objective measures of health or survey-based responses to identify positive deviants and focused primarily on identifying positive deviants within individual patient outcomes.10 At the organizational level, a study demonstrated that you can identify the best and worst performing primary healthcare centers by utilizing semi-structured and in-depth interviews with managerial and clinical staff from each of the primary healthcare centers.11 Additionally, several studies utilize systematic review, empirical studies, and interviews to apply the same concept of “performer identification” at the provider and staff level.4,12,13 Lastly, a review of literature is often used to determine commonalities, including things like strategies, procedures, and routines.14,15
As outlined above, the three primary methods used to identify strong performers include the review of methodology, performer identification, and the determination of commonalities, often using scoping literature review, surveys, or informant interviews. This manuscript describes an alternative method of identifying strong performers by repurposing quality outcome measures housed within EHR data. By re-purposing these data, one can identify individuals who are performing exceptionally well without implementing surveys or interviews. Of note, once the positive deviants have been identified, surveys and informant interviews are often the next best step to determine why each individual is performing at the level they are at.
A few limitations exist for this case study. Specifically, the study assesses providers within a single health system composed of six clinic sites, rather than comparing between organizations. Because of this, the data collected and repurposed all come from the EHR, athenahealth. Each EHR collects and stores data in different ways, with athenahealth specifically housing and labeling the quality outcomes measures utilized by many federal agencies for quality improvement reporting. Due to these limitations, we present this work as a generalizable method of determining positive deviance at the individual levels.
Conclusion
This case study represents a pragmatic, easy to implement, methodology aimed at any primary care health system tracking quality measures across a variety of providers, and aiming to understand the performance of their individuals, clinic sites, or organization. This protocol does not require the use of more time-consuming methods, such as surveys or interviews, and is instead based on repurposing data from quality measures likely already being documented in the EHR.
References
1. O’Malley, R., O’Connor, P., Madden, C, & Lydon, S. A Systematic Review of the Use of Positive Deviance Approaches in Primary Care; 2022. Family Practice, 39(3), 493-503.
2. Goff, S., Mazor, K., Priya, A., Moran, M., Pekow, P., & Lindenauer, P. Organizational Characteristics Associated with High Performance on Quality Measures in Pediatric Primary Care: A Positive Deviance Study; 2021. Healthcare Management Review, 46(3), 196-205. https://doi.org/10.1097/HMR.0000000000000247.
3. Foster, B. A., Seeley, K., Davis, M., & Boone-Heinonen, J. Positive deviance in health and medical research on individual level outcomes - a review of methodology; 2022. Annals of Epidemiology, 69, 48–56. https://doi.org/10.1016/J.ANNEPIDEM.2021.12.001.
4. Toscos, T., Carpenter, M., Flanagan, M., Kunjan, K., & Doebbeling, B. N. Identifying Successful Practices to Overcome Access to Care Challenges in Community Health Centers: A “Positive Deviance” Approach; 2018. Health Services Research and Managerial Epidemiology, 5, 233339281774340. https://doi.org/10.1177/2333392817743406.
5. Baxter, R., Taylor, N., Kellar, I., & Lawton, R. What methods are used to apply positive deviance within healthcare organisations? A systematic review; 2016. BMJ Quality & Safety, 25(3), 190–201. https://doi.org/10.1136/bmjqs-2015-004386.
6. Rose, A. J., & McCullough, M. B. A Practical Guide to Using the Positive Deviance Method in Health Services Research; 2017. Health Services Research, 52(3), 1207–1222. https://doi.org/10.1111/1475-6773.12524.
7. Lindberg, C. M., Lindberg, C. C., D’Agata, E. M. C., Esposito, B., & Downham, G. Advancing Antimicrobial Stewardship in Outpatient Dialysis Centers Using the Positive Deviance Process; 2019. Nephrology Nursing Journal: Journal of the American Nephrology Nurses’ Association, 46(5), 511–518.
8. D’Agata, E. M. C., Lindberg, C. C., Lindberg, C. M., Downham, G., Esposito, B., Shemin, D., & Rosen, S. The positive effects of an antimicrobial stewardship program targeting outpatient hemodialysis facilities; 2018. Infection Control & Hospital Epidemiology, 39(12), 1400–1405. https://doi.org/10.1017/ice.2018.237.
9. Lindberg, C., Norstrand, P., Munger, M., DeMarsico, C., & Buscell, P. (n.d.). Letting Go, Gaining Control: Positive Deviance and MRSA Prevention. Available at: https://static1.squarespace.com/static/5a1eeb26fe54ef288246a688/t/5df46b424b14fd7c686a67b2/1576299330698/Lindberg+-+Letting+Go+Gaining+Control+-+PD+and+MRSA+Prevention+-+Clinical+Leader+12-09+FINAL.pdf.
10. Foster, B. A., Seeley, K., Davis, M., & Boone-Heinonen, J. Positive deviance in health and medical research on individual level outcomes – a review of methodology; 2022. Annals of Epidemiology, 69, 48–56. https://doi.org/10.1016/j.annepidem.2021.12.001.
11. Lewis, T. P., Aryal, A., Mehata, S., Thapa, A., Yousafzai, A. K., & Kruk, M. E. Best and worst performing health facilities: A positive deviance analysis of perceived drivers of primary care performance in Nepal; 2022. Social Science & Medicine (1982), 309. https://doi.org/10.1016/J.SOCSCIMED.2022.115251.
12. Cohen, R., Gesser-Edelsburg, A., Singhal, A., Benenson, S., & Moses, A. E. Translating a theory-based positive deviance approach into an applied tool: Mitigating barriers among health professionals (HPs) regarding infection prevention and control (IPC) guidelines; 2022. PloS One, 17(6). https://doi.org/10.1371/JOURNAL.PONE.0269124.
13. Ellenbogen, M. I., Wiegand, A. A., Austin, J. M., Schoenborn, N. L., Kodavarti, N., & Segal, J. B. Reducing Overuse by Healthcare Systems: A Positive Deviance Analysis; 2023. Journal of General Internal Medicine. https://doi.org/10.1007/S11606-023-08060-3.
14. Singh, S., Mazor, K. M., & Fisher, K. A. Positive deviance approaches to improving vaccination coverage rates within healthcare systems: a systematic review; 2019. Journal of Comparative Effectiveness Research, 8(13), 1055–1065. https://doi.org/10.2217/CER-2019-0056.
15. de Kok, E., Weggelaar-Jansen, A. M., Schoonhoven, L., & Lalleman, P. A scoping review of rebel nurse leadership: Descriptions, competences and stimulating/hindering factors; 2021. Journal of Clinical Nursing, 30(17–18), 2563–2583. https://doi.org/10.1111/JOCN.15765.
Figure 1. Quartile scoring by primary care provider
Figure 1 (also available online) displays quartile scoring by primary provider. Each dot within the box and whisker plot represents a single provider and is colored according to the number of quality measures for which that provider placed within the top quartile. Darker green indicates higher scores for quality measures and becomes more yellow with fewer top quartile satisfied rates.
Author Biographies
Adam Baus, PhD, MA, MPH, (abaus@hsc.wvu.edu) is a research assistant professor in the Department of Social and Behavioral Sciences at the West Virginia University School of Public Health and the director of the West Virginia University School of Public Health, Office of Health Services Research.
Andrea Calkins, MPH, is a program coordinator senior with the West Virginia University School of Public Health, Office of Health Services Research.
Cecil Pollard, MA, is the assistant director of the West Virginia University School of Public Health, Office of Health Services Research.
Craig Robinson, MPH, is the executive director of Cabin Creek Health Systems.
Robin Seabury, MS, is the former hypertension care manager of Cabin Creek Health Systems.
Jessica McColley, DO, is the chief medical officer of Cabin Creek Health Systems.
Marcus Thygeson, MD, MPH, is the executive director of Adaptive Health.
Curt Lindberg, MHA, DMan, is the principal and senior consultant of Partners in Complexity.
Andrya Durr, PhD, is a research specialist with the West Virginia University School of Public Health, Office of Health Services Research.