Implementation and Impact of Psychiatric Electronic Medical Records in a Public Medical Center

by Anna Q. Xiao, MD, MHA, and Frank X. Acosta, PhD


Objectives: This study describes the efforts to implement electronic charting in a large public psychiatric outpatient clinic with the objective to improve clinical documentation.

Methods: Data made available through the quality review process are utilized to evaluate the effectiveness of the electronic intervention. The study is a comparative analysis of the three years before and three years after the point of implementation of electronic charting.

Results: Statistical analyses indicate significant findings (p < .0001) in the comparison of the periods before and after implementation in terms of note completion and documentation of medication management, supporting the study’s hypothesis that electronic intervention will improve the quality of clinical documentation.

Conclusions: This study contributes new knowledge to improve our understanding of the barriers and benefits of implementing and maintaining electronic charting in mental health settings.

Keywords: electronic charting, electronic medical records, quality improvement, psychiatric electronic medical records


This research reports on the implementation of psychiatric electronic charting (e-charting) and the assessment of its effects in a psychiatric outpatient clinic at a large public medical center. There is currently a paucity of studies in the psychiatric literature on the implementation and evaluation of e-charting and electronic medical records (EMRs).

The pace of adoption of EMRs has been extremely slow in the field of mental health and slow in the field of healthcare.1, 2 Recently, adoption of at least a basic EMR system among acute care hospitals has shown an upward trend.3 A similar trend has occurred in the past decade among office-based physician practices.4 However, the adoption of EMRs in psychiatry has continued to lag behind that of other provider types, with psychiatry having the lowest use of EMRs across 14 medical specialties according to a recent national survey.5

EMR systems can be classified on the basis of functionalities and can be described as basic (e.g., note writing, demographic data) or comprehensive (e.g., with clinical decision support and operational in every clinical service). How health professionals and the public view the use of EMRs in treatment settings can have major consequences on the acceptability of and motivation for the use of new technology.6–8 An example is seen in a recent international study of nurses’ perception and adoption of EMRs in three public hospitals, in which high underutilization of EMR functionality was found.9

Cogent concerns about privacy, confidentiality, and the misuse of EMRs (e.g., copying and pasting in note writing) have been expressed in the psychiatric literature.10–13 Strong ethical concerns about the inappropriate use of EMRs by medical students and by experienced physicians have been expressed in the nonpsychiatric literature.14–16 However, a literature review concluded that, in general, physicians perceived benefits of the use of new technology.17 In a 2013 survey of primary care physicians on how reviewing psychiatric notes changed their practice and perception of patients with mental illness, 79% reported that they were more likely to monitor and treat potential side effects (e.g., metabolic side effects) of psychiatric medications if they were able to view the psychiatric notes. Also, 97% agreed that reviewing psychiatric notes saved time in encounters with challenging patients.18

We encountered many of the barriers noted in the literature in our efforts to develop and implement e-charting in psychiatric outpatient services. To mitigate barriers and risks, we initiated meetings with medical center leadership in health information management, information technology (IT), and risk management. We also had meetings with attending supervisors, residents, and staff. Important areas such as fear of loss of confidentiality and anxiety over lack of computer skills were discussed in these meetings. We also reviewed state mental health laws, studied the mental health literature, and consulted with forensic experts and risk management experts. We then proceeded to develop a user-friendly e-charting program. Because no universal or standard psychiatric templates or quality improvement (QI) instruments were available, we designed the psychiatric electronic templates and modified our QI checklists that were already in use.

We held a series of meetings at our medical center with IT experts to develop electronic templates. These templates were drafted, circulated, discussed, revised, and finalized on paper by multiple clinical providers and disciplines. The IT experts then designed the electronic prototype, and lead clinicians reviewed the electronic templates together with IT staff and provided feedback to enhance the clinical utility. This process allowed for increased accuracy and completeness of the type of notes that we wanted to design. It also allowed active participation with an increased sense of ownership for providers.

We analyzed some of the findings from QI reviews of medical records after the implementation of the EMR process and compared these findings with those from a similar time period prior to the implementation. The QI review process at the clinic involves an interdisciplinary QI committee. The members are faculty in psychiatry, clinical psychology, social work, and pharmacy and chief residents. The QI committee completes quarterly medical record reviews using QI review checklists of documentation by providers. In addition to the medical record reviews, a pharmacy faculty member completes medication monitoring.

Guiding our analysis was the hypothesis that initiating and maintaining e-charting will improve the quality of clinical documentation.



This study was conducted at a large tertiary public medical center, which is the main training site for more than 870 medical residents in almost all medical specialties. The medical center is among the largest public health centers in the nation. It provides a variety of psychiatric services, including outpatient care for adults and children. The adult outpatient psychiatric clinic is the focus of this study. This clinic is the main rotation site for residents in adult psychiatry for an entire year in the third year of a four-year training program. The clinic is also one of the training sites for forensic fellows and pharmacy residents in the same academic training year.


To monitor the completion and quality of documentation required for different visits and the documentation of medication management, several checklists were designed. Each item in the checklists was then used for group analysis of the percentage of compliance on a quarterly basis. Table 1 lists all the items studied in this project. These items are grouped into three sections: Intake Evaluation Note, Progress Notes, and Medication Monitoring.

For the Intake Evaluation Note review, data for item 1 within the Intake Evaluation Note section in Table 1 were available for review both before and after implementation of e-charting. Items 2 and 3 listed within the Intake Evaluation Note section in Table 1 occurred only in the period after implementation because these elements were built into the e-charting templates. This functionality helped to capture more accurate additional information that is necessary for billing.

Similarly, for the Progress Notes review, items 1 and 3 within the Progress Notes section in Table 1 occurred only in the period after implementation in order to automatically record the use of e-charting related to visits and the timeliness of completion as reflected in the electronic record. For item 3 in the Progress Notes section (“Notes are completed in a timely manner”), the date and time of note completion are built into the electronic record. However, for paper charting, the date and time of note completion were reliant on the clinician’s report.

The Medication Monitoring checklist allows for reviews of the use and efficacy of medication treatment. The items reviewed are items 1 to 6 in the Medication Monitoring section of Table 1.

Period of Study

Two major periods were selected for this study in chronological sequence, consisting of the three years before and the three years after the July 2009 implementation of e-charting. We chose the academic years because of the training schedules of the majority of providers. The academic year was reviewed in four quarters. Available data in each of the years in the periods before and after implementation yielded seven quarters in each period. With the exception of the intake evaluation notes, each of the six years studied across the two periods are represented by at least one quarter. The distribution of quarters within each of the years in these two periods did not affect the statistical analysis of the data.


For the intake evaluation note review, the total number of charts reviewed and analyzed for percentage of compliance was 105 for the period before implementation and 141 for the period after implementation.

For the progress note review, the total number of charts reviewed and analyzed was 106 for the period before implementation and 158 for the period after implementation.

For the medication monitoring review, the total number of charts reviewed and analyzed was 57 for the period before implementation and 63 for the period after implementation.

Data Analysis

Two approaches for analyzing the data were employed. First, to assess the null hypothesis between the periods before and after implementation, the quarterly data for the percentage of compliance were aggregated for the three years before and the three years after the implementation of the EMR. Because the research data were processed and integrated by using total average percentages within each individual quarter, the standard deviation calculation was not easily accomplished with traditional statistical methodology. Thus, an advanced statistical analysis modeling technique, generalized linear modeling, was employed by using the chi-square test (the independent variable was dichotomized as two groups). A normal distribution was assumed for raw data, and the link function was set up as an identical function in the generalized linear model. The PROC GENMOD procedure in SAS 9.2 was implemented for comparison analysis between two time periods. Secondly, in order to study the trend across time, the generalized linear model was employed for both time periods.

For progress note review data, item 6 was not utilized in the analysis of the period after implementation because the factor of legibility of writing was no longer relevant.


The hypothesis for this study is supported by the findings. The hypothesis that initiating and maintaining e-charting will improve the quality of clinical documentation is confirmed through the analysis of the items measured.

Intake Evaluation Note

As shown in Table 2, the intake evaluation note statistic, which documents the completion of intake evaluation, including the signature of a supervisor, showed a significant difference between measurements before and after implementation (p <.0001). This finding indicates improvement in documentation of intake notes.

Figure 1 presents the trend analysis results from generalized linear modeling for the period after implementation. There was no significant finding in the period prior to implementation. However, each of the three intake note items in the period after implementation demonstrated an improvement trend toward full compliance. For example, the documentation of visit time and procedure code (item 3) improved from approximately 60 percent compliance to approximately 100 percent compliance. Item 1 had a slight downward movement in quarter two of year 2011–2012. The reason for this anomaly is not known. However, it did not change the overall trend of improvement.

Progress Notes

Table 2 also lists the parameter estimate, standard error, and p-value from the generalized linear model for five of eight progress note items (items 2, 4, 5, 7, and 8) for the years before and after implementation. Items 1, 3, and 6 were not included because comparison was not possible because of the introduction of the EMR. The findings for the five analyzed items were all highly significant (p < .0001), indicating strong improvement in documentation of progress notes. Table 1 lists the specific items reviewed in QI to assess the content and timely completion of progress notes.

Table 3 provides the generalized linear model analyses for each of the five items (items 2, 4, 5, 7, and 8) in progress notes that could be tested in the three years before and the three years after EMR implementation. Each item is highly significant (p < .0001) in the direction of improved documentation by clinicians. The average range of improvement in the comparison of the periods before and after implementation is considerable and ranges from 21 percent to 50 percent across items.

The trend analysis of progress notes in Table 4 for the period after implementation indicates three significant positive trends in the generalized linear model analysis for timely completion of notes (item 3, p = 0.01), co-signature of notes by supervisors (item 7, p < .0001), and documentation of interpreter services used (item 8, p = 0.004).

Medication Monitoring

The assessment of medication monitoring documentation across six years revealed good documentation in the three years before EMR implementation. However, as can be seen in Table 2, which presents findings from the generalized linear model analyses of all six items, there was further improvement that occurred in the period after EMR implementation that is highly significant (p <.0001) for the level of quality and completeness of notes relevant to medication management.


EMR Use in Different Settings

Our study revealed that EMR use increased the timely completion of medical records in the outpatient setting. Further, our study revealed a positive continuous trend across three years in QI reviews after EMR implementation. However, this finding may not reflect the impact of EMR use in different settings. For example, a study by Park et al. conducted in the emergency department (ED) of a large academic training center over a period of six months found that the use of EMRs increased documentation time fourfold to fivefold, which in turn significantly increased the number of incomplete charts.19 While our study did not measure the time required to complete a note, one possible explanation for the difficulties reported in the study by Park et al. is that the ED setting may not have been conducive to the type of EMR templates that were employed. It is also possible that time demands in the ED, in which patients must be seen rapidly, were a factor contributing to incomplete charts. In an outpatient setting, providers may have more time flexibility to complete documentation. Another factor that may explain the findings of Park et al. is the user learning curve needed to recognize the benefits of EMR use.20 This positive learning curve phenomenon was also found in our study.

EMR Use and Training

The introduction and monitoring of EMR use in residency training in our study proved to be a valuable asset for the training of psychiatrists. Our findings in the trend analysis showed continued learning through improved quality of clinical documentation. In addition, we observed that clinical supervisors had immediate access to review electronic records and to use the information in a more timely fashion as points of training in discussion with residents on clinical management.

E-charting allowed for greater facilitation of monitoring the work of trainees and also providing feedback. This benefit can be an asset for ongoing QI reviews as well.21 For the success of EMR implementation, close and regular monitoring of work completed is essential. For training purposes, the availability of the EMR also reduces the time spent on reading by eliminating legibility issues. In this study, we found that in the period before implementation an average of 37 percent of written records were not legible. In contrast, in the period after implementation of e-charting, legibility was no longer an issue.

Accessibility, affordability, and accountability are among the key demands from the public toward greater efficiency and effectiveness in medical care.22 EMR use is considered a major development in facilitating the achievement of these goals. Physicians play critical roles in evaluating and interacting with new technologies to ensure their proper and efficient use in medical care. Thus, it is important to integrate the use of EMRs early into the training of future physicians. Since recent studies and surveys have pointed more often than not to the enhanced clinical care of patients resulting from the use of electronic technology, it is critical that the current and future generations of physicians and other healthcare professionals become part of the leadership not only in implementation but also in continuous improvement of EMRs. The overall findings in this study will hopefully stimulate further discussion of the value of using EMRs early in the career development of mental health professionals.

Lessons Learned

  1. We found in the present study that engagement of providers in the early planning and implementing process was essential for successful buy-in by both faculty and resident providers. For example, providers offered ideas and feedback that helped lead clinicians to work closely with IT staff to design more user-friendly templates and programs. A few ideas that proved to be helpful in template design were the use of check boxes or the alternative of narrative writing in documenting mental status examinations, and the inclusion of narrative comments by supervisors. In addition, we found that ongoing feedback and problem solving with providers following implementation has made the use of the EMR a more positive experience. Several studies have previously demonstrated the importance of provider buy-in for successful implementation and continuing use of EMRs.23, 24
  2. Our experience showed that a high level of technical and training support by IT staff during the initial days of implementation proved to be invaluable and is recommended during the implementation phase. This finding is consistent with observations reported in the literature.
  3. The cost-benefit ratio consideration in our implementation and use of an EMR system indicated that the benefits have far outweighed the costs involved. For example, (1) considerable savings in labor and material expenses have been realized in the reduction of use of paper and time spent in filing, ordering, pulling, delivering, returning, and maintaining paper records; (2) the electronic templates that were implemented included prompts necessary for accurate billing, and the findings in this study showed continuous improvement in documentation after implementation of e-charting; and (3) the timely completion of records, which were significant findings in this study, resulted in enhanced clinical care and training.

Financial savings data from the use of EMRs were beyond the scope of this study. Our observations and feedback from staff suggest that accuracy in billing was enhanced after the implementation of the EMR system. The quantitative data of financial benefits from EMR use would be an important focus for future research.

A limitation of this study is that it did not assess the comparative amount of time required to complete documentation electronically or by handwriting. Such an assessment would contribute to a better understanding of cost-benefit ratios in the use of EMRs. Some of the potential disadvantages of EMR use include the possibility of more time needed to complete notes, interruption of workflow, and possible concerns of interference with patient rapport. These areas warrant future study.

We think it would be of further value for researchers to conduct qualitative assessment of the perceptions of EMR use by psychiatry residents, attending physicians, and other mental health professionals.


The hypothesis of this study that initiating and maintaining e-charting would improve the quality of clinical documentation was supported by the research findings reported here. The assessment across the three years after implementation of e-charting strongly points to a continual high level of improvement in the quantity and quality of medical records. Quality documentation provides valuable and necessary information for ongoing assessment, continuing care, and continuity of treatment. In this context, improvement in clinical documentation as found in this study will promote positive and promising effects in psychiatric services. The results are consistent in demonstrating the value of implementing and maintaining e-charting in a large outpatient psychiatric clinic and have encouraging implications for other healthcare settings.



The authors extend special thanks to the Quality Improvement Committee members and secretary Ida Flores, and to senior biostatistician Xiaowei Liu, for their assistance.

Authors’ Note

The preliminary data were presented at the 2013 American Psychiatric Association Annual Meeting, May 19, 2013, in San Francisco, CA.


Anna Q. Xiao, MD, MHA, is an assistant professor of clinical psychiatry in the Department of Psychiatry and Behavioral Sciences at the Keck School of Medicine, University of Southern California in Los Angeles, CA.

Frank X. Acosta, PhD, is an associate professor emeritus in the Department of Psychiatry and Behavioral Sciences at the Keck School of Medicine, University of Southern California in Los Angeles, CA.



  1. Jha, A. K., C. M. DesRoches, E. G. Campbell, K. Donelan, R. R. Sowmya, T. G. Ferris, A. Shields, S. Rosenbaum, and D. Blumenthal. “Use of Electronic Health Records in U.S. Hospitals.” New England Journal of Medicine 360 (2009): 1628–38.
  2. DesRoches, C. M., E. G. Campbell, S. R. Rao, K. Donelan, T. G. Ferris, A. Jha, R. Kaushal, D. E. Levy, S. Rosenbaum, A. E. Shields, and D. Blumenthal. “Electronic Health Records in Ambulatory Care—a National Survey of Physicians.” New England Journal of Medicine 359 (2008): 50–60.
  3. Charles, D., M. Furukawa, and M. Hufstader. Electronic Health Record Systems and Intent to Attest to Meaningful Use among Non-federal Acute Care Hospitals in the United States: 2008–2011. Office of the National Coordinator for Health Information Technology (ONC) Data Brief No. 1. February 2012. Available at (accessed December 11, 2015).
  4. Meigs, S., and M. Solomon. “Electronic Health Record Use a Bitter Pill for Many Physicians.” Perspectives in Health Information Management (Winter 2016): 1–17.
  5. Kokkonen, E., S. A. Davids, H .C. Lin, T. S. Dabade, S. R. Feldman, and A. B. Fleischer Jr. “Use of Electronic Medical Records Differs by Specialty and Office Settings.” Journal of the American Medical Informatics Association 20 (2013): e33–e38.
  6. Boyer, L., M. H. Renaud, K. Baumstarck-Barrau, M. Fieschi, and J. C. Samuelian. “Establishment of an Electronic Medical Record in a Psychiatric Hospital: Evolution of Professionals’ Perceptions.” Encephale 36, no. 3 (2010): 236–41.
  7. Gadd, C. S., and L. E. Penrod. “Dichotomy between Physicians’ and Patients’ Attitudes Regarding EMR Use during Outpatient Encounters.” Proceedings of the AMIA Symposium (2000): 275–79.
  8. Gaylin, D. S., A. Moiduddin, S. Mohamound, K. Lundeen, and J. A. Kelly. “Public Attitudes about Health Information Technology, and Its Relationship to Health Care Quality, Costs, and Privacy.” Health Services Research 46, no. 3 (2011): 920–38.
  9. Mahalli, A. E. “Adoption and Barriers to Adoption of Electronic Health Records by Nurses in Three Governmental Hospitals in Eastern Province, Saudi Arabia.” Perspectives in Health Information Management (Fall 2015): 1–16.
  10. Thielke, S., K. Hammond, and S. Helbig. “Copying and Pasting of Examinations within the Electronic Medical Record.” International Journal of Medical Informatics 76S (2007): S122–S128.
  11. Houston, M. “The Psychiatric Medical Record, HIPAA, and the Use of Electronic Medical Records.” Child and Adolescent Psychiatric Clinics of North America 19 (2010): 107–14.
  12. Clemens, N. “Putting Your Medical Record on the Line.” Journal of Psychiatric Practices 12, no. 4 (2006): 250–52.
  13. Peterson, D., and D. Wickeham. “New Challenge for Academic Psychiatry: The Electronic Medical Record.” Academic Psychiatry 35, no. 2 (2011): 76–80.
  14. Brisson, G. E., K. J. Neely, P. D. Tyler, and C. Barnard. “ Should Medical Students Track Former Patients in the Electronic Health Record? An Emerging Ethical Conflict.” Academic Medicine 90, no. 8 (2015): 1020–24.
  15. McLaughlin, K., and S. Coderre. “Finding the Middle Path in Tracking Former Patients in the Electronic Health Record for the Purpose of Learning.” Academic Medicine 90, no. 8 (2015): 1007–9.
  16. Bernat, J. “Ethical and Quality Pitfalls in Electronic Health Records.” American Academy of Neurology 80 (2013): 1057–61.
  17. Anderson, J. “Social, Ethical and Legal Barriers to E-health.” International Journal of Medical Informatics 76 (2007): 480–83.
  18. Bhe, E., S. Summers, M. Pakyurek, M. Soulier, and J. Ferranti. “Outpatient Psychiatric Documentation Use by Primary Care Physicians following De-sensitization in the Electronic Medical Record.” Academic Psychiatry 39 (2015): 639–44.
  19. Park, S. Y., S. Y. Lee, and Y. Chen. “The Effects of EMR Deployment on Doctors’ Work Practices: A Qualitative Study in the Emergency Department of a Teaching Hospital.” International Journal of Medical Informatics 81 (2012): 204–17.
  20. Xiao, N., R. Sharman, R. Singh, G. Singh, A. Danzo, and H. R. Rao. “‘Meaningful Use’ of Ambulatory EMR: Does It Improve the Quality and Efficiency of Health Care?” Health Policy and Technology 1 (2012): 28–34.
  21. Luo, J. S., D. M. Hilty, L. L. Worley, and J. Yager. “Considerations in Change Management Related to Technology.” Academic Psychiatry 30, no. 6 (2006): 465–69.
  22. Srinivasan, M., C. Keenan, and J. Yager. “Visualizing the Future: Technology Competency Development in Clinical Medicine, and Implications for Medical Education.” Academic Psychiatry 30, no. 6 (2006): 480–90.
  23. Fullerton, C., P. Aponte, R. Hopkins, D. Bragg, and D. J. Ballard. “Lessons Learned from Pilot Site Implementation of an Ambulatory Electronic Health Record.” Proceedings (Baylor University Medical Center) 19 (2006): 303–10.
  24. Vishwanath, A., S. R. Singh, and P. Winkelstein. “The Impact of Electronic Medical Record Systems on Outpatient Workflows: A Longitudinal Evaluation of Its Workflow Effects.” International Journal of Medical Informatics 79 (2010): 778–91.

Printer friendly version of this article.


Anna Q. Xiao, MD, MHA, and Frank X. Acosta, PhD. “Implementation and Impact of Psychiatric Electronic Medical Records in a Public Medical Center.” Perspectives in Health Information Management (Fall 2016): 1-14.

Leave a Reply