Optimizing the post-graduate institutional program evaluation process

Monica L. Lypson, Mark E. P. Prince, Steven J. Kasten, Nicholas H. Osborne, Richard H. Cohan, Terry Kowalenko, Paul J. Dougherty, R. Kevin Reynolds, M. Catherine Spires, Jeffrey H. Kozlow, Scott D. Gitlin

Reviewing program educational efforts is an important component of postgraduate medical education program accreditation. The post-graduate review process has evolved over time to include centralized oversight based on accreditation standards. The institutional review process and the impact on participating faculty are topics not well described in the literature. We conducted multiple Plan-Do-Study-Act (PDSA) cycles to identify and implement areas for change to improve productivity in our institutional program review committee. We also conducted one focus group and six in-person interviews with 18 committee members to explore their perspectives on the committee's evolution. One author (MLL) reviewed the transcripts and performed the initial thematic coding with a PhD level research associate and identified and categorized themes. These themes were confirmed by all participating committee members upon review of a detailed summary. Emergent themes were triangulated with the University of Michigan Medical School's Admissions Executive Committee (AEC). We present an overview of adopted new practices to the educational program evaluation process at the University of Michigan Health System that includes standardization of meetings, inclusion of resident members, development of area content experts, solicitation of committed committee members, transition from paper to electronic committee materials, and focus on continuous improvement. Faculty and resident committee members identified multiple improvement areas including the ability to provide high quality reviews of training programs, personal and professional development, and improved feedback from program trainees. A standing committee that utilizes the expertise of a group of committed faculty members and which includes formal resident membership has significant advantages over ad hoc or other organizational structures for program evaluation committees.