Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them

Jennifer N. Hill, Sara M. Locatelli, Barbara G. Bokhour, Gemmae M. Fix, Jeffrey Solomon, Nora Mueller, Sherri L. LaVela

The objective of this paper is to demonstrate the utility of the CFIR framework for evaluating broad-scale change by discussing the challenges to be addressed when planning the assessment of broad-scale change and the solutions developed by the evaluation team to address those challenges. The evaluation of implementation of Patient-centered Care and Cultural Transformation (PCC&CT) within the Department of Veterans Affairs (VA) will be used as a demonstrative example. Patient-Centered Care (PCC) is personalized health care that considers a patient's circumstances and goals. The Department of Veterans Affairs (VA) is working towards implementing PCC throughout its healthcare system, comprised of multiple interventions with a singular long-term goal of cultural transformation, however little is known about the factors influencing its implementation. This paper discusses the issues that arose using CFIR to qualitatively assess the factors influencing implementation of cultural transformation. Application of CFIR to this broad-scale evaluation revealed three strategies recommended for use in evaluating implementation of broad-scale change: (1) the need for adapted definitions for CFIR constructs (especially due to new application to broad-scale change), (2) the use of a mixed deductive-inductive approach with thematic coding to capture emergent themes not encompassed by CFIR, and (3) its use for expedited analysis and synthesis for rapid delivery of findings to operational partners. This paper is among the first to describe use of CFIR to guide the evaluation of a broad-scale transformation, as opposed to discrete interventions. The processes and strategies described in this paper provide a detailed example and structured approach that can be utilized and expanded upon by others evaluating implementation of broad-scale evaluations. Although CFIR was the framework selected for this evaluation, the strategies described in this paper including: use of adapted definitions, use of mixed deductive-inductive approach, and the approach for expedited analysis and synthesis can be transferred and tested with other frameworks.