Evidence-based medicine – an appropriate tool for evidence-based health policy? A case study from Norway

Kirsti Malterud, Anne Karen Bjelland, Kari Tove Elvbakken

Evidence-based policy (EBP), a concept modelled on the principles of evidence-based medicine (EBM), is widely used in different areas of policymaking. Systematic reviews (SRs) with meta-analyses gradually became the methods of choice for synthesizing research evidence about interventions and judgements about quality of evidence and strength of recommendations. Critics have argued that the relation between research evidence and service policies is weak, and that the notion of EBP rests on a misunderstanding of policy processes. Having explored EBM standards and knowledge requirements for health policy decision-making, we present an empirical point of departure for discussing the relationship between EBM and EBP. In a case study exploring the Norwegian Knowledge Centre for the Health Services (NOKC), an independent government unit, we first searched for information about the background and development of the NOKC to establish a research context. We then identified, selected and organized official NOKC publications as an empirical sample of typical top-of-the-line knowledge delivery adhering to EBM standards. Finally, we explored conclusions in this type of publication, specifically addressing their potential as policy decision tools. From a total sample of 151 SRs published by the NOKC in the period 2004-2013, a purposive subsample from 2012 (14 publications) advised major caution about their conclusions because of the quality or relevance of the underlying documentation. Although the case study did not include a systematic investigation of uptake and policy consequences, SRs were found to be inappropriate as universal tools for health policy decision-making. The case study demonstrates that EBM is not necessarily suited to knowledge provision for every kind of policy decision-making. Our analysis raises the question of whether the evidence-based movement, represented here by an independent government organization, undertakes too broad a range of commissions using strategies that seem too confined. Policymaking in healthcare should be based on relevant and transparent knowledge, taking due account of the context of the intervention. However, we do not share the belief that the complex and messy nature of policy processes in general is compatible with the standards of EBM.