Using the Results from Rigorous Multisite Evaluations to Inform Local Policy Decisions.
Evidence‐based policy at the local level requires predicting the impact of an intervention to inform whether it should be adopted. Increasingly, local policymakers have access to published research evaluating the effectiveness of policy interventions from national research clearinghouses that review...
| Publicado en: | Journal of Policy Analysis & Management Vol. 38; no. 4; pp. 978 - 1004 |
|---|---|
| Autores principales: | , , , , , |
| Formato: | Artículo |
| Publicado: |
Wiley-Blackwell
Sep2019
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=138457090&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 138457090 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 02768739 JPA jtl: Journal of Policy Analysis & Management issn: 02768739 maglogo: Y pubinfo: dt: Sep2019 vid: 38 iid: 4 pid: 480 pub: Wiley-Blackwell artinfo: ui: 138457090 10.1002/pam.22154 ppf: 978 ppct: 26 formats: tig: atl: Using the Results from Rigorous Multisite Evaluations to Inform Local Policy Decisions. aug: au: Orr, Larry L. Olsen, Robert B. Bell, Stephen H. Schmid, Ian Shivji, Azim Stuart, Elizabeth A. su: Sampling errors Educational evaluation sug: subj: Sampling errors Educational evaluation ab: Evidence‐based policy at the local level requires predicting the impact of an intervention to inform whether it should be adopted. Increasingly, local policymakers have access to published research evaluating the effectiveness of policy interventions from national research clearinghouses that review and disseminate evidence from program evaluations. Through these evaluations, local policymakers have a wealth of evidence describing what works, but not necessarily where. Multisite evaluations may produce unbiased estimates of the average impact of an intervention in the study sample and still produce inaccurate predictions of the impact for localities outside the sample for two reasons: (1) the impact of the intervention may vary across localities, and (2) the evaluation estimate is subject to sampling error. Unfortunately, there is relatively little evidence on how much the impacts of policy interventions vary from one locality to another and almost no evidence on the implications of this variation for the accuracy with which the local impact of adopting an intervention can be predicted using findings from an evaluation in other localities. In this paper, we present a set of methods for quantifying the accuracy of the local predictions that can be obtained using the results of multisite randomized trials and for assessing the likelihood that prediction errors will lead to errors in local policy decisions. We demonstrate these methods using three evaluations of educational interventions, providing the first empirical evidence of the ability to use multisite evaluations to predict impacts in individual localities—i.e., the ability of "evidence‐based policy" to improve local policy. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|