OECD Presentation by Stéphane Jacobzone on Mobilising Evidence for Good Governance. For further information see: http://www.oecd.org/gov/mobilising-evidence-for-good-governance-3f6f736b-en.htm

1. MOBILISING EVIDENCE FOR GOOD GOVERNANCE
TAKING STOCK OF PRINCIPLES AND STANDARDS FOR POLICY
DESIGN, IMPLEMENTATION AND EVALUATION
Stéphane Jacobzone
Head of Evidence, Monitoring and Evaluation
OECD Public Governance Directorate
With special thanks to Daniel Acquah and Lizeth Fúquene
Conference « Evidence for Policy Makers »
Science Works 16 December 2020, The Hague

2.  A critical role to improve public sector effectiveness
 Contemporary policy problems:
 Multidimensional challenges
 Social Fragmentation
 Institutional complexity,
 Scientific uncertainty.
 Trust in government has been eroded in many countries
 COVID 19 pandemic: Greater need for evidence and accelerated digitalisation
 Challenges of multiple information sources, social media, and Disinformation
(i.e. fake news)
WHY DO WE NEED QUALITY EVIDENCE ?

3. THE ROLE OF “WHAT WORKS” APPROACHES /
EVIDENCE SYNTHESIS
 Not all evidence is equal!
 The role of knowledge brokers is to figure out which evidence is
robust and communicate this to decision makers.
GENERATE
• Synthesis of
evidence
• Identifying &
addressing
knowledge
gaps
TRANSLATE
• Diffusion
• Dissemination
• Standards of evidence
ADOPT
• Capability
building
• Network
building
Strategic functions of Knowledge
Brokers to stimulate EIPM
BUT POLICY
MAKING WILL
REMAIN MESSY!
EIPM AT BEST!

4. Evidence plays a role throughout the policy cycle. It has a critical role to play in
improving the quality, responsiveness, and accessibility of public interventions,
services or programmes.
ACHIEVING THE PROMISE OF EVIDENCE-
INFORMED POLICY-MAKING
Policy design:
• ‘Policy memory’: an
understanding of where
challenges have been
experienced in the past and
what previous good practice
could be incorporated into the
current reform effort.
Implementation of public
interventions
•Policies require significant
planning and management.
Evidence provides an
understanding of how policies
should be adapted to meet local
needs.
Evaluation of public
policies
•To understand why some
complex policies work and why
others do not.
The need to create a supportive
institutional environment :
 Creating skills and investing in
capacity (“Building Capacity for
EIPM”)
 Creating conditions for
Institutionalisation, Use and Quality of
Evaluation (“Improving Governance
through Policy Evaluation”)

5.  Proliferation of different approaches to standards of evidence
 Stakeholders are confused about the difference between the standards of
evidence used by different organisations (Puttick, 2018[31]; Means et al.,
2015[32]). This can even happen within the same government, or across
two sides of the same ministry.
 NESTA existing attempt to provide a stocktaking of standards at UK
National level.
 Need to look at multiple dimensions: not only efficacy and effectiveness
but also design, implementation and potential for scale-up and many
more, including the governance of evidence!
TAKING STOCK OF STANDARDS AND PRINCIPLES for
the design, implementation and evaluation of public sector interventions

6. WHAT CAN THIS REPORT OFFER?
Serving the needs of central government agencies and of knowledge brokers
which are supporting greater uptake of standards and principles
The report offers:
 A first mapping of existing principles for the good governance of evidence
 A stocktaking of standards of evidence.
 Multiple detailed and concrete examples describing the use of principles and
standards in different jurisdictions.
 A self-assessment checklist for each principle and standard to help agencies
and organisations to think about key questions that they may consider including
in their own approaches
A descriptive and inductive approach…..
…. Which could serve to develop guidance in the future….

7. Conceptual
framework
•Literature review on evidence-
informed policy-making.
•Panel of high level experts from
OECD countries
Thorough Desk
research
Engaging
with
experts
• Two expert meetings.
• Close consultations
with experts
• OECD Committees of
senior officials for
public governance and
regulatory policy
METHODS AND PROCESS

8. COUNTRY COVERAGE
2
2
4
1
2
1
14
23
1
Australia Canada
EU Germany
New Zealand Spain
UK USA
Campbell Collaboration
Standards of evidence
50 approaches, 7 jurisdiction
Principles for the Good Governance of Evidence
29 approaches, 8 jurisdictions
13
5
4
2
1
1
1
1
UK EU USA
New Zealand OECD Japan

9. A CONSOLIDATED OVERVIEW
Transparency in the
use of evidence
Public
representation in
decision-making
Appropriate
evidence for the
policy concern
Accountability
Ensuring integrity
/Honest Brokerage
Contestability
Building evidence
through emerging
technologies and
mobilizing data
Evidence synthesis
Theory of
change/Logic
Model
Design and
Development
Efficacy
Effectiveness
Cost
(Effectiveness)
Implementation
and scale up
Source: Adapted from Justin Parkhurst

10. PRINCIPLES FOR THE GOOD GOVERNANCE OF
EVIDENCE
Appropriate
evidence for
the policy
concern
Ensuring
integrity
(honest
brokerage)
Accountability
Contestability
Public
representation
in decision-
making
Transparency
in the use of
evidence
Building
evidence
through
emerging
technologies
and mobilising
data (ethics)

11. STANDARDS OF EVIDENCE
Evidence
synthesis
Theory of change
and logic
underpinning an
intervention
Design and
development of
policies and
programmes
Efficacy of an
intervention
Effectiveness of
an intervention
Cost
effectiveness of
interventions
Implementation
and scale up of
interventions

12.  A unique mapping exercise
 Synergies with approaches to identify a data driven public
sector
 A contribution to a broader understanding of the underpinning
and conditions to make EIPM happen
 Also lexicographic analysis of evaluation guidelines
underway….
 An exploratory step in identifying and developing guidance at
the international level in the area of evidence and evaluation.
CONCLUSIONS

13. THANK YOU
Stéphane Jacobzone, stephane.jacobzone@oecd.org
Daniel Acquah, daniel.acquah@eif.org.uk