View all modules About the Innovation Skills Accelerator About the Office of Innovation Contact us

OFFICIAL SITE OF THE STATE OF NEW JERSEY

MODULE 9

RAPID EVIDENCE REVIEW



As Bill Clinton once said, "Nearly every problem has been solved by someone, somewhere.The challenge of the 21st century is to find out what works and scale it up."
In part 1 of this module, we are going to help you understand how to do a rapid review of research related to solving a public problem. To determine the best solution to our problem, we want to know what has been tried elsewhere and learn from both academic researchers and from social entrepreneurs. By the end of this module, we hope you will be able to: (1) Find what possible solutions already exist, (2) Evaluate whether a solution worked elsewhere, and (3) Determine whether what worked in another context will work in your context and community.
In Part 2, we review both traditional and innovative types of solutions in order to expand our readiness to solve problems.

PRACTITIONER INTERVIEW

Using Rapid Evidence Review for Problem Solving


In this interview, Peter Bragge, Director of Health Programs for MSDI's BehaviourWorks Australia, describes rapid evidence reviews and how they can be used for public problem solving. Using an example, he illustrates how one can use a rapid evidence and practice review to gather the critical information required to answer critical questions that arise while solving a problem or framing a policy.

ABOUT THE INTERVIEWEE

Thumbnail of interview with Peter Bragge

PETER BRAAGGE
Director of Health Programs, BehaviourWorks Australia

Associate Professor Peter Bragge specializes in translating research evidence into practice and policy to address challenges faced in health and sustainable development. This involves identifying, appraising and cataloging research evidence; exploring practice through analysis of behaviors and their context; consulting with practitioners and policymakers to design tailored behavior change interventions and evaluating their effectiveness.

As Director of Health Programs for MSDI's BehaviourWorks Australia, Peter manages BWA’s partnerships with a number of government and other agencies including the Department of Health and Human Services, Department of Premier and Cabinet, Victorian Managed Insurance Authority and the Victorian Transport Accident Commission. Many of these partnerships have drawn upon McMaster’s Forum method of evidence review and facilitated dialogues. Peter led the development of this methodology in Australia and has conducted over 25 Forums utilizing this approach since 2012.The Forum method has underpinned state-wide trials to optimize immunization in Victoria, a mass media campaign to promote the appropriate use of the Victorian emergency ambulance service (Save Lives. Save 000 for Emergencies) and a trial which improves management of urinary incontinence following spinal cord injury.

Peter is also leading Monash Sustainable Development Institute’s collaboration with McMaster University in Canada to build the world’s largest evidence resource for the Sustainable Development Goals (SDGs) – Social Systems Evidence (SSE). SSE will draw upon the established and successful Forum approach by applying it to social and other policy areas outside of health to aid governments around the world in achieving the SDGs.

Peter has published over 60 peer-reviewed journal articles encompassing academic journals including The Lancet, Lancet Neurology, Implementation Science and Public Administration Review. He recently published his first book, From Roadside to Recovery: The Story of the Victorian State Trauma System and has written over 80 reports for government. He consults regularly with healthcare, government and other organizations and regularly gives invited talks on behavior change. In 2019 Peter and colleague Geoff Paine began a now-regular radio segment on ABC Radio Melbourne, ‘This is BS’ (behavioral science).

Prior to his full-time research career, Peter worked for 10 years as a physiotherapist in public and private practice settings, including one year in the United Kingdom. He also holds a Licentiate from Trinity College London in piano performance (1991) and continues to play, perform and record music.

READINGS

Ten ways to optimize evidence-based policy

Peter Bragge

Future Medicine, Journal of Comparative Effectiveness Research

2019

The exponential rise in knowledge availability has greatly enhanced the ‘supply’ side of the evidence-into-practice equation – however, substantial gaps between evidence and practice remain. Policymakers are therefore increasingly looking to academia to optimize evidence-informed policy. This article presents ten considerations for optimising evidence-based policy, drawn from experience in delivering applied behavior change research to government.

Read the full article here.

A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving

Eugene Bardach

2009

Book is available online for purchase here.

Re-imagining “Action Research” as a Tool for Social Innovation and Public Entrepreneurship

Stefaan Verhulst

The GovLab

2020

Rethinking research is a vast project, with multiple components. This new essay focuses on one particular area of research: action research. In the essay, Stefaan Verhulst first explains what is meant by action research, and also explores some of its potential.

Read the full article here.

SELF-ASSESSMENT

QUESTION 1

Choose the incorrect answer

Conducting a rapid evidence review is important to:





Correct! We should have already worked to create a clear problem statement before we turn to exploring evidence of policy solutions

That’s not quite right.

QUESTION 2

Select the best answer:

Randomized controlled trials are particularly valued in the world of evidence-based policymaking because:





Correct! Randomized controlled trials help us estimate the effect of a policy or program on people’s lives allowing us to compare a group that received a program and a group that did not receive a program but are - on average - alike in all other respects.

Sorry, that’s incorrect.

QUESTION 3

Select all that apply:
Randomized controlled trials may not always be the best form of evidence for a given policy problem because:





Correct! Though RCTs can provide strong causal evidence when implemented correctly, we must be aware of their limits and must explore other forms of evidence as well.

Sorry, that’s not it.

You're right but there's more to this answer. Please try again!

QUESTION 4

True or False:

When searching for examples of good evidence, we should ignore articles from popular newspapers because many journalists are unreliable reporters of the validity of scientific studies.



Correct! Though true that we should read popular news articles with some skepticism, conducting a news search or setting up a news alert can be one of the best ways to find examples of evidence

Sorry, that’s incorrect.

QUESTION 5

Select the correct answer:

___________________, such as “Results First,” are useful resources to find catalogues of well-regarded studies and reviews of results.





Correct!

Sorry, that’s incorrect.

QUESTION 6

Select all that apply:

Which of the following are useful ways to find helpful people to learn from?





Correct! These are all ways to find people who will help you gather evidence in your review.

Sorry, that’s not it. Here we’re looking to ensure we remove the most practical barriers to participation.

You're right but there's more to this answer. Please try again!

QUESTION 7

Select the best answer:

Reducing class sizes in California might have failed to have the positive effects seen in Tennessee because _________________ were not present.





That’s right. To know if an intervention might work in a new context, it is important to understand the supporting factors that made it function successfully.

Sorry, that’s not it.

You're right but there's more to this answer. Please try again!

QUESTION 8

True or False

To determine if the positive impacts found in the study of a policy intervention will apply to another context, you can determine this through a clear, quantifiable calculation involving measured effect sizes and sample sizes.



Correct, it is important for a social entrepreneur to use judgment and engage with the community to decide if the factors needed for the intervention to be replicated locally are present and whether the risk is worth taking

Sorry, that’s incorrect.

You're right but there's more to this answer. Please try again!

QUESTION 9

Select all that apply:

Collective intelligence approaches can help with:





That’s correct!

Sorry, that's incorrect.

You're right but there's more to this answer. Please try again!

QUESTION 10

Select the best answer:

You are interested in gathering research and evidence on policy solutions for providing academic support to help struggling students improve their reading level. You should reach out to colleagues who work in early childhood education:





That’s correct! Though there are sometimes reasons to hold off on talking to some people, learning from content experts and the community is often helpful towards the beginning of your research review.

That’s not quite right.

You're right but there's more to this answer. Please try again!