Get Social Sector Leaders to Use Evidence With This 1 Weird Trick

1. Awareness, 2. Agreement, 3. Communication & Access, 4. Interact, 5. Skills, and 6. Structure & Process

Just providing people with evidence doesn’t mean they’ll do anything with it

One of the clearest takeaways from the experimental literature, which backs up the descriptive evidence I’ve previously written about, is that simply putting evidence in front of your audience doesn’t accomplish much on its own. Interventions to facilitate access to evidence were only effective if they were paired with parallel interventions to increase decision-makers’ motivation and opportunity to use the evidence, such as targeted and personalized reminders.

One-off workshops and training programs are (probably) a waste of time

Similarly, the skill-building activities studied only worked if the intervention design simultaneously targeted decision-makers’ capability and motivation to use evidence. Interventions that were passive or applied at low intensity (e.g., a one-off half-day training session) had no effect.

Increasing motivation is critical

As you can see from the above, a common theme running through the report was that many of the interventions only worked if decision-makers had sufficient motivation to use evidence. Yet of the three behavior change prerequisites, motivation was the one least often targeted by interventions included in the study. The authors surmise that the capability/motivation/opportunity framework is like a three-legged stool — you need all three of them in place for the design to work as a whole. So it stands to reason that if motivation is the most neglected of the three elements, interventions to increase motivation are especially valuable.

Making change requires structure and leadership

While not a universal rule, in general it seems to be the case that the more casual the nature of the intervention, the less likely it is to work. Of the six mechanisms studied in the first review, the closest that came to a failing grade across the board was M4 (interaction between researchers and decision-makers), as “a large majority of the…interventions that included an unstructured interaction component did not increase evidence use.” By contrast, the review found that authentically integrating interventions into existing decision-making systems and processes, rather than treating them as something separate or “extra,” appears to enhance their effects. Two specific examples that showed this kind of promise were evidence-on-demand hotlines and supervising the application of EIDM skills.

Learning from the broader literature

Arguably the most impressive contribution of “The Science of Using Science” comes from the second review, which examined the social science literature for more general clues about how to change behavior that could be applied to this context. This part of the project was breathtaking in scope — the authors not only identified some 67 distinct interventions across dozens of fields of study ranging from behavioral economics to user experience design, they were able to offer a preliminary assessment of the effectiveness of each of those interventions on outcomes related to evidence use. Below are some of the techniques and approaches that stood out as especially promising:

A call to action

I’ll close here with a finding that seems like a fitting summation of the lessons from “The Science of Using Science.” One review judged to be of high trustworthiness compared the effectiveness of different behavior change interventions to each other, and found that behavioral change interventions that worked best were those that emphasized action. The emphasis on action seems appropriate across the board when looking at the findings from the literature: over and over again, we are told that it’s not enough just to provide information and hope for the best, or get people talking to each other and hope for the best. Evidence is unlikely to be used, it seems, unless the entire intervention is designed with the act of using evidence as its center of gravity. As the authors write in the concluding sections, “the science of using science might be able to progress further by starting with the user of evidence and studying their needs and behaviours in decision-making,” and only then considering the role that research can play.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ian David Moss

Ian David Moss

Smarter decisions for a better world