Hi. My name is Kristi Pettibone and I am a Research Scientist with The MayaTech Corporation and Manager of our Center for Community Prevention and Treatment Research. I have been helping Tom Chapel as the AEA Summer Institute Program Co-Chair this year.

A few days ago, Tom posted about being clear about the primary demand for a product or service (milk is good for you… Got milk?) and secondary demand (buy [brand] milk because it is locally farmed/cheap/vitamin-reinforced etc), as it relates to the primary and secondary demand for evaluation services. I’d like to talk a bit about chocolate milk.

While I was at the AEA annual conference in Florida last year, I read an article in USA Today about a group of 5^th graders in Illinois who were advocating to get chocolate milk back onto the menu in their school district. The school district had dropped the chocolate milk product as an obesity prevention strategy. They soon found that the unintended consequence of this obesity prevention policy was that the students were not drinking milk at all – although the article did not indicate what they were drinking instead. The school relented and offered chocolate milk on Fridays and was going to evaluate changes in milk consumption on Fridays.

Lessons learned: We know that strategies that change the environment in which decisions are made have great potential to affect public behavior. Going forward, we need to understand how to best evaluate these environmental strategies and their effect on outcomes. One of the challenges with evaluating environmental strategies is determining what changes to measure. In the chocolate milk situation, evaluating changes in the sale of chocolate milk may help us understand elementary students’ cafeteria purchases. But if the true concern is childhood obesity, what outcomes can we evaluate, given typical resource limitations?

This is just one example of an environmental strategy implemented to affect behavior change, but regardless of topic (health, education, arts, etc.), we as evaluators have to be clear in our understanding of what it is we are evaluating and what is feasible to measure. There are many publicly available datasets for those interested in evaluating environmental change strategies, especially policy.

Resource: State Health Policy section of the Kaiser Family Foundation’s Website
http://www.kff.org/statepolicy/index.cfm

Resource: CDC’s Nutrition and Physical Activity Legislative Database http://apps.nccd.cdc.gov/DNPALeg/index.asp

Resource: The National Cancer Institute’s State Cancer Legislative Database
http://www.scld-nci.net/index.cfml

Hot Tip: Are you interested in environmental change strategies and how to evaluate them? Check out the AEA/CDC Summer Evaluation Institute, June 14-16 http://www.eval.org/SummerInstitute10/default.asp, where I’ll be conducting a session on evaluating environmental change strategies. There are lots of other great sessions available too!

Environmental Change, institute, milk

My name is David Devlin-Foltz and I direct the Advocacy Planning and Evaluation Program at the Aspen Institute, a Washington, DC, based think tank and convening organization. I am not an evaluator, but occasionally I play one at AEA conferences and client meetings. Why the confession? Because I am a career policy wonk with 25 years of experience helping foundations and nonprofits promote positive changes in policies. Over the past seven years or so, I have worked with “real evaluators” to think about how advocates can more systematically plan and evaluate their advocacy work. Changing policy is complex – just ask President Obama! But careful planning helps advocates think about meaningful outcomes and benchmarks that help them change course when necessary. Our team at the Aspen Institute has developed some web-based tools to help foundation program staff and advocacy groups learn together and sharpen their advocacy work. Our Continuous Progress Strategic Services consults to a number of major foundations and nonprofits, helping them use our tools to increase their impact – and to figure out what made the difference.

My able colleague and trusty sidekick Lisa Molinaro is teaming up Tuesday, March 23rd at 2 pm Eastern time with Astrid Hendricks of The California Endowment – one of our favorite “real evaluators – for an AEA “coffee break” demonstration of one Rad Resource that can help my fellow policy wonks and advocacy enthusiasts bring some careful planning and strategic learning to their work.

Rad Resource: The Advocacy Progress Planner is available free of charge – gratis! – for nothing! – on the web and is already assisting scores of advocacy groups and their funders on issues as diverse as promoting better family planning policies in Tanzania and closing the detention facility at Guantanamo Bay. Lisa and Astrid will walk you through a quick demo on Tuesday. You can sign up here: http://comm.eval.org/EVAL/coffee_break_webinars/Home/Default.aspx

Rad Resource: Julia Coffman of the new Center for Evaluation Innovation thinks it is “hilarious” that I call her my mentor, but – truly – Julia is an invaluable pioneer in this nascent policy and advocacy evaluation field. Check out the Center and its newsletter for new ideas and approaches.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

advocacy

My name is Julia Coffman and I am Director of the Center for Evaluation Innovation, a nonprofit effort that is building the field of evaluation in hard-to-measure areas such as advocacy, communications, and systems change.  I am also a strategy and evaluation consultant to nonprofits and foundations, specializing in advocacy and policy change efforts.

Advocacy to advance public policy can be a powerful way to achieve large-scale and lasting results for individuals and communities.  But sometimes a mismatch occurs between public policy goals and the strategies chosen to advance them.  For example, we know awareness alone typically doesn’t drive policy change, but how many public awareness campaigns have been funded with the expectation that they alone will drive policy? Before specific tactics are chosen, several important decisions must be made.

Hot Tip: Follow these five steps when developing a public policy strategy:

1. Choose the public policy goal: Select ambitious goals, but make sure they follow the SMART rules (specific, measurable, attainable, realistic, timely).
2. Understand the challenge: Assess where issues of interest currently stand in the policy process, along with why they are stuck.
3. Identify which audiences can move the issue: Determine who to engage to address the barriers identified in the last step.  Audiences may include the public (specific segments of it), policy influencers (politically-influential individuals or groups), or decision makers.
4. Determine how far audiences must move: Assess where audiences currently are in terms of their engagement, as well as how far they need to move in order to achieve policy success.  Audiences may be completely unaware that problems exist.  Alternatively, they might be aware that problems exist, but do not see them as important enough to warrant action.  Or, even if the willingness to act exists, audiences may not have the necessary skills to advocate.
5. Establish what it will take to move audiences forward: Identify the strategies and activities that will move audiences and support effective change.

Rad Resource: This visual framework was developed to support steps 3, 4, and 5 above.  The framework contains specific types of strategies and activities, organized according to where they fall on two strategic dimensions—the audience targeted (x-axis) and the outcomes desired for those audiences (y-axis).  The framework forces the consideration of audiences and outcomes before tactics are chosen.

Rad Resource: Read more about this framework and its application in the 2009 article I wrote with Martha Campbell in The Foundation Review. http://www.innonet.org/client_docs/File/center_pubs/public_policy_grantmaking.pdf

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

policy strategy