AEA365 | A Tip-a-Day by and for Evaluators

TAG | Evaluation Communication

I’m Catherine Rain of Rain and Brehm Consulting Group, Inc., a research and evaluation firm in Rockledge, Florida. Ever look at an evaluation project overflowing with new learning and fantastic results and wonder: What now? What can we do with this valuable information –beyond writing a report or peer-review publishing?

How about taking it public with a communications strategy, targeting various segments of the community and the field?

Such campaigns:

  • Educate policy-makers and decision-makers, as well as beneficiaries of services;
  • Leverage data, bringing recognition and benefactors to the program or service;
  • Positively re-frame attitudes about problems and risks affecting the community; and
  • Demonstrate accountability and transparency among tax-supported providers of programs and services.

Okay: sounds like you might need to hire an expensive marketing firm, right?  Wrong! With a couple of no-cost tools, you can become the marketing pro!

Lesson Learned: Planning a communications strategy is similar to planning an evaluation.  Carrying it forward, beginning to end resembles the process of designing, implementing and evaluating a program or service.  Communicating with various ‘markets’ requires you to shape messages in the same way you tailor those directed at an evaluated population: with relevance, in a language they understand, and with sensitivity to their culture, values and traditions!

Rad Resource: The Pink Book, otherwise known by its longer name Making Health Communication Programs Work developed by the National Cancer Institute (NCI) at the National Institutes of Health (NIH) provides you with everything you need to know about planning, designing, implementing and evaluating a communication strategy. While the book addresses health communications, the strategies easily transfer to any evaluated discipline. Note: hard copies of the book are no longer available; however, you can download the document or print it in HTML and PDF versions (option to print page or whole document).

Lesson Learned:  As with evaluation and program implementation: Planning remains the key to success!

Rad Resource: Under contract, our firm authored the publication Translating Evaluation Results to Published Documents which summarizes Stage 1–Planning and Strategy Development—contained in the Pink Book. We analogize the approach to steps taken when designing and developing a program management or evaluation plan.

Hot Tip: There is a time and resource cost to producing an effective communications strategy.

Rad Resource: Funders often require evaluators to disseminate results and lessons learned. Ask if you can fund the communications strategy as a dissemination budget line item.

Hot Tip:  According to our colleague and AEA Executive Director Susan Kistler, evaluation roles are changing! We need to redefine our approaches with clients in meaningful ways. What better than to extend our communication avenues?

Rad Resource: Susan Kistler The Future of Evaluation: 5 Predictions (building on 10 others!) .

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Felix Blumhardt and I am a Lead Evaluator at The Evaluation Group, an independent evaluation firm in Columbia, SC. My colleague, Michael Pesant, and I have had the pleasure of working with a Teacher Quality Partnership Project over the last year and one of the challenges that we have come across has been streamlining data collection across several school districts. The most important take away from this process has been ensuring that we communicate clearly with the people from whom we ask to provide data. Here are some tips that have helped us ensure that the data collection process goes smoothly:

Hot Tip: Customize your collection process. We developed a Data Collection Workbook that was distributed to each district. The collection tool is an excel file that has been formatted to look like a workbook. We removed gridlines and left only directions, tables, and graphs. Each worksheet clearly enumerates objectives, variables, and operational definitions. Cells that need to be filled in are highlighted; all others are locked. Once the data contact person from the district inputs the data into the highlighted cells, he/she is able to see the data instantly charted and graphed. Where applicable, conditional cells indicate whether or not objectives have been met. Because this workbook will be used from year to year (the project is five years) the districts will be able to track their own progress.

Hot Tip: Preset tables, charts, and graphs. Prior to data collection, we set up the workbook with directions, project objectives, tables, charts, and graphs. When all of the districts submitted their completed workbooks, we transferred that data into an overall project workbook. We instantly had all the tables, charts, and graphs needed for our reports. Because we linked our workbook in Excel to our reports in Word, our tables and graphs in our reports were updated. This makes for easy reporting in subsequent years.

Hot Tip: Communicate clearly. Because this Tip-a-Day is focused on communication, I would be remiss if I did not emphasize the need for continued verbal communication throughout the data collection process. We sent the initial draft of the workbook to all of the districts for feedback so that we could make sure that it was not too cumbersome and was clear to them. It’s important to assess your partners’ technological capabilities and make sure your tool is compatible. In our case, this involved making sure our workbook could run on all versions of Microsoft Excel. Once we finalized the Data Collection workbook, we held a meeting about the process and reviewed the workbook with the districts. This meeting was essential to the success of the collection process using the Workbook.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

·

Hello, I am Glenn O’Neil and specialize in evaluating communication programs and campaigns with my own company, Owl RE. My post today is about how to use the theory of change in evaluating communication programs.

Hot tip: there is nothing so practical as a theory of change! The theory of change maps out from activities to impact how the communications action would bring about change, often in a flow-chart like diagram. Here is a simplified example:


This should be done when designing a communications action but in my experience it is rarely the case. So you can reconstruct the theory of change at the start of the evaluation – what activities were undertaken? What was the desired short and long term effects – for example, raising awareness amongst whom? Getting people to act, but on what? Mobilizing publics – but what for? This helps clarify what you are then going to measure and how to go about it.

Rad resource: For more examples of how the theory of change is used in campaign evaluation for non-profits, check out this excellent paper from Julia Coffman of Harvard University: “Lessons in evaluating communications campaigns: Five case studies. Harvard Family Research Project, 2002 (pdf)”.

For those that would like a broad overview of ‘how to’ evaluate communication programs and projects, check out my presentation slides for a “One day training workshop for communication professionals on evaluating communication programmes, products and campaigns”.

· · ·

Archives

To top