“Creativity is intelligence having fun”, Albert Einstein.

Greetings! I’m Sara Vaca, independent consultant (EvalQuality.com) and recently appointed Creative Advisor of this blog. To start contributing I thought of writing some posts about how creativity intertwines with evaluation. This is Part I of a two-part post.

Lesson Learned: Evaluation is a rigorous, systematic transdiscipline. However, evaluators can (and already) use creativity to improve their practice in many different moments and levels.

Here are many examples, just digging in our aea365’s archives:

Hot Tips:

1. Advocating for evaluation

Evaluation is not as well-known as it should be for many citizens and politicians. Many of us find ourselves exploring ways to make evaluation more attractive, interesting and remarkable, at least at our local environment. Examples:

Kylie Hutchinson on “O Collective Impact,” an Evaluator’s Carol

Michael Quinn Patton on Using Children’s Stories to Open up Evaluation Dialogues

2. Making stakeholders engage

A demonstrated key factor in an evaluation, our fellows have already encountered this potential problem and shared tips to overcome it:

Alicia McCoy on Using Humor and Creativity to Engage Staff in Evaluation

Marybeth Neal on Using a Wall to Engage Stakeholders

Julie Poncelet, Catherine Borgman-Arboleda, and Jorge Arboleda on Using Participatory Video to Engage Youth in Evaluation in a Creative and Empowering Way

Jeanne Hubelbank on Assessing Audience or Client Knowledge in a Sweet Way

Jessica Foster on Maximizing Survey Response Rates

3. New ways of using data

Evaluation has always relied on data, but other sectors are catching up. Now evaluators have realized that and we are learning new ways in dealing with and using data:

Kimberly Kay Lopez on Getting Creative With the Data You Collect and Use for Evaluations!

Patti Patrizi on Using Existing Data in New Ways

Laura Pryor and Nichole Stewart on Data Science for Evaluators

Cameron Norman on The Evaluator-as-Designer

 

We would love to hear how YOU are using creativity in your evaluation work.

Please consider contributing your own aea365 post! (sara.vaca@EvalQuality.com)

Look for Part II with more examples of aea365 posts on creativity and evaluation! And even more about creativity and evaluation coming your way soon!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

creativity

Hi there! I am Deborah Levy, Principal of SuccessLinks, LLC and the new Chair of the Independent Consulting TIG. I have been evaluating for 15 years and have been independent for eight years. I have published two posts since I started my blog “Why Not Blog?” after attending the annual conference in October. As you can see, my start has been slow and the holidays didn’t help much. I started my blog because I wanted to insert my voice into the blogging world, specifically around evaluation and my work as an independent consultant. I really enjoy reading friends’ and colleagues’ blogs (evaluation-related and those that aren’t). They inspire me and also I really like that a blog helps you understand a different side to people. I wanted to produce the same effect for other readers and present a part of myself that people who know me personally or professionally don’t know.

Favorite Post –  (not many to choose from) My First Post, I Hope it’s a Good One: This post flowed just as I would want it to. It felt natural writing it and conveyed the excitement I was feeling as I put the words down. The response was positive and energized me to continue. It was a strong entry to the blogging world.

Lesson Learned- After my first two posts, I decided that a monthly blog was going to be more my speed. Many blogging experts suggest writing weekly or even multiple times a week, but for me that isn’t possible or even desired at this point. Everyone has a different writing schedule and that is okay. Just because you don’t post once a week does not mean your blog isn’t worthy or you should stop writing all together. I also have learned that graphics go a long way. Some of my favorite blogs use photos, charts, or cartoons. It makes reading them more fun. Lastly, don’t decide that your blog is going to serve one purpose (e.g. an evaluation blog) because you probably have many more stories to tell and thoughts to share that aren’t about that subject area. It would be a disservice to yourself and your readers to not write something you want to share because it doesn’t fit into the box you originally created for yourself.

This winter, we’re continuing our occasional series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

bloggers · Blogging

Hello! My name is Chithra Adams. I work as an evaluator at the Human Development Institute, University of Kentucky. I blog about design thinking and evaluation.

Rad Resource – Evaluation and Design Thinking: The blog explores the application of design thinking principles to the practice of evaluation. Each post includes a review of design thinking and design literature and ends with a discussion of possible applications to evaluation. I usually post once every two weeks. I view the blog as a community learning space and visitors are encouraged to provide their interpretation of the concepts discussed in the posts.

Hot Tips: Favorite posts: The blog is fairly new so there only a few posts. Most of the current posts focus on understanding the definition of design thinking. Here are two posts that will give you a sense of the blog:

• The Start of a Preoccupation: This posts talks about how I got into design thinking. It also talks about the questions I had after reading through many definitions of design thinking. The post includes some great introductory resources on design thinking.
• Definition Deconstruction Design Sensibility Part 1 of 3: This post describes design sensibility and what it means to evaluators. The post includes an article by design consultants. It is a pretty easy read and gives a glimpse of how designers’ view problems.

Lessons Learned: Why I blog: There are a lot of web resources on design thinking. I found these resources to be quite helpful to get me excited and interested in the concept. However, they were less helpful concerning how design thinking could be used in the practice of evaluation. At a cursory glance, design thinking will appear as a strategy to make products more user-centered. As a discipline, evaluation is rich with theories and practices that encourage being user centered (utilization focused evaluation, empowerment evaluation etc.). Evaluation has a strong tradition of implementing practices and developing products that are user centered. So what does design thinking offer evaluation? Does design thinking provide any added value to evaluation? What does it look like to practice design thinking in evaluation? The blog is the record of my journey to answer these questions.

Lessons Learned: What I’ve learned: Blogging to me is like exercising! Once I get started, I have so much fun.

This winter, we’re continuing our occasional series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

bloggers · Blogging

Call me Daniel. “Some years ago—never mind how long precisely—having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world”. (Herman Melville, 1851)

Rad Resource – Comprehensive Evaluation. A site where to find various information and resources on evaluation, public policy and evaluating public policies. Despite the limited time that I have, I started posting in October 2014 with the intention to include a post once a week, at least.

Hot Tips — Favorite posts: Comprehensive Evaluation is bilingual, it is written in Spanish and English. I have only a few posts, but here are those I think are most interesting. Also, I will include links to certain sections of the blog that I believe may be of interest to you.

• Evidence, Evaluation, and Effective Government (by Caroline Heider). Interesting opinion piece by Caroline Heider, Director General of the Independent Evaluation Group of the World Bank Group, originally published in The Diplomatic Courier.
• ENGAGE: Open Government Data. ENGAGE is a door for researchers that leads them to the world of Open Government Data. By using the ENGAGE platform, researchers and citizens will be able to submit, acquire, search and visualize diverse, distributed and derived Public sector datasets from all the countries of the European Union.
• Authors. Links to relevant information on authors and researchers.
• Training. Links to undergraduate and postgraduate degrees in design, management, analysis and evaluation of public policies around the world.

Lessons Learned — Why I blog: Comprehensive Evaluation is the result of my investigation for the preparation of my degree in Public Administration and Management Final Project, “The evaluation of public policies in Valencian Region: situation analysis and proposal of institutionalization”: many materials collected, pages visited or consulted sources. These data and information will be gradually incorporated into the web. I hope, someone might find it useful.

Lessons Learned — What I’ve learned: In addition to managing Comprehensive Evaluation, I have another personal blog. I am studying a master and also working at the Polytechnic University of Valencia so, I do not have much time for posting in my two blogs but I try never to forget … Post is an escape, a way to get away from daily routine and show the rest of the world who you are, what interests you, what worries you, what you can offer and someone may be interested in receiving… It is also a great way to make friends, if not … what would I do here? Thank you very much!!

This winter, we’re continuing our occasional series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

bloggers · Blogging

I’m Carrie Tanasichuk, a relatively new evaluation blogger. I currently work in the Program Research & Development department at the YMCA of Greater Toronto, where we support the Association to assess impact and improve programming across the YMCA as well as in the larger community.

Rad Resource – CarrieTanasichuk.com: Broadly speaking, my blog is about anything to do with research and evaluation. I’m a social psychologist by training and my favourite thing to write about is how social psychological theory can be useful to evaluation. I also find data visualization fascinating and I like to posts tips and tricks that I learn. I’m currently branching out and learning R, and I would like to start posting about that in the future.

Hot Tips: Favorite posts:

• Measuring attitudes that predict behaviours – Using attitudinal survey questions to predict future behaviour is something that comes up repeatedly when discussing the external validity of evaluation findings. In this post I look at the theory behind attitudes predicting behaviour.
• Developing valid self-report measures – Self-report measures are widely used in evaluation, but lately I’ve come across several people (not evaluators) who are quick to dismiss them. I wanted to do some background research on how to make self-report measures as valid as possible.
• A simple GIF illustrating dataviz principles – I didn’t create this GIF, but I love how it communicates a lot of principles very simply. I’ve shared it with a lot of people and they’ve all loved it, too.

Lessons Learned: Why I blog: I’ve kept a personal blog in one form or another for over 12 years. I’ve blogged about everything from cooking, running, knitting, and backpacking! I first came across evaluation blogs 3 years ago when I started reading AEA365. I toyed with the idea of starting an evaluation blog but I was hesitant – would I have anything valuable to add to the conversation? Would anyone even read my posts? I finally decided to take the plunge and I’ve been overwhelmed at how friendly and welcoming other evaluation bloggers have been!

Lessons Learned: What I’ve learned: I sometimes hesitate to publish a post if I don’t feel like I’m an “expert” on the topic. I read a tip from Susan Kistler that you can blog as a fellow learner, rather than as an expert. I think this is wonderful advice and something that I have tried to take to heart. Another lesson learned is to constantly draft posts. Whenever I have an idea for a post I quickly create a draft and jot down some notes. Sometimes I go back and flesh it out to be a full post (and sometimes the draft will sit there forever collecting dust).

This winter, we’re continuing our occasional series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

bloggers · Blogging

Hello! My name is Amanda Babine and I am the Director of Evaluate for Change, a program evaluation company that provides training exclusively to nonprofits dedicated to measuring their impact, and a researcher at Columbia University in the City of New York.

Rad Resources – Evaluate for Change’s Data-Driven Nonprofit Blog

The company started inviting both nonprofits and evaluators to share their personal stories of using data to improve programming in the social sector. We focus on the good, bad and ugly of using data. Our guests post blogs that are written as first hand accounts and lessons learned; whereas our staff find ways to connect the nonprofit sector to data-scientists and program evaluators who can help.

Favorite Post –As we get so many great perspectives from our bloggers, it’s hard to pick which post is my favorite. With that being said, I will share three that most people seemed to find extremely entertaining:

• Hottest Data Nerds – Working with something so serious all the time, data, we make sure we find time to have fun. And one of our most fun posts has been the Hottest Data Nerds. The title is exactly what you expect!

• Is Early Childhood Education Worth the Investment? What Does the Data Say? We happen to have a lot of members who are passionate about education. One of our frequent guest bloggers helped our nonprofits take a closer look at how to really look at complex issues with an eye for data.

• Shifting the Nonprofit Culture: Don’t Fear the Data – Our first guest blogger, which wasn’t that long ago, has been one of our most popular blogs so far. The topic brought more conversation than any of our other blogs.

Why We Blog – I use WE intentionally. Our blog wouldn’t be as popular and successful if it was not for our passionate community that submits amazing blog ideas and entries. And we continue our work because we want to give the community voice.

What We Have Learned – I have learned that graduate school ruined my creativity. I also learned that our biggest asset is our editor. I am sure whatever I produced before this would make you question my qualifications. On a more serious note, I learned to allow people to have a voice.

Hot Tip – Just because someone has not blogged before doesn’t mean they won’t produce amazing content. Contributors who have been hesitant to share since they don’t have a lot of experience submitted some of our most popular blogs! Not that getting “famous” guest bloggers is a bad thing, but we found that less known individuals can inspire us just as much, if not more.

This winter, we’re continuing our occasional series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

bloggers · Blogging

I’m Wendy Tackett, the president of iEval, a part-time faculty member at Western Michigan University, and a new blogger in 2014.

Rad Resource – Carpe Diem: Make Your Evaluations Useful! Three of my iEval colleagues and I started this weekly blog last June primarily to help evaluators make their evaluations more useful.

Hot Tips – Favorite posts: Here are some of our most referenced posts to date, listed chronologically:

• Week 5: Using a ‘Top of the Mind Memo’ – Sometimes even interim reports can be too cumbersome, so timely, relevant feedback presented in a short format can be very useful as a check point for clients.
• Week 16: Avoiding misuse: The evaluation client’s perspective – While most of our readers seem to be evaluators, it’s helpful to step back and think from the client perspective at times.
• Week 18: Miss America: The nation’s largest scholarship program for women – Evaluation really is all around us, and this comical perspective on creating a logic model reminds us of this, while also reminding us to humanize the work we are doing.
• Week 20: I was at the European Evaluation Society Biennial Conference – It’s always enlightening to listen to Michael Scriven and try to apply his ideas to your own work.

Lessons Learned – Why I blog: My focus has always been the meaningful use of evaluation findings. I don’t do evaluation for the sake of research, to merely fulfill funder requirements, or to have my work never to be used. I want clients to benefit from the work and use it to make improvements and decisions. The Carpe Diem blog is one way to help others achieve that. I also provide evaluation training by teaching graduate students in evaluation, presenting at conferences, and serving on the Michigan Association for Evaluation board…all with the purpose of helping others practically apply their high quality evaluation findings.

Lessons Learned – What I’ve learned: First, it’s been great using the team approach – we share ideas with each other and build off of each other’s work. It takes the weekly burden off of one person and lets the readers benefit from multiple perspectives. Second, the online interaction with readers hasn’t been what we expected. Most of our feedback is direct – emails, conferences, personal contact. While we may not get regular, immediate feedback on each post, we’ve gotten enough to think we’re resonating with some people and encourage us to continue!

Hot Tip – I think a good post is a mix of information, usefulness, and fun. We try to share useful information, provide tips that can immediately be used, and end with a silly picture of the blogger that week – just to remind people to have fun with evaluation!

This winter, we’re continuing our occasional series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

bloggers · Blogging

Greetings loyal aea365 readers AND authors! I’m Sheila B Robinson, aea365’s Lead Curator and sometimes Saturday contributor. A couple of weeks ago, I wrote this post asking readers what they would like to read about on aea365 in 2015.

Many thanks to those who offered their comments with ideas for posts and responses to each other. Now, I’m writing to ask YOU to consider contributing a post on one of these suggested topics, or any other evaluation-related topic.

Lesson Learned: Here’s what our readers said they would like to see:

• Some comments on developing, sharing, and storing lessons learned from evaluations.
• A post, or series of posts about evaluation from the perspective of practitioners for whom their primary job is not evaluation. Perhaps tips on how to best integrate evaluation into the myriad of other, seemingly more pressing, tasks without pushing it to the back burner.
• More about observational studies and the best practices in using secondary data sources.
• More of other evaluators’ ideas for how to make evaluation more useful for finding solutions to social problems and building a better world.
• Evaluators doing research on evaluation – what are they studying and what are they learning?
• More on collective impact and experiences in evaluation of partnerships.
• Hear from evaluators of all ages and those working with diverse communities
• More about how private consultants deal with the cost of SPSS…it causes a lot of angst. The other stat packs aren’t user friendly if you have always used SPSS.
• Case studies with operational survey questions and instruments.

Hot Tip: We’d love to hear from YOU! Do you have something to share on any of these or other evaluation-related topics? Please send us a draft post for consideration.

Hot Tip: You don’t have to be an expert to contribute! Notice that none of the comments ask for “expert” opinions. Many readers want to hear from everyday evaluators working in the field. You don’t need to be doing something unusual, or cutting edge, or revolutionary. Tell us how a strategy has worked for you in evaluation. Tell us what you’re learning about and experimenting with. Tell us about a lesson you have learned recently and are now applying in your work. Tell us about a book you have read, a course you took, or an experience you had that gave you new insight. What small lesson can you offer to teach others?

Cool Trick: Please follow contribution guidelines! You can find the link right up there…yes, just look up to the top of your screen and there it is! We can only publish posts that adhere to these simple guidelines.

Get Involved: It’s time to share YOUR insights with aea365 readers! We rely on the hundreds of generous authors who have contributed (many multiple times!) over the past 5 years to keep this daily blog going. As you can imagine, collecting 365 posts each year is no small task.

Is this the year you decide to contribute? We certainly hope so, and look forward to hearing from you soon.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

My name is Petra-Chambers-Sinclair and I’m a devout evaluation nerd and Biohacker from Victoria, British Columbia. Fellow evaluators understand when I explain that, as a Biohacker, I apply evaluative thinking to my personal life with the goal of maximizing my well-being. I run nutritional and lifestyle experiments, and then use data to fine-tune the strategies I use to achieve my goals.

Hot Tip: An evaluative mindset is all that Biohacking requires.

Summative Evaluation

Biohackers use summative evaluation when following an established dietary protocol or a lifestyle hack, such as one aimed at reversing autoimmune or improving sleep quality. Biohackers first gather data for the baseline, then engage with the protocol as prescribed, and gather data again at the end. As with any summative evaluation, the final stage is determining if it’s worth continuing with the hack.

Formative Evaluation

If, after engaging in a nutritional or lifestyle hack for a reasonable length of time, desired interim results aren’t being realized, biohackers assess fidelity to the protocol. If fidelity is perfect, formative adjustments can be implemented and subsequent results monitored.

Developmental Evaluation

Developmental Biohacking enables learning and engagement when there is no predetermined intervention; when conditions are complex and causality is hard to track. It enables biohackers to develop an intervention inside the mess of real life. Biohackers go developmental when nothing else seems to be working and when environments both inside and outside the body are unpredictable, such as with health issues that no one can explain or offer effective treatment for. Gradually, through self-experimentation and relentless reality-testing, an intervention might emerge that can be evaluated formatively or summatively. In the meantime, developmental evaluation allows for interaction, observation and adaptation.

The internet has allowed people who are innovating in this way to communicate the results of their experiments with each other, enabling Developmental Biohackers to accelerate learning & pattern-finding, thereby creating highly effective and scaleable interventions, such as the Autoimmune Protocol. Biohacking blogs remain a primary source of developmental information.

N=1

An N=1 is an experiment with one participant.

The assumption inherent to N=1 in the Biohacking world is that universal solutions to complex health problems have limited effectiveness, as we each have unique histories, genetic profiles, environments, and patterns of responding.

We are complex systems living in complex systems.

And an evaluative mindset is all we need to leverage this complexity on behalf of increased health & well-being.

Rad Resources:

• A Beginners Guide to Biohacking by Mark Moschel https://www.bulletproofexec.com/beginners-guide-to-biohacking-101/

And another one, if it’s not considered self-promotion:

• Biohacking: the ultimate New Year’s Resolution (because it’s not too late to make one!) http://petra8paleo.com/2014/12/26/biohacking-the-ultimate-new-years-resolution/

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

biohacking

Hey there. My name is Kath McNiff and I’m an online community manager at QSR International (the makers of NVivo).

Lessons Learned: We’re heading into a brand new year of evaluation (actually, THE year of evaluation). Oh, the joys of a clean slate! A chance to right the wrongs, sharpen the tools, clear the decks and morph into the best 2015 version of ourselves.

Here are some resolutions to consider:

• Do an Inbox detoxIs your cluttered inbox making you feel overwhelmed and out of control? Vital details can easily slip through the cracks as you flounder around in the digital debris. Get yourself a copy of How to be a Productivity Ninja and follow Graham Allcott’s simple steps for getting your inbox to zero.

• Decide on your Digital Strategy Have you been following the same old tweeters for the past two years? And what about those cobwebs on your LinkedIn profile? It’s time to make sure your digital footprint is polished and professional. If you’re looking at social media as a rich reserve of qualitative data, you need to make decisions about platforms, data collection and ethics.

• Get the right tools Seriously consider using the free version of Evernote. While you’re out in the field you can use your phone or tablet to record interviews, take field notes, snap photos and clip relevant content from the web. Then, back at your desk you can synch notebooks and have easy access to all your material (and then bring it into NVivo for analysis).

• Develop good NVivo habits Bring your research design documents into NVivo and refer to them regularly as you analyze your data. Start a project journal in NVivo and write, write, write – remembering to link to the data that supports your emerging insights. Then, when a client demands to know how you reached your conclusions – you can turn to your journal (complete with charts, word clouds and models). Check the QSR website for details about pricing.

Well, that should get us to February at least. Start the year feeling in control of your virtual world so you can spend more time celebrating in the physical one :

Hot Tip: Take a fresh look at the tools you use everyday. Are you missing out on some really useful feature because you always follow the same well-worn path? In the case of NVivo, spend some time watching the YouTube videos or read the help – and explore features you haven’t used before (framework matrices anyone?).

Rad Resources: If you want to get your ToDo list under control, try these free apps: Todoist or Trello

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags