AEA365 | A Tip-a-Day by and for Evaluators

Search

Hello! I’m Siobhan Cooney, Principal Consultant of Cooney Collaborative and avid podcast listener.

Once I transitioned into consulting and started working at home, podcasts became a “go-to” form of professional development. With podcasts, I can learn while taking care of mundane household tasks and carting my kids to and from school. Here I highlight my “Top Five” podcasts for evaluators.

Rad Resources:

  1. Consulting Success. This podcast, with a near constant focus on developing new business, is best suited for independent evaluators and for-profit firms. One of my favorite interviews is with Rochelle Moulton in which she discusses attracting clients on Twitter.
  2. Data Stories. Currently boasting 71 episodes, the Data Stories podcast is a great resource for evaluators with a keen interest in data science, data journalism, data visualization and analytics. These podcasts do not constitute DataViz 101 and assume a certain level of pre-existing knowledge. To get started, try episode #3, How Do You Evaluate Visualization?
  3. HBR IdeaCast from Harvard Business Review is amazing for its easily digestible and practical information on management, leadership, working in teams and virtually every “soft skill” an evaluator can bring to relationships with clients and coworkers.
  4. The Stanford Social Innovation Review website offers 4 podcasts billed as “audio talks from leaders of social change,” which feature Hallie Preskill, Lisbeth Schorr and Lucy Berholz, among others. These talks tend to be longer than your average podcast and are probably best for long car rides (and half-marathons).
  5. Adventures in Evaluation with James Coyle and Kylie Hutchinson is sadly no longer in production, but for the 35 episodes it ran, it covered the full gamut of evaluation-relevant topics from facilitation to mobile data collection to evaluation rubrics to effect sizes.

Hot Tip: Did I miss any of your favorite podcasts for evaluators? Leave them in the “comments” section below or tweet them to @Siobhan_eval.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Susan Kistler and I am the American Evaluation Association’s Executive Director and aea365 regular Saturday contributor. I’ve written before about podcasts of interest to evaluators, but they were from peripheral fields. Today, I’m excited to see evaluators entering the fray.

Hot Tip – Podcasts are for everyone: The word ‘podcast’ is associated often with iPods, Apple’s ubiquitous device. But you do not need to own an ipod to listen to podcasts. You can listen right on your computer – each has a play button for ready online listening.

Rad Resource –  Story by Numbers Podcast: The Story by Numbers podcast comes out about monthly and is from Maria Gajewski  and Ruth Terry. You might have seen Maria at the AEA 2011 or 2012 conferences. She’s an independent evaluation consultant at Changing River Consulting. The Story by Numbers podcast focuses on the nonprofit sector and their most recent offering is “The DataViz Extravaganza” a two part interview with Johanna Morariu and Ann Emery of Innovation Network.

Hot Tip – More from Ann and Johanna: You can also listen and learn and talk and engage with Ann and Johanna next month at Evaluation 2012 – Ann’s presentation list here and Johanna’s here.

Rad Resource – Adventures in Evaluation Podcast: The Adventures in Evaluation Podcast is hosted by two awesome Canadian evaluators, James Coyle and Kylie Hutchinson. You’ve probably heard from Kylie before in particular, she’s a regular contributor to aea365 and one of AEA’s dedicated professional development workshop presenters, most recently via eStudy on evaluation reporting. The Adventures in Evaluation podcast comes out every two weeks and focuses broadly on different aspects of the field of evaluation. Their most recent podcast features me! I had the opportunity to talk with James and Kylie this past week about what’s new at Evaluation 2012.

Lesson Learned – Accept the Invitation: I was a bit nervous about being on Kylie and James’ podcast, but it was easy and fun to be a guest. James and Kylie recorded their podcast via Skype. I was on my computer with my microphone headset and was quite pleased with how well the sound quality came out and K&J were gracious hosts who made it easy.

 

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

No tags

I’m Susan Kistler, AEA’s Executive Director, and regular aea365 Saturday contributor. I was inspired by Dorian’s 4/28 post on 5 must-read blogs for those working in nonprofits, and thought I would share a few must see/hear webcasts and podcasts with which you may not be familiar.

Rad Resource – Videos on Evaluation Methods from UNICEF: Recorded during a 2006 professional development training session, each video may be viewed in your browser and is over an hour in length. The evaluation topics include from AEA members include:

  • Jim Rugh on Real World Evaluation
  • David Fetterman on Empowerment Evaluation
  • Huey Chen on Theory Driven Evaluation
  • Bob Williams on Systems Approach to Evaluation
  • Sanjeev Sridharan on Multilevel Models in Program Evaluation

Rad Resource – Nancy Duarte’s YouTube Channel: Nancy Duarte, author of slide:ology and resonsate, is a vanguard in the post-Tuftean movement to create great presentations. Duarte urges us to tell stories, to make the audience care, to get rid of the extraneous and focus on a central message clearly conveyed. Sound like advice you could use for your AEA, client, or stakeholder presentation? Her short YouTube videos illustrate and expand the concepts from her books.

Rad Resouce – Social Good Podcasts from the Chronicle of Philanthropy: Hosted by AEA Member Allison Fine, a 2010 keynote speaker at the AEA/CDC Summer Evaluation Institute, this podcast focuses on “how charities and foundations can more effectively use social media tools.” While not aimed explicitly at evaluation or measurement, Allison’s background as the founder of Innovation Network, a leading evaluation consulting firm, is part of the lens she brings to her interviews and discussions with leaders in the nonprofit sector.

By the way, this year we’re pleased to welcome 2010 AEA President Leslie Cooksy speaking on Quality and the Good Enough Rule, and Zach Gemignani, founder of Juice Analytics, talking about 10 Steps to Data Vizardry, as our keynotes for the 2011 Institute coming June 12-15 to Atlanta. Hope to see you there!

Rad Resource – Filmspotting Podcast: Time for a break? Adam Kempenaar and Matty Robinson review and discuss films each week – always a new release or two (and rarely the blockbusters but something a bit more esoteric) and then a look back into the best from the past. They feed my film lust. I’ve always been intrigued by activities like film critique and wine tasting that hearken back to Eisner’s connoisseurship model of evaluation. They also remind me about the role of judgment and critical examination in our day to day activities – I’ve now come full circle back to last week’s post on thinking evaluatively in your everyday life.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators

· ·

Hi there!  I’m Liz Zadnik, member of the aea365 curating team and Saturday contributor.  I don’t know about you, but summer has started here on the East Coast of the United States and I could not be more excited.  I’m ready for some sun and time on the beach!

We all have our vacation rituals – we might bring books with us or some magazines to catch up on.  I personally love podcasts.  To be honest, I probably listen to podcasts more than music – I love how they teach and entertain and help me tap into my creativity.  I thought I would share some of my favorite evaluation-themed (and non-evaluation-themed) podcast episodes to really kick off the summer!

Rad Resource: TED Talks are a perennial favorite – and for good reason!  I found data journalist Mona Chalabi’s recent talk so insightful and practical.  I appreciated how she sought to make data meaningful and reflect the lives of communities – something we strive to do as we work with organizations creating change and generate accessible data visualizations that will resonate with folks.  She asks us to ask three questions – one of which is “Can I [see] myself in the data?”  She uses this question as a way to encourage folks to be mindful of axes labels and breakdowns of data points.

TED Talk Honorable Mention: Giorgia Lupi’s talk on data visualization and representation, “Because data are just a tool we use to represent reality.”

Hot Tip:  In my experience, folks often consider data as separate from them and their realities.  To help navigate this, I’ve talked about data as similar to constellations.  The stars are already there and we care connecting them to tell a story – a story that helps us make sense of our world.  

How have you personalized data so folks can relate and connect?

Image-of-night-at-dusk-with-stars-in-the-sky-and-a-silhouette-of-a-tree-in-background

Rad Resource: Another favorite podcast of mine is NPR’s Invisibilia which explores “the invisible forces that control human behavior – ideas, beliefs, assumptions and emotions.”  I’m currently saving up the most recent Season Three episodes for a work trip, but some of my favorites from earlier seasons explore different solutions to “problems” and ways we are and feel connected to one another.

What’s your favorite podcast?  Do you have a favorite episode that you go back to every now and again?

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi! I’m Myia Welsh of Welsh & Company Evaluation Consulting. I’m also the Chair-Elect of the Independent Consulting TIG. I’m going to share an independent consulting origin story that isn’t often talked about, but I suspect is rather common: dire necessity.

Years ago I was laid off. Yep. Lost my job. It was awful. I loved that job and those co-workers. But change is the only constant in this life, right? So, I had to turn that lemon into lemonade. I was really lucky to have been working with an evaluation consulting firm that let the junior staff in on the business development end of the operation, so I knew just a little about how things worked with proposals, contracts, and the like. Let’s be real here: what I didn’t know could fill volumes (and that is probably still the case). Going independent was terrifying, but I already had been doing evaluation consulting, so continuing to do it seemed like the quickest way from unemployment to a paycheck.

Lesson Learned: Clients can come from anywhere. My longest client relationship developed out of my interest in becoming a potential volunteer for an organization that does work I’m really passionate about. I thought I would have to tirelessly respond to Requests For Proposals, but it turned out that current relationships were far more fruitful.

Lesson Learned: There is so much help to start a business. I leaned very heavily on my state Small Business Development Center (SBDC). Most states have SBDC initiatives that include training, resources, and even one on one business coaching. My business coach walked me through my business license forms and my state LLC paperwork. The U.S. Small Business Administration has fantastic help as well: training, regional offices, partner networks, and more. All for free.

Lesson Learned: Everything comes to an end, even if it isn’t how we imagined. As it turned out, my lemonade is pretty good. I bet yours could be too.

Rad Resources: There is support out there in all kinds of forms!

  • The S. Small Business Administration has free online training, regional offices and partner networks to help fledgling businesses.
  • SCORE is a national nonprofit dedicated to helping small businesses get started. They have over 300 chapters across the country and will set you up with a business mentor.
  • There are many podcasts dedicated to starting and running a business. Two of my favorites are StartUp and Being Boss.

The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Hi, I’m Katherine Dawes from the U.S. Environmental Protection Agency. I’m currently on a year-long assignment as a Visiting Scholar at The George Washington University Trachtenberg School of Public Policy and Public Administration (find me at kdawes [at] gwu [dot] edu).

Earth Day 2016 theme is “Trees for the Earth. Let’s get planting.” Everyone knows that trees changing with the season are perfect metaphors for transitions. Every four to eight years, as spring trees start blooming, evaluators in the United States’ federal sector start contemplating our major upcoming seasonal change – the transition to a new Presidential Administration. We wonder: What will be our new federal evaluation goals and policies?  How will we change (or continue) our work to meet the needs and expectations of a new, energetic Administration?

Aside from tree leaves, what On Earth can an evaluator read to learn what the next Administration cares about (or is hearing from national experts) concerning evaluation, management, accountability, data… any issue that will directly or indirectly influence my work?

To understand the forest…err…big picture of U.S. presidential transitions and to learn what prospective federal leaders are considering planting, veteran transition watchers have many Rad Resources. Some of my favorites for evaluation-relevant info:

  • The White House Transition Project provides information to prospective federal leaders to help “[streamline] the process of transition from one administration to the next.” The Project coordinates with government agencies and non-government groups like the Partnership for Public Service and National Academy of Public Administration.
  • The National Academy of Public Administration’s Transition 2016 publishes articles and papers intended “to inform incoming national leaders about the policy and management challenges facing the nation.”
  • The Partnership for Public Service established the Center for Presidential Transition supporting the “Ready to Govern®” initiative. It has a repository for documentation from previous transitions and “shares management recommendations for the new administration to address government’s talent and operational challenges…”
  • As part of Ready to Govern, the IBM Center for the Business of Government joined with the Partnership in launching the Management Roadmap. The Roadmap presents “a set of management recommendations for the next administration – enhancing the capacity of government to deliver key outcomes for citizens.”

Daily news organizations and social networks with a federal focus supply fantastic transition information in short, readable bites – check out Government Executive and GovLoop. In addition to daily reporting, Federal News Radio co-sponsors longform interviews that are available as podcasts.  A recent interview with Professor Martha Kumar, a White House Transition project director, shares the rich history of U.S. presidential transitions. (You can also find fascinating interviews focused on program evaluation.)

Share your Rad Resources for government transitions. Let’s get reading!

The American Evaluation Association is celebrating Environmental Program Evaluation TIG Week with our colleagues in the Environmental Program Evaluation Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi there wonderful readers!  Liz Zadnik here with my first Saturday post of the new year.  I have been struggling with what to write and thought some of you might be feeling the same way, so I came up with some activities, resources, and ideas to get you started.  

Let me first say, that I love writing for this blog and being a part of this community.  It is tremendous and I am regularly inspired and motivated by contributors and commenters.  I also know, though, that writer’s block is real and a blog can be an intimidating endeavor.  So let’s brainstorm some other avenues for creativity…

Rad Resource:  I don’t know about you, but I need music or noise in the background when I’m working on something.  Songza offers playlists based on the time of day, activity, and mood!  Spotify is also great, as it’s so easy to create a playlist or create “radio stations” based on a song you like.  Sharing playlists and favorite tracks for a specific activity are great ways to engage readers or followers on social media.

Hot Tip:  Podcasts are a fantastic source of inspiration!  For me, they are a source of comic relief, information, and creative energy.  Some of my favorites are Stuff You Should Know, The JV Club with Janet Varney, and RadioLab.  Morning commutes wouldn’t be the same without them.  I’ve discovered books and artists I never would have otherwise without these generous folks sharing their research, insights, and stories.   

Interested in creating your own podcast or episode?  Great idea!  Try recording yourself talking for a few minutes about your favorite topic or current event or interview a colleague about a blog post, podcast episode, or recent study.  Podcasts can range from 15 minutes to an hour or two.  Some laptops and computers have great microphones built in, but you may want to borrow or purchase a quality microphone.  When it comes to editing, there are some free programs like Audacity and GarageBand that are user-friendly and intuitive.  This is a great way to personalize your blog or website.     

Cool Trick:  Who doesn’t love a library?!  Create your own personal curated list of 10 books on a certain topics.  Not a reading list per se, but your “go-to” resources on a specific topic.  Community readiness assessments?  Teaching evaluation?  Focus group facilitation?  You could even catalog your own personal library at home.  

And when all else fails, play around in Canva!  I created a desktop wallpaper for you to help kick the weekend off right.  

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Greetings loyal blog readers and Happy New Year! I’m Sheila B Robinson, Lead Curator and sometimes Saturday contributor for my favorite evaluation blog! It’s no news that I call myself aea365’s biggest fan, as this was the first evaluation blog I so fortuitously discovered on December 31, 2009. On that day, an announcement appeared on the AEA website that a daily blog would begin the next day, January 1, 2010.

Now, as we approach 2016 and aea365’s 6th birthday, I want to broaden your horizons and introduce you to other evaluation blogs. Since aea365’s inception (and even years before) evaluators became bloggers and began populating the evalusphere with generous evaluation know-how.

Hot Tip: Choose a day during which you would like to read up on some evaluation topics. Only after you’ve read the day’s aea365 blog post (well, what would you expect me to say?!), head over to one of the many other great evaluation-related blogs and take a look around. Here are three ways to find some gems:

1.) AEA’s list of evaluation-related blogs. AEA maintains a list with links to evaluators and their blogs.*

2.) AEA365 Bloggers weeks. Check out the aea365 archive for several weeks in previous years devoted to introducing readers to bloggers and their blogs.

3.) EvalCentral – Chris Lysy started aggregating evaluation blogs some years ago and his collection has now grown to more than 60! It’s like the Mall of America for blogs – one-stop shopping! Just one daily email brings you on average 1-4 blog posts from evaluation bloggers who post as often as once per week, to those we seem to hear from only once per year.

Cool Trick: Blogging isn’t just for evaluation, of course. Several (if not many) well-known evaluators maintain additional blogs on areas of interest well outside of evaluation! Do you know who they are?

Lesson Learned: There is a great range of opinion on the future of blogging, especially in the business and marketing world. Some say blogging will be eclipsed by other content delivery strategies such as audio (think podcasts and the like) and video. Others think the written word will stand the test of time.

Educator and marketing consultant Mark Schaefer is among these and predicts, “Written content is an important pillar of ‘rich content’ and will remain so although people will diversify the way they consume content” (see What Is The Future Of Blogging? for additional predictions).

What other evaluation-related blogs do you enjoy reading? Let us hear about them in the comments!

*Are you an AEA member with an evaluation-related blog not on our list? Write to info@eval.org with your blog’s title and URL.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

Hello! I’m Kate Tinworth, Program Co-Chair of the DVRTIG. I am excited to write about one of my favorite parts of my work as an evaluator— drawing.

Last year at AEA I co-presented a session called, “Drawing Them In: Graphic Facilitation & Evaluation to Strategically Visualize Change” with my friend Chris Chopyak (a rock star who literally wrote the book on using visuals to help businesses address challenges and create strategies). We reminded our audience that we all think visually, images are key to memory and learning, and that you—yes, you—can draw. And you should! Visuals can resolve ambiguity, cut across language and cultural divides, help findings become more salient, and kick start action.

Hot Tip: Find a Local Friend.

If you want to explore the potential of teaming with a graphic facilitator, it’s time to make some new friends. Check out the International Forum of Visual Practitioners (IFVP): http://ifvpcommunity.ning.com/. There’s bound to be someone near you!

Rad Resource: Go to a Class/Workshop.

There are great opportunities to try your hand at graphic facilitation, whether you plan to incorporate it into your evaluation practice or just want to stimulate your visual thinking. Though it can feel intimidating, I highly recommend signing up for a class. Learning graphic facilitation techniques have helped me to sketch out graph and chart ideas, think through report layouts, and get far more creative with methodology and instrumentation.

Cool Trick: Drink.

Some graphic facilitation practitioners, including Chris, do “drink and draws” where you can get some drawing practice over a cocktail. Amazing!

Hot Tip: Draw. All the Time.

To become more comfortable with drawing I draw, all the time. Try covering your dining room/kitchen table with butcher paper and put crayons or colored pencils out. When you sit down for coffee or a meal, draw. Tape paper to the wall and “live capture” TED talks or your favorite podcasts. Carry a notebook and favorite pen everywhere. Commit to drawing for just 2 minutes a day.

Tinworth 1

Cool Trick: Apps.

More of a techie? Draw on your tablet! I like Notability and iMotion.

Rad Resource: Get Inspired.

The DVRTIG is a great place to find inspiration and make connect with colleagues who care about visual thinking. Check out the AEA365 blog posts, the TIG website, and resources on p2i.

Hot Tip: Go Visual.

Whenever you can, try to “go visual” in your projects. Try a visual logic model. Engage stakeholders in drawing. I often get my stakeholders to draw during a training or as I present preliminary data. Lately I have also been experimenting with data placemats, which I learned about through AEA (thanks @VeenaPankaj).

Tinworth 2

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi! I’m Sheila B. Robinson, AEA365’s Lead Curator. I’m also an educator with Greece Central School District, and the University of Rochester’s Warner School of Education.

Today, I’ll share lessons learned about evaluation planning and a fabulous way to get ready for summer (learning about evaluation, of course!).

Rudyard Kipling wrote, I keep six honest serving-men, (They taught me all I knew); Their names are What and Why and When, And How and Where and Who.

The “5 Ws and an H” have been used by journalists, researchers, police investigators, and teachers (among many others, I’m sure) to understand and analyze a process, problem, or project. Evaluators can use them to frame evaluation planning as well.

Lesson Learned: Use these questions to create an outline of an evaluation plan:

What: What is your evaluand and what is the focus of the evaluation? What aspects of the program (or policy) will and will NOT be evaluated at this time? What programmatic (or policy) decisions might be made based on these evaluation results? What evaluation approach(es) will be used?

Why: Why is the evaluation being conducted? Why now?

When: When will evaluation begin and end? When will data be collected?When are interim and final reports (or other deliverables) due?

How: How will the evaluation be conducted? How will data be collected and analyzed? How will reports (or other deliverables) be formatted (i.e. formal reports, slides, podcasts, etc.) and how will these (and other information) be disseminated?

Where: Where is the program located (not only geographic location, but also where in terms of contexts – political, social, economic, etc.)?

Who: Who is the program’s target population? Who are your clients, stakeholders, and audience? Who will be part of the evaluation team? Who will locate or develop measurement instruments? Who will provide data? Who will collect and analyze data and prepare deliverables? Who are the primary intended users of the evaluation? Who will potentially make decisions based on these evaluation results?

Can you think of other questions? I’m sure there are many more! Please add them in the comments 

Hot Tip: Register for the American Evaluation Association’s Summer Evaluation Institute June 2-5, 2013 in Atlanta, GA to learn more about 20+ evaluation-related topics.

 

Clipped from http://www.americanevaluation.org/SummerInstitute13/default.asp

Hot Tip: Want to learn more about evaluation planning? Take my Summer Institute course It’s not the plan, it’s the planning (read the description here).

Rad Resource: Susan Kistler highlighted a few institute offerings here.

Rad Resource: I think this course: Every Picture Tells a Story: Flow Charts, Logic Models, LogFrames, Etc. What They Are and When to Use Them with Thomas Chapel, Chief Evaluation Officer at the Centers for Disease Control and Prevention, sounds exciting. Read the description here.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Older posts >>

Archives

To top