AEA365 | A Tip-a-Day by and for Evaluators

Hello, my name is Lindsey Stillman and I work at Cloudburst Consulting Group, a small business that provides technical assistance and support for a number of different Federal Agencies. My background is in Clinical-Community Psychology and so providing technical assistance around evaluation and planning is my ideal job! Currently I am working with several communities across the country on planning and implementing comprehensive homeless service systems. Much of our work with communities focuses on system change by helping various service providers come together to create a coordinated and effective system of care, rather than each individual provider working alone.

Lesson Learned:

  • The new HEARTH legislation includes a focus on system level performance versus program level performance. This has required communities to visualize how each program performance feeds into the overall performance of the system in order to identify how to “move the needle” at a system level. Helping communities navigate between the system level goals and the program specific goals – and the connections between them – is critical.
  • Integrating performance measurement into planning can help communities see the value of measuring their progress. All too often grantees or communities are given performance measures that they need to report on without understanding the links between their goals and activities and the performance measures. Presenting performance measurement as more of a feedback loop can help remove the negative stigma around the use of evaluation results and focus stakeholders on continuous quality improvement.
  • Working with agencies or communities to create a visual representation of the links between processes, program performance and system performance can really help to pull all of the pieces together – and also shine light on serious gaps. Unfortunately many federal grantees have had negative experiences with logic models and so finding creative ways to visually represent all of the key processes and outcomes/outputs/etc. can help to break the negative stereotypes. In several communities we have developed visual system maps that assist the various stakeholders in coming together to focus on the bigger picture and see how all of the pieces fit together. Oftentimes we have them “walk” through the system as if they were a homeless individual or family to test out the model and to identify any potential barriers or challenges. This “map” not only helps the community with planning system change but helps to identify places within the system and processes that measuring performance can help them stay “on track” toward their ultimate goals.

Rad Resources:

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, I’m Maureen Wilce, a founding member of the Atlanta-area Evaluation Association, and I’m Sarah Gill, president elect of AaEA. We’re both “true believers” in the power of evaluation to guide organizational learning. We’ve seen how good evaluation questions can help uncover important information to improve programs. We’ve also seen the opposite: how bad evaluation questions can waste time and resources – and increase distrust of evaluation in general.

Asking the right evaluation questions is critical to promoting organizational learning. Answers to good evaluation questions direct meaningful growth and build evaluation capacity. But what makes an evaluation question “good”? To get our answer, we reviewed the literature and then collected the practice wisdom of AaEA members and members of AEA’s Organizational Learning & Evaluation Capacity Building TIG. As we organized our thoughts, a checklist began to form. After more great discussions with our colleagues in AaEA and the TIG, we decided to structure the checklist around the standards. A few more refinements came as we used the resource in our work in CDC’s National Asthma Control Program, and finally, Good Evaluation Questions: A Checklist to Help Focus Your Evaluation was born!

Rad Resource: The Good Evaluation Questions Checklist, at http://www.cdc.gov/asthma/program_eval/AssessingEvaluationQuestionChecklist.pdf, is a tool to help ensure that the evaluation questions we create will be useful, relevant, and feasible. In keeping with the new accountability standard, it also provides a format for documenting our decisions when selecting evaluation questions.

Lesson Learned: Articulating what makes an evaluation question “good” requires thinking through several dimensions and assessing it against multiple criteria. A checklist can help us review evaluation questions to anticipate potential weaknesses and can also support communication with stakeholders during the question development process.

Rad Resource: While at the National Asthma Control Program website, check out our other evaluation resources, including our guides and webinars.

Get Involved: We received some great feedback from folks who attended our demonstration at AEAthanks to all who joined us! If you have additional suggestions about how to improve the checklist, please leave them in the comments below.

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Hi, my name is Travis Tatum and I currently work as an independent evaluator through my company Creative Research Solutions, LLC.  As President of the Atlanta-area Evaluation Association it is my pleasure to welcome you to our affiliate’s AEA365 week! This week, our members will be contributing AEA365 posts with advice, best practices, and new tools based on their experiences in evaluation.

Located in Atlanta, Georgia, the Atlanta-area Evaluation Association serves evaluators with a wide variety of backgrounds and areas of focus.  Of course, since the CDC is based in Atlanta, we have a particularly large population of public health evaluators among our members.  In recent years, AaEA has grown substantially. This year we have been working to develop our organizational processes to ensure that we can support and sustain our continued growth.

AaEA typically provides monthly events for our members, which alternate between professional development events and social activities.  We are volunteer based, and have several committees focused on different aspects of our activities:

  • The Programming and Professional Development Committee, led by Karen Anderson and Ayana Perkins, creates and organizes our monthly professional development and social activities.
  • The Membership and Networking Committee, led by Willliam Moore and Tekla Evans, handles new member recruitment and registration.
  • The Communications Committee, led by Linda Baffo and Linda Vo-Green, develops our email newsletter and manages our website, http://atl-eval.org.
  • The Finance Committee, led by Brandy Peterson and Judy Gibson, develops the budget, monitors the financial position of the organization, and helps identify ideas for fund raising.

In addition to the committee chairs, our board includes a President (myself), a President-Elect (Sarah Gill), and a Past President (Kari Cruz); we work together to guide the overall direction of the organization and support each of our committees however we can.

Hot Tip: Each of these committees are often in need of additional volunteers, so if you are a member in the Atlanta Area, we welcome your participation!

Hot Tip: Being part of a local affiliate can carry a lot of benefits for evaluators.

  • As an independent evaluator, I personally have made connections through AaEA that have led to new contracts and other business opportunities.
  • Members who work for larger companies can benefit from networking and professional development through AaEA.
  • Having a local community of people who care about evaluation makes it much easier to find partners to collaborate with on larger projects.

Rad Resource: The fastest way to find your local affiliate is to visit the local affiliates page on the main AEA website: List of local affiliates.

I am both grateful and excited for our members to share their insights.  I hope that you will also find our members’ contributions helpful every day this week!

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, I’m Sheila Robinson, aea365’s Lead Curator and sometimes Saturday contributor with reflections from Denver as we wrap up Evaluation 2014. I’ve enjoyed five AEA conferences now, each one as exciting a learning and community-building opportunity as the last. I spent time thinking deeply about our conference theme and discovering the connections among the various presentations to those ideas and ideals.

Beverly Parsons, our 2014 AEA president, kicked off the conference with an inspiring opening plenary, Visionary Evaluation for a Sustainable Equitable Future during which she described three key areas and how they apply to evaluation.:

Systems thinking: emphasizes seeing interconnections especially related to competing values and ripple effects of various actions.

Building relationships: emphasizes working across disciplines and partners in new ways.

Equitable and sustainable living: draws attention to matters such as the interface between human justice and the use of natural resources.

John Gargani, AEA president-elect for 2016, helped close out the conference in the final plenary by asking participants to consider three key questions:

1.) What should AEA’s role be in supporting a sustainable equitable future?

2.) How might AEA support your plans for visionary evaluation?

3.) How should AEA contribute to the global evaluation community?

Lesson Learned: Many sessions were overflowing with standing room only and some presenters were surprised and honored that their sessions drew such interest. Handouts were in short supply and I heard many, many participants ask for the presenter’s slides.

Get Involved: With that in mind, Evaluation 2014 presenters: Please upload your materials – Slides, handouts, etc. – to the AEA Public eLibrary. It’s easy to do and not only will your Evaluation 2014 participants appreciate it, but your reach will be extended to those who could not be at the conference.

Cool Trick: To extend your learning and enjoy a variety of perspectives, start looking in the coming days and weeks for evaluation bloggers to reflect on their conference experiences. Heres a link to our AEA member blogs.

Hot Tip: This same link will get you a list of evaluators on Twitter. Be sure to search the hashtag #eval14 for conference tweets. I maintain a twitter list of evaluators as well and it grew substantially during the conference closing in on 300. You can subscribe to that list through me – @SheilaBRobinson. Be sure to follow some of the newest #eval tweeters too, to continue to build community among evaluators.

And finally, many evaluators had the opportunity to enjoy all that Denver offers, while others stayed close to the conference sites – The Hyatt Regency and Denver Convention Center. We were perplexed and amused by the friendly but imposing 40ft big blue bear who peers curiously into the Convention Center as if to say, “Who are all these evaluators and what are they about?”

"I See What You Mean" (2005) Sculpture by Lawrence Argent. Photo by Billy Hathorn

“I See What You Mean” (2005) Sculpture by Lawrence Argent. Photo by Billy Hathorn

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Hello everyone! My name is Leigh M. Tolley, and I am an advanced doctoral student in the department of Instructional Design, Development and Evaluation at Syracuse University, and a Research Assistant at Hezel Associates, LLC in Syracuse, NY. I realized on the way to Denver that this is my fifth AEA conference!

Lesson Learned: I feel like AEA is my professional organization home, and also a professional branch of my family. Each year, I am thrilled to catch up with colleagues and friends in person and to continue to learn more about the field. At my first conference, I was amazed by the many aspects of evaluation that exist at AEA. As a graduate student new to the field, I decided to start by exploring sessions and visiting business meetings.

At the 2011 annual conference, I raised my hand right away when members of the PreK-12 Educational Evaluation TIG asked for volunteers interested in serving as Members-at-Large for the next year. The following year, I served as the TIG’s Program Chair-elect, and have been the Program Chair for 2014.

It can be scary as a student or as someone new to the field to jump in, but for me the initial fear dissipated quickly. I loved the opportunity to get more involved with AEA, and those little steps have evolved into something huge for me, both professionally and personally.

Get Involved: I would like to encourage those thinking about getting more involved to jump in. There are many opportunities to get involved, including volunteering to help a specific TIG at their business meeting, helping to review proposals for next year’s program, or even writing a blog post for aea365. Even just chatting with those seated around you in a session can be a great way to start a network or add to those you already have.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Hi! I’m Jayne Corso, Community Manager for AEA with some early perspectives from Day 1 of Evaluation 2014.

The twittersphere lit up this afternoon as Beverly Parsons, our current AEA president gave her plenary talk, Visionary Evaluation for  Sustainable, Equitable Future. Evaluators were especially impressed with young Xiuhtezcatl Roske-Martinez of Earth Guardians.


No tags

My name is Dan McDonnell and I am the Community Manager for the American Evaluation Association.

Ever wanted to be a professional conference insider? Social media offers a fascinating way to add an extra level of experience to any conference you attend, and provides you extra content, as well as the means to discover fellow attendee recommendations and conference secrets or ‘life-hacks,’ provided you know your way around Twitter and hashtags. Read on to see how you can take enhance your experience at the  next conference you attend (Evaluation 2014 for many of you!).

Hot Tip: Know Your Conference Hashtag

First things first. Check out the conference website or marketing materials to find out what hashtag will be used. In the case of Evaluation 2014, the official hashtag is #Eval14. Using the Twitter search client (or one of the many third-party Twitter management tools out there), search for the official conference hashtag and start reading. Consider this your conference command center! Whenever you have some downtime, or are interested in hearing your fellow attendee’s reactions to certain presentations or sessions, search the hashtag to see what people have to say/

Hot Tip: Share Your Experience

Part of getting the most value out of social media is by being, well, social. Use Twitter as your personal digital notepad by:

  • Tweeting out neat data points or insightful thoughts from speakers
  • Sharing your own reflections on the content and topics being discussed
  • Join the conversation by @replying to other users Tweeting on the hashtag
  • Posting photos from the event

With all of the above, be sure to include the conference hashtag to join in with the larger conversation. Not only will you have a digital record of some of your experiences from the event to review later, but you’ll open up opportunities to connect and meet with your fellow conference attendees, and give those who are unable to attend the conference a small taste of the experience they are missing.

Hot Tip: Connect with Others

Don’t miss an opportunity to expand your network and learn more from conference attendees and speakers. Follow people on Twitter that are using the conference hashtag, as chances are, you’ll have a lot in common. Search for conference speakers and presenters on Twitter (or just ask for their Twitter handle in person) to give them a shout out, especially if you enjoyed their session. You can also ask them follow-up questions via Twitter or simply to ‘subscribe’ to their feed and read up on more great evaluation content that interests you.

By following along with the conference hashtag, you may also uncover great recommendations on sightseeing, local cuisine and the best place to grab a coffee near the hotel or convention center.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello! I’m Sheila B Robinson, aea365’s Lead Curator and frequent contributor with a woman-on-the-street (well, woman lurking in conference room) report from first time attendees at #eval14.

This afternoon, I caught up with participants from Presenting Data Effectively: Practical Methods for Improving Evaluation Communication.

Tanya Hills, Evaluation Manager with the USTA Foundation wanted to learn more about technology to better display data she has to report. “I wanted to learn more about different options for graphs and charts.” She shared that she learned a lot of useful and applicable information about how to visually present data using visual learning theory.

Tanya decided to take full advantage of professional development sessions and is looking forward to Evaluation-Specific Methodology and Leading Through Evaluation later this week. Like many attendees, she chose sessions based on their descriptions.

Tanya downloaded the new conference app and is interested in using is as a networking tool. “I’m excited about being among a group of people who have similar interests who are excited about data and are trying to make a positive impact on the world.”

Next, I met Diane Mashburn, Instructor for Program Planning, Evaluation, and Accountability at the University of Arkansas Division of Agriculture. Before I could even ask her impressions of the workshop, she shared a “small world” story with me. Diane sat at a table with Mark Parman, Evaluation Outcomes Measurement Specialist of the Cherokee Nation. Conversing about bookstores and restaurants in California, Diane and Mark discovered that Diane was born in the same town as Mark’s wife. They then figured out that Mrs. Parman graduated high school with Diane’s mother and Mark remembers having been at their house in years past! Talk about networking!

Diane chose today’s workshop because she’s in charge of federal reporting. “I’m new in the position so I’m always looking for ideas for how to present all the data I’m in charge of collecting.” As for what she’s learned thus far: “I have so many ideas I’m going to have to make a priority list for what I’m going to tackle first!”

To choose sessions for the rest of the week, Diane found the TIG most closely related to what she does – the Extension Evaluation TIG – and decided to attend a number of their sponsored sessions. She started with the online catalog, but then downloaded the conference app to add more to her schedule.

As for networking, she jokes, “There’s no telling who else I might meet who knows my family!” She’s excited about professional networking, too. “I’ve already met a couple of other people that do extension work. I can tell that networking will be really good with this conference.”

Stay tuned this week for more #eval14 action!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi everyone! I’m Beverly Parsons, 2014 AEA president. I’m also executive director of InSites, a research, evaluation, and planning organization.

Evaluation 2014 is finally here! I’d like to kick off this week of conference-focused AEA365 posts by highlighting the conference theme, Visionary Evaluation for a Sustainable, Equitable Future. The graphic summarizes the key message. It was created by our fabulous conference chair, Matt Keene, and his amazing friend and colleague, Chris Metzner. Parsons 1

Here’s the basic idea.

Behind the “Visionary Evaluation Kaleidoscope” is a representation of a desired version of the Denver area—one in which natural and human systems are in a sustainable balance.

By thinking in terms of a desired future, evaluators are not trying to predict the future. Rather, having a picture in one’s mind of a desired future encourages us to use our professional capacities and personal commitments differently. Our world is experiencing sobering trend lines of unjust social conditions. They may be in areas such as health, education, and the economy. They may be related to diminishing natural resources such as clear air and water. We want to use evaluation to shift those trend lines in the direction of a sustainable and equitable future for many generations to come.

Here’s where Visionary Evaluation comes into play.

Visionary evaluation is not a particular method but rather is shorthand for encouraging evaluators to support movement toward a desired future. You might think of it as three creative turns of the Visionary Evaluation Kaleidoscope in the graphic:

 

Parsons 2 Parsons 4 Parsons 3

Systems thinking: emphasizes seeing interconnections especially related to competing values and ripple effects of various actions.

Building relationships: emphasizes working across disciplines and partners in new ways.

Equitable and sustainable living: draws attention to matters such as the interface between human justice and the use of natural resources.

My desire is that we all end this week with a renewed sense of what a sustainable, equitable future is and how we can use a visionary approach to evaluation to contribute to that future.

Hot Tip: Check out AEA Evaluation 2014 to review the program, the conference theme, and much more. Even if you are not attending the conference, you can be involved. Join the Twitter conversation at #Eval14. Check out the e-library as presenters post their materials. Contact the presenters whose work is of interest to you.

Rad Resources: There’s a new feature this year. Recordings of the Presidential Strand and Plenaries from the conference will be available for purchase after the conference! Information will be available online at AEA Evaluation 2014.

See you in Denver!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello, my name is Jayne Corso and I am a Community Manager for the American Evaluation Association (AEA). As the voice behind AEA’s Twitter presence (@aeaweb), I am always looking for new ways to connect with evaluators and surface online conversations focused on evaluation. I recently came across a tool called Riffle that is instrumental in helping to identify twitter users who are interested in evaluation and trending topics.

Riffle turns your browser into a pop-up informative Twitter analytics platform and allows you to quickly research users and read up on their Twitter habits. When you download Riffle (it’s probably easiest to add as a Google Chrome browser extension), a small triangular icon will appear on the right side of your browser. When you click this icon, Riffle will open up a sidebar that will allow you to search individual Twitter users. For example, if you search @aeaweb in Riffle, here’s what you’ll see:

Eval Riffle

When you’re looking at your home feed on Twitter, you’ll notice the Riffle icon now appears next to each Twitter user’s handle. Simply click     that icon and the sidebar will pull up their Riffle information.

Rad Resource: Expand your Twitter community

If you type a user name into the search bar, such as @aeaweb, you can easily see who AEA is mentioning in posts. As you can see, AEA often mentions @clysy (Chris Lysy), @evalu8r (Stephanie Evergreen), and @bettereval (Better Evaluation). All of these users are highly focused on evaluation best practices and are folks that we’d recommend you follow! Take Riffle one step further and search one of the usernames in AEA top mentions to find out who they follow.

Rad Resource: Uncover new trends

Similar to top mentions, the tool also returns the top hashtags used in Tweets by the Twitter user you search. Let’s look at another example, when you search for @evalu8r in Riffle, you can see that Stephanie Evergreen commonly uses these hashtags: #dataviz #eval #p2i #eval14. You’ve just found four evaluation focused hashtags that you can begin following on Twitter, opening the door to some great new content and information that you may have been missing in the past. You’ll find it easier to stay up-to-date on new trends in evaluation through reading these hashtags and participate in online conversations by using the hashtags in your own tweets.

Hot Tip: Get ready for Evaluation 2014

If you are joining us at Evaluation 2014, keep an eye on the #eval14  hashtag for the most up-to-date information about the event. Also use the hashtag in your Tweets to share your experiences, conference photos, or to connect with other attendees. We look forward to seeing you in Denver!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

Older posts >>

Archives

To top