AEA365 | A Tip-a-Day by and for Evaluators

Hi there! I’m Karen Anderson and I’m the programming Co-chair for the Atlanta-area Evaluation Association. After wrapping up my experience as the AEA Diversity Coordinator Intern last year I wanted to stay active and connected to the AEA community. During my short time working on the programming team I’ve learned a few tricks of the trade for successful events.

Hot Tip: Have an awesome communications team! Our communications team does so much, including managing our AaEA website, AaEA Facebook, AaEA LinkedIn and creating a newsletter, which really helps to get the word out.

Hot Tip: Just like blog posts and conference titles tend to pull you in, the title of your events are just as important. Last year we had an event, which included a “speed dating” format to facilitate getting to know members and visitors better. We could have easily named it: “Atlanta-area Evaluation Association Social”, but we used “Speed Networking for Evaluators”. Make the overviews of the events short and interesting as well to capture people’s attention.

Hot Tip: Build in time for socializing and networking. We set aside 30 minutes before events for mixing and mingling. This is a great way to set the tone for events.

Hot Tip: Provide refreshments! We all know that if you provide food, they will come. Fruit, cheese and meat tray, hummus, crackers, juice, beer, and wine make up a nice spread that will keep the crowd happy.

Hot Tip: Encourage council members to get to know visitors and members. By engaging visitors and members you can encourage visitors to become members, and inform members on volunteer opportunities.

Meeting new people and learning about their interests and areas of expertise helps with planning events in more ways that one. This is a great way to help you learn the pulse of the group and if an individual is bold enough you can ask them to facilitate or co-facilitate an upcoming event.

Hot Tip: Keep up with current trends in evaluation locally, as well and nationally and internationally. This may seem a bit daunting, but subscribing to AEA’s Eval Talk, AEA-LinkedIn, and aea365 blog are 3 ways to get regular updates on what’s going on in the world of evaluation. This will help when planning events and keeping topics fresh!

Have any tips on making affiliate events successful?  Please share in the comments.

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, my name is Ayana Perkins, the programming Co-Chair of the Atlanta-area Evaluation Association as well as Senior Research Analyst and Evaluator at Infinite Services and Solutions.

Lesson Learned: When evaluating organizations for empowerment one of the characteristics to explore is whether the agency is set up to be an opportunity role structure. Organizations structured in this manner are more likely to retain their members due to the high level of engagement that is inherently required. Opportunity role structures, a popular term in community psychology and conceptualized by Kenneth Maton and Deborah Salem, are egalitarian systems that encourage members to shape the direction of the organization, offer purposeful activities, create easy access to intellectual and material resources, and deliver guidance and information that would allow members to competently serve as a leader in this organization. In developing our local affiliate of the AEA, we shifted to become more systems oriented resulted in the development of an opportunity role structure. These system changes included branching off and, routine training.

Our first act was to more actively encourage branching off.   Seemingly a self-sabotaging act since this could naturally result in splintering, the opposite occurred—members now had a new way to pursue topics of interests with the affiliate gaining access to members who were more personally fulfilled; a predictor of future participation. We gambled and won on the logic that people appreciate most that which directly reflects them.   The first group was an evaluation consultant group, and was later followed by a pro bono group, economic evaluation, and finally facilitation. Each group retains a commitment to the AEA affiliate but also uniquely provides a space where members can problem solve within specific professional topics.

Our second act was routine training. Beyond the standard professional development and social networking activities that we offer monthly, we also wanted to make sure that members felt more comfortable about advancing their work in evaluation. Almost 8 years ago, we created a new evaluator’s series to help guide our less experienced evaluators in securing employment. Earlier this year, we trained graduate students on conference style presentations using Ignite format in preparation for their delivery in a showcase at Morehouse School of Medicine, even scheduling additional dry run sessions to reduce speaker anxiety. More recently, we have developed a voluntary 30 minute orientation for new members to directly match them with their interests and invite them into leadership. Each of these training activities is ongoing.

Rad Resource: For more information on opportunity role structure, see Kenneth Maton and Deborah Salem’s 1995 article in the American Journal of Community Psychology on empowering organizations.

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Hi, my name is Ayana Perkins, the programming Co-Chair of the Atlanta-area Evaluation Association as well as Senior Research Analyst and Evaluator at Infinite Services and Solutions. I am a qualitative enthusiast and often train other evaluators and researchers in these methods. What I have noticed is that participants are more likely to right to feel valued and engaged when sharing data using qualitative methods. In fact, interviews and focus groups are wonderful ways to encourage satisfaction and when done right, the evaluator can walk away with credible findings and the participant can leave renewed and excited about participating in the data collection event.

Hot Tips:

  • Practice, practice, practice. With each new project, every member of the evaluation team should have a firm understanding of what to expect in the data collection event. What contingency plans exist when the recorder doesn’t work or too many people show up? Investing time in learning how to respond to these worst case scenarios produces an investigator or an evaluation team that is well poised to resolve all unexpected issues.
  • Plan for informal conversation. Before any interview and focus groups, allow 5 to 15 minutes for informal conversation. This time should be built into the length of the focus group or interview. Further, the evaluator is able to shed the role of expert as well as imply that no greater effort than conversation is required. This strategy also increases an individual’s willingness to more fully participant in any icebreaker activity.
  • Create opportunities for success. Previous experience and personality differences can partially influence how a person’s will likely respond to the open ended format. Even with these influences, there are strategies to help the participant feel like their contribution was a successful effort:
    • Emphasize there is no right answer which helps to reduce social desirability in responses
    • Acknowledge that no response is a response and ask whether or not the question was meaningful to them or more time is needed before responding
    • Connect similarities in responses to enhance group dynamics

Focus groups and interviews do require a lot of preparation but this level of effort can be paid off with rich findings and satisfied participants. 

Rad Resource: Want to learn more about qualitative methods? Visit this website to identify ways to strengthen your project: http://www.qualres.org/. Although the site is not specific to evaluation, most of the recommendations would also apply for qualitative evaluation projects.

Have any of these strategies worked for you? Please share your experiences in the comments.

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello, my name is Lindsey Stillman and I work at Cloudburst Consulting Group, a small business that provides technical assistance and support for a number of different Federal Agencies. My background is in Clinical-Community Psychology and so providing technical assistance around evaluation and planning is my ideal job! Currently I am working with several communities across the country on planning and implementing comprehensive homeless service systems. Much of our work with communities focuses on system change by helping various service providers come together to create a coordinated and effective system of care, rather than each individual provider working alone.

Lesson Learned:

  • The new HEARTH legislation includes a focus on system level performance versus program level performance. This has required communities to visualize how each program performance feeds into the overall performance of the system in order to identify how to “move the needle” at a system level. Helping communities navigate between the system level goals and the program specific goals – and the connections between them – is critical.
  • Integrating performance measurement into planning can help communities see the value of measuring their progress. All too often grantees or communities are given performance measures that they need to report on without understanding the links between their goals and activities and the performance measures. Presenting performance measurement as more of a feedback loop can help remove the negative stigma around the use of evaluation results and focus stakeholders on continuous quality improvement.
  • Working with agencies or communities to create a visual representation of the links between processes, program performance and system performance can really help to pull all of the pieces together – and also shine light on serious gaps. Unfortunately many federal grantees have had negative experiences with logic models and so finding creative ways to visually represent all of the key processes and outcomes/outputs/etc. can help to break the negative stereotypes. In several communities we have developed visual system maps that assist the various stakeholders in coming together to focus on the bigger picture and see how all of the pieces fit together. Oftentimes we have them “walk” through the system as if they were a homeless individual or family to test out the model and to identify any potential barriers or challenges. This “map” not only helps the community with planning system change but helps to identify places within the system and processes that measuring performance can help them stay “on track” toward their ultimate goals.

Rad Resources:

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, I’m Maureen Wilce, a founding member of the Atlanta-area Evaluation Association, and I’m Sarah Gill, president elect of AaEA. We’re both “true believers” in the power of evaluation to guide organizational learning. We’ve seen how good evaluation questions can help uncover important information to improve programs. We’ve also seen the opposite: how bad evaluation questions can waste time and resources – and increase distrust of evaluation in general.

Asking the right evaluation questions is critical to promoting organizational learning. Answers to good evaluation questions direct meaningful growth and build evaluation capacity. But what makes an evaluation question “good”? To get our answer, we reviewed the literature and then collected the practice wisdom of AaEA members and members of AEA’s Organizational Learning & Evaluation Capacity Building TIG. As we organized our thoughts, a checklist began to form. After more great discussions with our colleagues in AaEA and the TIG, we decided to structure the checklist around the standards. A few more refinements came as we used the resource in our work in CDC’s National Asthma Control Program, and finally, Good Evaluation Questions: A Checklist to Help Focus Your Evaluation was born!

Rad Resource: The Good Evaluation Questions Checklist, at http://www.cdc.gov/asthma/program_eval/AssessingEvaluationQuestionChecklist.pdf, is a tool to help ensure that the evaluation questions we create will be useful, relevant, and feasible. In keeping with the new accountability standard, it also provides a format for documenting our decisions when selecting evaluation questions.

Lesson Learned: Articulating what makes an evaluation question “good” requires thinking through several dimensions and assessing it against multiple criteria. A checklist can help us review evaluation questions to anticipate potential weaknesses and can also support communication with stakeholders during the question development process.

Rad Resource: While at the National Asthma Control Program website, check out our other evaluation resources, including our guides and webinars.

Get Involved: We received some great feedback from folks who attended our demonstration at AEAthanks to all who joined us! If you have additional suggestions about how to improve the checklist, please leave them in the comments below.

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Hi, my name is Travis Tatum and I currently work as an independent evaluator through my company Creative Research Solutions, LLC.  As President of the Atlanta-area Evaluation Association it is my pleasure to welcome you to our affiliate’s AEA365 week! This week, our members will be contributing AEA365 posts with advice, best practices, and new tools based on their experiences in evaluation.

Located in Atlanta, Georgia, the Atlanta-area Evaluation Association serves evaluators with a wide variety of backgrounds and areas of focus.  Of course, since the CDC is based in Atlanta, we have a particularly large population of public health evaluators among our members.  In recent years, AaEA has grown substantially. This year we have been working to develop our organizational processes to ensure that we can support and sustain our continued growth.

AaEA typically provides monthly events for our members, which alternate between professional development events and social activities.  We are volunteer based, and have several committees focused on different aspects of our activities:

  • The Programming and Professional Development Committee, led by Karen Anderson and Ayana Perkins, creates and organizes our monthly professional development and social activities.
  • The Membership and Networking Committee, led by Willliam Moore and Tekla Evans, handles new member recruitment and registration.
  • The Communications Committee, led by Linda Baffo and Linda Vo-Green, develops our email newsletter and manages our website, http://atl-eval.org.
  • The Finance Committee, led by Brandy Peterson and Judy Gibson, develops the budget, monitors the financial position of the organization, and helps identify ideas for fund raising.

In addition to the committee chairs, our board includes a President (myself), a President-Elect (Sarah Gill), and a Past President (Kari Cruz); we work together to guide the overall direction of the organization and support each of our committees however we can.

Hot Tip: Each of these committees are often in need of additional volunteers, so if you are a member in the Atlanta Area, we welcome your participation!

Hot Tip: Being part of a local affiliate can carry a lot of benefits for evaluators.

  • As an independent evaluator, I personally have made connections through AaEA that have led to new contracts and other business opportunities.
  • Members who work for larger companies can benefit from networking and professional development through AaEA.
  • Having a local community of people who care about evaluation makes it much easier to find partners to collaborate with on larger projects.

Rad Resource: The fastest way to find your local affiliate is to visit the local affiliates page on the main AEA website: List of local affiliates.

I am both grateful and excited for our members to share their insights.  I hope that you will also find our members’ contributions helpful every day this week!

The American Evaluation Association is celebrating Atlanta-area Evaluation Association (AaEA) Affiliate Week with our colleagues in the AaEA Affiliate. The contributions all this week to aea365 come from AaEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, I’m Sheila Robinson, aea365’s Lead Curator and sometimes Saturday contributor with reflections from Denver as we wrap up Evaluation 2014. I’ve enjoyed five AEA conferences now, each one as exciting a learning and community-building opportunity as the last. I spent time thinking deeply about our conference theme and discovering the connections among the various presentations to those ideas and ideals.

Beverly Parsons, our 2014 AEA president, kicked off the conference with an inspiring opening plenary, Visionary Evaluation for a Sustainable Equitable Future during which she described three key areas and how they apply to evaluation.:

Systems thinking: emphasizes seeing interconnections especially related to competing values and ripple effects of various actions.

Building relationships: emphasizes working across disciplines and partners in new ways.

Equitable and sustainable living: draws attention to matters such as the interface between human justice and the use of natural resources.

John Gargani, AEA president-elect for 2016, helped close out the conference in the final plenary by asking participants to consider three key questions:

1.) What should AEA’s role be in supporting a sustainable equitable future?

2.) How might AEA support your plans for visionary evaluation?

3.) How should AEA contribute to the global evaluation community?

Lesson Learned: Many sessions were overflowing with standing room only and some presenters were surprised and honored that their sessions drew such interest. Handouts were in short supply and I heard many, many participants ask for the presenter’s slides.

Get Involved: With that in mind, Evaluation 2014 presenters: Please upload your materials – Slides, handouts, etc. – to the AEA Public eLibrary. It’s easy to do and not only will your Evaluation 2014 participants appreciate it, but your reach will be extended to those who could not be at the conference.

Cool Trick: To extend your learning and enjoy a variety of perspectives, start looking in the coming days and weeks for evaluation bloggers to reflect on their conference experiences. Heres a link to our AEA member blogs.

Hot Tip: This same link will get you a list of evaluators on Twitter. Be sure to search the hashtag #eval14 for conference tweets. I maintain a twitter list of evaluators as well and it grew substantially during the conference closing in on 300. You can subscribe to that list through me – @SheilaBRobinson. Be sure to follow some of the newest #eval tweeters too, to continue to build community among evaluators.

And finally, many evaluators had the opportunity to enjoy all that Denver offers, while others stayed close to the conference sites – The Hyatt Regency and Denver Convention Center. We were perplexed and amused by the friendly but imposing 40ft big blue bear who peers curiously into the Convention Center as if to say, “Who are all these evaluators and what are they about?”

"I See What You Mean" (2005) Sculpture by Lawrence Argent. Photo by Billy Hathorn

“I See What You Mean” (2005) Sculpture by Lawrence Argent. Photo by Billy Hathorn

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Hello everyone! My name is Leigh M. Tolley, and I am an advanced doctoral student in the department of Instructional Design, Development and Evaluation at Syracuse University, and a Research Assistant at Hezel Associates, LLC in Syracuse, NY. I realized on the way to Denver that this is my fifth AEA conference!

Lesson Learned: I feel like AEA is my professional organization home, and also a professional branch of my family. Each year, I am thrilled to catch up with colleagues and friends in person and to continue to learn more about the field. At my first conference, I was amazed by the many aspects of evaluation that exist at AEA. As a graduate student new to the field, I decided to start by exploring sessions and visiting business meetings.

At the 2011 annual conference, I raised my hand right away when members of the PreK-12 Educational Evaluation TIG asked for volunteers interested in serving as Members-at-Large for the next year. The following year, I served as the TIG’s Program Chair-elect, and have been the Program Chair for 2014.

It can be scary as a student or as someone new to the field to jump in, but for me the initial fear dissipated quickly. I loved the opportunity to get more involved with AEA, and those little steps have evolved into something huge for me, both professionally and personally.

Get Involved: I would like to encourage those thinking about getting more involved to jump in. There are many opportunities to get involved, including volunteering to help a specific TIG at their business meeting, helping to review proposals for next year’s program, or even writing a blog post for aea365. Even just chatting with those seated around you in a session can be a great way to start a network or add to those you already have.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

Hi! I’m Jayne Corso, Community Manager for AEA with some early perspectives from Day 1 of Evaluation 2014.

The twittersphere lit up this afternoon as Beverly Parsons, our current AEA president gave her plenary talk, Visionary Evaluation for  Sustainable, Equitable Future. Evaluators were especially impressed with young Xiuhtezcatl Roske-Martinez of Earth Guardians.


No tags

My name is Dan McDonnell and I am the Community Manager for the American Evaluation Association.

Ever wanted to be a professional conference insider? Social media offers a fascinating way to add an extra level of experience to any conference you attend, and provides you extra content, as well as the means to discover fellow attendee recommendations and conference secrets or ‘life-hacks,’ provided you know your way around Twitter and hashtags. Read on to see how you can take enhance your experience at the  next conference you attend (Evaluation 2014 for many of you!).

Hot Tip: Know Your Conference Hashtag

First things first. Check out the conference website or marketing materials to find out what hashtag will be used. In the case of Evaluation 2014, the official hashtag is #Eval14. Using the Twitter search client (or one of the many third-party Twitter management tools out there), search for the official conference hashtag and start reading. Consider this your conference command center! Whenever you have some downtime, or are interested in hearing your fellow attendee’s reactions to certain presentations or sessions, search the hashtag to see what people have to say/

Hot Tip: Share Your Experience

Part of getting the most value out of social media is by being, well, social. Use Twitter as your personal digital notepad by:

  • Tweeting out neat data points or insightful thoughts from speakers
  • Sharing your own reflections on the content and topics being discussed
  • Join the conversation by @replying to other users Tweeting on the hashtag
  • Posting photos from the event

With all of the above, be sure to include the conference hashtag to join in with the larger conversation. Not only will you have a digital record of some of your experiences from the event to review later, but you’ll open up opportunities to connect and meet with your fellow conference attendees, and give those who are unable to attend the conference a small taste of the experience they are missing.

Hot Tip: Connect with Others

Don’t miss an opportunity to expand your network and learn more from conference attendees and speakers. Follow people on Twitter that are using the conference hashtag, as chances are, you’ll have a lot in common. Search for conference speakers and presenters on Twitter (or just ask for their Twitter handle in person) to give them a shout out, especially if you enjoyed their session. You can also ask them follow-up questions via Twitter or simply to ‘subscribe’ to their feed and read up on more great evaluation content that interests you.

By following along with the conference hashtag, you may also uncover great recommendations on sightseeing, local cuisine and the best place to grab a coffee near the hotel or convention center.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Older posts >>

Archives

To top