Hello AEA members! Sheila B. Robinson, aea365’s Lead Curator and sometimes Saturday contributor with an important message about the AEA elections. The  ballot is now open for our 2018 President-elect and Board Members-at-Large.

AEA member voting has been historically low. Let’s change that this year!

Hot Tip: Don’t recognize some of the names? Get to know your candidates and cast your vote! Check out candidate bios here so you can familiarize yourself with their backgrounds and their visions for AEA. These candidates are accomplished leaders not only in AEA but in their own fields as well. Do your due diligence and give them each a few minutes of your time to learn about what they have to offer our association. They’ve worked  hard to get themselves on the ballot through a rigorous nomination process that involved a great deal of time and effort.

Many of us consider AEA to be our professional home and leadership is important to who we are as an association and what we have the potential to become.

Rad Resources: Our 2018 nominees (in alphabetical order in each category):

Image credit: tzu kwan valino via Flickr

Hot Tip: Please check your recent email from elections@vote-now.com with information on the election and a link to your ballot. You may vote at any time between now and 11:59 PM Eastern Time on Friday August 4. Reminder notices have been sent to those who have not yet cast their ballot.

Email info@eval.org or call the AEA office at any time with questions or concerns related to the ballot or any of your member benefits.

Get Involved: Play a role in this year’s election and make your voice heard. Let’s rock this vote!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

My name is Brian Yoder, and I am the Director of Assessment, Evaluation, and Institutional Research at the American Society for Engineering Education (ASEE). I also serve as chair of EvalAction 2017, which is co-sponsored by the American Evaluation Association (AEA), Evaluation Policy Task Force (EPTF) and Washington Evaluations (WE). This initiative coordinates AEA members attending the conference in Washington, D.C. to visit the office of their congressional representative to discuss the value of evaluation in government and to drop off a packet of materials created by EPTF and AEA.

The idea of engaging congressional members in evaluation comes from six years ago when I was working as a temporary civil servant as evaluation manger in the education office of a science and technology focused federal agency. I was charged with providing subject matter expertise for a data system that tracks the agency’s education investments, nationally; and developing capacity to evaluate national education programs. Working for this agency, I found the guidance put out by AEA’s Evaluation Policy Task Force (EPTF) useful to share with colleagues. I would share the documents to help explain the merits of program evaluation, and I believed that government evaluation could be improved by agencies following the program evaluation guidance of AEA and the EPTF.

I was also active in Washington Evaluators, the local Washington, D.C. AEA affiliate which hosts monthly brownbag sessions on a variety of evaluation topics important to government that were often well attended by staff from federal agencies, but staff from congressional offices never attended. I thought there should be a way to engage congressional offices in discussions of the value of evaluation in government, but living in D.C. didn’t provide many opportunities. Visits to congressional offices by D.C. residents likely would be ignored and visits by civil servants were discouraged. However, if congressional offices were visited by residents they serve, the visitors would have the opportunity to speak with congressional staffers about evaluation and promote AEA’s principles of evaluation.

When I learned that the 2013 AEA conference would be in Washington, D.C. I approached AEA’s executive director with the idea of WE working with EPTF to coordinate AEA members coming to Washington, D.C. to visit the office of their congressional representative. After several meetings to discuss the scope of the initiative, Evaluators Visit Capitol Hill, as it was called at the time, was launched in the summer of 2013. In the fall of 2013, despite a government shut-down, a total of 69 AEA members from 31 states and the District of Columbia participated in the initiative. The promotion of quality evaluation and evidence-based policy making in government is more important than ever.

Rad Resource: Want to make a difference? For more information visit, visit our EvalAction registration page.

We’re looking forward to November and the Evaluation 2017 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.

lawg

Hi, I’m Giovanni Dazzo, Program Committee Chair for Washington Evaluators (WE) and Chair of the Local Arrangements Working Group for Evaluation 2017.

Last year at Evaluation 2016 in Atlanta, we learned that the Atlanta-area Evaluation Association (AaEA) piloted a pro bono evaluation program. This program connects AaEA members with nonprofits in their area by offering in-kind program planning and evaluation services. Many professional associations also have similar groups, such as the American Statistical Association’s Statistics Without Borders program, which connects statisticians that can provide pro bono services in statistics and data science to organizations that may not have access to these resources.

In the spirit of the Evaluation 2017 theme, ‘From Learning to Action’, Washington Evaluators would like to continue building on the momentum of AeEA’s great program with a new initiative for this year’s conference: Evaluation Without Borders.

This pilot initiative seeks to connect conference attendees with local community-based organizations in need of program planning, measurement and evaluation services. While conferences, and evaluation research, can often be extractive in nature, this effort aims to connect conference-goers in a way where they can meaningfully connect and give back to the Washington, DC community.

In line with AEA’s goals, we hope to not only create opportunities where local nonprofits can begin to build their knowledge and skills to engage in evaluation, but to also create opportunities where evaluators can learn more about Washington, DC and the extraordinary work of those contributing to its vibrant communities.

Rad Resources:

Why not take an extra day off from work and get involved? We’ve created a form on the WE website for those interested in volunteering. Just let us know a bit about your professional experience, program planning and evaluation skills, and volunteer interests. We’ll be scheduling pro bono consultations for November 6-7, but do let us know if you’re interested in volunteering the weeks before or after the conference. We’ll then start the volunteer matching process, so you have enough time to plan your trip for Evaluation 2017.

We’re looking forward to November and the Evaluation 2017 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.

lawg

Hello! I’m Jonathan Jones, co-chair of the AEA Local Arrangements Working Group (LAWG). As the LAWG is responsible for mobilizing local expertise and resources to enhance the annual AEA conference, a big focus of our volunteer work is ensuring that international attendees feel welcomed to the conference and to the Washington, DC area. The LAWG is undertaking a number of tasks on this front that we want to share with you.

Hot Tip: Consider participating in the International Buddy Program. This is an excellent program that connects US-based evaluators with international evaluators to share experiences and ensure that our international participants feel welcomed at the conference. Please visit the conference website if you are interested in participating in the program for AEA 2017. (Thank you to Michelle Tarsilla, who has managed this program for many years).

Hot Tip: Have you attended the annual AEA silent auction? If not, you are really missing out! The proceeds go towards supporting evaluators from developing countries to attend the conference. Last year, AEA was able to support 5 international participants! We want to do even better this year by launching an initiative for corporate sponsorship of the silent auction. If you are interested in learning more about this initiative, including how your company might support the silent auction, please contact Jonathan Jones (jonathanjonesjdj@gmail.com). (And, thank you to Hubert Palmer who has managed the silent auction for many years).

Rad Resources:

• The LAWG is also working on a number of other initiatives, such as organized tours and transportation from the airport for international attendees. If you’re an international attendee, please let us know your expected arrival time on this form. We’ll then see if one of our LAWG volunteers can meet you at the airport, or connect those arriving at the same time so you can share the cost of a taxi.
• The LAWG’s ‘Welcome to Washington DC’ committee is busy preparing a number of resources to help ensure that out-of-town participants have a great experience in Washington, DC. In addition to our local resource guide, we will also have an ‘Ask me about D.C.’ table set up near the registration area of the conference. If you know the Washington, DC area well and are interested in volunteering at the table for a 2-hour time slot, please fill out our volunteer interest form.

The LAWG is excited to welcome all AEA 2017 participants to our wonderful city during November 6-11!

We’re looking forward to November and the Evaluation 2017 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.

lawg

Hello! I am Hilary Cook, a sociologist and member of the Washington Evaluators Program Committee. I am excited to organize group tours during AEA’s Evaluation 2017, so conference attendees can meet each other and experience some of the amazing museums and institutions that DC has to offer! There are also useful links that may be of interest for individuals to organize museum visits on their own.

Rad Resource: Organized Group Tours

The LAWG will be setting up group tours of 15-20 people at several places. Tour times may change and will be scheduled to avoid conflicts with Evaluation 2017 concurrent sessions. If you are interested in joining a group tour, please SIGN UP so I can gauge interest and schedule tours as the time gets closer. Once you sign up for a given tour, I will keep you posted about the time and date! Here is a preliminary group tour schedule. All of these places can also be visited independently on your own schedule as well.

Rad Resource:

• Group tour of the Hirshhorn Museum of contemporary art and culture on November 6 at 10:30 AM.
• Group tour of the Postal Museum on November 6 at 11:00 AM.
• Group tour of the Renwick Gallery of contemporary craft and decorative art November 6 at 10:30 AM.
• Group tour of the National Portrait Gallery, which has both classic presidential portraits and wild installation pieces, on November 7 at 6:00 PM.
• NPR Headquarters on November 11 at 3:00 PM.

Please sign up to receive more information about group tours as the schedule is finalized!

Hot Tips: Suggestions for Independent Visits

There so many museums and institutions to visit in DC. Here are links to some of the public places that may be of particular interest.

Every museum that is part of the Smithsonian Institution has free access. Check out the website to plan your visit to any of the 19 museums, gardens, or the zoo while you are in town! The newest SI museum, the National Museum of African American History & Culture, is wonderful if you can land tickets! Timed entry passes for November will likely open late in July or early August, so conference attendees should keep an eye out for that on the website, and be aware that these free tickets go quickly! Please click on the link to learn about how to get timed entry tickets to this amazing museum. The National Archives is free to visit, although tickets can also be reserved online for a nominal fee.

There are also plenty of non-public museums and institutions that visitors may find interesting. Please feel free to make suggestions in the comments to your fellow Evaluation 2017 attendees on your favorite places to visit in DC!

We’re looking forward to November and the Evaluation 2017 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.

lawg

I’m Patricia Moore Shaffer, Deputy Director for Research & Analysis at the National Endowment for the Arts and Communications Chair for Washington Evaluators (WE). I’m excited to welcome you to Washington, DC, for the American Evaluation Association conference this November. Our city offers many interesting diversions, including arts and cultural destinations and outdoor experiences. Plan to arrive a day early or extend your visit by a day to take advantage of some of the Washington, DC, area’s bountiful attractions. Here are some of the city’s many attractions that might entice you to stay that extra day.

Hot Tips: Suggestions for Independent Visits

We think of Washington, DC, as a city of memorials and museums, but there are plenty of opportunities for stunningly beautiful urban hikes and nature walks. If you want an outdoors experience in the heart of the nation’s capital, Rock Creek Park has over 32 miles of hiking trails and paths to explore. After visiting the John F. Kennedy gravesite and the Tomb of the Unknown Soldier at Arlington National Cemetery, walk up to the Arlington House, Robert E. Lee’s former home, for a stunning vista of downtown Washington, DC. Great Falls is my go-to hiking destination in the Washington, DC, area. Early November is a perfect time to enjoy the late fall colors and the cooler temperatures on the trails. If you’re an experienced hiker, try the Billy Goal Trail with nearly a mile of fun rock-hopping and wonderful views of the Potomac River.

One of the premier performing arts centers in the nation is the John F. Kennedy Center for the Performing Arts. The Tony Award-winning musical, The Book of Mormon, is scheduled during Evaluation 2017, but if musicals aren’t your taste, you’re sure to find other options. Check out the free performance at the Kennedy Center’s Millennium Stage every evening at 6pm. The Kennedy Center is accessible via a free shuttle from the Foggy Bottom Metro Station. Other performing arts companies and venues, including Woolly Mammoth Theatre Company, Arena Stage, Shakespeare Theatre Company, and Ford’s Theatre, round out DC’s cultural offerings.

Washington, DC, is rich in museums of every type. Most of the national museums, including the Smithsonian and the National Gallery, offer free admission. Watch this week for an upcoming post by Hilary Cook about organized group tours during Evaluation 2017 to some of the city’s premier museums.

There are so many more attractions in Washington, DC, than can be described in a short post. For more information on transportation to the conference site and nearby restaurants, please visit the Evaluation 2017 local resource guide on the WE website.

Safe travels!

We’re looking forward to November and the Evaluation 2017 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.

lawg

As the American Evaluation Association’s 2017 conference returns to Washington, DC, this fall, on behalf of the Washington Evaluators affiliate allow me to welcome you to DC for #Eval17!  I am Nick Hart, current president of Washington Evaluators, AEA’s DC-based affiliate.

Washington Evaluators launched in 1984 and has grown to more than 300 local evaluators today. Our goal is to strengthen the evaluation community in the Washington, DC area. We pride ourselves on having a diverse representation of government, non-profit, academic, and independent evaluators that comprise our membership.

This year our membership worked to produce a new strategic plan to ensure the services and professional development opportunities offered truly serve our community. We now have four key strategic goals: strengthen the sustainability of the evaluation community; enhance evaluation relationships and interactions; support individual evaluators’ professional development needs; and ensure strong administration of the organization. Each of these four strategic goals is a core component of the Washington Evaluators mission. In implementing our ambitious strategic plan, Washington Evaluators is working to create more opportunities to engage new evaluation professionals, further the professional development of long-time evaluation professionals, and offer the 30+ years of experience of our evaluation organization to other communities of practice throughout the country.

As the seat of the United States government, Washington, DC is perhaps best known for its influence in evaluation policy. But beyond the government, DC is home to leading evaluation organizations and the brightest evaluation minds in the U.S.  Building on this broad evaluation expertise, as we prepare for an exciting #Eval17 this fall, over the course of this week on AEA365 we will be showcasing local resources, sites to visit, volunteer opportunities, a major advocacy event on Capitol Hill, and other tips for your trip to DC.

Rad Resource:  Follow Washington Evaluators on Twitter or check out our website to learn more about the many opportunities available in the DC area.  Many of our events are open to non-members as we support the entire DC evaluation community.

Lesson Learned:  Book your travel for the conference early. There are three airports in close proximity to DC (Dulles, Reagan, and Baltimore).  From any of these airports, the conference site is just a short Uber ride away.  All are also reachable by DC’s public transit options.

Hot Tip:  In addition to the resources we will share in advance of the conference, Washington, DC has an excellent tourism website that explains the sites to see in America’s Front Yard, provides tips on accessing the many free museums, and explains the neighborhoods in the city.

Get excited for a great conference this fall. We look forward to seeing you in DC!

We’re looking forward to November and the Evaluation 2017 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.

lawg

Hello community of evaluators. I’m Salima Bhimani, Founder and Director of Diversity, Equity and Inclusion at Relational. Relational offers research and evaluation and consulting and education services to organizations, companies and institutions. My focus as an evaluator and researcher over the last two decades have been on addressing bias, discrimination and barriers faced by marginalized people in communities and institutions. Here I share the centrality of getting underneath the language of diversity so that evaluations can reveal how social inequities are designed into institutions and operate within them. 

Case for consideration:

Recently I conducted an evaluation for a higher education institution. They wanted to understand how to make their curriculum and pedagogies more accessible to the linguistic, gender, racial, ethnic and economic diverse constituencies they serve in more than 10 countries. These constituencies all fall under the same religious community. The institution already had a conceptualization of accessibility. Their understanding foregrounded that everyone should be able to obtain their resources and relate to them.  It was clear to me that their approach to accessibility was intimately connected to how they thought about what diversity means. In this circumstance, their benign conception of diversity was obscuring the connection between the social subjectivities of their constituencies and their relative power, voice and positioning in relation to their institution and the broader community.  That is, there was no analysis of the historical and contemporary dynamics of unequal relations between their constituencies that were implicitly and explicitly defining the curricular content and pedagogical approaches. What was required is an awareness of how their approaches and content were already shaped for those unquestionably thought to be the norm.

Hot Tip: Break open taken for granted notions of diversity

• A benign concept of diversity flattens difference. It undermines and diminishes histories and cultural forces that design inequities within institutions and which relationally shape individual and group identities, positions, interests and needs
• A more critical conception of diversity understands how people and their experiences are socially and politically constituted in relation to each other, even within a community with a shared identity
• Such analysis is foundational to a more nuanced conceptualization of what the curriculum and pedagogies need to be and for whom
• Accessibility then is directly entangled with social realities and the biases, barriers, and inequities experienced differently within social minority groups
• Accessibility must be framed with a clear view of how social markers of difference intersect to inform experiences of access

Rad Resources:

As I have written before diversity is often used as a ‘safer’ concept within institutions. Yet, those researchers that have examined the limits of diversity as an institutional marker, make an incredibly strong case for why we should understand the function of its uses. We need be cautious and as evaluators ask whether the use of diversity in fact undermines goals towards equity and social justice.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

diversity

Hi! I’m Melissa Rivera, an evaluation consultant with Habitat for Humanity International.

I provide evaluation consulting to local Habitat organizations and am pleased to share with you a conclusion we’ve reached about feedback loops as an approach to increasing community engagement.

Over the past 40 years, Habitat for Humanity has worked with people around the globe to help build or improve a decent and affordable place they can call home. The most important element of our mission is the partnership between Habitat and the homeowner, and we continuously seek to keep homeowners and their input at the center of what we do. Through Habitat for Humanity’s Neighborhood Revitalization approach, Habitat engages with local residents to help them respond to community aspirations, revive their neighborhoods and enhance their quality of life.

Over the past three years, we have piloted a systematized approach — feedback loops — to collect and analyze data and share findings with community residents to co-create strategies and projects that address their aspirational goals. We started with 12 local Habitat organizations in the United States.

Our goal was to examine if integrating standardized feedback mechanisms increases the level of resident engagement in these neighborhoods. We evaluated as follows:

• Explored whether technology that allows for real-time feedback could help facilitate this process.
• Developed a rubric to track the progression of integrating feedback loops.
• Evaluated data associated with the feedback loop prototype.
• Completed focus groups and key information interviews.
• Synthesized all data to provide quarterly updates on the level of resident engagement.

Lessons Learned:

• Feedback loops work!
• Having a rubric to monitor the coalition’s integration of feedback loops helps Habitat provide clarity and to track continuous progression.
• The feedback loop methodology can increase or sustain the level of community resident engagement. This occurred in each of the 12 pilot communities. As an example, prior to the pilot, 11 residents of a Missouri neighborhood were engaged in community projects. Now, more than 46 residents are actively participating.
• Integrating the feedback loop methodology during community conversations can change the dynamic between community residents and the people and agencies that support them. One example from our pilot is a Georgia neighborhood where community residents said they felt as if they did not have a major role in decision-making regarding community improvement.  Now community residents and partners are both co-creating projects and planning on their future efforts with equal input.

These learnings provide insight for Habitat for Humanity on how collecting shared metrics has strengthened the evaluation of community engagement. The methodology helps us to continuously refine programs, projects, or systems.  Our next step is to undertake a detailed exploration of the sustainability of feedback loops.

Rad Resource: Survey Gizmo has built in reporting mechanisms that enabled community residents to see how their community responded. Another great technology is Scantron’s Class Climate product. This system is especially helpful for agencies that work in communities where residents prefer to provide their feedback through a paper form.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

feedback loops

Hi! I’m David McCarthy, a 4^th year medical student at the University of Massachusetts Medical School. I had the opportunity to get involved in the Prevention Wellness Trust Fund (PWTF), a project run by the Massachusetts Department of Public Health that works to combat treatable chronic medical conditions by integrating clinical and community interventions. I chose to focus on the pediatric asthma intervention of the City of Worcester’s PWTF, which utilized a series of Community Health Worker (CHW) home visits. As part of this project’s evaluation, I interviewed CHWs and Care Coordinators about their experiences providing home visits for patients with pediatric asthma and their families. In this blog, I summarize some tips and tricks that I learned that could help refine a community-based care model and be used as benchmarks for future care model evaluations.

Hot Tip: Let those with the contacts help with the networking

Initially, getting patients referred for enrollment in the intervention was difficult due to lack of medical provider education about the program. The solution had two components. First, increasing the frequency of Worcester PWTF asthma workgroup meetings improved coordination between the different groups involved and overall program engagement. Second, provider champions at each site reached out directly to other providers taking care of patients within the focus population, which expanded the project reach. Eventually, referral numbers improved, as they were coming in from nearly all care team members.

Hot Tip: Think outside of office hours when coordinating visits with families

We needed to be flexible scheduling home visits outside of typical business hours, including weekends, to accommodate families’ schedules. CHWs also needed to be available to patients by cell phone for calls and text messaging. This scheduling and options for availability helped to build trust with families and further helped retention of patients in the program.

Hot Tip: Consider care provider’s safety

As with any intervention that requires home visits or meeting parents/families in their own space, it’s always good to remember that the safety of study team members is paramount when going to unfamiliar sites. As part of this project, we provided personal safety training for CHWs who were entering patient homes. Where possible, a team of 2 CHWs conducted each home visit and CHWs confirmed dates and times with families before each visit.

Lesson Learned: Account for the varied needs of patients and families

CHWs provided a standardized set of asthma management supplies to families at each visit, including medication pill boxes, trash cans, mattress and pillow covers, and vacuums. This was designed to incentivize their engagement and compliance with their asthma management plan. However, these supplies didn’t always match individual families’ needs. Future intervention efforts should tailor supply sets for each family based on their existing individual home environment.

Overall, our evaluation efforts identified that an integrated clinical program to address social determinants of health through CHWs represents an innovative healthcare delivery system and is very feasible to implement.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

interviewing · interviews