AEA365 | A Tip-a-Day by and for Evaluators

Hi! My name is Denise Ramón. I am a doctoral student in education at the University of the Incarnate Word in San Antonio, Texas and work at the Center for Civic Leadership that focuses on civic engagement and leadership. More specifically, I help to connect my university to the community. I am interested in Asset Based Community Development (ABCD).

Lessons Learned: While at the AEA 2014 Denver conference, I attended a session that was of particular interest to me, Altschuld, Hung, and Lee’s Getting Started in an Asset/Capacity Building and Needs Assessment Effort. Two dichotomous philosophical approaches were presented, needs assessment and asset / capacity building (A/CB). One of the main ideas stemming from this presentation was to create a hybrid framework between needs assessment and asset mapping. If evaluation is evolving to be visionary and sustainable, mixing traditional models, such as needs assessments, with newer ideas, such as capacity building and asset mapping, seems rather logical. This way, the best of both worlds can be extracted and can fill each other’s gaps, one can complement the other rather than being at odds. With this innovative notion, more research is needed to see if a model can really be developed and effectively implemented.

Coming to my second AEA conference enhanced my network system. I participated in most of the social events hosted by AEA, such as the TIG social events, the poster presentation session, and the silent auction. Getting to know others in the field gives me confidence to participate in more evaluation activities because I know I can ask for help and turn to other veterans with more expertise. Lesson learned: Jump in to AEA with confidence and an open mind. Reach out to others. Network.

Rad Resource: Using the AEA Public elibrary to find the presentations was so very useful for me. I was able to download the presentations and can now possibly use the document as a reference for my research. I highly recommend using the AEA e-library. You can also upload your own presentation and documents. It is another way to promote your work.

As a doctoral student and novice to the evaluation field, the mere experience of attending the conferences has enhanced my overall learning and understanding of evaluation. Not only have I learned about new resources to tap into, like the e-library, but I have been able to relate newly learned evaluation concepts to other parts of my professional and academic life and research. This has been in part to having made new connections.

We’re celebrating Evaluation 2014 Graduate Students Reflection Week. This week’s contributions come from graduate students of Dr. Osman Ozturgut of the Dreeben School of Education at the University of the Incarnate Word, along with students from other universities. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello! We are Çigdem Meek, Bashar Ahmed, and Marissa Molina, PhD students at the University of Incarnate Word in San Antonio, Texas. As novice evaluators, we would like to share what we have learned from our experience of attending the 28th Annual Conference of the American Evaluation Association in Denver.

Lessons Learned:

  • Attending the conference as a group of PhD students from the same university eased our anxiety of being among expert evaluators. Plan with your peers to attend the next conference in Chicago!
  • Stay at the conference hotel (and make your reservation as soon as possible). You will not regret the networking opportunities it provides!
  • Attend pre-conference and post-conference workshops! Evaluation 101 is a great workshop to understand the basics of evaluation.
  • Join Topical Interest Groups (TIGS) business meetings. Meet with like-minded evaluators!
  • Look for volunteer opportunities, especially if this is your first time. This helps you meet with other evaluators with ease (and also helps with the registration cost).
  • Participate in panel discussions. This is an excellent way to meet and learn from other evaluators.
  • Do NOT miss the opportunities to learn from the best through panel discussions, workshops, and conference sessions! (i.e. Donna Mertens, Robert Stake, Stafford Hood, Rodney Hopson, Hazel Symonette, Jody Fitzpatrick, Michael Scriven, Michael Patton, Art Hernandez, Karen Kirkhart, and Cindy Crusto have facilitated excellent sessions and provided exceptional insights for novice evaluators).
  • Make sure you have your business cards (a lot) with you and exchange! Remember to take notes on cards you receive (I thought I could remember all!). In order to stay connected send them a brief email within 10 days after conference.
  • Take notes to review later during the sessions and reflect on what you learn. Remember, reflection is what makes learning meaningful.

Rad Resources: Check out these resources before attending the conference!

We’re celebrating Evaluation 2014 Graduate Students Reflection Week. This week’s contributions come from graduate students of Dr. Osman Ozturgut of the Dreeben School of Education at the University of the Incarnate Word, along with students from other universities. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, my name is Erica Roberts, an AEA GEDI scholar, doctoral candidate at the University of Maryland School of Public Health, and an intern at the National Cancer Institute Office of Science Planning and Assessment. As a graduate student who is approaching the transition from student to professional in the field of public health evaluation, I would like to share with you the lessons I learned from attending the AEA conference in the hope that these lessons can be used by other graduate students planning to attend next year’s conference.

Lesson Learned: Prepare to build your professional network. The AEA conference provides an expansive and rare opportunity to meet evaluation experts, future mentors, and possible employers. Prior to attending the conference, use the Topical Interest Groups (TIG) to navigate the conference program and identify experts in your field of interest. Remember to pack business cards and update your resume or vitae. Once at the conference – be bold! Introduce yourself to presenters from organizations or fields of practice that interest you and have a few talking points or questions prepared. Once you’ve connected, add their information into an Excel spreadsheet and, after the conference, note if and when you follow-up via email and the outcome of your discussion. This will help for professional networking down the road!

Lesson Learned: Prepare to be overwhelmed (but in a good way). Before arriving at the conference, figure out a way to stay organized that works best for you. I brought my iPad to each session and used the EverNote app to take notes. Most importantly (to my organization), I kept a “to-do” note where I listed everything I wanted to do when I returned home (e.g., articles to read, experts to connect with, student scholarships or job opportunities to apply for). It is likely that you will encounter a lot of information that you want to know more about but do not have the mental space to process – this is where making a “to-do” list for home comes in handy!

Lesson Learned: Prepare to be inspired. You may find at the AEA conference that the ways to approach evaluation are endless – depending on the field, the context, the purpose, etc. Do not let this discourage you; rather – let it inspire you. Take these ideas and put them in your back pocket and know that at some point you may be asked to conduct an evaluation and you will have a myriad of methods and approaches to look to. I encourage you to use the AEA conference to learn about approaches to evaluation that you are not familiar with, and identify ways in which those methods could be adopted to your work!

We’re celebrating Evaluation 2014 Graduate Students Reflection Week. This week’s contributions come from graduate students of Dr. Osman Ozturgut of the Dreeben School of Education at the University of the Incarnate Word, along with students from other universities. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

 

 

 

Hello! My name is Danielle Cummings. I am a member of the AEA Graduate Education Diversity Internship (GEDI) program and a graduate student at NYU Wagner School of Public Service studying public policy analysis.

Attending AEA’s annual conference was inspiring and edifying. As a graduate student learning analytical methods and research design, the conference was a wonderful opportunity to see practical applications of many of the tools I learn about in the classroom. I came away from the experience with both a refined vision of what a career in evaluation might entail, as well as a wealth of theories, frameworks, and skills to integrate into my work.

I’m anxious to put so many of the things I learned at AEA into practice, but the technique I’m most excited about is called solution-focused qualitative interviewing. Dr. Emily Spence-Almaguer and Shlesma Chhetri introduced session participants to an innovative approach to qualitative inquiry that they believe has improved interview participants’ candor, thereby increasing the richness of their qualitative data. Trained as a social worker, Dr. Spence-Almaguer adapted a therapeutic technique called solution-focused therapy to enhance qualitative inquiry.

Hot Tip: There are two key elements of solution-focused dialogues: 1) People are experts on their lives, and 2) interviewees filter their responses based on expectations. In practice, this means that when we approach qualitative interviews with humility, treat interviewees as the experts, and frame our questions in a way that encourages creativity, interviewees’ responses will be more frank, dynamic, and provide sufficient context to require low levels of inference by the researcher.

Here are examples of typical vs. solution-focused qualitative questions asked by Dr. Spence-Almaguer’s research team in her research on solution-focused questions, and examples of interviewee responses:

Traditional approach:

What would you recommend to improve the program?

“Nothing, they are dong an outstanding job.”

Solution-focused approach:

If I were going to give this program another $100,000 next year, what would you recommend that the program administrators do with the money?

“Put more of the [initiative’s] programs together and coordinate them to make them work more effectively.”

By constructing a question that placed the interviewee in a position of authority and invoked imagery, the interviewer elicited a response that not only provided a critique of the program, but also a potential solution to a programmatic problem.

Rad Resource: This post just scratches the surface of solution-focused interviewing. For more information on this approach, check out the slide deck from Dr. Spence-Almaguer’s AEA presentation, available for free to AEA members on AEA’s eLibrary. Make sure to check out the list of additional solution-focused literature and resources on Slide 24!

We’re celebrating Evaluation 2014 Graduate Students Reflection Week. This week’s contributions come from graduate students of Dr. Osman Ozturgut of the Dreeben School of Education at the University of the Incarnate Word, along with students from other universities. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I am Kristin Woods, 2013-2014 co-chair for the GSNE TIG; I am a PhD student at Oklahoma State University in the Research, Evaluation, Measurement, and Statistics program and a faculty member at Southwestern Oklahoma State University.

I attended my first AEA conference in 2012 and was overwhelmed by the number of people and sessions as well as trying to learn about AEA. My advisor, Dr. Katye Perry, encouraged me to attend the Graduate Student and New Evaluator’s TIG business meeting. This drastically changed my experience with AEA because I was voted in as a co-chair even though I did not really know what I was getting into. Over the past two years, this experience has granted me many opportunities that have made me a better evaluator.

Opportunity 1: AEA Involvement. As a co-chair, I have further developed my skills as an evaluator through the vast amounts of resources on the AEA website. I have worked with AEA members, members of the board of directors, and staff on various tasks for the conference. For example, I served as a member and then chair of the Student Travel Awards Working Group and as a conference volunteer.

Opportunity 2: GSNE TIG Involvement. I have developed leadership skills through working with other members of the leadership team to coordinate the conference program, serve as a reviewer, run the business meetings, coordinate social outings, communicate with members, and develop a peer-mentorship program that connects novice evaluators with peers to aid navigating AEA and offer advice on evaluation.

Rad Resources:

  • GSNE TIG website has specific information geared toward novice evaluators and those new to AEA.
  • GSNE TIG Facebook Community Page is a place TIG members informally network throughout the year. We share resources, ask questions, and celebrate our successes as well as commiserate over our struggles.

Opportunity 3: Networking. These opportunities have allowed me to expand my network to include novice to more experienced evaluators from all over the world. I have co-authored several accepted submissions at the 2013 and 2014 AEA conferences, chaired sessions, and been asked to speak at another TIG’s business meeting. This has led to the past two years conferences being drastically different from my first conference. I speak with people I met the previous year, have engaged with through the Facebook page, e-mail, or on the phone. It allows me to put a face with a name, get to know them, and connect with another evaluator that has different experiences, therefore, becoming another resource in my toolbox to pull from when needed, which I do often.

We’re celebrating Evaluation 2014 Graduate Students Reflection Week. This week’s contributions come from graduate students of Dr. Osman Ozturgut of the Dreeben School of Education at the University of the Incarnate Word, along with students from other universities. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Andrea Guajardo and I am the Director of Community Health for CHRISTUS Santa Rosa Health System and a doctoral student at the University of the Incarnate Word in San Antonio, Texas. I have been a member of AEA for exactly one year, and in that year, I was selected for the Minority Serving Institution Fellowship program, elected as the Co-Chair of the Multi Ethnic Issues in Evaluation (MIE) Topical Interest Group (TIG), and functioned as a core planning member of La RED (Latino Responsive Evaluation Discourse) TIG. In addition to these positions within AEA, I have also presented two poster sessions and one paper at AEA2013 and AEA2014.

I would like to use this platform to encourage other graduate students to seize opportunities for professional development afforded by active participation in AEA.   Your graduate education related to evaluation can be tremendously supplemented by creating relationships with experienced evaluators and by providing leadership for groups within AEA.

Hot Tip 1: Become an official member of AEA, join a TIG, and volunteer your time. TIGs are an essential part of the AEA experience both at the annual conference and throughout the year. TIGs are responsible for coordinating the review of proposals in their area of interest and developing a strand of conference sessions at the AEA annual conference, so volunteer help with this process is always appreciated.   TIG membership can help you create relationships with top evaluators and might afford the opportunity to learn about emerging ideas in your field or discipline.

Hot Tip 2: Don’t just attend the annual conference – participate in it. Submit your own evaluation work for a poster, paper, roundtable, or birds of a feather.   Even if your research is in progress, it is still an appropriate occasion to perfect your presentation skills.

Hot Tip 3: Find a mentor. Many experienced AEA members are very willing to provide guidance about how to become more involved and to help map out a path in evaluation at AEA. Their expertise and guidance is valuable as you begin to navigate which activities will benefit you the most in your evaluation career.

Rad Resources: Where do I find more information about joining a TIG or participating in the next annual conference?

For more information about Topical Interest Groups: http://www.eval.org/p/cm/ld/fid=11

For more information about how to submit your evaluation work at AEA2015 in Chicago, Illinois: http://www.eval.org/p/cm/ld/fid=170

We’re celebrating Evaluation 2014 Graduate Students Reflection Week. This week’s contributions come from graduate students of Dr. Osman Ozturgut of the Dreeben School of Education at the University of the Incarnate Word, along with students from other universities. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello loyal aea365 readers! I’m Sheila B Robinson, aea365’s Lead Curator and sometimes Saturday contributor. Thank you for being “here” today and reading this post! As I was searching for inspiration for this Saturday’s post, I turned to our aea365 archives and found a post from fall 2010 by Susan Kistler (thank you Susan!) that really resonated with me and her message bears repeating.

Lesson Learned: Whenever you feel appreciation for people, let them know. That’s it. Just do it.

  • Thank the person who helped you tweak that survey question to get it just right.
  • Thank the people who gave up their time to participate in a focus group.
  • Thank the interviewee who trusted you enough to be so forthcoming with information.
  • Thank the people who opened their doors and gave you access to the site for your observation.
  • Thank the professor, workshop facilitator, or  book author from whom you learned important skills that contribute to your professional success.
  • Thank the research assistant, data analyst, or statistician who pored over the data to help you make sense of it.
  • Thank your client for choosing to hire you for the job, or your employer for giving you the opportunity to do evaluation work.
  • Thank the blogger whose post gave you insight, taught you something, or inspired something in your work.
  • Thank those who freely share their resources and materials.
  • Thank the Tweeters, Facebookers, LinkedIn-ers, Google+ers, and others on social media who offer innovative ideas, experiential wisdom, and links to great content.

I want to take this opportunity to thank the hundreds of wonderful authors and volunteer curators who have so graciously contributed to make aea365 my very favorite evaluation blog (and yours too, I hope!) and a tremendous crowd-sourced treasure trove of evaluation know-how!

Hot Tip: Make someone’s day today by saying “thank you.”

Rad Resources: Some insights on the importance of saying thank you:

The Two Most Important Words

Why It is Important to Say “Thank You”

Why ‘Thank You’ is More Than Just Good Manners

Image credit: Chris Piascik via Flickr (thank you Chris)

Image credit: Chris Piascik via Flickr (thank you Chris)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

I am Andrea Guajardo, MPH and I am the Director of Community Health at CHRISTUS Santa Rosa Health System in San Antonio, Texas. I am also a doctoral student seeking the PhD in Education at the University of the Incarnate Word, and I enjoyed the distinct honor of being selected to participate in the Minority Serving Institution (MSI) Faculty Fellowship Program in 2013-2014.

My acceptance to the MSI Faculty Fellowship Program came as a surprise because I am not a faculty member of an institution that possesses this designation. I am not a faculty member of any institution. Despite the fact that I did not fit the typical mold of previous MSIs, I applied for the fellowship based on my experience in evaluation within the context of hospital operations, grant administration, and community-based programming.   My year as an MSI produced personal growth as an evaluator, and it also allowed me to share my perspective with my fellow MSIs – all of whom are accomplished evaluators and faculty from around the United States and Puerto Rico.

Interaction within our cohort often led to discussions of evaluation theory, models, and applications that I had not considered as a hospital-based evaluator. In return, I offered my pragmatic, “real-world” approach to evaluation. My year as an MSI has facilitated a rapid development of a skill set as an internal evaluator that I could not have acquired from workshops and textbooks and enabled me to share a more robust understanding of evaluation among my clinical colleagues in the hospital.

AEA is a dynamic organization comprised of evaluators from multiple disciplines, philosophies, and theoretical perspectives. It is also home to evaluators whose backgrounds are not rooted in academia and theory. My year in the MSI program allowed me to experience evaluation from multiple viewpoints and to clearly identify my own perspective and path as an evaluator in hospital operations and in academia.

Lessons Learned: Don’t label yourself. Seek opportunities for growth in evaluation even if you do not necessarily fit into a defined category as an evaluator.   Don’t pass up an opportunity at AEA because you don’t “fit the mold.” Take a chance and you might find that others can learn as much from you as you learn from them.

Lesson Learned: Seek out colleagues from different disciplines and perspectives. Although the MSI program is a formal process, you don’t have to be an MSI to engage evaluators whose experiences, backgrounds, and skill sets differ from yours. Shared knowledge benefits then entire AEA community.

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Edilberto A. Raynes. I am an Associate Professor from the Tennessee State University. I am one of the five MSI Fellows from 2013-2014. I am coming in to the fellowship program with minimal background in evaluation. However, I have broad experiences in conducting social research in an academic institution. I have noticed that there were differences between conducting evaluative research and purely evaluative research, which include constructing a survey, writing a report, and conducting a presentation.

Lessons Learned:

  • It is best to know the background of the stakeholders, the recipient of the report, the timeline, and the budget.
  • Rubrics are utilized to facilitate framing of survey questions and to interpret quantitative, qualitative, and mixed methods.
  • Logic models include five core components: inputs, outputs, outcomes, assumptions, and external factors. These components are very valuable in mapping out the evaluative process.
  • There is culture in the art of evaluation. There is a need to increase awareness of creating a culturally responsive evaluation.
  • The art of writing non-evaluative questions. The use of proper verbiage such as “Was it any good? To what extent?
  • The use of focus group as a form of evaluation.

Hot Tip: Attend the workshops that you are interested in exploring because you will find new information. These workshops include, but are not limited to AEA pre-conference workshops or professional development workshops in general. In my case, I learned a lot in the pre-conference workshops. Collaborate with colleagues because that is the only way one can establish relationships. Once relationships have been established, one is able to build tools in evaluative thinking.

Rad Resources:

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Denise Gaither-Hardy. I am an Assistant Professor at The Lincoln University in Pennsylvania.

I want to share a few thoughts about my experience as a member of American Evaluation Association (AEA) and Minority Serving Institution Fellow (MSI) during the past year. My five member cohort was led by Art Hernandez, Professor and Dean of the College of Education at Texas A&M University and my individual mentor was Kevin Favor, Professor at Lincoln University and both are former MSI fellows.

This Fellowship has offered me an opportunity to acquire and enhance evaluation skills while becoming a member of an organization that continues to provide me with future opportunities needed to advance my career.

I have been fortunate that some of the skills I have refined and acquired are as follows:

  • the art of survey development, which is truly is an art
  • the theoretical understanding of culturally responsive evaluation and the need to increase awareness and advancement
  • the utility of logic models in evaluation

Rad Resource: Susan Kistler on Lessons Learned Using Online Survey SoftwareGo to the AEA website and read more about Dr. Kistler and her work on survey design and development at AEA.

Rad Resource:  Stafford Hood, Rodney Hopson, & Henry Frierson’s book on culturally responsive evaluation, The Role of Culture and Cultural Context – A Mandate for the Inclusion, the Discovery of Truth and Understanding in Evaluative Theory and Practice. Read more about cultural responsive evaluation at the Center for Culturally Responsive Evaluation and Assessment.

Rad Resource: Thomas Chapel is the Chief Evaluation Officer at the Centers for Disease Control and Prevention. He serves as a central resource on strategic planning and program evaluation for CDC programs and their partners. He is a frequent presenter at national meetings, a frequent contributor to edited volumes and monographs on evaluation, and has facilitated or served on numerous expert panels on public health and evaluation topics. Go to the AEA website and read more about Dr. Chapel and his work on logic models.

Lesson Learned: The profession of evaluation has increasingly become recognized as essential in both the private and public sectors. Training is critical and empowering. The MSI Initiative allows you to put into context components like culture and equality, while giving enough room for self-exploration. You will inherently develop and/or refine technical skills which most assuredly will include theory and methodology.

Hot Tip: No matter how often you attend workshops and speak with colleagues there is always something new to be learned. The program evaluation skills and competencies that can be learned through AEA will not only benefit you, but students, faculty and administration at your home institution.

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Older posts >>

Archives

To top