AEA365 | A Tip-a-Day by and for Evaluators

TAG | interviews

Hello, my name is Deborah Grodzicki and I just received my Masters in Organizational Behavior and Evaluation from Claremont Graduate University.  I plan to pursue a PhD in Evaluation at UCLA in the fall.  Prior to attending Claremont Graduate University, I investigated complaints against New York City police officers.  During my time as an investigator, I gained experience questioning civilian complainants and police officers about extremely sensitive issues.  Drawing on this experience, I will give some tips on how to obtain essential information without compromising evaluator – stakeholder relationship.

Hot Tip: Do not be a prisoner of your question list. At their most basic, interviews and focus groups consist of the evaluator asking stakeholders a list of questions.  To make these qualitative measures most effective, however, it is critical to maintain flexibility in your questioning and establish a conversational atmosphere.  Do not use the questions as a crutch, but rather as a directional tool for the conversation. Otherwise, you risk casting yourself as an interrogator, which could result in the individual withholding vital information.

Hot Tip: Check your biases at the door. It is natural to come into a situation with personal biases that may affect how you approach an interview and/or focus group. It is important to be mindful of these inevitable biases and make a conscious effort to prevent them from affecting how your questions are phrased and delivered. Faced with a biased or leading question, a stakeholder is more likely to provide more restricted answers that mirror the bias and unduly skew the results.

Hot Tip: Withhold judgment. When conducting interviews and/or focus groups, never give someone the impression that you disapprove of their thoughts, feelings, or actions.  It is up to you as the evaluator to generate a safe, comfortable, and above all, accepting atmosphere. Only then will a stakeholder freely share their impressions about the evaluand.

Hot Tip: Look them in the eye. During my time as an investigator, I was taken aback by how many of my colleagues broke eye contact when a complainant spoke about a sensitive issue. Though seemingly insignificant, this small action can have substantial consequences. Failing to maintain eye contact at the stakeholder’s most vulnerable moment gives the impression that you are uncomfortable hearing what they have to say.  Sensing this can lead the stakeholder to feel self-conscious and promptly shut down.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

My name is Bob Spencer and I work for the Riverside, California Department of Mental Health. I serve as an evaluator for a program that aims to keep children with mental health and substance abuse problems in their natural home with their family. My goal for this blog post is to share some of my experiences from my recent job search. Finding a job in this economy is an unenviable position, but with enough persistence and tenacity, it is certainly not impossible.

Ted Williams, widely regarded as one of the best baseball players ever, owns one of the most famous quotations in the game. He said, “Baseball is the only field of endeavor in which a man can succeed three times out of ten and be considered a good performer.”

“Teddy Ballgame” never had to look for a job.

As anyone who has gone through an extended job search can attest, receiving a call back on just one of ten applications can be promising, let alone three. If baseball is known as the “Game of Failure,” what does that make job hunting?

Here are some of the things I focused on when looking for my first “grown-up job.” Hopefully they will help you as well.

Hot Tip: Be Persistent!  Hiring managers are extremely busy—filling open positions is but one of their many responsibilities. If you submit an application or resume and do not hear back within several days, do not hesitate to follow up with a polite e-mail or phone call. Applicants are sometimes afraid to seem to “pushy,” but a friendly reminder can often help your chances of getting an interview, and may help you stand out from the other candidates. I actually looked forward to that follow up call, because it gave me the chance to connect personally with my potential employer instead of being just another name on a page.

Hot Tip: Give Them No Reason NOT to Hire You! Double-negative notwithstanding, this was my philosophy for any job interview I got. A well-crafted resume and cover letter are essential for “getting you in the door,” but it is up to you to make a lasting impression and convince the hiring manager that you are the right person for the job. Be prepared to talk about your experience instead of just your coursework and research history. Be true to yourself—the worst thing you can do is misrepresent yourself to your future employer. In my experience, adaptability is key. If you do not have experience with a certain skill or resource, emphasize that you are willing and able to learn anything they need you to learn. Always have questions for your interviewer! Have several questions, prepare them ahead of time, and write them down if you need help remembering. Your interviewers will be pleased that you are taking a genuine interest in the position, and will certainly not mind the role reversal of having you interview them for a few minutes.

Hot Tip: Most importantly, Don’t Take It Personally! Ted Williams hit 521 career home runs, but he also struck out 709 times. Job hunting is undoubtedly a frustrating experience. However, the key is to not take it personally when you get turned down, and instead continue looking for your next opportunity. As in baseball, patience and persistence are the tools for success in finding a job. Don’t get discouraged if you strike out. Instead, step up to the plate again, knock the dirt off your cleats, and swing for the fences!

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

·

Hello!  My name is Stefanie Leite and I am a Research Assistant for Advanced Empirical Solutions and an Independent Evaluation Consultant.  As program evaluators, we seem to find ourselves in the role of interviewer more often than not.  However, job seeking transfers our role to interviewee.  While it may take a different set of skills, I daresay that as evaluators, we make the best interviewees!  This is because we tend to be sympathetic toward interviewers (having been in their shoes), and hyper-sensitive to answering questions in ways that provide the exact bits of information that the interviewers seek.

Hot Tip: For the purpose of this blog, I’d like to share with you some interview questions I have come across as an interviewee for jobs in the evaluation field, with the intention of helping those of you out there who are job seekers to prepare.

  1. What computer skills do you have and how have you applied them in research and evaluation?
  2. What steps do you take to ensure the integrity of the data?
  3. What experience do you have in quantitative/qualitative data analysis?
  4. Describe a problem you encountered in analyzing data and how you handled it.
  5. Describe your experience in [project management/developing data collection instruments/report writing/presenting results and recommendations to stakeholders].
  6. [The interviewer reads a case study of a program.]  How would you evaluate this program?
  7. Describe an accomplishment that you are especially proud of.
  8. Describe a mistake you made.  How did you handle it?  What did you learn from it?
  9. What is the most challenging aspect of program evaluation for you?  What is the most rewarding aspect?
  10. Describe your ideal work environment.
  11. Describe your ideal supervisor.
  12. Describe your experience working in a team.  What do you like most about working in a team? What do you like least?
  13. Why are you interested in working for us?
  14. In what ways would this position contribute to your long-term career goals?
  15. In what capacities would you like to grow professionally?

One more thing—at the end of the interview, the interviewer always asks, “Do you have any questions for me?”  An excellent strategy is to refer to your own list of questions that you prepared prior to the interview.

Read Resource: These two books by Tony Beshara have been indispensible in helping me prepare for job interviews: The Job Search Solution (2006), and Acing the Interview (2008).

Best wishes to you all in securing the job of your dreams!

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

·

My name is John LaVelle, I am an advanced graduate student at Claremont Graduate University.  When I worked as the Jobs Coordinator for my department, I would encourage the students to develop a personal statement about evaluation.  This is important because when they would go to interviews, they would often be asked to describe their understanding of evaluation and explain it to people that may or may not have an background in evaluation.  This exercise eventually became an important element in the Evaluation Procedures course.

Hot Tip: Develop a personal statement of what evaluation means to you and how it can and should be practiced in dynamic, fluid, and political organizational and community environments and how it differs from basic research. In other words, if a client asked you to explain your understanding of evaluation, your approach to evaluation, how you would work with stakeholders, and so on, what would you tell him or her? In your statement, explain what processes you think are important for designing and implementing an evaluation, and how you would approach determining an evaluation’s design and data collection methods.

What might your personal statement of evaluation look like?

· ·

Hi, my name is Dreolin Fleischer.  I am a doctoral student at Claremont Graduate University. I would like to share resources, at different price points, I have used to capture and organize qualitative and quantitative telephone interview data.

One resource I have used in the past is Microsoft Office Access. You can create a form in Access that mirrors the interview protocol you are using. You control where each field on each form is located and you can create multiple tabs for different interview questions.  As you conduct the interview you enter the interviewee’s responses directly into the field associated with the question you posed.  Cost: $$

I have used online survey programs (e.g., SurveyGizmo, SurveyMonkey, etc.) for the same purpose. You can create an online survey that mirrors the interview protocol you are using. You log into the online survey (as if you are taking the survey yourself) and enter the interviewee’s responses directly into the survey. At the completion of the interview, you can import the data (most of these programs allow you to import into Excel or SPSS files).  Cost: Free to $$ (depending on the online survey program you use)

I’ve yet to explore the tool myself, but I heard from a colleague that Google Documents now offers a way to develop online surveys for free: http://www.google.com/google-d-s/forms/.  Cost: Free

I prefer using the aforementioned resources because:

  • The forms I create help me stay organized and guide me through the interview when I am on the phone.
  • I have flexibility about how I organize the questions on the form (i.e., I can cluster questions together or isolate a single question according to my preference).
  • I can easily record both open-ended qualitative responses and close-ended quantitative responses using these resources.
  • It saves me time because the data is immediately available in a spreadsheet/table format at the conclusion of the interview.

I also use an audio recorder to record many of my interviews.  You can purchase audio recorders that connect to your landline phone or cell phone.  Of course you should always ask the interviewee permission before recording the interview.  I have had very few people refuse to be recorded. Keep in mind that recording interviews may not be appropriate for all data collection contexts.  You must weigh the pros and cons of using an audio recorder in relation to the information you are inquiring about.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· · · · · ·

<< Latest posts

Archives

To top