Happy LGBTQ+ Evaluator Week! I’m Joseph Van Matre, an institutional research analyst at the University of California*.
Surveys are an integral part of evaluation, and when designing a survey, gender is often the second question after name. We add it to our surveys without a second thought. But do we need to know?
Evaluators are, by our nature, curious people, but evaluations are not fishing expeditions. Look carefully at the program’s theory of action and your own evaluation plan.
Cool Trick: Ask yourself:
- Is this program targeted at a particular sex or gender (they’re different!)?
- Is there reason to believe that this program will have a differential impact on participants/subjects of different genders or sexes?
- Do you need to collect gender/sex information of everyone involved in the program, or just some participants (e.g. teachers and students?)
- Do you have the time, resources, and mandate to evaluate and report on gender/sex differentials?
While our first instinct is to collect as much data as possible about the programs and interventions that we evaluate, it is our responsibility not to collect and store personal information that we do not need. If you will not or cannot use information on someone’s sex or gender, you do not need to ask in the first place.
Ask for what you need to know. Unless you are evaluating a health-related program, you probably only want to ask about a person’s gender.
It is often best to allow people to identify their gender in an open ended way: I identify my gender as ___________. With large-scale surveys, you can provide male, female and an open response so that you only manually code responses that are not male or female.
Hot Tip:
Some government agencies or funders may require rigid gender reporting. The Integrated Postsecondary Education Data System (IPEDS) requires colleges and universities to report the gender of every student as either male or female. Even unknown or missing is not an option, leading to some very strange reporting outcomes.
While dreadfully cliché, the adage, “say what you mean and mean what you say,” is an important rule when reporting evaluation outcomes related to sex and gender. Your forethought and planning will make your communication inclusive and accurate.
Cool Trick:
For example, when you ask people to identify their gender on a survey, the phrase, “there were 24 men in our sample” can be replaced by, “27 people in our sample identified themselves as male.”
It is our job to continually educate ourselves about the people and programs we evaluate, and any population is likely to include people who do not fall within the gender binary. Asking for exactly what you need (or not asking at all!) is a simple way to create more inclusive evaluations.
* The views and opinions expressed in this post are those of the author and do not necessarily reflect the views, positions or policy of the University of California.
The American Evaluation Association is celebrating LGBT TIG Week with our colleagues in the Lesbian, Gay, Bisexual & Transgender Issues Topical Interest Group. The contributions all this week to aea365 come from our LGBT TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for
Hello, I am currently a Masters student attending Queens University and I am currently involved with a board wide evaluation project of a new math program.
I know that one of the questions we ask teachers is the amount of males and females in their classroom. It seems to be a standard question that has always been included on a typical report but after reading your post I am wondering why the school board would need this information.
Within my school I know of students who don’t identify with either and I now wonder what their teacher decided to put down as an answer. Why is knowing a students gender or how they identify important when evaluating a math program the school board wants to use? I never really questioned why this was something we ask but now I am curious as to why this was included. Gender has nothing to do with your ability to solve math problems so why should it be a question we ask teachers to answer during an evaluation of a math program? I think we need to go back and update out boards evaluation program.
As an elementary/special education teacher who is new to program evaluation, it is insightful to see how LGBTQ issues and concerns are being addressed from this perspective. Our school board has put together a Positive Space Committee to inform and empower staff/students in addressing LGBTQ topics/concerns. Like Jessica has mentioned in her comment, students register either as “male” or “female” and there is no option currently for anything otherwise.
With that said, I think the idea of using more inclusive is very important and educating others on the difference of sex vs. gender. I like your “Cool Trick” of phrasing of sample participants (i.e. “sample identified themselves as…..” As a teacher this is something that I can keep in mind when sending forms or newsletters home. Hopefully over time this inclusivity and knowledge will reach “higher up” with the Ministry and School Boards.
As something to remember and be mindful of, I completely agree with your statement: “It is our job to continually educate ourselves about the people and programs we evaluate, and any population is likely to include people who do not fall within the gender binary.”
I appreciate the “hot tip” to be aware when data systems only allow specific genders. But then what do people choose to do?
We work in schools where children are either “male” or “female,” which comes from parent report when they register their child in the District. Apparently, it’s not possible to have “other” or missing data in this field. I appreciate that it can be very important to allow students to self-identify. However, to match to the District, we often have to drop the write in data and it doesn’t get used (analyzed or reported) anyway. I often wonder if it is respectful or ethical to give people the chance to self-identify but then not be able to truly use that information anyway. Notably, this also comes up – and more often – when we ask students or other stakeholders to identify their race/ethnicity but the District provides a narrow list (such as no Middle Eastern or Southwest Asian option, for example).
I would be curious to hear how others have navigated this issue and what you all have learned from your experiences?
What do folks think about having three closed-ended options: male, female, and non-binary? Just saw California is using these for state ID: goo.gl/TxzpH4
Is “non-binary” too reductive? I think it could work well for large surveys or reporting situations where we have to reduce things to a finite number of categories. And it’s certainly better than forcing people into a binary category.
Great points. Before asking any question on a survey, it’s always good to consider whether or not you need the information for some useful purpose. If not – don’t ask it.