NA TIG Week: Measuring Necessity and Sufficiency by Ryan Watkins

I’m Ryan Watkins and I teach needs assessment at George Washington University and for The Evaluator’s Institute. Last year I emphasized the importance of applying the concepts of necessary and sufficient conditions when doing needs assessments. This year I want to extend that contribution to introduce aspects of measuring necessity and sufficiency, which (when data are available) allow us to quantify these relationships rather than leaving them as assumptions.

As a foundation, it’s important to note that statistical tools based on central tendency (correlation, regression, etc.) are not appropriate for measuring necessity or sufficiency. For measuring these we have to apply set theory, fuzzy set theory, Boolean algebra, and Bayesian probabilities as tools — which are less common in evaluations and evaluation trainings.  We won’t get into the details here, but Charles Ragin’s writing on QCA are a great starting place.

When you have data (for example, that includes those among your sample who received an intervention and those who did not (let’s call that condition X) and data on who achieved the desired outcome (let’s call that condition Y), you can then estimate the necessity of X for achieving Y within that sample. That’s to say, was it necessary for everyone in the group that achieved Y to have been in the group X that received the intervention?  The consistency of data with the notion that it was necessary then provides evidence for necessity. To keep this article brief, here’s a formula for calculating consistency with necessity, but you have to click the link below and read to get the context for what is X and Y, and to see an example.

https://www.socsci.uci.edu/~sgsa/docs/fsQCA_thomas_elliot.pdf

In set theory-based logic, necessity is largely the inverse of sufficiency. That is to ask, was it sufficient to be in the group X that received the intervention in order to get into the group that achieved Y? The coverage of data with the notion that it was sufficient then provides evidence for sufficiency. Here’s the formula, and again you can click the link below for more information.

https://www.socsci.uci.edu/~sgsa/docs/fsQCA_thomas_elliot.pdf

Calculating necessity and sufficiency in this manner can quite helpful when attempting to distinguish between needs and wants. For instance, in the past has it been necessary and/or sufficient to achieve a desired increased level of public transportation in order to achieve the desired traffic congestion relief.  Next is to ask, what is the probability that not achieving public transportation goals would have led to not relieving congestion (the counterfactual). Judeo Pearl (1999) provides the probability formulas for making such calculations:

https://ftp.cs.ucla.edu/pub/stat_ser/r260-reprint.pdf

There are, of course, some assumptions that must be met to use these formulas, and I strongly suggest reading the original articles/books (cited above) as background. Here I’m introducing these as tools available to needs assessors (and evaluators) as we work to measure results and guide decisions about what should happen next.

Rad Resources:  

• Necessary Conditions Analysis website and software.
• Software for identifying necessary and or sufficient relationships, with free Excel app.

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.