NA TIG Week: Measuring Necessity and Sufficiency by Ryan Watkins

I’m Ryan Watkins and I teach needs assessment at George Washington University and for The Evaluator’s Institute. Last year I emphasized the importance of applying the concepts of necessary and sufficient conditions when doing needs assessments. This year I want to extend that contribution to introduce aspects of measuring necessity and sufficiency, which (when data are available) allow us to quantify these relationships rather than leaving them as assumptions.

As a foundation, it’s important to note that statistical tools based on central tendency (correlation, regression, etc.) are not appropriate for measuring necessity or sufficiency. For measuring these we have to apply set theory, fuzzy set theory, Boolean algebra, and Bayesian probabilities as tools — which are less common in evaluations and evaluation trainings.  We won’t get into the details here, but Charles Ragin’s writing on QCA are a great starting place.

When you have data (for example, that includes those among your sample who received an intervention and those who did not (let’s call that condition X) and data on who achieved the desired outcome (let’s call that condition Y), you can then estimate the necessity of X for achieving Y within that sample. That’s to say, was it necessary for everyone in the group that achieved Y to have been in the group X that received the intervention?  The consistency of data with the notion that it was necessary then provides evidence for necessity. To keep this article brief, here’s a formula for calculating consistency with necessity, but you have to click the link below and read to get the context for what is X and Y, and to see an example.

consistency equation

https://www.socsci.uci.edu/~sgsa/docs/fsQCA_thomas_elliot.pdf

In set theory-based logic, necessity is largely the inverse of sufficiency. That is to ask, was it sufficient to be in the group X that received the intervention in order to get into the group that achieved Y? The coverage of data with the notion that it was sufficient then provides evidence for sufficiency. Here’s the formula, and again you can click the link below for more information.

coverage equation

https://www.socsci.uci.edu/~sgsa/docs/fsQCA_thomas_elliot.pdf

Calculating necessity and sufficiency in this manner can quite helpful when attempting to distinguish between needs and wants. For instance, in the past has it been necessary and/or sufficient to achieve a desired increased level of public transportation in order to achieve the desired traffic congestion relief.  Next is to ask, what is the probability that not achieving public transportation goals would have led to not relieving congestion (the counterfactual). Judeo Pearl (1999) provides the probability formulas for making such calculations:

probability calculation

https://ftp.cs.ucla.edu/pub/stat_ser/r260-reprint.pdf

There are, of course, some assumptions that must be met to use these formulas, and I strongly suggest reading the original articles/books (cited above) as background. Here I’m introducing these as tools available to needs assessors (and evaluators) as we work to measure results and guide decisions about what should happen next.

Rad Resources:  

The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

2 thoughts on “NA TIG Week: Measuring Necessity and Sufficiency by Ryan Watkins”

  1. Hi Ryan. Another good thing about a set theory/QCA tye perspective is its emphasis on linking up cross-case and within-case analysis. Within-case analysis should inform how cross-case analysis is done and vice versa.

    EvalC3 can help in this area, as well as the necessity and sufficiency analysis. While the EvalC3 workflow starts with cross-case analysis it ends with ways of strategically selecting cases for within-case analysis.

    (BTW: I am happy to provide skype video support for anyone wanting to use EvalC3)

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.