Johanna Morariu on Proofiness: The Dark Arts of Mathematical Deception and the Evaluation Profession

I’m Johanna Morariu, co-director at Innovation Network, an evaluation consulting firm that works with nonprofit organizations and foundations.

Rad Resource: Every now and then I come across an article or book that makes me fundamentally rethink what I thought I knew. One that I want to share with you all is the book Proofiness: The Dark Arts of Mathematical Deception by Charles Seife.

While the entire book is a great read—especially for evaluators—the first chapter, Phony Facts, Phony Figures, is especially apt. Through many examples and discussions the author makes the point that when numbers are attached to arguments or “facts,” we as humans are more likely to believe the statement. Seife opens the chapter by asserting, “If you want to get people to believe something really, really stupid, just stick a number on it.” Essentially, humans are wired to believe in numbers—numbers don’t lie.

I couldn’t help but think about how this applied to evaluation consulting: So often the individuals and organizations I work with naturally prioritize the supposedly objective, quantitative data over the seemingly fluffy, qualitative data. This is especially evident in two phases: evaluation design and when presented with findings. Before reading Proofiness, I chalked up this point of view to lack of evaluation experience. But since reading Proofiness (and years spent puzzling over the preference for quantitative data over qualitative), I agree with Seife; We as humans have an innate belief in the infallibility of numbers.

Lesson Learned: Now, when I work with an individual or organization to design an evaluation approach or present qualitative findings, I realize the increased need for me to explain the merit and appropriateness of qualitative designs. Sometimes it is not enough to lay out the pros and cons of a variety of quantitative and qualitative approaches—sometimes the rigor of qualitative approaches needs to be specially called out and affirmed.

Have you read Proofiness? If so, how did the book make you think differently about evaluation?

And if you haven’t read Proofiness, I heartily recommend it!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea3365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators