Hi! I’m Jonathan Scaccia, principal of the evaluation firm The Dawn Chorus Group and founder of PubTrawlr. What connects these two strands of work is finding better ways to efficiently and effectively extract meaning from text-based data. There is so much qualitative data out there that is laborious to analyze. Techniques from Natural Language Processing can help us to accelerate and scale the analyses.
Sentiment Analysis. Words don’t just communicate ideas. They can express emotion. And, emotion is a critical aspect of how people relate to one another. Sentiment Analysis is an NLP technique that looks at the emotional content behind the words.
Sentiment analysis is well-established in customer service. It can be an essential technique to learn how customers perceive the value of a product. Although many research studies have explored the potential of these methods in behavioral health, they haven’t yet gotten a significant foothold in social-services evaluation.
For example, I looked at the overall sentiment communicated on Twitter by the two U.S. congressional representatives who represent my community: Rep. Dan Meuser and Rep. Chrissy Houlahan. Both were recently re-elected to their second term, so their official accounts are nearly the same age.
After pulling all the tweets using Twitter’s API, I used a pre-coded group of words called a lexicon to identify words as either positive or negative. For this, I used the bing lexicon. I then averaged the overall sentiment for each month and plotted these since the beginning of 2020, when they took office.
We can see that both generally use positive messages, though Rep. Houlahan communicates far more of them. Interestingly, we don’t see as much of a dip during the early days of the pandemic (April-ish 2020), which shows up in other accounts. Rep. Meuser has also been trending down since his party lost power in the November 2020 election.
There are other methods out there that can get at more nuanced emotions. For example, the NRC lexicon looks for words that signify emotions like Trust or Anticipation.
Text Mining in R. This open-source text by Julia Silge and David Robinson is a great entry-level way to explore implementing these methods in R.
It’s not that difficult to learn and can yield super interesting insights. Have fun with it!
The American Evaluation Association is celebrating Behavioral Health (BH) TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to aea365 come from our BH TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.