AEA365 | A Tip-a-Day by and for Evaluators

TAG | wordle

Hello! I’m Sheila B. Robinson, aea365’s Lead Curator and sometimes Saturday contributor. It’s Independence Day in the US, the anniversary of the day our Declaration of Independence from Great Britain was signed in 1776. As I thought about what topic to tackle for this week’s post, I wondered what insights a word cloud created from the text of the Declaration of Independence might yield. Since I’m familiar with two word cloud generators, I tried each of them.

Here is what I created in Worldle:

Dec of Ind wordle

And here is what I created in Tagxedo:

Dec of Ind Tagxedo

Both generators allow for some creativity with regard to color, and font. Wordle allows you to choose directionality – whether you want all text horizontal, all vertical, or some mix of the two. Tagxedo is best known for its shapes, allowing you to choose from a bank of available shapes that can help you illustrate a point. Tagxedo can also use custom fonts from popular font sites.

Lesson Learned: Word clouds can be fun and have a number of appropriate applications. Use them with caution, however, as they never substitute for analysis, and can potentially detract from important themes. Stuart Henderson and Eden Segal tackle word clouds in a chapter on qualitative data visualization in a recent issue of New Directions for Evaluation.  “Word clouds create a dramatic visual, which likely accounts for their popularity,” the authors posit. “Despite concerns with word clouds, their ease of creation and striking visuals make them a useful tool for evaluators if they are used sparingly and their challenges are acknowledged” (p. 57-58). 

Hot Tip: Henderson and Segal also mention another word cloud generator, TagCrowd, and I decided to give that one a try as well. Here is the result:

Dec ind TagCrowd

While TagCrowd has fewer opportunities for creativity, it does give the user the option to see word counts after each word. I simply checked this option as I created this one.

Rad Resources: A few other aea365 authors – Susan Kistler, Stacy Carruth, Sue Griffey, Jaquelyn Christensen, and Sarajoy Pond – have also shared their perspectives on word cloud generators.

Did you know there is an advanced version of wordle? Learn about this and other Wordle tips, such as how to keep word phrases intact, and how to get numbers to show in Wordle here!

Read the entire article Visualizing qualitative data in evaluation research, by Stuart Henderson and Eden Segal, to understand more about qualitative data visualization.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

We’re Tara Gregory, Director of Research and Evaluation, and Bailey Blair, Youth Leadership in Kansas Program Associate, at Wichita State University’s Center for Community Support and Research (CCSR).  CCSR was awarded a grant last year by the Kansas Department for Aging and Disability Services to provide technical assistance and research services to leadership groups for youth with a mental illness diagnosis and their parents.  Our first task was to better understand these groups, so we conducted focus groups with each site, asking questions about the nature of the groups and roles of the youth. Their qualitative responses and our observations indicated that while they highly value their groups, the adults tend to be in charge and youth perform tasks like choosing food, picking up trash, etc.

So our question was: How could we honor what the members love about their groups but also move them toward best practices for positive youth development/leadership (e.g., Eccles and Gootman, 2002)?  Our approach was to gently present “what is” in their own words alongside “what could be” as a way to respect the members’ voices but also offer ideas for enhanced experiences.

Hot Tip:

  • Instead of just giving the leadership groups a written report, we displayed a Wordle™ graphic, which contained all of their qualitative responses, during a gathering of all groups. This was an engaging method that showed them the results “in their own words” without inserting our thoughts.
  • We then displayed a model of meaningful youth participation and asked them to compare it to the  Wordle™ visuals.  This spurred a very energetic and insightful discussion among youth and adults.
  • Next, we gave them an opportunity to incorporate their ideas into a visual representation of their vision or aspirations for the groups as a whole.
  • Finally, we’re following up with technical assistance and written guidelines on options to further incorporate true youth leadership in their groups.

Lessons Learned:

  • By presenting the qualitative data juxtaposed with the ideal model, the groups had an “A-ha!” moment that was totally theirs. They were not left feeling like they had done something wrong and were even able to laugh at the discrepancies. This appeared to be a moment of genuine empowerment.
  • Highlighting the discrepancy between “what is” and “what could be” wasn’t enough. It was essential to make sure they had concrete ideas about how to move toward their self-determined vision.

Figure 1.  Wordle graphic for responses to: “What are youth in charge of in this group”

Gregory Blair 1

 

Figure 2. Ladder of youth voice

Gregory Blair 2

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org

· ·

My name is Stacy Carruth and I am a Community Health Specialist at a Regional Center for Healthy Communities in Massachusetts.  Much of the work we do is supporting community coalitions working on substance abuse prevention.  Two communities we work with are funded to reduce fatal and non-fatal opiate overdose, an issue of concern in the Northeast.  In one of these communities, we used Wordle to look at how people in the community were talking about overdose. I will be describing how we did this.

In their work to reduce opiate overdoses, the coalition conducted a comprehensive community assessment, which included many stakeholder interviews.  The stakeholders were asked:  What does overdose look like in your community? What’s being done and what’s working? What would help or harm the work (overdose prevention)? The coalition was interested in different ways to present this information to a larger, more diverse audience in a visually engaging way.

Rad Resource: I had recently learned about Wordle at the AEA/CDC Summer Evaluation Institute, and so I created a Wordle document with stakeholder interview transcripts as an example for the coalition staff.  Wordle allows you to create word clouds out of text. The more frequently a word is used in the text, the more prominent it is in the word cloud.  This creates a visual representation of the information that is easy to share with others.

Some of the words that were prominent in the Wordle document that we created were:  Need, Women, Public, Person, Detox, Overdose, Drugs.  It was a powerful way to visualize the thoughts of community members.

While using Wordle, I wondered whether it could make data more easily accessible for those with low literacy levels.  Wordle has the potential to engage members of the community that might otherwise not be engaged.

Hot Tip:  Wordle is very user friendly.  You literally copy and paste your text into a box.  You can change the orientation of the text, and the color scheme quickly and easily.  If there is a term that appears in the Wordle document that you want to delete (for example, in our document, the word “mentioned” was prominent, but it did not add to understanding the issue) you simply put your cursor on the word and right click and then you can delete the term. Wordle is an accessible way to share data and engage the community in your work.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Good Morning! Susan Kistler here, AEA’s Executive Director. We’ve had a number of posts talking about creating and using word clouds, primarily in terms of creating them through wordle. Word clouds take a set of text and create a cloud of the most frequent words in that text with word size within the cloud based on frequency.

Here’s an example that I made today using the AEA square logo and the text from the AEA Guiding Principles for Evaluators.

Guiding Principles as Word Cloud

AEA Guiding Principles for Evaluators as a Word Cloud

Rad Resource – Try Tagxedo for Customizable word clouds: I’ve turned to wordle time and again to make a word cloud, but I recently was introduced to Tagxedo and it solves a lot of the shortcomings in wordle and extends the functionality. Tagxedo is free (although a pro account is being considered on the horizon). Here are six key features of Tagxedo that make it my new go-to application for clouding:

  • You can upload your own image and wrap text around the image (or the inverse of the image).
  • You can save to multiple formats including high-resolution formats appropriate for printing as well as the standard lower resolution options good for the web.
  • You can modify the algorithm used to decide the size of words so that one word does not dominate the others in the extreme.
  • You can create your own custom color palette (and also upload your own font if you want to go all out).
  • Tagxedo incorporates word stemming (which may be turned off or on) so that similar words (such as “evaluator” and “evaluators”) are treated as one.
  • From here, you’ll just need to go play – the customizability is extensive.

Hot Tip – Uses for Word Clouds: While word clouds have limited application for data analysis, they do have many other uses, including:

  • Discussion starters among stakeholders
  • PowerPoint slides
  • Presentation and report covers & illustrations
  • Take a look at the Tagxedo slideshow “101 Ways to Use Tagxedo” – it is a bit slow to load, but loaded with great ideas

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Mar/10

16

Sue Griffey on Wordle

I’m Sue Griffey and I direct the Evaluation Center in the Public Health Research group at Social & Scientific Systems.

I really like technology and the tools it’s given us – and I am always trying out new things. (I love the EZAnalyze program that Paul Pope blogged about a few days ago.)  So, when I read about Wordle (http://www.wordle.net/ ) a couple months ago, I just had to try it – but on what? I decided to put the text of my biopara into it – and see what would result.

It was very instructive and helpful, as you can see from the following picture. (In the interest of full disclosure, I did edit out common words, as Wordle does allow you to remove words.)
Of course, I was glad to see that, as an evaluator, the word “evaluation” came out quite prominently. Since I came to my evaluation career through a variety of experiences and pathways, it was useful to see how those were reflected as well.

It was instructive and helpful for these reasons:

  • It helped me see where some themes in my career and life came out as quite large visually. I reexamined how I wrote about those so that they didn’t override other information.
  • I applied different templates in Wordle which also helped me see different visual themes and identify what I wanted to emphasize.
  • I had fun doing it. (Don’t we all like to focus on our favorite subject?!)
  • At the same time, it gave me ideas of how I could apply it in my work to see if I got the result I wanted:
    • Apply it to an executive summary of a business proposal. Does the major theme you’re emphasizing come through visually?
    • Apply it to a conclusion section of a report or an article. Does the Wordle of the conclusion show proportionally what you think you said?

What ideas do you have for how to use it?

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

·

My name is Jacquelyn Christensen and I am an Evaluation Associate at a non-profit mental health agency in Los Angeles, as well as an advanced graduate student in Applied Developmental Psychology at Claremont Graduate University. I work closely with my organization to report program outcomes and monitor service quality. When working directly with staff or clients, I must find creative ways to involve them in the evaluation process and promote accuracy during data collection.

Rad Resource: When presenting information to staff, I often try to make seemingly uninteresting information visually stimulating. For a recent presentation of data from staff feedback regarding new software, I compiled all of the qualitative comments regarding their positive and negative experiences with the software and created a “word cloud” using Wordle. This “picture” acted as the cover page and introduction to the presentation. I received overwhelmingly positive feedback from staff who felt it not only piqued their interest, but accurately represented their thoughts and ideas. www.wordle.net

Rad Resource: Surveys are an inevitable part of my data collection, and, despite clear wording and simple organization, I have found people often read quickly, resulting in a discrepancy between their item responses and qualitative feedback. For Likert-type scale responses on our client (and some staff) surveys, I have placed a smiling face above “Strongly Agree” and a frowning face above “Strongly Disagree” as a visual reminder of the direction of the scale. Since doing this, I have noticed a great improvement in the congruence of the quantitative and qualitative responses.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

My name is SaraJoy Pond, and I am a doctoral candidate at Brigham Young University.  I’m particularly interested in evaluation capacity-building for social change.

Cool Trick: Chances are, if you’ve worked at all with qualitative data sets, you know the feeling of “drowning in data.” The end of an evaluation engagement often leaves me swimming in reams of interview transcripts, gigabytes of video, and hours of audio (somehow always virtually inaudible) wondering “where do I even start!?”  Wordle is a free web-based word cloud generator, has come to my rescue many times. Simply paste in your transcripts or notes and in seconds, you’re looking at an intuitive visualization of “themes” from the text.

Though the tool itself relies on “quantification” (as compared to true qualitative analysis, my professors inform me) I find it invaluable for wrapping my head around that first stage of qualitative data analysis.

TIPS 2 TRY:

  • Sort the responses to a question by stakeholder group, then compare the resulting wordle images to see differences in how each group responded.
  • Make periodic wordles of your own field notes to see how your impressions, interpretations and judgments varied or evolved over the course of a study.
  • Include a wordle image in your evaluation report to help stakeholders see how the themes you discuss “emerged.”

Wordle is available at http://www.wordle.net/

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.


· ·

Archives

To top