AEA365 | A Tip-a-Day by and for Evaluators

TAG | google

Building Evaluation Capacity. David and I have teamed up to apply empowerment evaluation concepts and principles to build evaluation capacity at Google and beyond. We are using rubrics to focus learning, and student ratings to identify areas that are strong or merit attention. We are using a 3-step approach to empowerment evaluation and an evaluation planning worksheet (building on my graduate school courses with Nick Smith) to help our colleagues assess their program’s performance.

The worksheet has 4 parts:

  • describe the program to be evaluated
  • define the evaluation context (purpose and audience)
  • plan the evaluation (questions, data sources, procedures)
  • create an evaluation management plan

With little or no evaluation background needed, teams dive into the worksheet to focus on their program’s purpose and goals before setting up metrics. Laying out the evaluation plan is often illuminating — leading to refined program logic, alternative (and more meaningful) programmatic plans, and more useful ideas about how to measure processes and outcomes.

Beyond Google. We are also sharing our work with nonprofits and higher education. Through the Computer Science Outreach Program Evaluation Network (CS OPEN) Google is supporting evaluation for 12 nonprofits through a partnership with the National Girls Collaborative Project.

David and I are also co-teaching at Pacifica Graduate Institute. David highlights the 3-step approach to empowerment evaluation, including: 1) mission; 2) taking stock; and 3) planning for the future. I follow-up with our worksheet to answer questions such as:

What is the overall program purpose?

Who are the audiences for the evaluation?

How will the result be utilized and by whom?

Rubrics and Technology for Peer Review and Self-assessment. Students in our course are developing evaluation proposals that can help them conduct evaluations, solicit funding, and/or guide their doctoral dissertations. The class meets face-to-face, but includes a virtual classroom strategy that has worked well in the past. Students use rubrics to guide their self- and peer-feedback to refine and improve their work and understanding. This improves the proposals, guides instruction, and models our focus on empowerment and capacity building.

Computer screen snapshot of proposal posted online (using Doctopus) with our rubrics (in Goobrics) above to rate or evaluate the proposal.

Computer screen snapshot of proposal posted online (using Doctopus) with our rubrics (in Goobrics) above to rate or evaluate the proposal.

Rad Resources: We are using our evaluation rubric with tools that require Chrome and free extensions:

This is a partial version of the rubrics used in the empowerment evaluation at Pacifica Graduate Institute

This is a partial version of the rubrics used in the empowerment evaluation at Pacifica Graduate Institute.

Doctopus: A tool for teachers to manage, organize, and assess student projects in Google Drive.

DF3

Goobrics: This rubrics based assessment tool works with Doctopus, allowing teachers to evaluate student’s work in Google Drive.

DF4

Goobrics for Students: Allows students to use a rubric to assess peers’ documents.

DF5

Google Forms: Enables students to self-assess their work and their peers’ work using an online survey.

DF6

DF7

Please contact us for additional information!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

Hi! I’m Laura Sefton, Project Analyst in the University of Massachusetts Medical School’s Center for Health Policy and Research. The school’s Lamar Soutter Library gives me access to a database of peer-reviewed journals and resources like Science Citation Index and PubMed to inform the literature searches and survey instrument research that I conduct for many evaluation projects. I have found Google Scholar to be an complementary source for access to materials that might not be available in the library’s databases. Below are a few tips and tricks for getting the most out of this resource.

Google Scholar is free to access and available anywhere you have an internet connection. Keywords, author names, and/or titles can all be used as search terms. Results include scholarly works as well as grey literature, including peer-reviewed and non-peer-reviewed journal articles, conference presentations, technical reports, and theses. Google searches across the web and pulls its results from publishers (free and subscription-based), professional associations, university repositories, and anywhere materials are available.

Hot Tip: Use the options listed in the left sidebar to narrow your search to specific time periods or to sort results by date. Results are, by default, shown in order of their relevance to your search terms, and the top articles may not be the most recent.

Hot Tip: Use the down arrow located to the right of the search box to refine your search. Options include searching by exact wording or without certain words.

Rad Resources: Below each search result are two features that can enhance your searches.

  • The Cited by function denotes how many times the article has been cited in a journal. Frequently cited papers will be closer to the top of your search results. Click on the hyperlink to see those articles, which may or may not be related to your topic of interest, depending on why they were cited.
  • To see other articles that are similar to the search result, click on Related articles.

Cool Trick: The Cite function, which also appears below each search result, shows the result in several citation formats, including MLA, APA, and Chicago, which can be copied and pasted directly into your report. You can also import the reference directly into a bibliography manager via the Cite function’s pop-up box. You’ll want to review each entry, however, since it may not import complete citation information.

Hot Tip: Set up email alerts to receive notifications when new articles related to your search terms are available. This can be particularly helpful if you are interested in the latest information.

Rad Resources: See previous posts by Molly Higgins and Len Levin, my colleagues in the Lamar Soutter Library, for additional literature resources.

Happy hunting!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Hi, I’m Paul Bakker. I run an independent evaluation consultancy, Social Impact Squared.  As an independent consultant, I don’t have a call centre to conduct population based surveys. However, a colleague (Brian Cugleman of Alterspark) introduced me to Google Consumer Surveys. I would like to share with you my research and trials into how Google Consumer Surveys could be used for evaluations.

Rad Resource: Google Consumer Surveys embeds your survey questions into a network of websites, and people answer your questions in exchange for access to the websites’ content.   You can target specific populations by:

  • The location of respondents’ IP addresses,
  • Demographics such as age or gender,
  • A custom screening question.

Google Consumer Surveys allows you to ask up to 10 questions to any one group of respondents for a relatively low cost – 10¢ per complete for 1 question surveys, and $1.10 to $3.50 per complete for 2 – 10 question surveys.

Lesson Learned: Google Consumer Surveys’ accuracy and reliability is comparable to other surveying methods. You can view their paper on how Google Consumer Surveys compare to other internet surveys here, and you can view how well their predictions on the 2012 U.S. Presidential race compare to other internet and phone surveys here.

Hot Tip: You can use Google Consumer Surveys to help answer evaluation questions around the relevance and effectiveness of population level programs and policies.

As a trial, I ran a couple of 1 question surveys:

  • The first found that only about 29% of Canadians remember reading a Material Data Safety Sheet outside of a training session. That metric could be used as a possible performance measure for Canada’s Workplace Hazardous Materials Information System.
  • The second found that 70% of recipients of Canada’s Universal Child Care Benefit (UCCB) felt that their child would continue to receive no or the same child care if they did not receive the benefit. While there may have been some issues with the survey’s response options, the results could inform an evaluation of the UCCB’s performance.

You can view the overall results and break them down by demographics by clicking on the links above.

survey-wxau5xt2anryq-question-1 survey-wz5nv34j6khxs-question-2

We’re celebrating 2-for-1 Week here at aea365. With tremendous interest in the blog lately, we’ve had many authors eager to share their evaluation wisdom, so for one special week, readers will be treated to two blog posts per day! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

I’m Ann Martin, an evaluator working with a team of science educators and outreach professionals in Hampton, VA. I frequently employ Google Formsfor evaluation and even other projects. Forms are free, simple, intuitive for users, and get the job done.

In the past, though, Google Forms had several notable limitations. If you found that Forms didn’t meet your needs in the past, you might not be aware of great new features that represent significant improvements. It’s worth taking another look at this free resource!

Hot Tip: Customize the visual look, feel, and branding of your survey! In September 2014 new Forms functionality allowed survey designers to add background and header images and to customize fonts and other display options. Before, theme options were limited. You can use this functionality to make your survey more readable and inviting. A custom header image with a logo may make your users feel more comfortable responding, or can make your survey a seamless part of a website in which you embed it. You can also embed images and videos within the body of the survey itself, which is handy for quizzes or assessments.

Figure 1 – Customization options range from a header image, page and form background, and fonts.

Figure 1 – Customization options range from a header image, page and form background, and fonts.

Cool Trick: Google Forms now support more complex survey design and administration options, including progress bars, data validation, logic/path branching, and randomizing the order of options in multiple choice questions. It’s also easier now to set up your survey’s questions. For instance, if you have a long list of options to include in a question, you can now copy-and-paste in a list from a word processor or spreadsheet table and automatically populate. (I wish that option had existed a few years back, when I created a drop-down with 200 alphabetized options!)

Cool Trick: New Add-ons enable even more behind-the-scenes functionality. The latest Add-ons includenifty widgets like Form Notifications, which will send automatic emails to your survey respondents, and Form Publisher, which will use survey responses to fill in a new document from a template.

Figure 2. Example Add-ons for Google Forms (screencapture from the Google Drive Add-ons Store).

Figure 2. Example Add-ons for Google Forms (screencapture from the Google Drive Add-ons Store).

Rad Resource: The Google Drive blog shares updates to Forms functionality so that you can always be aware of new features. I’m also more than happy to share tips if you contact me.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org .aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello, my name is Dan McDonnell and I am a Community Manager at the American Evaluation Association (AEA). In the fast-paced world of social media, things are always changing. Just as soon as you stop to take a breath, Facebook has tweaked its algorithm again. Or Twitter has updated its design. Recently, there have been a slew of changes that hit just about every major social network, and I thought I’d take the opportunity to give a few quick hits on what’s new, and how and how it affects evaluation professionals.

Hot Tip: Facebook Design Changes & More

Do you run a Facebook fan page? Facebook took a cue from Twitter who recently gave a facelift to the layout of fan pages. As of June 5th, the size of fan page  cover photo images has been adjusted to now require 851 x 315 pixels. In addition, Facebook removed what used to be called ‘page tabs’, replacing them with a simple menu of the major sections of your page – Timeline, About, Photos, Likes and More.

If the above changes weren’t enough, Facebook also gave users more freedom to customize the leftmost column of their fan page. Want your page ‘Reviews’ to be front and center? You can do that now! The entire left sidebar can be ordered entirely to your liking.

Hot Tip: Google + Authorship Limited

Remember that post I made a few months ago about Google Authorship? Well, it turns out, Google changed the game when it rolled out some major changes this week. While Google Authorship still exists, the biggest benefits have now been removed. Unfortunately, pictures are no longer supported in Google search results, nor will the author’s Google + circle information be shared. Now, if Authorship is correctly implemented into a blog post, only the name of the author will be added to the search result. Bummer.

Hot Tip: LinkedIn Premium Gets an Update

LinkedIn has become an essential tool for job seekers these days, as well as an excellent way to network. For the power users of the world, LinkedIn has a service called LinkedIn Premium, which I would highly recommend to anyone actively in job search mode. It’s a bit pricey though at $23.99 or $47.99 a month options, but with the addition of LinkedIn Premium Spotlight, a starter package that runs at just $7.99 a month, evaluation professionals can enjoy many enhanced benefits of LinkedIn without breaking the bank.

With Premium Spotlight, LinkedIn will make your profile stand out more among search listings, offer you suggestions for keywords to include in your profile to make yourself more visible for hiring managers and more. Check it out!

Hot Tip: Twitter Changes Fonts

Ok, so unless you’re a Helvetica purist, this one isn’t too big of a deal. Back on May 30th, Twitter angered (or delighted, depending on who you ask) font geeks around the world by changing the default typeface of Tweets from Helvetica Neue to Gotham. Some users have reported that the new font makes it more difficult to read, while others have embraced it fully. What’s your take?

Twitter Font Change

Twitter Font Change

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Dan McDonnell on Google + | Dan McDonnell on Twitter

 

· · · ·

David Fetterman“Ok, glass.” That’s how you activate Google Glass. I’m David Fetterman and that’s me to the right wearing Google Glass. I’m an empowerment evaluation synergist and consultant, busy father and spouse, and owner of Fetterman & Associates.

Rad Resource – Google Glass: Google Glass is a voice and gesture activated pair of glasses that lets you connect with the world through the internet. You can take a picture, record a video, send a message, listen to music, or make a telephone or video call – all hands free.

Hot Tips – Redefining Communications: Google Glass is not just another expensive (currently about $1500) gadget. It can free us up to do what we do best – think, communicate, facilitate, and, in our case, assess. Here is a brief example.

I said “Ok, Glass,” then “make a call to Kimberly James.” She is a Planning and Evaluation Research Officer I am working with at the W.K. Kellogg Foundation.

Kimberly asked how the evaluation capacity building webinar is coming along. Via Glass, I took a screenshot and mailed it to her so we can discuss it. When a colleague is mentioned, with a few swipes of my finger on the frame, I find a picture on the web, and miraculously remember who we are talking about.

Mid-conversation, Kimberly needed to step away briefly. While on hold, I sent a note to colleagues in Arkansas to ask them to check on the data collection for our tobacco prevention empowerment evaluation.

Kimberly returned to the call and we discussed a recent survey. With a simple request, the display of our results appeared, reminding me what the patterns look like.

Did I mention that I did all of these things while making lunch, picking up my son’s clothes off the floor, letting the dogs out, and emptying the dishwasher?

Later in the day, with a tap on the frame, I confirmed our scope of work with Linh Nguyen, the Vice President of Learning and Impact at the Foundation, while dropping my son off for piano lessons.

Later in the week I plan to use Google Hangout to videoconference with another colleague using Glass. When she connects during a project site visit, she will be able to take pictures and stream video of her walk around the facilities, bringing me closer to the “hum and buzz” of site activities.

Lessons Learned:

Respect people’s privacy – do not wear Google Glass where it is not wanted, will put people off, or will disrupt activities. Do not take pictures without permission. Remove it when you enter a bathroom.

Rad Resources

Hot Tip: Stay tuned for Part II tomorrow when I will cover using Google Glass as an evaluation tool.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Enhanced by Zemanta

Hi, I’m Audrey Rorrer and I’m an evaluator for the Center for Education Innovation in the College of Computing and Informatics at the University of North Carolina at Charlotte, where several projects I evaluate operate at multiple locations across the country.  Multisite evaluations are loaded with challenges, such as data collection integrity, evaluation training for local project leaders, and the cost of resources. My go-to resource has become Google because it’s cost-effective both in terms of efficiency and budget (it’s free). I’ve used it as a data collection tool and resource dissemination tool.

Lessons Learned:

Data Collection and Storage:

  • Google Forms works like a survey reporting tool with a spreadsheet of data behind it, for ease in collecting and analyzing project information.
  • Google Forms can be sent as an email so that the recipients can respond to items directly within the email.
  • Google documents, spreadsheets, and forms can be shared with any collaborators, whether or not they have a gmail account.
  • Google Drive is a convenient storage source in ‘the cloud.’

Resource Dissemination:

  • Google Sites provides easy to use website templates that enable fast website building for people without web development skills.
  • Google Groups is a way to create membership wikis, for project management and online collaboration.

Rad Resource: Go to www.google.com and search for products. Then scroll down the page to check out the business & office options, and the social options.

For a demonstration of how I’ve used google tools in multisite evaluation, join the AEA Coffee Break Webinar on February 27, “Doing it virtually: Online tools for managing multisite evaluation.” You can register here.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

· ·

My name is Dan McDonnell and I am the Community Manager for the American Evaluation Association.

In my last post, we touched on how evaluators can use Google + Authorship to add photos to their blog posts that will appear in Google search results. Besides making search results prettier, these photos can help you with grow your personal brand, as well as increase the likelihood that someone clicks through to read your post from Google. In this post, I will outline a few more tips and tricks for making Authorship work for you.

Hot Tip: Choose Your Photo Carefully

The photo that displays in search results comes directly from your Google + Profile Picture. As such, you’ll want to ensure that you choose a photo that shows your entire face – no avant-guard snapshots! In addition, Google will not display mascots or logos, so a real, human photo of yourself is the best way to go.

Hot Tip: Easy Authorship with Email Verification

The method I described in my last post demonstrated how to set up authorship on individual posts with a rel=author tag. If you have an email that is on the same web domain as your blog (for instance, if I had dmcdonnell@aea365.org) , it’s as simple as adding that email address to your contact information in your Google + Profile. Once you’ve done that, any post you make will simply need to link back to your Google + profile, no rel=author tag needed.

Hot Tip: Even Easier Authorship with WordPress

If your blog is hosted on WordPress, the process is even more automatic. Sign into WordPress.com with your Google + login information, and any WordPress post you make will automatically ‘author-ized.’ Hopefully, Google makes this a more integrated and automated procedure for other blogging platforms down the line.

Hot Tip: Guide to Troubleshooting Authorship

Having trouble getting Google + Authorship to work? Searchengineland.com has created a really nifty guide to troubleshooting issues – see below.

How To Troubleshoot Authorship Issues: Step by Step Flowchart

Do you use Google + frequently? If not, why not?

Dan McDonnell on Google +

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Dan McDonnell and I am the Community Manager for the American Evaluation Association.

Have you ever Googled something and noticed a photo that has made its way into the search page (see fig. a)? This is a recent Google + feature called Authorship – a neat tool that allows Google + users to have their profile photo appear alongside search results for their blog posts. Not only does this create a link between your blog content and your Google + profile, but it also can increase the likelihood that someone clicks the link to your blog post via search results (by as much as 150%, by some sources), no small feat!

Google + Authorship Example

Fig. A

Here’s how it’s done. Start by opening up a new Google search session. If you’re logged into your Google account (for Gmail and all other Google properties), you’ll see a button in the upper right hand corner that says + . Click this button to visit your Google + homepage.

From here, click the drop down menu in the upper left marked ‘Home,’ and select ‘Profile,’ then ‘About.’ As you scroll down, you should see all of your profile information: the area you want is a tab labeled ‘Links.’ Click the ‘Edit’ button at the bottom of that area. Here, you can add in any blog to which you are a current contributor. If you’re a contributor to AEA365, for instance, you’ll click ‘Add Custom Link,’ and type in AEA365 under’Label,’ then ‘www.aea365.org’ under URL.

Next, if possible, under Contact Information, add in an email address on the same domain as the blog to which you contribute . For instance, if your blog lives at www.greatwidgets.org, you’ll want to add in your dmcdonnell@greatwidgets.org email address. This automates the process. Once you’ve done this, click Save. Most of the heavy lifting is done!

The next time you create a post on a blog to which you’re listed as a contributor on your Google + page, include a link to your Google + profile in the byline, and add in “rel=author” at the end of the hyperlink. This tells Google to display your Google + profile picture alongside search results that show your blog post. See the example at the bottom of this post.

So there you have it! Give it a go, and try Googling the blog post in question to see your smiling face in the search results. Congratulations on your Google + Authorship!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Dan McDonnell on Google +

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Jan/12

14

Susan Kistler on Tracking aea365 Growth

I’m Susan Kistler, the American Evaluation Association’s Executive Director, and I post each Saturday to aea365. On January 1, 2010, we started aea365 with three subscribers – myself, John LaVelle our intrepid intern responsible for overseeing aea365 startup, and a relative who shall remain unnamed but wanted to support our efforts.

In the first few months, John worked diligently on two fronts. First, he encouraged authors to contribute to aea365, knowing that the readership was low and the forum untested. I strongly encourage you to check out the aea365 archive to read some of the earliest contributions (as well as over 700 others). Second, he worked with a number of people to reach out to AEA members and nonmembers alike to encourage people to subscribe – to receive a tip-a-day right in their email boxes. And subscribe you did. Approximately a year ago, in January of 2011, I reported that by December of 2010, 1,560 people viewed aea365 via email or RSS each day. The rise in subscriptions has continued steadily in our second year.

 

Lesson Learned: We use Feedburner to track subscribership. This May 2010 post will tell you more about it if you haven’t heard of Feedburner. The chart above shows our subscribership growth from aea365’s start in January of 2010 through to the end of 2011 when we recorded 2821 aea365 subscribers on an average day.

Hot Tip: Share you [blog] data. We’re sharing here so that association colleagues may have a case example of a blog’s growth trajectory, so that our potential authors will gain an understanding of the likely size of the readership, so that we are transparent regarding aea365’s readership, and finally as an entrée to saying ‘thank you.’ Thank you to every single subscriber, for taking the time to read and learn and share (lots of posts are passed along!). And a double thank you to all of our writers, over 500 to date, who have contributed their knowledge and expertise to aea365.

Get Involved: If you are an email-based subscriber, who joined on or before December 31 of 2011, you’ll be receiving a very short survey (I promise – 5 minutes max!), asking you about how often you read aea365 and how, if at all, you have used aea365 content. For one example, stay tuned for Monday’s post from Sheila Robinson Kohn and Kankana Mukhopadhyay talking about how they have used aea365 as a teaching tool.

And, if you have an example that you would like to share with the world, consider posting it in the comments or sending me a note at susan@eval.org regarding possibly submitting it as an aea365 contribution.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Older posts >>

Archives

To top