AEA365 | A Tip-a-Day by and for Evaluators

CAT | Distance Education and Other Educational Technologies

My name is Vanora Mitchell and I am a professional independent evaluator working in Washington, DC.

How do we evaluate online learning? About a year ago, a school that I volunteer with asked me to help them to evaluate their new online learning initiatives. They were in the process of developing online courses and wanted to know how to identify success and if the evaluation process was different than for the traditional classroom. Here are three resources I found particularly useful as I did my background research:

Rad Resource: Evaluating Online Learning: Challenges and Strategies for Success: Written in 2008 by WestEd for the U.S. Department of Education, this 80 page report was my go-to guide. It had concrete examples, came from a reliable source, and was research-based.

Rad Resource: E-Learning Concepts and Techniques: This online collaborative ebook was developed in 2006 by a class at Bloomsburg University of Pennsylvania’s Department of Instructional Technology. The entire book is useful and Chapter 9 is devoted to E-Learning Evaluation.

Rad Resource: eLearn Magazine: Focusing mostly on the online classroom context, this magazine is web-based and free and full of articles that helped me to understand more about electronic learning. I read a number of background articles, but here are a few that were directly evaluation related:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

My name is Andrea Velasquez, and I am a doctoral student at Brigham Young University. For the last four years, I have been an instructor of an undergraduate class that teaches pre-service teachers how to use technology effectively in elementary and secondary education settings. One of the principal frameworks that we use to teach pre-service teachers how to distinguish between all the facets of designing effective instruction with technology is TPACK, or Technological Pedagogical Content Knowledge (Mishra & Koehler, 2006). This framework states that in any effective technology mediated instruction, technology, pedagogy and content are three components that not only co-exist, but also interact and have an impact on each other. The research examining TPACK can be useful to the field of evaluation by applying it to evaluations of technology-mediated instruction. Distinguishing between these three components- technology, instructional content, and instructional strategies- can help evaluators identify appropriate questions and alleviate the complexity of evaluating e-learning.

Hot Tip: When designing an evaluation of technology-mediated instruction, after determining context and stakeholders, consider technology, pedagogy and content as evaluands. Then, identify criteria and questions for judging each evaluand. Before continuing the evaluation, also identify criteria and questions that take into account how each component impacts the others. These questions should address the compatibility between these components. For example, if an online high school uses video technologies to communicate with students, an evaluation of such a program should take into account the video technologies, the strategies the teacher uses to teach the class (i.e. group work, field experiences, presentations), and the content of the instruction he is teaching. Besides addressing each of these three components, the evaluation should address the relationships that exist between each of these components at each stage of the evaluation process. This approach to evaluation ensures a more holistic evaluation of the technology use in relation to the context and the needs of the students and stakeholders.

Rad Resource: This site is a resource that is maintained by the developers of the TPACK framework. It has updated research articles and many other resources for understanding the practical applications of the framework

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Want to learn more from Andrea? She’ll be presenting as part of the Evaluation 2010 Conference Program, November 10-13 in San Antonio.

· ·

<< Latest posts


To top