Hi! I’m Dana Wanzer, doctoral candidate at Claremont Graduate University and an avid #EvalTwitter user!
Many people new to evaluation—students and clients alike—struggle with understanding what evaluation is, and many evaluators struggle with how to communicate evaluation to others. This issue is particularly difficult when evaluation is so similar to related fields like auditing and research.
There are many great resources on what evaluation is and how it differs from research, including John LaVelle’s AEA365 blog post, Sandra Mathison’s book chapter, and Patricia Rogers’ Better Evaluation blog post. I wanted to examine these findings more in depth, so I conducted a study with AEA and AERA members to see how evaluators and researchers defined program evaluation and differentiated evaluation from research.
In this study, I recruited members of AEA (who were primarily members of the PreK-12 Educational Evaluation and Youth-Focused Evaluation TIGs) and members of Division H (Research, Evaluation, and Assessment in Schools) of the American Educational Research Association (AERA). A total of 522 participants completed the survey which, among some other questions, had them define evaluation, choose which of the below diagrams matches their definition of evaluation, and rate how much evaluation and research differs across a variety of study areas (e.g., purpose, audience, design, methods, drawing conclusions, reporting results).
Lesson Learned: Evaluators and researchers alike mostly define evaluation like Scriven’s definition of determining the merit, significance, or worth of something—essentially, coming to a value judgment. However, many evaluators also think evaluation is about learning, informing decision making, and improving programming, indicating the purpose of evaluation beyond simply the process.
Lesson Learned: Mathison described five ways in which evaluation and research could be related:
Half of participants thought research and evaluation overlap like a Venn diagram, which is similar to the hourglass model from LaVelle’s blog post, and a third thought evaluation is a sub-component of research. However, evaluators were more likely to think research and evaluation intersect whereas researchers were more likely to think evaluation is a sub-component of research. Evaluators are seeing greater distinction between evaluation and research than researchers are!
Lesson Learned: Participants agreed that evaluation most differs from research by the purpose, audience, providing recommendations, disseminating results, and generalizing results and are most similar in study designs, methods, and analyses. However, more evaluators thought evaluation and research differed greatly across a multitude of study-related factors like these compared to researchers.
Hot Tip: If your students or clients already know about research, describing how evaluation is similar to and different from research might be a great approach for teaching what evaluation is!
I believe this study will be useful in helping propel the field of evaluation forward, at least by helping our field better describe evaluation but potentially in situating our field as a distinct discipline and profession.
Rad Resource: Hungry for more information? All the study materials, data, and manuscript are posted on the Open Science Framework, a free and open-source website that allows scientists to collaborate, document, and share research projects, materials, and data.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi there,
I am currently in a Learning Design and Technologies Master’s program at ASU and just started the course Introduction to Research and Evaluation in Education. This week’s discussion is about the difference between research and evaluation and your post did a great job of explaining the difference between the two. It seems that my understanding matches what you indicated that the main difference between the two is the purpose and what is going to be done with the information collected.
This week we were also introduced to the concept of paradigms and how they may influence the research and evaluation process. We were asked to determine which paradigm seems most useful, but I’m curious – based on your experience in this field, is there one paradigm that is more frequently used than others? Is there a process for selecting a paradigm or does it depend on where you are working or who you are doing research/evaluation for?
Thank you for your post and for any insight you can provide into this specific question.
Hello Dr. Wanzer.
Thank you for this information post on research vs. evaluation. I am a graduate student at Arizona State University and have been tasked with determining the differences between research and evaluation. I understand that these two words have been used interchangeably. They essentially seem to be the same idea, but there are many differences between the two. I found your image of the 5 ways research and evaluation could be related very interesting to dissect. The green circle seems interesting to me. Can these two ideas be the same when ideology is so different between the two? The purpose between the two are so different, I can’t see how they would be considered the same. Have you thought about writing a paper on the differences between research and evaluations when it pertains to literacy education?
Thank you for sharing this piece.
Hello! I am a graduate student at ASU studying the difference between evaluation and research and I found your graphic to be the most useful on explaining how these two topics are interrelated. The last green circle claims they are the same thing and that makes me question my own understanding of them both. I thought the main difference between research and evaluation were their goals but the more I try to elaborate the harder it becomes differentiate them. My favorite point you wrote was how some educators saw the purpose of evaluation as learning – so true and so powerful. This article has made me think!
Thank you,
Monique
Hello Dr. Wanzer,
I am so glad to have found your post. I appreciate how succinct this information was. In my current graduate course, we are taking some time to look at not only the differences, but also similarities, between research and evaluation. Your post gave me a new perspective from the evaluator itself which was new and exciting. I am fairly new to this subject and seeing different perspectives is truly helping to discover my own beliefs and opinions within the field. As a classroom teacher, I want to encourage my students to take risks, problem solve, and “research”/ discover, rather than be blatantly told information. However, research looks very different in first grade. I am wondering if you have any suggestions on how I could incorporate research and/or evaluation in first grade? Maybe in relation to reading/literacy?
I appreciate your work!
Dannica Lyon
Hello Dr. Wanzer,
First, please allow me to introduce myself. I am Hind a graduate student at Arizona State University, earning my master degree in Teaching English for Multilingual Students. I am currently enrolled in COE 501: Introduction to Research and Evaluation, one of my task is to seek for assistance for proper understanding of the difference between research and evaluation. I saw Patricia Rogers blog post then I headed to yours and I have a better understanding for Research vs. Evaluation concept. Your post is really beneficial and inclusive, It was a lot helpful for me me to differentiate between them, thanks for that!
Relying on Mertens (2022), Patricia Rogers’ blog post and your blog post, I can understand that research is a general theoretical knowledge controlled by researcher, but evaluation is a more specific applied knowledge controlled by funders.
If I want to add an example to see if I am understanding correctly, when my students are asking about something frequently I do a research to answer their questions and inquiries. When the technical advisor is questioning a lesson plan or any other education product I made we do an evaluation so they approve its publication.
I appreciate if you approve or correct my example,
Thanks again!
Hind
Hello Dr. Wanzer,
Thank you for your post! As a graduate student and a curriculum designer, I am interested in research and evaluation fields and how I can use them in my career to make students` learning experience even better.
Checking Mathison`s Venn diagram, I can tell that in my previous experience with evaluation and research, I always consider them to be the same thing. After learning more about these two fields, I can clearly see that evaluation and research differ in their purpose. But you also mentioned that they differ by the audience. I can see that research is more focused on the researcher and the evaluation is stakeholder-focused. But are there any other differences in evaluation and research audience?
Thank you!
Hello Dr. Wazner,
I am a graduate student with ASU. I appreciated your post, as it clearly shows your passion and dedication to explain the distinction between research and evaluation. As you mentioned, many evaluators think there are many more facets to the evaluation process than just the process for determining a value judgement. It is exciting to know the purpose of evaluation is to make informed decisions and learn from the process in order to make improvements. Mathison’s Venn Diagram models certainly helped me to visualize the distinctions of research and evaluation. As an educator, many of our science units are designed to teach students how to conduct research. I know research and evaluation is a growing field as we start to implement many new systems in our growing society. I understand there is a growing need to develop data scientists for our future generations. I wonder if you have any recommendations or resources to equip upper elementary or high school students with the knowledge of the similarities and differences between research and evaluation?
Best regards,
Alyssa Gonzales
Dr. Wanzer,
Hello, this is Ho Jung Yu from South Korea, working at a university as a faculty member. I am currently taking an online graduate program at ASU. I have read chapters explaining and discussing research and evaluation. I have not thought about evaluation critically until this point. However, it is very interesting how people view them differently. The Venn diagrams shown in your post are very practical to understand research and evaluation. In fact, research and evaluation can be more varied. The perceptions of them are different, depending on who defines them, a researcher or an evaluator. I think the fundamental intention of research lies in finding new knowledge; instead, that of evaluation lies in promoting change. However, I have to admit that they share common intentions and processes. Thank you for sharing your thoughts via authentic responses. – Ho Jung
Hi Dana!
It is interesting to read how differently evaluators and researchers perceive the terms “Research” and “Evaluation.” Currently, I understand that research is more rigorous than evaluation in practicing methodologies. Also, their purposes are different. Evaluation is for decision-making or directions, but research is conducted to produce new knowledge. I think my understanding will be more sophisticated after reading the thoughts that you have posted. Thank you for your thought-provoking post with authentic data.
Hello Dr. Wanzer,
I am currently a graduate student attending Arizona State University taking a course on research and evaluation in education. In my undergraduate years, I used the words “research” and “evaluation” interchangeably, thinking that they were the same thing. However, after reading your post, I gained a better understanding of the difference between them. Now that I understand, I can think about past experiences I’ve had when conducting evaluations. I agree with you that evaluation is done to determine the merit, significance, or worth of something. I also like that you included how evaluators look at evaluation as being about learning, informing decision making, and improving programming, indicating the purpose of evaluation beyond simply the process. It is interesting to me that evaluators identify a greater distinction between evaluation and research than researchers do.
Hello Dr,
Thank you for sharing us your thoughts. I hoped to know more about how the participants see large differences between the research and the evaluation, from which sides
As I had read, Mertenz (2020) differentiates between them saying that evaluation:
• brings science and policy together.
• is based on authoritative evidence “that enables the timely incorporation of its findings … into decision-making processes” (p.48).
• obliges the evaluators to respond to the motivations of the beneficiaries (stakeholders).
• its data arises from a comprehensive list provided from the stakeholders.
• is used to reduce uncertainty about decisions about the program to be implemented due to the presence of many influencing factors.
• requires managerial, group and political skills, and we may find a specialization for it in itself.
Did the participants in your research added more points?
Best regards,
Hind
Reference:
Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.). SAGE Publications. ISBN: 9781544333762
Dr. Wanzer,
I am a student of Education- Literacy Education at Arizona State University studying the difference between research and evaluation. I appreciate your post as it gave me a clear insight into the differences as well as the similarities between them. I find it interesting that research can be considered a subset of evaluation, but that evaluation is not considered to be a part of research. I always assumed that research was a means of finding evidence so that it could benefit others in whichever field that research was in. However, I now see that evaluation is used to determine the effectiveness and importance of something whereas research is mostly scholarly and for the purpose of gaining new knowledge in an area of study. I appreciate the different ways that research and evaluation could also be viewed as interrelated not just as dichotomies. Thank you for writing an informative blog post that helped me understand the dense information from the textbook I am reading. – Cecilia
Hello Dr. Wanzer,
I am a graduate student attending Arizona State University currently taking a course on research and evaluation in education. Initially, I had thought research and evaluation are the same thing. Your post helped clarified the distinction between research and evaluation. Based on my experience with evaluations of learning systems in a previous course, I agree with the many evaluators who think evaluation is a process of determining the merit, worth, or value of an evaluand with the purpose of making judgements, improving effectiveness, increasing understanding, and/or informing decision making. It’s interesting that evaluators are able to identify a greater distinction between evaluation and research than researchers can; I wonder why. Thank you for your work!
Hello Dana,
I enjoyed reading your post; it was short, simple, and got straight to the point. I personally work in the corporate environment doing learning and development. I would be interested to see research into how a persons personal experience and profession could define their understand and definition of evaluation vs. research. Being in the corporate business world (and seeing this from a personal standpoint) evaluation is commonly used to look at internal programs and learning with a political undertone that is always present. However, research and evaluation might mean something different within higher educational system or, say, the medical field. Each industry will be looking to utilize evaluation and research for various reasons with different objectives and methodologies. It is understandable, then, that clearly defining the differences between research and evaluation will forever be an ongoing debate and each will see the role of research vs evaluation differently.
Thank you!
I found your blog post on distinguishing between evaluation and research interesting. I have felt that research and evaluation can be so similar yet completely different. I found the LaVelle image you reference to be similar to the image of comparing evaluation and research I have in my head. Specifically having the beginning steps/ purpose different and following similar steps to get the information portion of the image. In one of your lessons learned you mentioned that a similar thought, that the purpose of evaluation goes beyond the process as a way evaluation and research differ. I found this to be a great summation of how I view them to differ as well. Thank you for your insight and ideas.
Hi Dana!
Thank you for your overview of the differences between evaluation and research. I found it enjoyable and informative utlizing your fellow peers to show the differing opinions of what people think are the differences and similarities between evaluation and research.
I had to ponder this myself recently for a graduate course. At first I thought it was quite obvious the difference, but as I tried to actually define it I did struggle to have clear difference. The largest factor I found was one you pointed out, “An evaluation should provide…recommendations and lessons into the decision making process of organizations and stakeholders,” (Meterns, 2020, pg 47-48). One could argue that research does this through a discussion section of a paper, but most of the time these focus on elaborating further on the results and possible steps the research team may take in the future, not necessarily recommendations that stakeholders should implement.
In the end, I think it is like what some of your peers stated, that the two are overlapping practices like a venn diagram, sharing a great deal of similarities and practices, but some very distinct differences.
Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (Kindle ed.). Sage Publications, Inc.
Hello. Dr. Wanzer, I am a graduate student attending ASU currently taking research and evaluation in education. I found your research on how Researches and Evaluators define evaluation to quite insightful for future projects. After reading chapters one and two of our text (Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.). , I can fully agree with your findings about evaluations and research and how they differ based on purpose, audience, providing of recommendations to stakeholders, disseminating results, and the generalization of those results. As stated in our text, “Evaluation is an applied inquiry process for collecting and synthesizing evidence that culminates in conclusions about the state of affairs, value, merit, worth, significance, or quality of a program, product, person, policy, proposal, or plan.” Mertens, D.M. 2020 pg. 47. Their similarities are more likely the two processes get confusing to the novice. According to Mertens, (2020) “Both make use of systematic inquiry methods to collect, analyze, interpret, and use data to understand, describe, predict, control, or empower.”
This post was insightful for helping complete my assignment. I appreciate and thank you for your work.
RDAllen5
Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.). SAGE Publications. ISBN: 9781544333762
Hello Dr. Wanzer,
This was an interesting post and helped me to continue thinking about research and evaluation. I had taken a course in evaluating learning programs so I was familiar with the process of evaluating programs but have been asked in my current class to explain the difference between evaluation and research.
As you stated above, many evaluators find that an evaluation is meant to not just determine effectiveness and judgement but also to learn and gain education. This is where I have been struggling the most in defining the two. From my previous course, I had also seen the similarity between research and evaluation in that there will be information gained and learned from.
In the Lessons Learned sections, it sounded like a lot of the differences and similarities of evaluation and research was through the output, meaning the information gained from the work. Could another difference be from the questions posed or the purpose for the research or evaluation? An evaluation could be using a more focused question relying on a value judgement or that the purpose may be more focused for the evaluation is what I am wondering.
Thank you for your work and it helped me to think more deeply!
Hello. Dr. Wanzer, I am a graduate student attending ASU currently taking research and evaluation in education. I found your research on how Researches and Evaluators define evaluation to quite insightful for future projects. After reading chapters one and two of our text (Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.). , I can fully agree with your findings about evaluations and research and how they differ based on purpose, audience, providing of recommendations to stakeholders, disseminating results, and the generalization of those results. As stated in our text, “Evaluation is an applied inquiry process for collecting and synthesizing evidence that culminates in conclusions about the state of affairs, value, merit, worth, significance, or quality of a program, product, person, policy, proposal, or plan.” Mertens, D.M. 2020 pg. 47. Their similarities are more likely the two processes get confusing to the novice. According to Mertens, (2020) “Both make use of systematic inquiry methods to collect, analyze, interpret, and use data to understand, describe, predict, control, or empower.”
This post was insightful for helping complete my assignment. I appreciate and thank you for your work.
RDAllen5
Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.). SAGE Publications. ISBN: 9781544333762
Hello Dana!
I thoroughly enjoyed reading your take on evaluation and research. I am a student at ASU in the graduate program. We are currently on the topic of defining research and evaluation, and recognizing the differences between the two, if any. I initially saw research and evaluation on two different spectrums, and in reading your article most feel the same way. When looking at your 5 diagrams, I like that you showed how many may view how evaluation and research could work in relationship to each other. I’ve always saw evaluation as a sub-component of research. My thinking was once the research was completed (graphs, charts, discussions, studies) that results were analyzed and summarized to then be evaluated for suggestions of what could be changed and made better, or omitted completely. Thank you for your insight it was really helpful!
Hello Dana,
I appreciate and agree with your suggestion that evaluation has two main components that differentiate it from research. To start, you pointed out that your participants emphasized the value that evaluation adds. The “merit and worth” (Mertens, 2020, p. 49) then bring on a whole new level of politics because of how this value can be interpreted in the public eye. Secondly, your participants included that evaluation differs from research because it encompasses a planning process of what to do next with the information that was found. Evaluations include some sort of a result, follow up, or suggestion for change as what to do with this information now that it has been analyzed. This post clarified these two major differences between evaluation and research clearly, making it easy to digest and learn from.
Thank you!
Zoitza
Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.).
Hello, I am currently taking my Masters in Curriculum and Instruction at ASU. We are learning about research and evaluation and how it pertains to education. After reading, I understand the difference between research and evaluation and how they both are significant. Research is analyzing and interpreting data to determine interventions, while evaluation is a final decision backed up with quantitative, qualitative, or mixed methods. I also enjoyed the visual to distinguish the importance of both. I enjoyed reading your differences and similarities between research and evaluation.
Hi Dana,
Thank you for your post. I found it interesting that Mathison described five ways in which evaluation and research could be related, and that all of them could be correct. On the whole, society or people have been conditioned to find one answer and apply it generally. This approach complicates the desire for that straightforward one size fits all, but encompasses far more. I appreciated Erika’s comments about the cyclical nature of research and evaluation. The impact that both can potentially have on the other is interesting.
Thank you!
Melanie Hunsaker
Hi Dr. Wanzer,
I’m currently a grad student at Arizona State University pursuing my MA in Literacy Education. I am currently taking a course on research and evaluation in education. According to Mathison’s “five ways in which evaluation and research could be related,” I’m torn between research and evaluation overlapping and where evaluation is a sub-component of research. Regarding the latter, I feel something had to be researched to gain new knowledge in order for what was learned to be evaluated. Whereas, I also feel one without the other would make the process incomplete. In regard to research being sub-component of evaluation, I’m not sure I can grasp this concept – unless Mathison was inferring that evaluators may find that more research is warranted. Would you please clarify?
This article furthers my new understanding of research v evaluation. I can see from this blog and others, that the two terms, while serving different purposes, also overlap in some regard. Especially, in regards to enhancing knowledge. I am a graduate student and I have many questions that I want to research regarding literacy interventionalist in a school setting. I have begun to evaluate the effects of certain strategies being taught in an intervention program at a school setting. I see that we are falling short of our goals and results. I plan on articulating my question, or hypothesis and then I can begin research, specifically at the school I am working. I am particularly interested in strategies that are most beneficial in improving reading fluency in the primary levels. After that research is done, I plan on re-evaluating our progress. I hope I am on the right path as I learn the differences and likenesses of research and evaluation. Any feedback is appreciated. Thanks again for the information!
Dr. Wanzer,
Thank you for this post about people views of research and evaluation. I am a graduate student at ASU and currently taking my first class related to research and evaluation, so your post really helps! It can be hard to distinguish between the differences of research and evaluation. Personally, up until a few days ago I thought the terms were synonymous. Your post helped to identify the differences between the two, although they are closely related. What I am still wondering if the two processes can be effective when applied separately, or if both need to be used. Again, thank you for an insightful article!
Hi Dana,
I have recently began my study of research and evaluation as part of my master’s degree pursuit at Arizona State. I appreciated the graphic of the 5 different thoughts of how research and evaluation may be connected. I do wonder if if the two have a symbiotic nature. For example, research may lead to new programs or systems, which are then evaluated for effectiveness, resulting in new research opportunities based on their effectiveness. I think I see overlap in the disciplines, but I am wondering if this line of thought goes with the continuum theory you showed in that graphic.
Dr. Wanzer,
Thank you for sharing the interesting results of your study. Prior to beginning specific inquiry into the fields of research and evaluation, I was unaware of the debate and lack of consensus on the two topics. Quite frankly, I am a bit shocked at the collective effort that has been focused on finding new and unique ways of describing the relationship between the two.
You mention in your closing thoughts the possibility of evaluation moving towards existing as a separate profession. What do you believe needs to occur to move in that direction? Would there be any negative ramifications for either the research or evaluation field if evaluation did evolve into a unique and distinct discipline?
Best,
Stacey
Hello Dr. Wanzer!
I am a graduate student at Arizona State University studying curriculum and instruction for gifted education. I find research and evaluation fascinating, but from reading your article I have found that they are not as related as I thought they were. I firmly believe that research and evaluation are needed – especially in the education world. I am currently a second grade teacher and am always looking for new insights from research studies to bring into my classroom. Something I have been wondering since starting this course is if research and evaluation would have much effect without each other. Can we have research without evaluation? Can we have evaluation without research? I have an idea on where I stand for this, but would love to know how you stand!
Thank you for your time,
Kayla Boling
Personally, I don’t think either is effective without the other. Researchers use evaluative tools to determine what is worthwhile to study, the best way to study the concepts of interest, and how to interpret the results; furthermore, research needs to be evaluated for its quality and other criteria (e.g., the field of research evaluation). Similarly, evaluation needs research to better inform both how we do our evaluations, such as through research on evaluation (RoE) and the program theory underlying what programs due (e.g., theory-driven evaluation). I’m not sure which would be more effective on its own.
Thank you for sharing the interesting results of your study. Prior to beginning specific inquiry into the fields of research and evaluation, I was unaware of the debate and lack of consensus on the two topics. Quite frankly, I am a bit shocked at the collective effort that has been focused on finding new and unique ways of describing the relationship between the two concepts.
You mention in your closing thoughts the possibility of evaluation moving towards existing as a separate profession. What do you believe needs to occur to move in that direction? Would there be any negative ramifications for either the research or evaluation field if evaluation did evolve into a unique and distinct discipline?
There is a large literature on professionalization in evaluation which would help answer your first question. Negative ramifications might include putting boundaries on the profession that are not inclusive of people who currently do evaluation but do not fit the profession’s idea of what evaluation is or who evaluators are.
Hello Dr. Wanzer,
I am currently a graduate student at ASU. This article really helped me to clarify the difference between research and evaluation. As a visual learner, the diagram of the 5 ways research and evaluation could be related helped me to understand how parallel research and evaluation can be. I also enjoyed the link to LaVelle’s article. My question for you is do evaluators collaborate with researchers on their findings and how does that in turn affect researchers?
Thank you!
Yes, a prime example would be National Science Foundation (NSF) or Institude of Education Science (IES) grants which require researchers collaborate with evaluators. Outside of federal and state grants, I’m not sure of as many examples. Some people do both research and evaluation and are perhaps best situated to span the boundary between the two.
Hello!
I’m currently a graduate student at Arizona State University and this blog post was very interesting! The image displaying the multitude of ways that evaluation and research can overlap with each other was fascinating and very helpful. I was surprised that evaluators saw greater distinction between evaluation and research than researchers did! As someone who is studying Gifted Education, I can’t help but imagine how this applies to my field. I find that those who are evaluating gifted programs aren’t always aware of the research behind the programming, or even that those crafting the programming aren’t fully aware of the research! I wonder if part of this is that evaluators tend to think of evaluation and research as more separate than they perhaps should be.
In my personal opinion, I think it makes sense for them to be continuous subsets of each other. Research influences evaluation and evaluation influences research. Both have the similar general aim to improve something. However, it often takes evaluators to put research and change into motion as they evaluate programming. The cycle continues as researchers continue to seek more answers as they observe the results of the programming. They are intertwined and I think it’s a good thing for all educators to be aware of! The process of researching, applying, evaluating, and then researching again is a very human thing.
Thank you for a fantastic blog post!
Liesel Lutz
I completely agree with your sentiment that they should influence one another. In some ways they do, with research on evaluation, theory-driven evaluation, and when evaluators publish their findings for researchers to better find. I just wonder how we can better intertwine the two.
Dear Dana,
Thank you for posting a very detailed article on the differences between Research and Evaluation. As part of my graduate course, I have been reading on this topic, and would appreciate it if you could share your views on whether Research should be considered as a subset of Evaluation, or should it be considered vice versa? Do you think if both perspectives are worth considering? Thank you for your time. I look forward to your valuable thoughts.
Kind regards,
Ainee.
I think ideally, if we can get to the alpha discipline Scriven describes, we would eventually get to a place in which research is considered a subset of evaluation. Currently, given the newness of the professional field and our lack of general recognition as a field, I don’t think we are there just yet.
Dr. Wanzer,
I enjoyed reading your blog post as it was quite informative and helped me understand the difference between evaluation and research. As a researcher on an early childhood project for Arizona State University, I admit, like your own research, my initial thoughts were that evaluation was a sub-component of research. I can say my team’s ultimate goal is to improve something, using research to identify and prove where and what needs improvement. So, after reading your blog post along with Mathison’s chapter, I see how research and evaluation differ, but also how the terms are interconnected.
I’m curious, of Mathison’s five ways in which evaluation and research could be related, what do you believe is the most accurate description?
Thank you,
Joanne
I think I would go with the majority of my fellow evaluators and vote for the overlapping Venn diagram, with an eye towards figuring out how we can get to the alpha discipline Scriven describes which would suggest research as a subset of evaluation.
Hi Dr. Wanzer,
My name is Emily and I am a graduate student at the University of Arizona. Like so many others, I found your article very helpful and thought provoking. I am fairly new to learning about the topic of research and evaluation and never knew there was such a distinction between the two until now. I grasp the idea that in general, the audience of researchers and evaluators differs. But what about social researches and evaluators? Is the purpose of both a social researcher and evaluator to ultimately meet the needs of various people and environments? I am interested to hear what your understanding of this is.
Thank you for your informative post and the incredible work that you put into this topic.
Given that I did the study with education researchers at AERA, I think it already answers for social researchers. I would be curious to compare to those in economics or in auditing because I think they might have very different conceptualizations of what evaluation is.
Hello Dr. Wanzer,
My name is Heather Keeley and currently I am a graduate student. I am learning the difference between research and evaluation for the second class. We are now going more into the process, but what was strange for me was that before graduate school, I thought I knew the difference. What has happened is that I have gotten more confused. Confusion just means that my perspective is changing and I am being challenged!
I wanted to first tell you how much this blog post has helped identify my own beliefs with research and evaluation. I looked at Mr. Lavelle’s blog post, and like him I am a very visual learning. The hourglass resonated with what I believe for research and evaluation based off personal experience and the textbook for our class “Research Methods in Education and Psychology: Integrating Diversity with Quantitative and Qualitative Approaches” by Donna Mertens. I believe that they overlap in similarity, but ultimately it is about the stakeholders involved. I am starting to see that there is more stake in evaluation than in research. I teach middle school, so how I am connecting this is to “bullying.” I was bullied, so I see that my own “merit,” “value,” and “worth” were based on those who were evaluating me. I listened to their “results” and changed to fit their definition, so that way I could be more accepted and effective at life. If they spent more time getting to know me and doing their research on me, the results would have changed, but if they just researched me with no evaluation, it would not have affected me. I would not have known their opinions on how to be better. It is not exactly the best metaphor, but it is how I am connecting to it personally.
I have been researching myself during this quarantine and evaluating my selfworth based on old information and facts. It is time I evaluatd from a different perspective. When my professor asked which paradigm I think is important, I picked constructivist. I have to connect personally to what I am learning and create a narrative to learn. Connecting it to the job is just a little harder currently because my job is not about learning, but completion right now. The skills my students are learning right now are the ones not taught in the classroom.
Thank you for taking the time to read my input on your blog post! Thank you for giving me an insight into this definition and helping with the clarification process!
Dr. Wanzer,
I am a graduate student in Learning Design and Technology at Arizona State University. Your article was very helpful in defining research vs evaluation. I initially thought that both terms were related. Upon reading your article and exploring the links provided, I think I have a better definition of what the two terms mean. I originally thought that they were the same because they both deal with identifying data and information. As an experienced evaluator, do you think that research and evaluation is necessary?
Thanks again for your insight.
Sincerely,
Carl
Hi Carl,
I don’t think I’d be in this profession if I didn’t think that research and evaluation is necessary! How else do we know what works and for whom? How else do we expand the knowledge base? Research and evaluation help us in both endeavors.
-Dana
Dr. Wanzer,
I am studying Curriculum and Instruction at Arizona State University and currently enrolled in a research course. I really appreciated your article and visuals you had to help explain the five ways research and evaluation could be related. I also appreciated the links to other pages that support your information and findings on research and evaluation. Thank you for the resources, such as Open Science Framework. Do you believe that evaluators are seeing the difference between the two more than researchers are?
Thank you again!
Lauren Foley
Hi Lauren,
Yes, my research found that evaluators see more differences between research and evaluation than researchers do. In other words, we see the uniqueness of our work more than researchers do.
-Dana
Dr. Wanzer,
I completely agree that educators see more differences in evaluation and research. I teach fourth grade and I am currently attending graduate courses through ASU. We are studying the differences between evaluation and research, and the best way for me to break it down was to think how it applies to my classroom. As teachers, we are constantly evaluating our students through collection of artifacts. I use what I learn from evaluating my students to research ways to better help them meet their learning goals. Thank you for providing another source of information for my coursework and classroom!
Sincerely,
Megan Corder
Dr. Wanzer,
I’m enrolled in a graduate course at Arizona State University that is currently discussing the differences between research and evaluation. Upon searching for resources to inform my discussion post, I came across your article. Thank you for including helpful links, diagrams, and resources. I found it fascinating that evaluators are more likely to view evaluation and research as only intersecting whereas researchers leaned more towards evaluation as a sub-component of research. Before taking this course and learning about these differences, I honestly thought they were one and the same. Now that I know how evaluation goes beyond the analysis of data into determining value (worth and merit) of the findings, I feel as though I agree with the comment above mine as research and evaluation being cyclical in nature with both research and evaluation informing one another. I’m a classroom teacher and I have definitely had my students evaluate their findings but used the verbiage of research! Now that I can distinguish between the two, I want to further my own understanding on the differences and pass that along to my own students.
Thank you again for your wonderful article- I found it to be incredibly helpful and straightforward.
Sincerely,
Paige
I’m glad you found it useful, Paige!
Dr. Wanzer,
I couldn’t agree more with your claim that many people new to evaluation struggle to understand it and explain the relationship it has to research. I am a middle school teacher pursuing a graduate degree in Learning Design and Technologies. It wasn’t until I started taking graduate courses that I even tried to tackle the concept of evaluations. As a teacher, and a pretty new one at that, I am used to assessing students and having administrators evaluate me, so I’ve never really done the evaluating myself. This makes it very difficult for me to fully understand what evaluations are, what they involve, and how they relate to research. However, after reading what you have to say about it, combined with the text I read for my course, I have a much better grasp of it.
The graphic you included was very helpful when it came to my perception of evaluation relating to research. I can see how all five are plausible relationships and I think they can change depending on the content that is being researched and evaluated. Because of this, I tend to gravitate toward the idea that they exist on a continuum.
Something I’ve been thinking about is whether or not research can be effective without an evaluation component and vice versa. I would love to hear your thoughts on this because I’m having a hard time deciding. I am inclined to think that research can be effective without an evaluation, but an evaluation cannot be effective without research. However, I don’t know if this is an accurate claim or if I’m considering all possibilities. I would appreciate your perspective on this idea.
Thank you for your valuable input on this topic.
Jennifer Baron
Hi Jennifer,
Great questions! I think they’re more synergistic that many believe. We see research being used in evaluation a lot, especially in theory-driven evaluation. However, I think we should also be thinking about research on evaluation to inform our evaluation practices.
The incorporation of evaluation into research may be less obvious, especially with research tending to want to remain ‘value-neutral.’ Yet researchers embed their values into their research when they choose what and who to study, with what methods, with what data analysis tools, etc. (these might be the “merit” of a study). Approving a study for a grant or a paper for publication indicate the value we place on research (these might be the “worth” of a study). And statistical significance and effect sizes indicate the value we place on the findings (these might be the “significance” of a study). Research has evaluation all throughout it, which is why we need to embrace and advocate for the role of evaluation as a transdiscipline (and perhaps, as Scriven argues, the alpha discipline).
-Dana
Hi Dr. Wanzer,
I am currently in a graduate program that focuses on learning design and technology, which involves me taking courses on both research and evaluation. I would definitely consider myself a novice when it comes to evaluation considering I have only taken one formal course on the topic and in my career life, I have evaluated learning programs, but not to the scope of a true evaluation.
I would have to say that study you conducted and the topic mentioned is extremely common since I really do struggle to this day to explain the difference between research and evaluation. I know from my courses and experience that they’re not one in the same, but it does become difficult to explain the differences to others. I would say I connect with the participants that believe research and evaluation are similar in scope to a venn diagram. I have always felt that they overlap in some regard because there is a bit of research necessary in the evaluation process to either understand the program being evaluated or the company being evaluated. I believe that research can further lift up the evaluators in their endeavor to evaluate.
I also will say that it is important to distinguish the difference between evaluation and research when working with a client or organization so that they understand the scope of exactly what you are there to do as an evaluator. Your tips are extremely helpful!
Thanks for your insights and even the graphics help explain the different beliefs around this topic even further.
Sincerely,
Jackie Arthur
I’m glad you found this article and the research helpful!
Dr. Wanzer,
I wanted to let you know how much I enjoyed this article. I’m a graduate student and between my recent class about evaluation and current class about research, this topic is coming up a lot for me. Until I took the course on evaluation last semester I thought evaluation and research were basically the same things. I know better now. This article added more to my knowledge in this area by pointing a lens at the evaluator’s perspective. I think it is a great perspective to have. Thank you for including the links to the data, diagrams, and other information of the study. I downloaded all of them and, as a beginner to the field of evaluation loved seeing the things I was taught in my class “come to life” in a real-life scenario. I thought the data in your figures and tables were easy to understand and the results are enlightening.
As I am a novice in the field of evaluation, do you have any textbooks or other books you suggest to teach someone about evaluation? Any favorite resources you use (blogs, books, other publications) that you consider “must-read” information to stay up-to-date in the world of evaluation? I’ve already read the article from Patricia Rogers suggested in your post and will read the others you list above as well but I’m very interested in both Research and Evaluation and would appreciate any suggestions you might have from your experiences.
Thank you!
I find AEA365, #EvalTwitter, and following my favorite evaluation journals (e.g., American Journal of Evaluation, Canadian Journal of Program Evaluation, Evaluation and Program Planning, Journal of MultiDisciplinary Evaluation) the best way to stay up-to-date in the world of evaluation! There are a ton of resources out there, and the community is always sharing in a wide variety of mediums including blogs, podcasts, journal articles, webinars, and more.
Thank you so much for your help!
Pingback: #EvalTuesdayTip: Identifying relationships between evaluation & research - Khulisa
Hi Dana,
Your post is of particular interest since it has prompted me to reflect on the relationship between evaluation and research within the context of teaching. I am a high school science teacher, and I have always regarded myself primarily as a deliverer of information that I hoped would become knowledge to the receiver. I did not consider myself an evaluator at the same time, and it was only until recently that my thought process began to change.
When reading your post, I was immediately inclined to support the Venn diagram, and thus the hourglass model that LaVelle suggested in his post, to represent my idea of how evaluation and research are related. I can appreciate that research and evaluation both have their respective purposes, research being to discover “generalizable knowledge” and evaluation to discover “context-specific knowledge” (Alkin, M. C. & Taut, S. M., 2003), but that they share common methodologies and analysis techniques to render their conclusions.
However, when I think of my role as a teacher, I see myself as using the results of an evaluation to illicit more research, to then test out and re-evaluate. I see myself learning through this process. When you say, “many evaluators also think evaluation is about learning, informing decision making, and improving programming, indicating the purpose of evaluation beyond simply the process”, it validates my belief that evaluation can elicit more research and can result in learning on behalf of the evaluator for the benefit of the stakeholders: my students.
For example, I am evaluating whether a specific set of laboratory skills have been internalized by my students, and whether they are then transferrable to another scenario. Through the evaluation, I discover that the transfer of knowledge did not occur. Therefore, I research different methods for ensuring transferability of knowledge and then try the evaluation again to see whether my new approach has worked. Within my context of teaching, I am seeing evaluation and research as cyclical in that there really is no beginning or end; whether we begin with research that we then evaluate, or an evaluation that leads to more research. Either research or evaluation prompts the other in a cyclical manner. I wonder what your thoughts are in regards to my perspective.
Thank you for a great thought provoking post, and congratulations on your work in the field of evaluation.
I definitely agree that research and evaluation can and should be a cyclical process! I think we think of research informing evaluation more so than the other way around, but they can both improve how we think about the other. Research can inform us on what might be useful programs or interventions, but evaluations can elucidate things that might need further research to determine whether it can generalize to other people, situations, contexts, etc.