AEA365 | A Tip-a-Day by and for Evaluators

Nov/16

30

Ed Eval TIG Week: Anane Olatunji on Assessing Student Engagement

Allow me to introduce myself as Anane Olatunji, president of Align Education, LLC, a consulting R & D firm. Having worked with all types of educational agencies over the last two decades, I’d like to share one important tip that I’ve found particularly helpful when evaluating educational program evaluations. Assess student engagement!

Although there is no agreed upon definition among researchers for the term student engagement, it has to do with the quality of students’ involvement in school based on their behaviors and feelings or attitudes (see Yazzie-Mintz and McCormick, 2012). To underscore the need for assessing engagement, I’d like to borrow a line from a document recently used in my work on a state-level evaluation of charter schools.  A Report from the National Consensus Panel on Charter School Academic Quality contends that student engagement is “a precondition essential for achieving other educational outcomes.” In other words, engagement is a bellwether of academic achievement, the critical outcome educational concern. Whether engagement is high or low, achievement usually follows in the same direction. This information thus enables a program to make modifications, if needed, prior to summative evaluation. It is precisely for this reason that assessing engagement adds value to program evaluations. Here’s a simplified illustration of the role of engagement:

olatunji

Unfortunately, even though engagement is an antecedent of achievement, it often is not assessed in evaluations. This omission may in part be due to program managers rather than evaluators. If managers don’t explicitly express an interest in assessing engagement, we as evaluators may be inclined to leave it at that and not push any further. My hope, however, is that you will take “program evaluation destiny” into your own hands. Through your awareness and use of this knowledge, you can improve quality of not only an evaluation, but also and more importantly – an educational program as a whole.

So how do you move from knowledge to implementation? Student attendance is one of the most common measures engagement. A shortcoming of this indicator, however, is that it doesn’t give a good indication about why students go to school. If most kids goes to school because the law or their parents force them to, then attendance alone can be a poor measure of engagement. Other measures therefore might include tardiness rates, rates of participation in school activities, or student satisfaction rates. For examples of survey items, see national surveys of middle and secondary school students. It’s especially important to assess at these levels because engagement declines after elementary school.

Of course, we’ve only scratched the surface on the topic of assessing engagement, but at least now you can move begin moving forward better than before. Good luck!

The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

3 comments

  • Shelley Greenwood · August 9, 2017 at 4:31 pm

    Hello, Anane Olatunji.

    Thank you for shedding light on the necessary topic of student engagement when evaluating a program. I find it imperative to measure the effectiveness of a program based on outcomes but ALSO on engagement. Our students are more likely to benefit from a program they are engaged in. If we are truly going to impact their lives through their participation in a program do we not want them to be engaged? It is unfortunate that many measures of programs are quantitative measures that are not inclusive of engagement. I agree that it would be helpful to take the evaluation destiny into our own hands and include valuable data on engagement. A program is not worth much if it is not engaging. Student engagement increases achievement. That is documented. Therefore, engagement in programs should increase positive outcomes.

    Have you had found any more appropriate measures of engagement past attendance and tardiness that would be appropriate measures? I would be curious to add this information in the evaluation of a program we are implementing in a grade 5-9 school. Attendance is not an appropriate measure in our school because we are a rural school where students are bussed and it is not possible to “skip” school as there is nowhere to go.

    I look forward to hearing back from you.
    Thank you for your time.
    Shelley

    Reply

  • Heidi Ashton · March 6, 2017 at 1:56 am

    Dear Anane,

    Thank you for your post on the importance of evaluating for student engagement. As you identified, there is a direct causal link between engagement and achievement and more students would benefit if teachers and school leaders evaluated the level of student engagement and responded promptly to the data they collected.

    I fully agree that including student engagement in programme evaluation would improve the quality of the evaluation; it would also improve the usability of it. Teachers who are made sensitive to how deeply their students are engaged in their learning, would be much better equipped to respond to a lack of engagement in a timely manner. By going through a process of regularly collecting data on student engagement, most teachers would feel compelled to make changes in their teaching practice before these practices became too ingrained and part of “how we do things”.

    Including the collection of this data, particularly as part of a process evaluation, would certainly help stakeholders identify problems and correct them before investing more resources into a programme and losing more instructional time.

    Reply

  • Jindra Cekan · December 1, 2016 at 12:43 pm

    Anane – you’re so right! As a parent of teens I can’t underscore the need for this (and teacher/ school adjustments needed) as a result of such data.
    As an evaluator, we need such engagement data in int’l development… that’s a precursor to capacity built and motivation needed to sustain projects after we leave.
    Thanks so much,
    Jindra

    Reply

Leave a Reply

<<

>>

Archives

To top