AEA365 | A Tip-a-Day by and for Evaluators

TAG | student learning

My name is Sean McKitrick, Vice President with the Middle States Commission on Higher Education.

In higher education settings, “assessment” is a term that can mean both institutional research and student learning assessment and usually refers to institutional efforts to provide accurate data and reports to oversight bodies such as federal and state governments or to systems offices, to efforts to evaluate overall institutional effectiveness, and to efforts to assess student learning. In recent years, pressures to assess have their origins in pressures by state and federal governments, accreditors, and by a public requiring more accessible information for prospective applicants.

With regard to assessment in higher education settings, the following points, among others, appear salient:

  1. Accountability demands will only increase, but a debate is brewing about whether these demands should focus on reporting or institutional improvement. Some parties argue that accreditors should not be required to link assessment of student learning and other measures with recommendations regarding an institution’s future eligibility to dispense federal funds, while others argue that measures such as graduation rates and student salary information (in aggregate) are sufficient measures of institutional quality.
  2. Support for requiring institutions to report additional data, such as the aggregate salaries of students, engenders further debate regarding the reliability of such information. Some important questions to ask include: How effectively might institutions be able to contact students for salary information? Should the government be allowed to link federal databases in order to find such information independent of institutional involvement?
  3. The validity of assessment information continues to be debated. Although graduation and retention rates are important measures of institutional effectiveness, some argue that these can serve as proxy measures of student learning. Others argue that these measures do not directly evaluate student learning and other measures be taken to do this, although this increases reporting burdens on institutions.
  4. Pressures to assess student learning continue. However, given a lack of a common core of learning outcomes from institution to institution, it appears that the current trend is to focus on how institutions are using assessment processes (and evaluation information) to manage and improve student learning rather than to focus solely on the measurement of outcomes.

Hot Tip: Assessment and evaluation in higher education continue, but expectations regarding methods of evaluation and assessment are changing as well as expectations regarding what information to report and use by governments and accrediting organizations.

RAD Resource: The College Navigator site, sponsored by the National Center for Education Statistics, is the primary site where institutional data required by the U.S. Department of Education can be found, http://nces.ed.gov/collegenavigator.

The American Evaluation Association is celebrating Assessment in Higher Education (AHE) TIG Week. The contributions all this week to aea365 come from AHE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

I am Veronica Smith, principal of data2insight, an evaluation and research firm specializing in STEM program evaluation. We love working with organizations to design, develop and improve curricula for STEM teaching and learning. Examples of curricula we have evaluated include the NWABR Science and Ethics of Humans in Research, TechStart, and BioQuest Global Health Curriculum.

Lesson Learned: When answering questions aimed at gathering evidence of student learning, it is essential to carefully design the evauation to meet time and budget constraints, and to set stakeholder expectations regarding method limitations. Refer to What Works Clearinghouse standards as an aid for communicating the differing levels of rigor for different evaluation designs.

Rad Resource: Understanding by Design is a framework for improving student achievement that emphasizes the teacher’s critical role as a designer of student learning. We have found the UdB text and professional development workbook to be assets in design and development of standards-driven curricula. The UbD approach helps teachers clarify learning goals and devise revealing formative and summative assessments.

Capture1

Hot Tip: Engaging STEM organizations early in conversations RE: evaluation alongside program and/or grant proposal development improves evaluation quality and stakeholder satisfaction and use. We offer evaluation plan development prior to submission of grant applications as a free business development service. We craft a letter of understanding indicating that if the grant is funded, our firm will be hired as the evaluator for that project. This upfront work saves time and money.

Lesson Learned: Whether a curriculum gets into the hands of teachers who can put it to work to improve STEM teaching and learning is largely dependent on where the digitial version of the curriculum lives once the curriculum is published. Program leaders are wise to develop a 1-3 year strategy for sustaining access to and updating curriculum products past the end of grant funding in order to expand their work’s reach and impact. Organizations like the American Chemistry Society track and monitor curricular resource use in order to increase and broaden the use of those resources.

Hot Tip: Northwest Association of Biomedical Research recently conducted a teacher survey asking about the best places to post and/or present curricula. National Science Teachers Assocation was one of the faves. NSTA’s website has a Freebies for Science Teachers page that might be a great location for your STEM curriculum.

Capture2

Hot Tip: Just a few more days until the March 15 proposal submission deadline for Evaluation 2013. If you are having a hard time deciding between two TIGs for your submission, you can suggest that the TIGs co-sponsor your session. Choose one primary TIG and add a comment in the “other information” box suggesting that the second TIG may also be interested.

The American Evaluation Association is celebrating STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to aea365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Archives

To top