My name is Kirsten Rewey and I am a Senior Research and Evaluation Associate at ACET, Inc. Two years ago ACET was selected to provide evaluation services for a federally-funded Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) implemented by the Minnesota Office of Higher Education (OHE). As part of the services, OHE asked ACET to create a research plan to determine the impact of GEAR UP on students’ academic preparedness using Minnesota’s statewide academic test (Minnesota’s No Child Left Behind assessment).
Hot Tip: ACET used the following approach and finds that it facilitates staff development and client selection of a research design:
- Identify data available from the client: In order to develop a solid research plan, ACET needed to know what data OHE/GEAR UP were currently collecting. One of our first planning meetings was to discuss the types of data available in the GEAR UP database, the format of the data, and retrieving the data. During the meeting ACET obtained the list of variables maintained in the database, their descriptions, and the timeline for data entry and retrieval.
- Identify data available from other sources. Once ACET knew the data that would be available from GEAR UP, other sources of data needed to be identified and catalogued. From previous work, ACET staff knew there was a substantial amount of school-level, public-record data available from the Minnesota Department of Education. For example, the Minnesota Department of Education publishes demographic information for the state’s public schools on their website including number of students enrolled, number eligible for free or reduced-price meals, and number who have limited English proficiency. For some of the variables, demographic data are available by grade-level and ACET can hone in on a specific grade of interest. Other data, such as individual demographic data and test score results, are only available pending approval from the district or the state. In order to obtain this data OHE and ACET wrote a formal application to the district to receive selected data from their databases.
- Create an array of research options and present them to the client. After ACET identified available data, the staff created an array of research options for the client. ACET typically creates a variety of designs that vary in research/experimental control, types of conclusions which can and cannot be drawn, and cost. The options are presented to the client in a matrix with a brief description of the design, the advantages, and challenges for each. And because clients have an array of research options and are alerted to the advantages and challenges of each design, they can select the research design that best meets their needs and budget.
This aea365 contribution is part of College Access Programs week sponsored by AEA’s College Access Programs Topical Interest Group. Be sure to subscribe to AEA’s Headlines and Resources weekly update in order to tap into great CAP resources! And, if you want to learn more from Kirsten, check out the CAP Sponsored Sessions on the program for Evaluation 2010, November 10-13 in San Antonio.