Hi, my name is Rhodri Dierst-Davies, an evaluation specialist with Deloitte Consulting LLP working out of our federal practice. Many times federal programs are, by design, implemented differently across states and municipalities. These programs rely on inputs from local stakeholders and policy makers to ensure they are tailored to the needs of the communities they serve. While this can help maximize benefits to beneficiaries, it creates challenges for federal evaluators as they try to demonstrate generalizable benefits across an entire system. With an increased emphasis on evaluations that can provide both national and local benefits, I will explore potential solutions that may help solve common national evaluation challenges.
Lesson Learned: Generate common goals and objectives specific to all programmatic aspects. This way individual jurisdictions can create tailored evaluation frameworks that focus on what is relevant to them.
Hot Tip: Consider offering capacity building grants that are directly focused on evaluation. Such grants are effective at helping individual jurisdictions build their evaluation instruction, as some requirements may be difficult to implement.
Lesson Learned: Offer a data collection warehouse that contains a set of easily accessible common data collection measures. While there are always a set of core variables that must be collected (e.g. socio-demographic characteristics), offering a set of validated measures focused on other factors important to local jurisdictions (e.g., needs assessments, benefits utilization, mental health, stigma), from which a jurisdiction may pick from to measure local benefits, can help facilitate analysis of individual programmatic elements which are not uniformly implemented.
Hot Tip: Consider creating a secure web-based data collection portal that providers can easily use to collect and store evaluation data. This may reduce burden on local jurisdictions who will not have to rely on creating their own local systems. It may also help reduce reporting burdens of individual jurisdictions, as data collection will be semi-automated.
These ideas may help streamline the evaluation efforts of national programs in multiple ways. For an individual jurisdiction, it can reduce burdens around data collection while still providing implementation flexibility. At the national level, it can streamline data collection and reporting methods. Taken together, these suggestions can facilitate both the timely reporting and integrity of data, aspects important for successful federal evaluation.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.