We are Marty Henry and Keith Murray, lead evaluators for M.A. Henry Consulting, LLC in Fairfax, Virginia, and St. Louis, Missouri. The issues noted here stem largely from math and science education program evaluations we have done, but they apply to many evaluation contexts.
The federal funding programs supporting these projects aim at improving teacher and student content knowledge and enhancing teachers’ classroom performance. We collect a wealth of qualitative and quantitative data, especially since partnership, leadership, and school-based team-building are project priorities.
Lessons Learned: Issues often arise about who can access evaluation data containing personal identifiers, particularly with the university-based project teams. We have encountered several challenging situations dealing with providing access to evaluation for purposes outside the original project plans.
In one case, a graduate student wanted data with identifiers for new research in the third year of a five-year project. Participant consent forms signed by the university’s PI stipulated that only the evaluators could access the identifier keys. Even though the university IRB agreed with our interpretation of restrictions, our contract was terminated.
In another case, the project team wanted to use data for analyses outside the specifications of evaluation protocols and consents. Universities usually include a clause in their master contract that the university owns any data or products produced as a result of contractor activities. They insist that any project-derived data can be used for any purpose, regardless of consent specifications. IRBs have been inconsistent in addressing this concern.
These are only two of the issues we’ve faced about the “ownership” of evaluation data.
Hot Tips: We can suggest three steps to help alleviate the ethical issues that can arise for evaluators in such circumstances.
1. Include language in the consents stating that project leadership as well as evaluators may use identifiable data. This approach may reduce participation in evaluation activities, but will enable projects to access evaluation data.
2. Include language in the consents and contracts that evaluators and the institution own the data jointly. Institutional contracts are usually boilerplates that disallow customization. An internal addendum can sometimes be added to institutional contracts to allow this approach.
3. Finally, universities often state that no risk exists in their obtaining and using evaluation data with personal identifiers. While the largest issue here concerns maintaining ethics and the integrity of the human subjects system, we assert that risks potentially are involved, given the qualitative nature and frank sharing that is a part of effective evaluation. In the end, we would prefer that evaluation-specific data not be subsumed in the larger research data pool. Ultimately, our purposes differ, and participants deserve sensitive consideration.
Rad Resource: The EvalTalk list has discussions about data ownership issues.
The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.