AEA365 | A Tip-a-Day by and for Evaluators

CAT | Uncategorized

Hi! This is Juan J. DelaCruz, an Associate Professor of Economics and Business at Lehman College (Bronx) and Associated Faculty of the School of Public Health and Health Policy (Harlem) of CUNY. I am proud to have been selected for the 2017 Minority Serving Institutions Program. I am a health economist studying the impact of HIV on older adults in New York City. My work assesses rival interventions for HIV-infected individuals and its economic choices that are cost-effective. Economics informs CRE. Researchers and evaluators using economic tools should have a solid perception of how diverse cultural norms and practices influence their own worldview. Economics faces challenges in the evaluation process due to its quantifiable nature. Economics for CRE is constrained by quantitative methods.

I want to contextualize the role of economics in evaluation using economic stability as a social determinant of health. Good health requires efforts that go beyond epidemiological factors, and the position in the social ladder depends on finding good jobs and staying healthy, but income is unequally distributed in society. Some factors keep vulnerable groups from reaching full economic potential making them more vulnerable to negative health outcomes. Quality jobs are associated with education, gender and age, which determine health status and job stability (better health leads to better earning and vice versa). Unemployment is linked to alcoholism, crime, drug-use, incarceration as well as housing and food insecurity. Poverty creates ill-health and persists even when constraints are alleviated by social policy. Raising healthcare costs distress people, as they forgo basic needs to choose healthcare. These difficulties are enhanced by the intersection of gender, race/ethnicity, age and other factors; needless to say, health inequities are rooted in historical, economic and political factors. The social context (criminal justice system, social segregation, income inequality and gender gaps) influences individual behaviors and determines health outcomes.

Lessons Learned:

The most valuable lesson is that we can achieve cultural responsiveness when the design, execution and appraisal of any program are rooted in cultural inclusion and cultural context. Economics approaches need to develop the analytic techniques needed for evaluation studies that allow for cultural inclusion and cultural context. We need to identify analytical frames that fit the goals of CRE. Community-based and community-based participatory research helps engage and empower communities during the evaluation process.

Rad Resources:

  • Adimora, A. & Schonenbach, V.J. (2005), “Social Context, Social Networks and Racial Disparities in Rates of STI’s”, J of Infectious Diseases, 191(Supplement 1):S115-S122
  • Benach, J. et al. (2014), “Precarious Employment: Understanding an Emerging Social Determinant of Health”, Annual Review of Public Health, 35:229-253
  • Godin, I. et al (2004), “Differential Economic Stability and Psychosocial Stress at Work: Association with Psychosomatic Complaints and Absenteeism”, Social Science & Medicine, 58(8):1543-1553
  • Mosier, S and Clayton, PF (2015), “Economic Instability: A Social Determinant of Health”, Kansas Department of Health and Environment, Bureau of Health Promotion, March 2015
  • Schulz, A & Northridge, ME (2004), “Social Determinants of Health Implications for Environmental Health Promotion”, Health Education and Behavior, 31(4):455-471

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello!  My name is Imelda K Moise, Assistant Professor in the Departments of Geography and Public Health Sciences at the University of Miami, Florida. Prior to this, I was a GIS /Global Health M&E Advisor, a Research Specialist and spent six years as a Peace Corps technical trainer in Zambia. Because of these experiences, much of my work has primarily focused on utilizing multi-method approaches, community-based participatory research (CBPR) and evidence-informed interventions that are culturally responsive to a specific problem identified for a given context and practice (e.g., in the areas of diffusion and distribution of disease, health care/utilization and geographical targeting). As a 2016-17 MSI Fellow, our cohort examined the Intersection between Social Determinants of Health (SDOH) and Culturally Responsive Evaluation (CRE).

My contribution to the group project focused on how geographic thinking can help us understand a wide range of SDOH and identification of at-risk groups in affected communities to provide tailored interventions and services to the right people, in the right places and in a timely manner. The following example from one of my projects highlights how I have applied geographical thinking to support neighborhood scale interventions.

Lessons Learned: How would you go about identifying the most at-risk populations and neighborhoods to provide tailored interventions and services in a timely manner? This was the issue in post-Katrina New Orleans.  What we found was that although the hurricane itself was a huge trauma, for those who lived in the affected areas, the mental strain did not stop after floodwaters receded. To help identify at-risk areas and populations in need, we examined hospital data from the Louisiana Department of Health and Hospitals in New Orleans from 2004 (pre-Katrina) and 2008 (post-Katrina), looking for a change in the rate of hospitalizations for substance abuse. “What we saw was that geographic patterns of hospitalization for substance abuse disorders shifted in post-Katrina from flood-exposed areas to less exposed areas located in the center of the city, areas used for evacuees displaced by the hurricane.”

The generated information can inform public health officials deploy targeted interventions and treatment for substance use disorders to those affected individuals and neighborhoods in a timely manner. Further, physicians and other health care providers can use these findings as evidence to attend to the patient’s state of mind after such trauma.

As evaluators, we can contribute to research on CRE by teasing out connections between place effects and health disparities by utilizing geographic tools and methodologies to explore these associations. If you do not have expertise in using geographic tools and methodologies, you can leverage local resources in your community such as universities and county health departments. 

Rad Resources:

Hospitalizations for substance abuse disorders before and after Hurricane Katrina: Spatial clustering and area level predictors, New Orleans, 2004 and 2008.  

“A Process Guide to Monitoring and Evaluation for Informed Decision Making” provides evaluators with an overview of geospatial analysis techniques and ways to apply geospatial analysis within the context of M&E, along with additional resources.

 

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello!  My name is Nicole Henley, an Assistant Professor and Health Care Management Program Coordinator in the Department of Health Science and Human Ecology at California State University, San Bernardino (CSUSB).  My research interests are access to health care for vulnerable populations and social determinants of health.  The main courses I teach are: Health Services Administration, Statistics, and Social Determinants of Health.  As a 2016-17 MSI Fellow, our cohort examined the Intersection Between Social Determinants of Health (SDOH) and Culturally Responsive Evaluation (CRE).

My contribution to the group project focused on the Health and Health Care domain of the SDOH framework, and the importance of incorporating CRE in the theoretical framework of health-related programs addressing the complex needs of vulnerable populations. 

Lessons Learned: Vulnerable populations have different needs than the general population; Therefore, it’s important to examine the roles of structural and environmental factors, and their affect and effect on this group’s overall health and health outcomes.  Their health and health care challenges intersect with social determinants of health and when “culture” is embedded in the theory, design, and practice of evaluation, systematic errors, cultural biases, and stereotypes are reduced (AEA, 2011), and as a result, the program produces valid and reliable results, and improved population health outcomes and quality of life for this population.

Rad Resource:

If you’re interested in learning more about culturally-appropriate theory that takes into account the complex needs of vulnerable populations, read the article, “Behavioral Model for Vulnerable Populations: Application to Medical Care Use and Outcomes for Homeless People” (Gelberg, L. et al, 2000).

Rad Resource:

Time for Change Foundation (TFCF) is a non-profit organization in San Bernardino, CA that has integrated the “culture” of the vulnerable population they serve in the theory and design of their Homes for Hope Program, which is a permanent supportive housing program that assist homeless families in becoming self-sufficient by placing them directly into their own apartment and providing intensive case management and support services.  TFCF currently has 13 scattered-site locations throughout San Bernardino, CA. TFCF is one of many community-based organizations making a difference in the lives of vulnerable populations.  To learn more about TFCF’s success stories, please visit their website: http://www.timeforchangefoundation.org/.

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

My name is Arthur Hernandez and I am a Professor at the University of the Incarnate Word. I have served as evaluator and teacher of evaluation and am very interested in the processes of cultural responsiveness in practice.

Lesson Learned: It seems to be generally accepted that cultural context influences the manner in which individuals perceive and understand reality and cultural context is a matter of development, identity and the foundation for making judgments about value.  Thus, the inclusion of means of and mechanisms for Culturally Responsive Evaluation (CRE) in our practice is requisite to competent, ethical practice.

Hot Tip:  It is essential to invest in self-assessment.  All too often the effort to engage in CRE involves solely examining participants to determine the impact of identifiable culture on their perception, perspectives and behavior.   While this is certainly important, it is equally important to be aware of or to engage in inquiry to ascertain not usually recognized elements of culture and of how these potentially alternative ways of knowing and valuing may be in influencing the dynamic or interest much less how they might conflict with the culture of the evaluator as well as other important stakeholders.  At a minimum, this self-assessment should be concerned with knowledge, attitudes, familiarity and acceptance.  Knowledge about the community in which the evaluation will take place is essential for development of meaningful metrics and methods.  Gauging attitudes about the community and its values and expectations is essential to ensure that contact with the members is respectful and reasonable (to them).  Familiarity deals with the necessity of establishing meaningful (as opposed to utilitarian) relationships with members of the community and acceptance is the requirement that evaluators understand differences as legitimate and of value in their own right.

Rad Resources: Some good starting point references include:

  • C. Griffith & B. Montrosse-Moorhead (Eds.), Revisiting truth, beauty, and justice: Evaluating with validity in the 21st century. New Directions for Evaluation, 142.
  • Thompson-Robinson, M., Hopson, R., SenGupta, S. (Eds.), In search of cultural competence in evaluation. New Directions for Evaluation,

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello Loyal aea365 readers! I’m Sheila B Robinson, aea365’s Lead Curator and sometimes Saturday contributor with one question for you: What is it that YOU would like to read about on this blog?

I first posted this article in 2015 (and every year since) and we receive excellent responses from readers each time it is published. I typically share responses in a subsequent post and as a result we receive blog articles on some of the suggested topics from authors willing to answer the call. Here we go again with some minor updates to reflect the times:

Lesson Learned: AEA365 has been going steadily since January 1, 2010 with 2850+ contributions (Wow!) from hundreds of evaluators across the globe. We accept individual submissions at aea365@eval.org on a rolling basis, along with inquiries about sponsored or themed weeks. Posts are about any and all evaluation-related topics, and anyone with something to share with fellow evaluators is welcome to contribute! If you are interested in sharing a tip, please be sure to check out our *updated* contribution guidelines here.

As a key learning tool for evaluation, aea365 can also be a fabulous vehicle for promoting evaluation and evidence-based policy. With that in mind, we would like to include your voice as we head into the new year as our aea365 team considers inviting authors and groups to contribute.

Hot Tip: Let’s crowdsource some ideas for aea365 in 2018 and make it the best year ever.

Please let us know what you would like to see in aea365 by responding to these questions in the comments (click the word “Comments” just under the title of the post and scroll down to add yours):

1. What do YOU want to read or learn more about on aea365 in 2018?

2. Who do YOU want to hear from on this blog?

Thanks very much for your input and your loyal readership.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello, my name is Michel Laurendeau, and I am a consultant wishing to share over 40 years of experience in policy development, performance measurement and evaluation of public programs. This is the last of seven (7) consecutive AEA365 posts discussing a stepwise approach to integrating performance measurement and evaluation strategies in order to more effectively support results-based management (RBM). In all post discussions, ‘program’ is meant to comprise government policies as well as broader initiatives involving multiple organizations.

This last post discusses the creation and analysis of comprehensive and integrated databases for ongoing performance measurement (i.e., monitoring), periodic evaluation and reporting purposes.

Step 7 – Collecting and Analysing Data

Program interventions are usually designed to be delivered in standardized ways to target populations. However, standardization does not often take into consideration variations in circumstances that may affect the results of interventions, such as:

  • Individual differences (e.g., demographic and psychological factors);
  • Contextual variables (e.g., social, economic and geo-political factors/risks);
  • Program and institutional variables (e.g., type and level of services, delivery method, accessibility).

Focusing exclusively on program delivery (i.e., economy and efficiency) through the assessment of the achievement of delivery targets or compliance with delivery standards may be quite appropriate when programs are mature, and the causal relationships between outputs and outcomes are well understood and well established. But this is not always the case, and definitely not so when programs are new or in a demonstration phase (e.g., pilot project) and relying on uncertain or unverified underlying assumptions. In those situations, more robust and adapted analytical techniques should be used to measure the extent to which programs interventions actually contribute to observed results while taking external factors in to account. This is essential to the reliable assessment of program impacts/outcomes.

It is well known in econometrics that incomplete explanatory models lead to biased estimators because the variance that should have been taken by missing variables is automatically redistributed among the retained explanatory variables. Translated for evaluation, this means that excluding external factors from analytics creates a risk of incorrectly crediting the program with some levels of impacts that should instead have been attributed to the missing variables (i.e., having the program claim undue responsibility for observed results).

Dealing with this issue would require collecting appropriate microdata and creating complete data sets, holding information on all explanatory variables for each member of target populations, which can then be used to:

  • Conduct robust multivariate analysis to isolate the influence of program variables (i.e., reliably assessing program effectiveness and cost-effectiveness) while taking all other factors into account;
  • Explore in a limited way (using the resulting regressive model to extrapolate) how adjustments or tailoring of program delivery to specific circumstances could improve program outcomes;
  • Empirically assess delivery standards as predictive indicators of program outcomes (rather than rely exclusively on benchmarking) to determine requisite adjustments to existing program delivery processes.

Developing successful program interventions will require the evaluation function to successfully deal with the above challenges and more effectively support management decision-making processes.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello, my name is Michel Laurendeau, and I am a consultant wishing to share over 40 years of experience in policy development, performance measurement and evaluation of public programs. This is the sixth of seven (7) consecutive AEA365 posts discussing a stepwise approach to integrating performance measurement and evaluation strategies in order to more effectively support results-based management (RBM). In all post discussions, ‘program’ is meant to comprise government policies as well as broader initiatives involving multiple organizations.

This post discusses how to identify comprehensive sets of indicators supporting ongoing performance measurement (i.e. monitoring) and periodic evaluations, from which subsets of indicators can be selected for reporting purposes.

Step 6 – Defining Performance Indicators

When a logic model is developed using the structured approach presented in previous posts, and gets validated by management (and stakeholders), it can be deemed to be an adequate description of the Program Theory of Intervention (PTI). The task of identifying performance indicators then requires determining a comprehensive set of indicators that includes some reliable measure(s) of performance for each output, outcome and external factor covered by the logic model. In some cases, as discussed in the previous post, the set of indicators may extend to cover management issues as well.

Most performance measurement strategies (PMS) and scorecards also require the identification, for each output and outcome, of success criteria such as performance targets and/or standards, which are usually based on some form of benchmarking. This is consistent with a program design mode (i.e. top-down approach to logic models) based on inductive logic where each result is assumed to be a necessary and/or sufficient condition (as discussed in the TOC literature) for achieving the next level of results. This is however very limiting as it reduces the discussion of program improvement and/or success to the exclusive examination of performance in program delivery (as proposed in Deliverology).

Additional useful information that may be required includes the following:

  • Data type (quantitative or qualitative);
  • Data source (source of information for data collection);
  • Frequency of data collection (e.g. ongoing, tied with specific events, or at fixed intervals);
  • Data owner (organization responsible for data collection);
  • Methodology (any addition al information about measurement techniques, transformative calculations, baselines and variable definitions that must be taken into consideration in selecting analytical techniques);
  • Scales (and thresholds) used for assessing and visually presenting performance;
  • Follow-up or corrective actions that should be undertaken based on performance assessments.

Many organizations further require that performance measurement be designed with a view to address evaluation needs and adequately support the periodic evaluation of the relevance, efficiency and effectiveness of program interventions. However, evaluation and performance measurement strategies are most often designed separately, with evaluation strategies usually being arrested just before the actual conduct of evaluation studies. Evaluations are then constrained by the data collected and made available through performance measurement. In order for evaluation and performance measurement strategies to be coordinated and properly integrated, it would be necessary to develop them concomitantly at an early stage of program implementation.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Hello, my name is Michel Laurendeau, and I am a consultant wishing to share over 40 years of experience in policy development, performance measurement and evaluation of public programs. This is the fifth of seven (7) consecutive AEA365 posts discussing a stepwise approach to integrating performance measurement and evaluation strategies in order to more effectively support results-based management (RBM). In all post discussions, ‘program’ is meant to comprise government policies as well as broader initiatives involving multiple organizations.

This post discusses how the relation of performance measurement to results-based management should be articulated and incorporated into logic models.

Step 5 – Including the Management Cycle

Some logic models try to include management as a program activity leading to corporate results (e.g., ‘financial/operational sustainability’ and ‘protection of organization’) that are presented as program outcomes. Indeed, good management can help improve program delivery and thus contribute to program performance. However, that contribution is indirect and normally achieved through the ongoing oversight and control of program delivery (and the occasional revision of program design) with requisite adjustments to operational or strategic plans being informed by the ongoing measurement (or monitoring) and the periodic assessments of program performance (see Figure 5a).

Results-based management (RBM) then depends on the identification of relevant indicators and the availability of valid and reliable data to correctly inform players/stakeholders and adequately support management reporting and decision-making processes. The quality and use of performance measurement systems for governance is actually one of many elements of Management Accountability Frameworks (MAF) in the Canadian Federal Government, with other elements covering expectations regarding stewardship, policy and program development, risk management, citizen-focused service, accountability and people management. However, MAFs are the object of development and an assessment process that is totally separate from the one used for Performance Measurement Frameworks (PMF) based on delivery process models and/or logic models.

Indeed, the management cycle is relatively independent from actual program operations, with management standing in a relation of authority above program staff to provide oversight and control at each step of the delivery process (see Figure 4b in yesterday’s AEA365 post).

Trying to build the management cycle as a chain of results (or as a part of it) in a logic model is then totally inappropriate as it creates unnecessary confusion between management and program performance issues. Presenting the results of good management as program outcomes also blurs the distinction between efficiency (i.e., the internal capacity to deliver) and effectiveness (i.e., program impacts on target populations). Figure 5b below shows how to properly situate the management cycle in a logic model itself, essentially as an authoritative or facilitative process without direct causal links to specific program results.

This does not mean that management issues should be excluded from PMFs. Relevant indicators of management performance should also be identified for monitoring purposes whenever they are identified by management itself as internal factors or risks that do (or may) influence program delivery.

The next AEA365 post will discuss ways of addressing indicators and actual measures of performance.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Hello, my name is Michel Laurendeau, and I am a consultant wishing to share over 40 years of experience in policy development, performance measurement and evaluation of public programs. This is the fourth of seven (7) consecutive AEA365 posts discussing a stepwise approach to integrating performance measurement and evaluation strategies in order to more effectively support results-based management (RBM). In all post discussions, ‘program’ is meant to comprise government policies as well as broader initiatives involving multiple organizations.

This post articulates the approach to the development of delivery process models that ventilate the individual activity-output sequences of a logic model in order to allow for the management oversight and control of program implementation.

Step 4 – Including Delivery Processes

Delivery process modelling can easily be accomplished by adapting a computer-assisted Integrated Definition (IDEF) methods developed in the ‘90s by the National Institute of Standards and Technologies (NIST). The IDEF 0 function modelling was initially based on authority links of controls ensuring that activities of multiple players are coordinated and undertaken only when proper authorities have been issued. In the IDEF 0 model, inputs and support mechanisms are used to perform operations and produce outputs through operations or transformation processes that are subjected to management controls and oversight (see Figure 4a). These nodes can be detailed further (by digging into the steps of each function) and/or sequenced to provide an exhaustive view of all program operations.

The delivery process model presented in Figure 4b is an adaptation of the IDEF 0 approach that is achieved by redefining operations and mechanisms as successive sub-activities supported by various players and stakeholders, with distinct products that are delivered at each step of production and that are subject to direct management authority (i.e. oversight and control). The final step is then the one that actually generates the product (or service) that is being identified as the output of a specific activity in the logic model.

In this modelling approach, inputs are not used to define consumed resources (e.g. funds and human resources), but rather to identify the mechanisms and sources of support for program delivery. Further, since delivery is done in a strict stepwise manner, the strong conditionality between sub-activities can actually be redefined as dependencies. The model also makes it possible to take into account internal and external factors/risks that can have an influence on delivery at each step of production.

The above model is essentially a working tool to help analysts validate their understanding of delivery processes. In performance measurement frameworks, the description of the process models can actually be limited to the sequence of operations and their related products in the narrative of each activity of the logic model, with a view to support the identification of indicators for monitoring purposes.

Figure 4b already suggests how to position the management cycle in relation with program delivery processes, and how best to articulate management issues from a program perspective. The next AEA365 post will address these features in greater detail.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello, my name is Michel Laurendeau, and I am a consultant wishing to share over 40 years of experience in policy development, performance measurement and evaluation of public programs. This is the third of seven (7) consecutive AEA365 posts discussing a stepwise approach to integrating performance measurement and evaluation strategies in order to more effectively support results-based management (RBM). In all post discussions, ‘program’ is meant to comprise government policies as well as broader initiatives involving multiple organizations.

This post articulates how logic models should be structured when program designs include multiple strategic elements (or program activities) supporting a common objective.

Step 3 – Addressing Conditionality

Program interventions rarely rely on a single product or service to achieve intended results. In fact, program strategies are most often designed using multiple interventions from one or more players. In these situations, there normally exists some conditionality between separate program activities as they support and interact with each other. Addressed this way, the notion of conditions (also used in the TOC literature) allows structuring logic models by properly sequencing the multiple program interventions (i.e. converging results chains) deemed to contribute to a common final result that is specific to the program.

To an outside observer being exposed to multiple interventions, program activities may appear to be delivered in a sequential manner (from left to right) based on some observable results (e.g., outputs or immediate outcomes) until some final outcome is achieved (see Figure 3a). This would be the case of a person arriving at a hospital emergency or an employment centre and being subjected to a series of treatments or services.

However, from a program perspective, all activities are actually implemented in parallel with different clients and/or players. In the examples of the hospital emergency and the employment centre, it is the clients who are moving from left to right across activities as they are exposed to various program services. In programs that reach clients only indirectly (e.g., environmental programs or economic policies), it is rather the projects or client files that are shifting across activities while being processed and/or subjected to various program interventions.

Conditionality then allows taking into account the relationships between the strategic elements (or activities) of program interventions without the need to clutter the logic model with an exhaustive mapping and display of all possible interactions and feedback processes. Implicitly, all program activities are (or may be) influenced to some extent by previous activities situated at the left of the diagram (see Figure 3b). Thus, when conditionality exists and is properly taken into consideration, the positioning of program activities in the logic model becomes important for the description and understanding of the program theory of intervention (PTI).

The next AEA365 post will dwell further into program implementation and discuss how to best integrate delivery processes into logic models in order to effectively support management oversight and control.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Older posts >>

Archives

To top