Rapid-Cycle Evaluation Project

This is an EEI Emerging Learning and all projects are works in progress. Subscribe to our newsletter for the latest updates on this and other EEI projects.


Colorado health foundation - EEI Design Labs

By: Nick Stuber, Kelci Price, Yen Chau

BRIEF BACKGROUND OF PROJECT

A rapid cycle evaluation of the Foundation’s new funding opportunity focused on supporting youth resiliency in out-of-school settings. We are working with our evaluation partner to deliberately explore how to bring an equity lens to the evaluation. 

STATUS OF PROJECT AS OF WRITING

Underway, in its early stages. We are exploring with our evaluation partner how to collect data and develop conversations that will help us understand how the work is impacting different groups of youth who are experiencing health inequities. As part of the evaluation, we are interviewing organizations in Colorado that are focused on helping youth develop coping skills and resilience. We have been intentional about including not only larger organizations who clearly identify themselves as doing this kind of work, but also seeking out smaller, grassroots organizations who are doing this work, but who don’t describe it as building resilience/coping skills.

HOW COLORADO HEALTH FOUNDATION IS MAKING THE CASE FOR EE WITHIN THEIR ORGANIZATION

Racial equity work is ongoing in the organization. The Foundation’s new cornerstones challenged the Learning & Evaluation team to put equity at the center of how we design and carry out our work as a team. The Foundation’s equity focus has been led by our CEO and other executives who see equity work as core to where we are moving as an organization. Having the support of our executives made it fairly easy to introduce and integrate the EE framework. In addition, all staff are participating in the Foundation’s Diversity, Equity and Inclusion (DEI) work. This created an organization-wide openness where we could share and explore the EE work with colleagues as part of our conversations and reflection of how we are applying DEI to the Foundation as a whole.

WHICH EE PRINCIPLE HELPED MAKE THE CASE FOR EE

The second principle that highlighted the importance of answering questions about the effective of a strategy on different populations was most prominent in our thinking about the rapid cycle evaluation work. It aligned with questions we had about the populations identified as the focus of the work, and questions about how the programs/interventions had been adapted to the needs of those groups (i.e., cultural responsiveness). We’ll be exploring application of the third principle as we move into the next phase of the evaluation and engage directly with youth.

WHICH ORTHODOXIES COLORADO HEALTH FOUNDATION CAN (OR SHOULD) PUSH AGAINST TO ADVANCE EE

We have been exploring our thinking about a number of orthodoxies in this evaluation project. The ones that are top of mind are: evaluations providing generalizable lessons (#4), credible evidence (#7), and evaluators as objective (#8). We have strongly rooted our thinking around this evaluation as informing us about our early strategic steps drawing on the perspectives and experiences of experts, practitioners, and the youth that are served. We are actively seeking to draw on the perspectives of those who engage in this work, and especially understand the multiplicity of perspectives around what youth need from this type of programming. As we move forward with the second phase of the evaluation, we will be deeply exploring the issue of the key users of evaluation (#3), and exploring with our grantee partners issues around how the evaluation is designed and used, and by whom.

INSIGHTS, CONSIDERATIONS OR CAUTIONS YOU HAVE FOR THE EEI AUDIENCE

One tension we’re still exploring is around the orthodoxy related to the Foundation as the primary user of evaluation. As a Foundation we want to get better at what we do because we believe that will help us have greater impact. To do that, we need information about the way we’re enacting our work and what impact that is having - but some of those questions are very Foundation-centric and neither useful nor meaningful to our grantees. So we wrestle with questions of how we use the evaluation to answer questions that are important for our own progress, and what part of the evaluation we should share with our partners so they can address their own questions. We try to think about how the evaluation can help answer the questions of several sets of actors in the ecosystem - including the Foundation - because we all have a need for evidence that will help us improve our practice so that we can best serve community.