Reflecting on your own practice is difficult, but it is as valuable, if not more so, than a "product"
This is an EEI Emerging Learning and all projects are works in progress. Subscribe to our newsletter for the latest updates on this and other EEI projects.
The Oregon Community Foundation - EE Coaching
By: Kim Leonard, Senior Research Officer & Sonia Worcel, VP for Strategy and Research at The Oregon Community Foundation
Brief background of project
We are exploring how equitable evaluation can be applied to our work in the Research Department at The Oregon Community Foundation (OCF).
To do so, Research Department team members are reading about various aspects of equitable evaluation, reflecting on existing evaluation activities, and engaging in quarterly coaching calls with Jara Dean-Coffey. Through this work we are developing a shared understanding across our team that will influence evaluative practice within and beyond the foundation.
Status of project as of writing
Though we’ve been thinking about the relationship between equity and evaluation for some time, we feel like we are in the very early stages of working towards equitable evaluation as a way of practicing. Right now this looks like learning about what equitable evaluation can look like—lots of reading and thinking about things like “what does multi-culturally valid” mean and look like? And getting comfortable with definitions not being precise or contained.
It also looks like reflecting on pieces of our work—a single component of an evaluation at a time. We’re now intentionally reflecting on those components that are ongoing, where there is still opportunity to shift or reconsider what we’re doing, because that helped us feel like we could take action in response to our reflections. To do this we’re using a simple reflection tool designed by EEI staff that asks us to describe how each principle shows up, or doesn’t, in a given evaluation component.
How The Oregon Community Foundation is making the case for EE within their organization
The mission of The Oregon Community Foundation is to improve lives for all Oregonians through the power of philanthropy. The “all” in the above statement has come to be particularly important as the foundation works to focus and connect its work to address the opportunity gap in Oregon, in conjunction with a report completed by the Research Department in 2017: https://www.oregoncf.org/Templates/media/files/research/top_report_2017.pdf.
OCF’s commitment to equity, diversity, and inclusion (EDI) was made official in 2014, though foundation staff and leaders have worked actively toward EDI for much longer. You can read about our commitment to EDI here: https://www.oregoncf.org/explore-ocf/about-us/commitment-to-edi.
Foundation-wide, this work is and will always be ongoing. Staff and leaders are putting in hard work as individuals and there are examples of practice being pushed throughout the foundation (e.g., efforts to do better outreach to particular communities). However, it often feels as though there is more appetite for action than action itself. It is hard work to let go of the way we’ve always done things.
The Research Department itself and its internal evaluation efforts are relatively new. We feel open to being challenged, even when that is uncomfortable, and often have at least some control over the breadth, focus, and depth of our efforts. Ultimately, we strive to help the foundation and its donors be more responsive and effective, so infusing equitable evaluation principles feels like a very appropriate fit.
WHICH EE PRINCiple helped make the case for ee
For us, the three EE principles seem so interwoven they are hard to even think about separately! Jara’s early advice to think of them as mission, strategy, and methods respectively has been very helpful in thinking about how they connect and can play out in a given project.
which orthodoxies The oregon community foundation can (or should) push against to advance ee
We began our exploration of the equitable evaluation framework with discussion about the orthodoxies, and how they show up in our work. We were happy to identify several ways we were already pushing back against these -- we certainly don’t claim to be objective, and we most often use qualitative, non-experimental methods, for example.
Two are particularly ripe for us to continue to push against—“the foundation defines what success looks like” and “the foundation is the primary user of evaluation.” Though OCF has established goals for its various initiatives, we can better incorporate, measure, and reflect the ways that our grantees define success, deferring to those definitions wherever possible. While we aim to make our evaluations useful to those working in the field, and not just those inside the foundation, there is much we can do to further share and promote use of what we’re learning. This all goes hand-in-hand with seeing our grantees as THE experts in their own work and communities.
Insights, considerations or cautions you have for the eei audience
For philanthropy and researchers/evaluators, though these likely apply to anyone:
Because our work is collaborative, we find that reflecting on the principles is most useful when done as a team. The conversation we have is at least as valuable, if not more valuable, than the “product” of that reflection (notes about possible next steps).
Reflecting on your own practice is difficult, and it should be. We can’t take credit for this insight, which was shared by one of our grantees (Marna Stalcup of the Regional Arts & Culture Coalition) during a meeting where we debriefed a grantee reflection process we’d just piloted. This is just one of many ways that we’ve realized how hard we ask our grantees to work at times!
Reflecting on our work is helping us notice and challenge assumptions about what information is useful and how we should gather or report on it. We are asking “to what end” or “why” much more often. At times, this means NOT doing something we thought we wanted to (e.g., collect this or that piece of data).
And we’re also grappling with the tension between what is in our control—or part of our charge—and what is not and what that means in terms of the limitations of what we are doing as a team. We are realizing that there is only so much we can do without bringing others—including key program officers and foundation leaders—into our conversations; they have to be ready and willing to take action too, otherwise our efforts won’t be reinforced on the programmatic side at the foundation.