The Annie E. Casey Foundation's EE Journey to Date
The Casey Foundation’s research and evaluation staff have embarked on a journey to elevate equitable evaluation in their work. The seeds of this journey began with the implementation of Casey’s pipeline evaluation training program for researchers from historically underrepresented racial and ethnic groups – Leaders in Equitable Evaluation and Diversity (LEEAD). Launched in 2015, the LEEAD program includes a semester of online-based evaluation coursework with an emphasis on equitable evaluation approaches, ongoing mentoring, and a remote evaluation residency placement at a research organization, think tank, foundation, or private firm.
More recently, the research and evaluation staff launched a process to embed equitable evaluation approaches in planning, grantmaking, and execution of evaluation projects. The first step involved taking an inventory of the equitable evaluation approaches currently utilized in Casey’s evaluation investments. The Equitable Evaluation Framing Paper provided valuable insights, and informed the development of categories and definitions of equitable evaluation approaches. The results of the inventory revealed the quantity and depth of equitable evaluation approaches across different categories.
Findings from the assessment revealed that nearly all the 31 projects inventoried included team members who were evaluators of color.
Commonly used approaches included:
The analysis of structural and systems-level drivers of inequity and
The analysis of racial and ethnic disparities, typically by employing disaggregated data.
Less common approaches included:
Employing culturally appropriate methods,
Ensuring the community has a role in shaping and owning research/evaluation, and
Applying an equity lens in the dissemination of findings.
In the next phase, research and evaluation staff met to discuss how they define and use the approaches. Staff reflected on the inventory by responding to a series of probing questions such as: Do you rely on certain approaches more than others? Are there certain situations where you find it more advisable to use some approaches? What are the benefits and challenges? What is missing from the categories and definitions?
Highlights of the discussion include:
The racial and ethnic diversity of project teams is common, particularly in small firms. Staff can build on these efforts by articulating their values around diversity and clarifying what it means to have diversity of lived experience and a team that is representative of a community.
Culturally appropriate methods emerged as an area staff most want to learn more about. In particular, the team expressed interest in exploring ways evaluation teams can employ less traditional methods that respect culture and draw on the strengths of the community.
The community’s role in shaping and owning research/evaluation can take many forms. It is important to define up front what is meant by “community” and “community members,” particularly in system-level evaluations.
Although data disaggregation by race and ethnicity is common, there is room to build on this approach (e.g., by examining intersections with age, sexual orientation, and gender identity).
In the future, research and evaluation staff will further discuss capacity building needs to advance this work in their grantmaking as well as to increase awareness within the Foundation of equitable evaluation and the value of integrating equitable evaluation approaches in the work.