The Annie E. Casey Foundation's EE Journey to Date

The Casey Foundation’s research and evaluation staff have embarked on a journey to elevate equitable evaluation in their work.  The seeds of this journey began with the implementation of Casey’s pipeline evaluation training program for researchers from historically underrepresented racial and ethnic groups – Leaders in Equitable Evaluation and Diversity (LEEAD).  Launched in 2015, the LEEAD program includes a semester of online-based evaluation coursework with an emphasis on equitable evaluation approaches, ongoing mentoring and a remote evaluation residency placement at a research organization, think tank, foundation or private firm.  

More recently, the research and evaluation staff launched a process to embed equitable evaluation approaches in planning, grantmaking, and execution of evaluation projects.  The first step involved taking an inventory of the equitable evaluation approaches currently utilized in Casey’s evaluation investments. The Equitable Evaluation Framing Paper provided valuable insights, and informed the development of categories and definitions of equitable evaluation approaches.  The results of the inventory revealed the quantity and depth of equitable evaluation approaches across different categories.

Findings from the assessment revealed that nearly all the 31 projects inventoried included team members who were evaluators of color.  

Commonly used approaches included:

  • The analysis of structural and systems-level drivers of inequity and
  • The analysis of racial and ethnic disparities, typically by employing disaggregated data.

Less common approaches included:

  • Employing culturally appropriate methods,
  • Ensuring the community has a role in shaping and owning research/evaluation, and
  • Applying an equity lens in the dissemination of findings.  

In the next phase, research and evaluation staff met to discuss how they define and use the approaches.  Staff reflected on the inventory by responding to a series of probing questions such as: Do you rely on certain approaches more than others?  Are there certain situations where you find it more advisable to use some approaches? What are the benefits and challenges? What is missing from the categories and definitions?  

Highlights of the discussion include:

  • The racial and ethnic diversity of project teams is common, particularly in small firms. Staff can build on these efforts by articulating their values around diversity and clarifying what it means to have diversity of lived experience and a team that is representative of a community.
  • Culturally appropriate methods emerged as an area staff most want to learn more about. In particular, the team expressed interest in exploring ways evaluation teams can employ less traditional methods that respect culture and draw on the strengths of the community.
  • The community’s role in shaping and owning research/evaluation can take many forms. It is important to define up front what is meant by “community” and “community members,” particularly in system-level evaluations.
  • Although data disaggregation by race and ethnicity is common, there is room to build on this approach (e.g., by examining intersections with age, sexual orientation and gender identity).

In the future, research and evaluation staff will further discuss capacity building needs to advance this work in their grantmaking as well as to increase awareness within the Foundation of equitable evaluation and the value of integrating equitable evaluation approaches in the work.


What Would It Take to Align Foundation Mission and Evaluation Practice?

We are grateful for Dr. Chera Reid and The Kresge Foundation's leadership around equitable evaluation. See her recent blog post on GEO's Perspectives, "What Would it Take to Align Foundation Mission and Evaluation Practice?" Dr. Reid shares her journey towards this work and how she is bringing an EE perspective to Kresge.

At the upcoming GEO Conference, the EE team is co-facilitating a pre-session workshop with Dr. Reid entitled, "Confronting Orthodoxies for Equitable Evaluation Practice". We will also share early insights and updates on who else is taking up EE. We look forward to seeing some of you there!

EE is Getting Around

It is affirming and exciting to see who is self identifying with the emerging EE principles and exploring what is next. In the future the voices and experiences of these early adopters/curious folks will be part of our communications but for now here is where we have been and who has been in the “room:”

  • LeftCoast Evaluators, San Francisco (3/26) - 30+ evaluation staff from foundations with a presence on the west coast heard about the opportunity EE offers.
  • NNCG Webinar (3/27) - 50+ consultants from across the country participated in a 1-hour webinar as part of the DEI Webinar series.
  • Colorado Webinar (3/30) - Sponsored by the Colorado Trust and the Colorado Health Foundation and as part of professional development offered by COEN (the AEA regional), 125+ consultants/evaluators from non profits, foundations, research and independent firms participated in a webinar on EE. It was the first step in a longer term engagement including 2 design labs for evaluators to identify how they will evolve practice over the summer to advance EE principles.
  • Walton Family Foundation, Strategic Learning and Evaluation Department (SLED) Exploratory Session (4/3) - As part of staff retreat, SLED spent a few hours learning about EE including its origins, findings from the EE Project and reflection on the EE principles and orthodoxies.

Moving From an Idea to an Initiative

We know from history that change is challenging and evaluation practice in philanthropy will be no different.  For that reason, the EE Team is dedicating time to develop an Impact, Influence, Leverage, and Learning agenda (thanks ORS Impact) which will help us determine the who, the how and the what which will advance this work. A new website live in the Spring will be an element of a more robust communication strategy. Generous funding from Ford and CalEndowment is resourcing EE to move from an idea to a multi-year initiative.

EE: A Reason to be Optimistic

We likely all agree - 2017 was a challenging year for many reasons, but we were delighted to learn that the National Center for Responsible Philanthropy (NCRP) thinks that Equitable Evaluation was one reason for optimism.  NCRP highlighted EE in its “Speaking Truth to Power” section, describing we are taking on the “sacred cows of research and evaluation to challenge how foundations measure performance and impact.” It’s true and we have our work cut out for us, but we are up for the challenge.

"So What?" Aspen Institute Blog

It’s great to see other people talking about Equitable Evaluation. In the Aspen Institute’s December 17 bi-weekly “So What?” they offered a recap of an end of year conversation around effectiveness and who defines it with perspectives from Vu Lee and Kathleen Enright.  The entry ended with “we salute the like-minded funders and evaluators teaming up to define and promote Equitable Evaluation.” We agree.

Our Work is Growing

We are THRILLED to share that the journey towards Equitable Evaluation will continue in 2018 with new and returning funding partners and a growing team of practice partners. 

The timing of this conversation and the implications for evaluation practice is resonating with foundations, consultants, evaluators, and funded partners reflecting various geographic and issue areas of focus, sizes, and ages of maturity. In early 2018 we will share more leaving you in suspense for the remainder of the year.  

On behalf of the team and even more importantly, for the world we all hope we get to live in.  Thank you!

Equitable Evaluation and Creative Measurement

In Fall 2017, the Creative Measurement Lab (CML) pilot at Arizona State University launched. At the request of Dr. Maria R Jackson who leads the CML, Dr. Chera Reid of The Kresge Foundation provided an overview of Equitable Evaluation.

The CML Summit hosted a wide variety of participants -- national institutions, researchers, students, local community development organizations, and community organizers interested in comprehensive place-based strategies in vulnerable communities. A participant shared “I believe it gave a really important framing to something we were all hungry for, and contributed to a space in which folks were not only getting creative about the ways we might measure community expression of things like agency, narrative of place, social cohesion, etc., but also about the ways we might creatively and productively turn our gaze on systems and institutions that impact community planning and development.”

These are exactly the types of opportunities and experiences we want to hear about and influence. We look forward to learning from ASU.


Portland State University and EE

Last fall after the Roundtable, The Nonprofit Institute at Portland State University reached out to the EE Team wanting to highlight the work to date and our aspirations in their blog The Nonprofit Nerd. Of course, we said yes. Read the full interview here.

We are exploring what might be possible with the Nonprofit Institute because we know that an integral partner in moving towards EE are non profit leaders and the communities they represent and serve.


Race and Evaluation - A Difficult Conversation

We have been to CREA since its inaugural convening in 2013. We attend because it has been a place where we evaluation is situated within context and culture. There is vibrant and honest conversation about colonialism, imperialism and how that has affected people, communities, systems and countries. Values are front and center.  

The draw for us this year was Dr. Ernie House whose paper Evaluation and The Framing of Race in AJE earlier this year was a provocative contribution to the conversation about race and evaluation. If you have not read it yet, do so. This conversation is in its early stages and will continue at this year’s American Evaluation Association annual conference, “Continuing the Dialogue Evaluation’s Call to Action, 21st Century Perspectives in Addressing Race.”* Our team member Jara Dean-Coffey will be on that panel.

This is all new territory. It will take time and patience to get it right but at least efforts are being made to bring it to the forefront.


* To see the panel abstract, you'll need to use the search function. For example, type "Race" into the Title area, or "Jara" into the Presenter area. 

Finding our Tribe

In September, the EE Team got around. We spread the word about equitable evaluation in multiple venues all of which were primarily focused on philanthropic practice and purpose. In general, people were intrigued, inspired, and interested in learning more and curious about how to seed the conversation in their organizations. We think we might be onto something.

Here is a quick overview:

New Grantmakers Institute - Introduced EE to a group of 70+ new grantmakers as one of the big ideas influencing strategy and evaluation in foundations.

Unity Summit  - Attended by primarily philanthropic serving organizations and their funding partners, EE was introduced as part of a panel on how consultants can support DEI efforts.

Kresge Foundation - We had the opportunity to introduce EE to the Program Team as a part of a launch of a larger effort with Kresge around this work. More on that in a later post.


CREA 2017 - Evaluation and the Framing of Race

If you don’t know about CREA, you should. The Culturally Responsive Evaluation and Assessment conference hosted by the University of Illinois at Urbana-Champaign is where you go when you want to center evaluation and assessment in culture and context. EE team members have never missed a conference and the next one is September 27-29 in Chicago, IL.

This year Ernest R. House, PhD will be speaking to and about his paper “Evaluation and the Framing of Race.” It’s a thought provoking and personally revelatory piece that explains how racial framing influences evaluation practice.

Additionally the 3rd American Evaluation Association Race and Class Dialogue will take place during the conference. This series hosted by AEA is a series of national dialogues designed to reflect and promote positive actions on the deeply rooted, and structurally intertwined issues behind the headlines that propel racial, ethnic, and class disparities in our society.

EE Inaugural Roundtable - 30+ Gathered on August 1st

On August 1st, 30+ learning and evaluation staff from foundations and consultants from across the country came together in Detroit at The Johnson Center to explore three paradigm shifting questions:

  1. How can our evaluation mindset and practices better reflect and advance the principles and values of equity?
  2. What does it take for foundations to build their organizational capacity, will, and wherewithal to engage in equitable evaluation?
  3. What can we do at a field level to ensure that more evaluators are prepared to practice equitable evaluation?

Starting with reviewing the 3 Equitable Evaluation Principles and unpacking the 9 Orthodoxies we quickly moved to naming additional orthodoxies before we shifted to identifying strategies to undo habits preventing equitable evaluation. The group was engaged and eager to begin to move practice forward in their own foundations/places of work as well as how to elevate this conversation in philanthropy more broadly.

See our working documents:


Forum 2017 - EE in the House

The EE Team was invited as a guest discussant for “Inside Out: Doing Race, Equity, Diversity, and Inclusion Work Internally” session moderated by Edward Jones from ABFE and focusing on the research by Putnam Consulting Group (see full paper or brief).  A key finding was that foundations are interrogating the ways in which equity is influencing practice, internally and externally. However, as Jara Dean-Coffey shared with the audience both the “The Road to Achieving Equity: Findings and Lessons from a Field Scan of Foundations That Are Embracing Equity and the EE Project Framing Paper” revealed that “ few if any are looking at the ways in which equity is influencing the framing, purpose and practice of evaluation.”  The audience reaction was primarily that of unrealized opportunity which suggests there is a readiness and willingness to deepen this work.

See the Agenda here.

Equitable Evaluation Framing Paper

We are excited to share our Framing Paper to set the stage for further conversation around equity in philanthropy and evaluation. It reflects briefly on the current “state of practice” of equity in both philanthropy and evaluation, and poses questions to consider. The content is based on a combination of a year-long research process that included a thorough review of philanthropic and evaluation literature, environmental scanning of equity work in philanthropy, and interviews with foundations and evaluators, as well as our own experiences in the field and conversations with other practitioners.


GEO 2017 Learning Conference: Incorporating Equitable Evaluation in Your Practice

In a packed session, a group of 75 (standing room) foundation staff delved into the challenges of evaluation programs focused on promoting equity, explored how evaluation practices can advance racial equity goals, and discussed how equitable evaluation related to their own work. It was a lively discussion with quite a bit of activity on Twitter suggesting that the core principles of Equitable Evaluation resonated strongly. This was evident in being invited to submit a session for GEO 2018.

See the session materials here:

The session was led by Teresa Behrens, Dorothy A. Johnson Center for Philanthropy, Jara Dean-Coffey, Luminare Group and Tom Kelly, Hawai’i Community Foundation.


Incorporating Equitable Evaluation in Your Practice, Grantmakers for Effective Organizations Learning Conference

Dr. Teri Behrens, Jara Dean-Coffey, and Tom Kelly will facilitate a world cafe session delving into the challenges of evaluating programs focused on promoting equity, exploring how evaluation practices can advance racial equity goals, and how equitable evaluation relates to foundation work.

Learn more: