EEF Expansion: Elements of the EEF—Orthodoxies

Orthodoxies are tightly held beliefs to be questioned/challenged. They can undermine the Equitable Evaluation Framework™ Principles.

Over time, the philanthropic sector developed a set of “orthodoxies,” or tightly held beliefs, about evaluative practice.* Orthodoxies are often invisible and unspoken, masquerading as “common sense.” They are believed to be foundational and affect the undercurrents of organizational culture. They are shaped by the actors in the philanthropic ecosystem.

Many of the Orthodoxies act like a drag on any evaluation effort and, even more so, on those efforts related to equity. In some cases, they reinforce inequities. They reflect a mix of capitalism and white dominant framing. As evidenced in practice and praxis, these get in the way of advancing the EEF Principles.

In reflecting on these Orthodoxies, six areas surfaced. These areas are expressed and experienced differently based on the actor in the philanthropic ecosystem: foundations, consultants, nonprofits & public/government agencies, and philanthropic serving organizations (PSOs).

Objectivity, Rigor, Evidence

• Evaluators are objective.

• Credible evidence comes from quantitative data and experimental research.

• Grantees and strategies are the focus of the evaluation, but not the foundation.

• Objectivity is necessary for consultant credibility.

• There are specific methods and tools for evaluation that center equity.

• Stories are the best evidence of our impact. Numbers strip away the complexity and humanity of what we do.

• Programs are the focus of evaluation, not the organization.

• Credible evidence comes from program attendance.

Resources: Money, Time, People

• Evaluation funding primarily goes to data, collection, analysis, and reporting.

• Evaluation resources primarily support data collection, analysis, and reporting.

• Centering equity requires resources (time, money, humanity, etc.) often not tended to sufficiently in evaluation.

• Resources go to into evaluation, but not much comes out of it

• Evaluating the real impact of our work is beyond our capacity and no one wants to fund it.

• Evaluation requires money, expertise, and time we don't have.

• We are competing for funding with community-based organizations (scarcity mindset).

Expectations, Roles

• Evaluators are the experts and final arbiters, grantees are beneficiaries.

• Evaluators should be selected based on credentials that reflect conventional notions of expertise.

• Evaluations should provide generalizable lessons.

• Consultants must specialize in certain areas or approaches to remain competitive.

• The foundation holds all of the power and decision-making authority.

• The responsibility to support consulting practices that centers equity rests on consultants alone.

• Evaluation is something we have to do to obtain or maintain funding.

• What people are willing to pay for and show up for is what matters most.

• We are both gatekeepers and grantees and need to keep both in mind.

• We are so far removed from impact that we can't attribute change to our work.

Definitions, Decisions, Perceptions

• The foundation defines what “success” looks like.

• Evaluation is in service of foundation brand.

• The foundation is the primary user of evaluation.

• Consultants accept/perpetuate definitions and norms that pit rigor against equity rather than understanding how they are intertwined.

• Consultants are often motivated by the desire to help/save communities.

• Consultants conceal personal and/or business values to be successful.

• Evaluation is a gamble, and we could lose.

• Evaluation is expensive, time consuming, and takes away from our mission work.

• We could influence what success looks like, but could also perpetuate foundation truth.

• Conferences and publications change the field.

Relationships, Trust

• Trust/relationships come from doing the work, but are not the starting point.

• Relationships are secondary to the technical responsibilities that consultants hold.

• We shouldn't work together.

Productivity, Accountability

• Timeframes/short-term outcomes serve as indicators of good stewardship.

• The ways in which business models are often exercised in capitalism are at odds with knowledge sharing, co-creation, and collaboration.

• Funder satisfaction matters.

• Accountability is a one-sided set of expectations rooted in compliance, generally expressed as foundation sets expectations and contractual obligations of grant partners, consultants, etc.

• The fast-paced production of deliverables demonstrates “impact” and “outcome.”