---
title: The Measurement Gap
number: 7
summary: Counting activity does not establish what changed because of it. Measuring that change can require resources and methods beyond routine reporting.
evidence: debated
node: outcomes
flip: Measuring what changed is paid for as part of the work, and small charities share measurement instead of each building their own.
sources:
  - title: The Prize in Economic Sciences 2019, press release
    publisher: The Royal Swedish Academy of Sciences
    year: 2019
    url: https://www.kva.se/en/news/ekonomipriset-2019-2/
  - title: Understanding and misunderstanding randomized controlled trials
    publisher: Angus Deaton and Nancy Cartwright, Social Science & Medicine 210
    year: 2018
    url: https://doi.org/10.1016/j.socscimed.2017.12.005
---

## What it looks like

A hypothetical charity reports 10,000 meals served, 500 students tutored and 2,000 nets handed out. Those are *outputs*. Whether anyone is healthier, better educated or less hungry a year later is a different and harder question. Those are *outcomes*.

## Why it happens

Counting activity can be simpler than estimating its effects. Establishing causal impact may require comparison groups, follow-up surveys or specialist help. Routine outcome monitoring does not always need a trial. The 2019 Nobel Prize in economics went to Abhijit Banerjee, Esther Duflo and Michael Kremer for an experimental approach to measuring what anti-poverty programs really change. Their work shows how much careful measurement can reveal, and how much was unknown before.

## Why it’s debated

Rigorous evaluation isn’t free, and not every program can or should run a randomized trial. Angus Deaton and Nancy Cartwright argue that trial results don’t carry over between places and populations as easily as people assume. Charitology takes both sides seriously: measure what matters, with methods proportionate to the decision being made. The sources here do not establish how large the measurement gap is across the sector.

## No one is to blame

A small team may need partners or shared services to evaluate its work. Limited evaluation capacity does not establish negligence.

## What we don’t know yet

What share of headline impact findings replicate? See [Open Problem 10, the Replication Rate](/open-problems/the-replication-rate/), and for source-level work, which is hardest of all to measure, [Open Problem 3, Measuring the Source](/open-problems/measuring-the-source/).