Why people believe it
Much of what charity offers feels beyond numbers: dignity, hope, a sense that someone cares. And charities often report activities, such as meals served or classes held, rather than what changed afterward, so it can look as if change itself can’t be counted. Measuring change well costs money, and few funders pay for it.
What’s true
Some of the most important work is genuinely hard to measure. Advocacy, research, shifts in culture and prevention, where success is a disaster that never happens, rarely fit a simple before-and-after count. Measurement has limits even where it works. Angus Deaton and Nancy Cartwright argue that a randomized trial’s result applies to the people in the trial, and extending it to other places takes further evidence and reasoning.
What’s misleading
The claim that the good can’t be measured at all. The 2019 Nobel Prize in economics went to Abhijit Banerjee, Esther Duflo and Michael Kremer “for their experimental approach to alleviating global poverty.” Their method breaks big questions into smaller ones that careful experiments can answer, and the Royal Swedish Academy of Sciences notes that one of their studies led to remedial tutoring programs that have reached more than five million children in India. Researchers affiliated with J-PAL, a global research center based at MIT, have led more than 2,700 randomized evaluations, and J-PAL says programs informed by them have reached more than 850 million people.
The results can be concrete. A randomized trial of unconditional cash transfers in rural Kenya found that monthly household consumption rose from about $158 to $193, adjusted for local prices, nine months after the transfers began, along with large gains in psychological well-being. Measurement also catches harm that good intentions miss, as in the case of Scared Straight.
What to ask instead
What has this charity measured, how, and what result would tell it that it is failing? Look for outcomes, not just outputs. For the hardest cases, see Open Problem 3, Measuring the Source.