Why people believe it
From the outside, charities look alike. They tell similar stories, publish similar annual reports and report similar overhead ratios. Results, when they are published at all, are rarely in a form you could compare. So it is natural to assume the differences are modest. In studies by Lucius Caviola and colleagues, members of the public guessed that the most effective charities helping the world’s poorest people were about 1.5 times as effective as the average one. Experts put the gap at about 100 times.
What’s true
Effectiveness depends on more than the cause. The same program can be run well or badly, in a place where it fits or one where it doesn’t. The estimates are uncertain, too. Disease Control Priorities in Developing Countries, known as DCP2, is the most comprehensive compendium of health cost-effectiveness estimates, in the philosopher Toby Ord’s description. In 2011 GiveWell found errors in its estimate for deworming that overstated cost-effectiveness by a factor of about 100. And some kinds of good are very hard to compare at all, such as a museum and a vaccine, because they are measured in different units.
What’s false
That the gaps are small. In a 2013 essay, Ord charted DCP2’s estimates for 108 health interventions. They spanned more than four orders of magnitude, from 0.02 to 300 years of healthy life gained per $1,000, with a median of 5. Moving money from the least to the most effective would produce about 15,000 times the benefit, and moving it from the median to the best about 60 times. The estimates are rough, but Ord shows that even if the best were ten times worse and the worst ten times better than estimated, a factor of 150 would remain.
The gaps are between interventions, not named charities, and Charitology doesn’t rate charities. But a charity can only be as effective as what it does. In Caviola’s experiments, telling people how large the differences are moved their donations toward the most effective option.
What to ask instead
What does this charity achieve per dollar, compared with other ways of reaching the same goal, and how does it know? See cost-effectiveness analysis, the Giver’s Checklist, and for comparisons across causes, Open Problem 2, the Exchange Rate Problem.