Open Problem 03

Measuring the Source

How do you measure the impact of source-level work, such as changing a story, a norm or an institution, when the effects are diffuse, shared and delayed?

Status: open
What would count as progress

A measurement approach validated on at least one historical source-level change, recovering an effect that independent historians agree took place.

Why it matters

If funders require measurements that source work cannot provide, useful work may be missed. That is one possible mechanism behind the Symptom Trap hypothesis. Hard-to-measure is not the same as small.

Where to start

Work backwards from changes everyone agrees happened, such as the spread of oral rehydration therapy or the end of smallpox (see Historical Case Studies), and test which measures would have detected them early. Contribution analysis, process tracing and forecasting tournaments are candidate methods.