What happened
In November 2005, at the World Summit on the Information Society in Tunis, United Nations Secretary-General Kofi Annan unveiled the first working prototype of a “$100 laptop,” alongside Nicholas Negroponte, cofounder of the MIT Media Lab. The machine came from One Laptop per Child, a nonprofit created by Media Lab faculty to give schoolchildren and their teachers a personal, connected computer. It was rugged, ran on very little power and could share one internet connection across many machines. Governments would buy the laptops, and schools would hand them out, one per child.
Governments did buy. Peru ran the largest deployment, with a commonly reported 902,000 laptops. Uruguay’s Plan Ceibal was, according to the economists who studied it, the only deployment to cover a whole country.
Then came the evaluations. In Peru, a team led by researchers at the Inter-American Development Bank ran a randomized evaluation in 318 rural primary schools. After 15 months, treatment schools had gone from about one computer for every eight students to more than one each, and children used computers far more at school and at home. But the study found no effect on math or language test scores, and only inconclusive evidence of gains in general thinking skills. In Uruguay, economists found the program had no effect on math or reading scores in its first two years, possibly because students mainly used the laptops in class to search the internet.
Reactions split on what the results meant. Michael Trucano of the World Bank called the habit of handing out hardware first “classic worst practice,” and also asked whether standardized tests were measuring the right things.
Where it sits in the system
The program aimed at structure (level 3) on the depth scale: changing what every school gives every child. But a new input changes the structure of learning only if it changes what produces learning, such as what children do in class and how teachers teach. The evaluations suggest that, on its own, it didn’t.
This is the Measurement Gap in its classic form: outputs and outcomes are different things. Laptops delivered were the output. Learning was the outcome. A bold, simple promise, unveiled at a world summit, made the output easy to sell (Story Bias). The randomized evaluation during Peru’s rollout is the reason anyone knows the difference.
What carries over
Write down the theory of change, the chain from what you deliver to the change you want, and test its weakest link at small scale before buying for a whole country. Decide in advance which outcomes count, so a disappointing result can’t be explained away and a promising one isn’t missed.
Lessons for the system
- Delivering a thing is not the same as delivering what the thing is for. The laptops reached children, and better test scores did not follow.
- Test the theory of change before scaling it. The big evaluations reported after governments had already bought laptops for whole school systems.
- What you measure shapes what you learn. Peru’s evaluation found no effect on test scores and inconclusive gains in general thinking skills, and observers disagreed about which outcomes mattered most.
- Randomizing during a national rollout is what made these lessons knowable. That choice deserves as much attention as the results.