--- title: Replication Program group: Research order: 6 summary: Preserve replication plans, describe versioned artifacts and connect sourced, self-recorded results to their exact protocol and deviations. status: live statusNote: Local protocols, artifact manifests, results and descriptive comparisons can be saved and exported. Browser timestamps and checksums are not public registration or verified execution; studies, safeguards and independent replications still need people and resources. conditions: [silence-about-failure, measurement-gap] firstMilestone: A list of 20 influential charity impact findings, chosen by published rules, with a replication plan for each registered before any results are known. help: - Open an issue labeled program proposing a study to re-test, with a link to its data if the data is public. - Re-run the analysis of a study whose data and code are public, and publish what you find, whatever it is. - Critique the rules for choosing which studies to re-test. related: [/open-problems/the-replication-rate/, /programs/evidence-reviews/, /programs/failure-lab/, /record/evidence-library/] sources: - title: Estimating the reproducibility of psychological science publisher: Open Science Collaboration, Science 349(6251) year: 2015 url: https://europepmc.org/article/MED/26315443 - title: 'Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a pure replication' publisher: Alexander M. Aiken, Calum Davey, James R. Hargreaves and Richard J. Hayes, International Journal of Epidemiology 44(5) year: 2015 url: https://pmc.ncbi.nlm.nih.gov/articles/PMC4681107/ - title: 'Commentary: Deworming externalities and schooling impacts in Kenya: a comment on Aiken et al. (2015) and Davey et al. (2015)' publisher: Joan Hamory Hicks, Michael Kremer and Edward Miguel, International Journal of Epidemiology 44(5) year: 2015 url: https://europepmc.org/article/MED/26203170 --- ## Use the live research desk Choose a claim from a primary study and fill in the [replication planner](#workspace-replication-program). State the population, outcome, analysis, exclusions and stopping rule before inspecting the result. Freeze a local version, fork any amendment and export it for independent review or an external registry. This software does not conduct experiments or independently timestamp a preregistration. Use the artifact and results register to connect an exact protocol to a versioned manifest of code, data or reports. You can start there or preview saved records from an existing Methods or Replication JSON export. Only deliberately selected saved records become protocol snapshots; an unfinished source draft does not. Files stay on your device. A selected file's checksum identifies bytes, while an entered external checksum or registration reference stays a user-supplied declaration. Record the later result and its date separately from the time you entered it. Preserve the original estimate, reported estimate, missing values, source references, deviations and limitations. Reported null, inconclusive, stopped and not observed remain distinct. A descriptive comparison can show an arithmetic difference only for explicitly matched, known units and scope; it never determines whether a replication succeeded. Changed plans or manifests do not silently inherit earlier outputs. The same register opens in [Methods Lab](/programs/methods-lab/), and its full JSON backup includes all projects and the saved raw form draft. Your workspace stays in this browser. Save records, edit them or fork frozen versions, export JSON, a research memo or CSV, and import a backup with an explicit replacement step. Deleting or clearing offers an undo while the page remains open. No notes are submitted to Charitology. The institutional goals below remain the program’s longer-term charter. ## What it is A single study can be informative without settling a question. In 2015 the Open Science Collaboration repeated 100 published psychology studies. Ninety-seven percent of the originals had reported statistically significant results. Only 36 percent of the repeats did. Charitology has not established an equivalent sector-wide rate for charity impact research. That is [Open Problem 10, the Replication Rate](/open-problems/the-replication-rate/), and the Replication Program exists to answer it. ## How it works A replication can mean re-analyzing the original data or running the study again somewhere new. The program would do both. It would choose studies by fixed rules, such as how often they are cited and how much money relies on them, and publish the list and the analysis plan before any results come in. Every result would be published, including the ones that hold up. Charity research has done this before, in public. In 2015 a team re-analyzed an influential 1998–99 school deworming study from western Kenya, using the original data and methods. It was possible because the original researchers shared their data and analysis files. Most results were reproduced. Some were not, including an effect on attendance at nearby schools, for which little evidence remained once coding errors were corrected. The original researchers replied in the same issue of the journal. The program aims for the same openness every time: shared data, published plans and replies printed next to results. ## Why it matters for the system The [Silence About Failure](/diagnosis/silence-about-failure/) applies to research too. A standing program makes re-testing routine, so a finding that doesn’t hold up is published as ordinary news rather than buried or treated as a scandal. It also treats the [Measurement Gap](/diagnosis/measurement-gap/) from the other side. Measurement helps only if the measurements can be trusted, and replication is how a field learns which ones can. ## Risks and limits A failed replication doesn’t mean the first team did anything wrong. Places change, programs change and chance plays a part. The program would report results without blame and invite the original authors to respond alongside every result. ## What success looks like Within five years, the program has published replications of at least 20 influential findings and an estimate of the replication rate with confidence bounds, and donors can see a replication status next to the studies they rely on.