The Experiment Societyexperimentsociety.org⌕Get involved
EducationHuman capitalPositive

Boston Charter School Lottery Studies

MIT (Joshua Angrist, Parag Pathak) / Harvard (Susan Dynarski) · Boston, Massachusetts, USA · 2009

Summary

The Boston charter school lottery studies are the methodological gold standard in modern US education research and the strongest single body of evidence on what high-performing urban charter schools can accomplish. The lottery design — applicable wherever charter demand exceeds supply — eliminates the selection bias that has plagued earlier charter research (where attending students differed systematically from non-attenders in motivation and family characteristics that themselves predict outcomes). The Boston effects are unusually large by education-research standards: 0.4 SD per year in math is roughly twice the typical effect size of even effective educational interventions. The findings have been used both to support charter expansion and to caution that the effects depend on specific implementation features (extended time, structured curriculum, intensive teacher coaching) that not all charters share. The methodological contribution — using admission lotteries as natural randomization — has been adopted in dozens of subsequent education-policy evaluations and has substantially raised the standard of evidence in the field.

Research question

"Do oversubscribed urban charter schools produce better academic outcomes than traditional public schools serving similar students, isolating the school-quality effect from selection bias by exploiting admission lotteries?"

Methodology

Intervention

Students applied to admission lotteries at oversubscribed Boston-area charter schools. The schools varied in pedagogical approach but shared features common to 'no-excuses' charter models: extended school day and year, strict behavior expectations, intensive tutoring, frequent testing and feedback. Comparison was between lottery winners (who enrolled) and lottery losers (who attended other Boston-area schools).

Assignment

Randomized controlled trial via admission lotteries — students who applied to oversubscribed charters were randomly assigned by lottery; researchers used the lottery as natural randomization

Sample size

Multiple lotteries across years; combined sample of 1,800+ Boston-area students with valid lottery offers; tracked through middle school and high school outcomes

Primary outcome

Standardized test scores (Massachusetts MCAS), high school graduation, college enrollment, and college persistence

Effect estimate

Boston charter middle schools produced gains of approximately 0.4 SD per year in math and 0.2 SD per year in reading — among the largest effects observed in K-12 education research. High school effects were similar in magnitude. Lottery winners were significantly more likely to enroll in 4-year colleges (74% vs. 56% for losers) and to persist.

Decision

Findings have been heavily cited in the broader US debate over charter school expansion. The Boston results have been confirmed in lottery studies of New York City charter schools (Hoxby, Murarka, Kang), Washington DC (Mathematica), and other urban charter systems. Critically, the strong positive effects appear concentrated in the 'no-excuses' charter model and in specific urban contexts; charter effects in suburban and rural areas have been smaller or null.

Result

Positive

Boston charter middle schools produced gains of approximately 0.4 SD per year in math and 0.2 SD per year in reading — among the largest effects observed in K-12 education research. High school effects were similar in magnitude. Lottery winners were significantly more likely to enroll in 4-year colleges (74% vs. 56% for losers) and to persist.

Evidence strength

Strong

Randomized trial, replicated across multiple sites or studies.

Replication status

Replicated

Institution

MIT (Joshua Angrist, Parag Pathak) / Harvard (Susan Dynarski)

Location

Boston, Massachusetts, USA

Year

2009

Policy area

Education

Mechanism

Human capital

More from these institutions

Other trials of this mechanism

Cite this entry

MIT (Joshua Angrist, Parag Pathak) / Harvard (Susan Dynarski). (2009). Boston Charter School Lottery Studies. The Experiment Society Registry. Retrieved from https://www.experimentsociety.org/registry/boston-charter-school-lottery (primary report: https://www.nber.org/papers/w17332)

Citation network

Cited by 1 other entry in the registry

Cross-references

Adjacent experiments — same domain, parallel pilots, or alternative mechanisms.

Related experiments

United States · 2018

Year Up Young Adult Job Training Program

Positive

Chicago, IL, USA · 2016

Chicago STAR Scholarship — Dual Enrollment

Positive

Wales, UK · 2007

Universal Free School Breakfast Program

Positive

← Back to registryWhat the evidence shows →