Discipline report “plagued by logical fallacies, overly simplified interpretations of findings, and inflammatory language

The Academic and Behavioral Consequences of Discipline Policy Reform: Evidence from Philadelphia, published by the Fordham Institute, investigated the impact of a reform in the School District of Philadelphia that eliminated suspensions for certain low-level misbehaviors.

Yolanda Anyon of the University of Denver and Kathryn E. Wiley of the University of California San Diego reviewed the report and found it “plagued by logical fallacies, overly simplified interpretations of findings, and inflammatory language.”

Find the review by Yolanda Anyon and Kathryn Wiley at:
http://nepc.colorado.edu/thinktank/review-discipline-reform

The report considered whether the change in discipline policy was associated with any of the following: (a) district-wide out-of-school suspension rates, (b) academic and behavioral outcomes for students (looking separately at students who had a record of prior suspensions and those with no prior suspensions), and (c) racial disparities in suspensions.

While the report concluded that the reform was a failure, the actual results were mixed, with the positive trends for students who were earlier suspended being much stronger in magnitude than evidence of negative outcomes for students who were not. A strength of the report is the use of advanced statistical methods and a longitudinal dataset to answer the questions of interest.

However, Anyon and Wiley explain, the report uses misleading causal (“consequences”) language in the title and to describe study results, even though the study design is limited by unmeasured confounding factors and inappropriate comparison groups. Thus, while the analyses upon which the report is based have some technical merits, the narrative seems more of an attempt to advance a political agenda opposed to the reform studied than to improve understanding of complex policy issues.