But Some Publications
are “Spinning” the Outcomes and Twisting these Results
[CLICK
HERE for the full Blog message]
Dear
Colleagues,
Introduction
In order to be successful in today’s
schools, we all need to be scientist-practitioners and critical consumers of
both past and present research. This is
essential because, for those of us working in the schools with real students
in real classrooms experiencing real challenges, we need to
identify and implement “high probability of success” services, supports,
strategies, and interventions.
This requires not just an understanding of
research methods and outcomes. It also requires
an understanding of how (and whether) reported research is relevant,
meaningful, and applicable to specific students, staff, and schools.
_ _ _ _ _
In all of these contexts, educators are
confronted—far too often—with research reports of programs and interventions
that sound too good to be true.
Many times, the original technical reports report
with the research with a high level of integrity and accuracy. Thereafter, the research
sometimes— inadvertently—enters “The Spin Zone.”
In The Spin Zone, others take the original
results, publishing an article that describes the research from the author’s own
perspective. Sometimes, the “spin”
simply occurs as a provocative headline that “teases” the reader into reading
the article, but that over-simplifies or misrepresents the real results.
At other times, the article “spins” the original
research—by omission or commission—in ways that do not truly represent the
study and its data, results, and/or applications or implications. The new version, then, is a bastardization of
the original research, and unless the original research is read, the
reader could accept the “spin” as reality.
Sometimes, the “spin” is naïve or
ignorant. It occurs because the “new
author” does not understand the science underlying sound research, or because
the author’s attempt to simplify the research for his/her audience results in
an inaccurate or overgeneralized summary.
This often occurs when (popular press)
reporters or journalists are untrained in research methodology, do not do their
due diligence, assume that they know more than they know, or use “experts” who
truly are not expert.
Sometimes, the “spin” is conscious and
intended. Its goal is to
misrepresent or “flip” the results of the original research for the purposes of
(a) softening or reframing results that are counter to the “new author’s”
beliefs or agenda; (b) putting equivocal results in the author’s “positive
light,” or (c) “pivoting” past the results to opine about an issue that the new
author’s really wants to publicize or emphasize.
Without getting too political, think about
what occurs after a Presidential debate as the operatives on each side “spin”
their version of what their candidate did or said during the debate to influence
the press and the public on “who won.”
_ _ _ _ _ _ _
_ _ _
A New Study of Restorative Practices
in the Pittsburgh (PA) Public Schools
So. . . why this mini-dissertation on “Spin”?
Today’s Blog analyzes a 132-page Report
published by the Rand Corporation on December 27, 2018 titled:
Can Restorative Practices Improve
School Climate and Curb Suspensions? An
Evaluation of the Impact of Restorative Practices in a Mid-Sized Urban School
District
CLICK HERE for the Report
The Report describes the results of the Rand
Corporation’s study to determine the efficacy of the International Institute
for Restorative Practices (IIRP) use of its SaferSanerSchools
Whole-School Change restorative practices program in the Pittsburgh Public
Schools.
This study came about after the Pittsburgh
(PA) Public Schools received a National Institute of Justice grant, selected
the IIRP as its restorative practices program, and then separately selected the
Rand Corporation to conduct the program evaluation.
_ _ _ _ _
The Rand Corporation used a highly
sophisticated randomized controlled study to evaluate the two-year
implementation of the IIRP’s Restorative Practices Program. Indeed, the Program was implemented in 22
randomly-selected Pittsburgh schools, with 22 other randomly-selected Pittsburgh
schools serving as non-participating Control schools.
By way of
background:
The IIRP, a Partner with the Collaborative
for Academic, Social, Emotional Learning (CASEL), is (according to its
website) “the world’s first graduate school wholly devoted to
restorative practices. Our faculty — all scholar/ practitioners — are dedicated
to helping individuals find new ways to empower people and transform
communities.”
The Rand Corporation is a highly
regarded non-profit and non-partisan group that “develops solutions to public
policy challenges to help make communities throughout the world safer and more
secure, healthier and more prosperous.”
The Rand Corporation publicly asserts that its “publications do not
necessarily reflect the opinions of its research clients and sponsors.”
_ _ _ _ _
The Rand
Study Results:
The results of the study (see more detail
below) indicated that, while the District’s suspension rates had been declining
prior to the implementation of the study, the suspension rates in the
Restorative Practices schools declined even more than the rates in the Control
schools.
In addition, in the Restorative Practices
schools (a) alternative school placements decreased; (b) students were less
likely to be suspended multiple times; (c) disparities in suspension rates
between African-American (vs. Caucasian), and low-income (vs. higher-income)
students, respectively, decreased; and (d) suspension rates for female students
declined.
However, more in-depth analyses revealed
that:
- While suspension rates in the Restorative Practices schools declined by 36% during the two-year study, suspension rates in the Control schools also declined 18% during the same time period.
- The overall suspension results were driven by lower rates in the Restorative Practices elementary schools.
- Fewer suspensions were not found in the Restorative Practices Middle schools (Grades 6 to 8).
- Fewer suspensions were not found for male students or students with disabilities.
- There were no reductions in student arrests, or for incidents of violence or weapons violations.
- In the Restorative Practices Middle schools, academic outcomes actually worsened when compared with the Control schools.
- Survey results from staff in the Restorative Practices schools indicated that they did not think the IIRP program was affecting student behavior. They did, however, report that their relationships with students had improved because of program involvement.
_ _ _ _ _
What is
the point of this Blog?
First of all, it is interesting that a
Report of such significance was released on December 27th—at a time
when schools were on vacation, and most people were in the midst of their
holiday and New Year celebrations.
More important is the fact that a
handful of national educational (and other) news outlets published high-profile
articles—immediately after New Year’s—that seemed to “spin” the outcomes
described in the Rand Report— through either their headlines or their content.
The result is that the educators who
read the “spinned” headlines or articles (but not the original Report) might draw
incorrect conclusions about what really happened in the Pittsburgh School
District. Indeed, they might conclude
that Restorative Practices “worked” in the Pittsburgh School District—even
though the more-detailed results delineated above suggest otherwise.
And as a result of their inaccurate
conclusions, they might then invest precious time, training, and resources on a
Restorative Practice framework in their schools, only to replicate the same
“underwhelming” results that actually occurred.
Among the articles of concern were those
published in:
- The Smartbrief’s—specifically the ones sponsored by the Association for Supervision and Curriculum Development (ASCD) and the National Board for Professional Teaching Standards (both January 4, 2019)
[Smartbrief is the leading digital media publisher of targeted,
industry-specific business news and information—serving nearly 6 million senior
executives.]
- The Atlantic (this article, published on January 3, 2019, was the one cited in the above Smartbriefs)
- Education Dive (January 3, 2019)
- US News & World Report (January 4, 2019)
In the remainder of this Blog, we identify
our concerns from the articles above—demonstrating the main thesis of this
Blog. First, however, we provide a brief
overview and critique of Restorative Practices.
[CLICK
HERE for the full Blog message]
Critically, many educators trust the publications
above, and depend on them for news about “promising programs.” As such, many educators feel no need to read
the original research described in these publications— especially when they are
132 pages long.
This places a burden on the publications to
exercise caution in how they write their headlines and select their content.
It also places responsibility on our
educational colleagues . . . to read not just the descriptions of
recently-published studies, but to read and analyze the original studies themselves.
This is critically importance as it relates
to Restorative Practices—as these programs have been “pumped up” by the popular
press, even though the data-based research validating these practices and
programs is incredibly thin.
_ _ _ _ _ _ _
_ _ _
An Overview of Restorative Practices
Restorative
Practices have been pushed—by the U.S. Department of Education, some publishers,
and selected professional development companies—as a “failsafe, go to” program
for school discipline and behavior management, and as a key to solving the
disproportionate disciplinary referrals of students of color and with
disabilities, respectively.
But it is hard to
know what “Restorative Practices” are—as they are a collection of strategies, and
there is no sound science-to-practice research that has validated how these
strategies should be integrated, sequenced, or evaluated.
Moreover, the
Restorative Practice “push” has fostered a “cottage industry” of organizations
and vendors who similarly have not independently or objectively validated their
approaches using sound research—and who are using their own versions of
Restorative Practices.
Indeed, the
Pittsburgh School District implemented the International Institute for Restorative Practices’ SaferSanerSchools
Whole-School Change restorative practices program. Thus, other restorative practice developers
may claim that the Rand Corporation study results do not apply to their
program. . . that the Rand Report can only be applied to the IIRP program, or
programs that have similar restorative practice elements.
[Amazingly (but, predictably), on the IIRP’s
current website Homepage, there is a direct link to its IIRP News Page
where a banner proclaims,
Research shows restorative practices
improves school climate, reduces student suspensions and discipline disparities (! ! !)
The story that follows emphasizes only the
parts of the Pittsburgh study that appeared to support the IIRP’s
Restorative Practices program.]
_ _ _ _ _
National Concerns with
Restorative Practices
Restorative
Practices were described in an undated (but circa 2016—based on its citations) Issue
Brief published by the national Now Is the Time Technical Assistance
Center:
“Restorative
Practices: Approaches at the Intersection of School Discipline and School
Mental Health”
This Center is
funded by the U.S. Department of Health and Human Services’ Substance Abuse and
Mental Health Services Administration.
[CLICK HERE
for the full Blog message that quotes this Issue Brief, delineates additional
national concerns regarding Restorative Practices, and especially details
how to integrate restorative practices into a comprehensive
scientifically-based, multi-tiered school discipline, classroom management, and
student self-management model.
_ _ _ _ _ _ _ _ _ _
Returning to the
“Spin”
Returning now to
what “triggered” this Blog: The Rand Corporation’s
Report that evaluated the International
Institute for Restorative Practices (IIRP) use of its SaferSanerSchools Whole-School
Change restorative practices program in the Pittsburgh Public Schools.
As detailed above,
the Rand Report objectively documented the data-based positive (or affirmative)
and negative (or contra-indicative) results, respectively, of the two-year
restorative practices implement. But, a handful of national educational (and other)
news outlets published high-profile articles—immediately after New Year’s—that
seemed to “spin” the outcomes described in the Rand Report—through either their
headlines or their content.
The concern, once again, is that the
educators who read the “spinned” headlines or articles (but not the original
Report) might draw incorrect conclusions about what really happened in the
Pittsburgh School District. Indeed, they
might conclude that Restorative Program was so successful in the Pittsburgh
School District that its practices should be seriously considered and implemented
in other, similar school districts. . . even though the more-detailed results
delineated above suggest otherwise.
[CLICK HERE
for the full Blog message that provides both inappropriate and appropriate
examples of how different national educational news outlets publicized the Rand
Corporation Report.]
One of the “appropriate”
examples was published on
January 9, 2019 by the Thomas B. Fordham Institute. This article did a nice job of summarizing the
Pittsburgh results and implications:
The study has some limitations that may
affect its findings. First, RAND only studied implementation over two years. It
is possible that stronger results will take longer to show. Second, they do not
have any direct measures of student experiences with (the IIRP Restorative
Practices program), such as student interviews or the number of student
referrals to the office. And finally, researchers had little insight into how
each restorative practice was used daily at the classroom level.
Despite these limitations, the results of (the
IIRP Restorative Practices program) are underwhelming. A drop in suspensions is
good as far as it goes, but researchers could not identify a cause for the
change, and teachers expressed confusion as to whether restorative practices
were supposed to take the place of other disciplinary actions. Additionally, to
see no effect on arrest rates and potentially negative academic effects is
concerning, especially since RAND does not include any information about the
cost of the program. We can’t shake a stick at wanting struggling students to
feel like part of the community, but as a comprehensive approach to discipline
reform, restorative justice does not seem promising.
_ _ _ _ _ _ _ _ _ _
Summary
This Blog addresses
three inter-related topics: (a)
Restorative Practices; (b) the “spin” that some educational (and other)
publications employ when reporting on some research; and (c) the reality that
some educators read the “spinned” versions of a research study, do not read
and analyze the original research, and then implement some programs or take
student- or staff-focused actions on misrepresented or inaccurate information.
Restorative Practices. Relative to Restorative Practices, and to set
the record straight, please understand that I believe that:
- It is critically important to decrease the number of students being suspended from our schools nationwide, and to eliminate suspensions that are arbitrary, unnecessary, steeped in prejudice, and that do not match the intensity of the offense.
[We just need to do it the right
way.]
_ _ _ _ _
- Legitimate decreases in student suspensions and even discipline referrals to the principal’s office do not always result in simultaneous increases in positive school and classroom climates, student engagement, and prosocial student behavior.
[While we may successfully decrease
the intensity of some students’ challenging behavior—such that they no longer require
office referrals—this does not mean that they are engaged and learning in their
classrooms.]
_ _ _ _ _
- Suspensions are administrative responses, and they rarely result in decreasing or eliminating students’ future inappropriate behavior, while simultaneously increasing their appropriate behavior.
[In other words, without the
psychoeducational interventions that change the underlying reasons for
specific students’ behaviors, these students typically return from their
suspensions with the same problems.]
_ _ _ _ _
- Some teacher referrals to the principal’s office and some administrative suspensions are arbitrary, capricious, and/or mean-spirited on one end; or—on the other end—due to a lack of sensitivity, knowledge, understanding, and/or skill on how to handle specific student conditions (e.g., students from trauma, with a disability, coming from poverty, with a history of academic failure).
[Thus, given these circumstances, the
“intervention targets” will necessarily involve these adults—along with the
students involved. If inappropriate
office discipline referrals or suspensions are educator errors, the adults must
be changed if the inappropriate disciplinary actions are going to be changed.]
_ _ _ _ _
- Restorative Justice practices (not programs) are useful when implemented as available strategic strategies or interventions within a comprehensive scientifically-based, multi-tiered school discipline, classroom management, and student self-management model.
[However, within this model,
restorative practices will only be successful when, based on functional assessment/root
cause analyses, they are matched to the students who will most benefit from these
practices.]
_ _ _ _ _
- Ultimately, schools need to focus on teaching and reinforcing students’ interpersonal, social problem-solving, conflict prevention and resolution, and emotional control and coping skills—while also providing the multi-tiered assessment and intervention services, supports, strategies, and programs that the most challenging students need to address their inappropriate behavior.
[Without school-wide prosocial
skill instruction programs and approaches that motivate students to “make good
choices,” we will never know how many challenging student behaviors we can
prevent.]
_ _ _ _ _
As always, I look forward to your thoughts
and comments. I am always available to
provide a free hour of telephone consultation to those who want to discuss
their own students, school, or district needs.
Feel free to contact me at any time if there is anything that I can do
to support your work. . . now, or as you prepare for next year.
Best,
Howie
