What the Report Says. .
. Why RtI is Not Working. . . Recommendations for Improving the RtI
Process
Dear Colleagues,
This past week, a new federal report, Evaluation
of Response to Intervention Practices for Elementary School Reading- -
commissioned by the U.S. Department of Education’s Institute of Education
Sciences, and completed by the National Center for Education Evaluation and
Regional Assistance - - was disseminated across the country.
The largest federal investigation of its
kind, the study involved approximately 24,000 first through third grade students
in 13 states. More importantly, it
statistically compared 146 schools that had been implementing key elements from
the U.S. Department of Education’s Response-to-Intervention (RtI) framework in
literacy for at least three years, with 100 randomly-selected comparison
schools in the same 13 states- - that were not implementing RtI.
As a quick review: RtI is a multi-tiered instructional and
intervention system that has its historical roots in the 2004 reauthorization
of the federal Individuals with Disabilities Education Act (IDEA). Its framework has been promoted (if not
mandated during the Reading First era) by the U.S. Department of Education- -
especially through a number of its federally-funded national Technical
Assistance and Dissemination Centers.
Typically, RtI involves assessing students (in
reading for this new study) with a screening tool, putting moderately or
significantly deficient students (identified by the tool) into intervention
groups (called “Tier II”), and monitoring their progress over time. If the student makes good progress, then the
(Tier II) intervention is discontinued.
If the student does not progress or falls further behind, then more
individualized Tier III interventions are attempted.
Significantly, this RtI framework has
always had many critical flaws- - flaws that violate psychometric and
psychoeducation principles of sound practice. . . flaws that:
* Have delayed services to students
* Have resulted in the wrong interventions
being implemented, and that
* Have actually added to, increased, or made
some students’ academic problems more resistant to change
I have discussed these issues in at least
two previous Blogs and one extensive Technical Assistance paper:
National Concerns about RtI and PBIS: A Review of Policy and Practice
Recommendations Not Based on Research or Effective Practice
_ _ _ _ _
September 7, 2015 “When Kids Can’t Read: Policy and Practice Mistakes that Make it
Worse”
_ _ _ _ _
February 15, 2015 “Your State’s Guide to RtI: Some Statutes Just Don’t Make Sense (What
your Department of Education isn’t Sharing about its
Multi-tiered/Response-to-Intervention Procedures)”
_ _ _ _ _
Below is a summary of the most important
results from this important RtI Report, and then recommendations (and
resources) to help explain and improve RtI processes across this country.
_ _ _ _ _ _ _ _ _ _
What Did the RtI Report Find?
While this extensive, 308-page Evaluation
of Response to Intervention Practices for Elementary School Reading report
has many primary and related outcomes, here is a brief summary of the most
essential “take-aways.”
* First of all, the study compared the
literacy progress of 1st through 3rd grade students
during the 2011 to 2012 school year primarily using individually-administered
norm-referenced tests and their state’s high-stakes proficiency test.
* The students in the 146 “RtI Schools”
qualified for RtI Tier II interventions, and the students in the 100
“Comparison Schools” barely made or just missed the cut-offs for Tier II
intervention- - all based on a Fall screening test (the DIBELS or AIMS were
clearly the most-used screeners).
Due largely to different school site
configurations, data from 119, 127, and 112 RtI schools were collected at the
Grade 1, Grade 2, and Grade 3 levels, respectively.
The Results? Based on Fall to Winter interventions and
assessments:
* The 1st graders receiving Tier
II interventions performed 11 percent lower on the reading assessments
than the comparison students who barely missed qualifying for the Tier II
intervention approaches.
* The 2nd and 3rd
graders receiving Tier II interventions experienced no significant reading
benefits- - although they did not lose ground.
* At Grade 1, only four of the 119 schools
studied found data-based benefits for their Tier II students, while 15 schools
had negative effects for their Tier II students. [100 schools showed no benefits for all of
the staff and student time- - and resources- - expended.]
* At Grade 1, 86% of students who began in
Tier I remained in Tier I; 50% of the students who began in Tier II remained
there; and 65% of the students in Tier III remained. Across the Grade 1 student sample, 13% of the
students moved to a more intensive Tier, and 14% moved to a less
intensive Tier. [The percentages of
students moving were smaller in Grades 2 and 3.]
* Students already receiving special
education services or who were “old for grade” (probably due to delayed entrances
or retentions) had particularly poor results when they received Tier II
interventions.
* For all students, the reading results did
not significantly differ for students from different income levels, racial
groups, or native languages.
_ _ _ _ _ _ _ _ _ _
Additional “More Subtle” Report Findings
In addition to the primary results above, a
“close reading” of the RtI Report reveals some important additional findings
for the RtI schools:
* 79% of the schools for the Grade 1
students, 75% of the schools for Grade 2, and 80% of the schools for Grade 3 used
only ONE screening test when placing their students in Tier II
interventions in the Fall.
* Once again, two curriculum-based
measurement tools (the DIBELS or the AIMS) were the most-used screeners.
* Between 31% (Grade 3) and 38% (Grade 1) of
the students in the study were placed into Tier II or III interventions using
no other information but the screening test.
* The “interventions” tracked by the RtI
Report were simply small-group instruction or one-on-one tutoring.
While the schools were surveyed on the focus
of the interventions (e.g., phonemic awareness, phonics, vocabulary, fluency,
or reading comprehension), the Report did not identify or track the specific
skill-based interventions received by the students.
* However, in 1st grade for example, 45% of
the schools provided Tier II interventions to groups of students at all reading
levels- - not just for students reading below grade level. Moreover, 67% of schools provided Tier II
interventions during the core reading instruction- - not just in addition to
it.
* Across Grades 1 through 3, teachers
provided Core Instruction for approximately 102 minutes per day. 97% of the schools provided Tier II
interventions at least 3 times per week- - for approximately 39 minutes per
day. 68% of the schools provided Tier
III interventions at least 5 times per week- - for approximately 49 minutes per
day.
* Finally, for the Below Grade Level
students in intervention groups, 37% of them in Grade 1, 28% in Grade 2, and
22% in Grade 3 were receiving their interventions from paraprofessionals-
- not certified teachers or reading or other specialists.
_ _ _ _ _ _ _ _ _ _
Some Notable Quotes about the Report
While a number of national experts have yet
to “weigh-in” regarding the Report, here are some interesting early quotes.
* Regarding the 1st grade Tier II
intervention results, Rekha Balu- - the lead author of the RtI Report- - said
that these students’ academic losses "(were) the equivalent of losing one-tenth
of a year of learning."
* Fred Doolittle, a co-author of the Report
said, "We're looking at this framework that has developed over the years
and how it has really played out in classrooms... We weren't expecting to see
this pattern."
He went on to say, "We don't want to
have people say that these findings say these schools aren't doing RTI right;
this turns out to be what RTI looks like when it plays out in daily life."
* Finally, Sarah Sparkes, a writer for Education
Week, who interviewed a number of national experts about the RtI Report,
concluded that the study “raises questions about the evolution of a model originally
designed to target students as soon as they started to struggle, and prevent
their difficulties from escalating to the need for a special education
evaluation.”
She also noted that both Doolittle and
another national expert suggested that school leaders using RtI for
early-grades reading should re-examine how they identify students and what
interventions they provide to them.
_ _ _ _ _ _ _ _ _ _
Recommendations to Improve RtI Processes across the
Country
This RtI Report is critical because our almost-annual
surveys now estimate that (a) more than 70% of school districts across the
country are using RtI processes in at least some classrooms, (b) it has become largely
a general education approach, but that (c) most schools are using the same
RtI approaches reflected in the RtI Report.
While the Report spent precious little time
discussing the hypothesized reasons for the bleak results, a number of the
results are because the RtI process- - largely used since the mid-2000s as
promoted by the U.S. Department of Education- - has (as suggested earlier) many
critical flaws- - flaws that violate psychometric and psychoeducational
principles.
Please understand: my purpose here is not to bash RtI. My purpose is to correct the bad practices
that have been advocated, and improve the outcomes for students, staff,
and schools.
At the same time, please understand that nothing
in education should be utilized in a “one-size-fits-all” manner. We need to strategically use RtI to attain
its best outcomes. But RtI needs to be
implemented in the context of school improvement at the systemic level, and
sound data-based problem-solving at the student level.
And so, here are some recommendations to
improve RtI processes around the country:
Recommendation 1. Academics is Interdependent with Behavior.
I often ask teachers with
struggling students two critical questions:
* Do you have students who are
behaviorally acting out because of academic frustration?
* Do you have students who are
academically not learning (or not learning quickly enough) because they do not
have certain behavioral skills (sitting in their seat, paying attention,
working in interpersonally effective ways with others)?
When they answer "Yes"
to both questions they are demonstrating that academic instruction, learning,
and mastery is interdependent with classroom discipline, behavior management,
and student self-management.
Thus, it does not make sense for
our RtI processes to focus only on academic skills. . . to the exclusion of
students' social, emotional, and behavioral skills.
Indeed, if a student does not
(a) have the social skills to get along with others (e.g., in a cooperative
learning group); (b) feel emotionally secure in class (e.g., due to teasing or
school safety issues); or (c) have the behavioral skills to organize themselves
(e.g., to work independently), then the even best teachers, curricula,
technology, and instruction will not result in the desired academic outcomes.
_ _ _ _ _ _ _ _ _ _
Recommendation 2. Intervention Occurs along a Continuum of
Instruction.
Many state RtI guidebooks and
systems do not provide a research-based continuum of services and supports that
help to organize and differentiate the difference between
"instruction" and "intervention." These guidebooks talk
about the need for intervention, but rarely provide any specificity.
Over the past decade (or more),
I have presented the continuum below to states, districts, and schools across
the country---organizing it as the PASS (Positive Academic Supports and
Services) model.
As is evident in the slide, RtI
starts with an effective teacher providing sound, differentiated instruction,
supported by good classroom management, and the data-based monitoring of
students' academic and behavioral learning and mastery.
When students are not learning
(or learning quickly enough), an assessment process must be conducted to
determine why the progress is missing (see Recommendation 3 below). This
assessment could be done (a) by the teacher, (b) with the support of
grade-level colleagues as part of a Grade-level RtI Team, or (c) with the
support of the multidisciplinary Building-level RtI Team. How the teacher
assesses the problem is determined largely by his/her skills, and the duration
or intensity of the problem (see Recommendation 7 below).
Once the underlying reasons for
the problem have been validated, the teacher (once again- - by him/herself,
supported by grade-level colleagues, and/or with members of the Building-level
RtI team) strategically decides how to solve the problem (see Recommendation 4).
As in the slide above, the problem may
be solved through strategically selected:
* Assistive support technologies
* Remedial approaches
* Accommodation approaches
* Curricular modification
approaches
* Laser-targeted Interventions
* Compensatory strategies
When students are demonstrating
social, emotional, or behavioral problems, a comparable continuum is used
(after completing the needed functional assessments) that consists of
strategically selected:
* Skill Instruction strategies
* Speed of Learning and Mastery
Acquisition strategies
* Transfer of Training
strategies
* Emotional Coping and Control
strategies
* Motivational strategies
* History of Inconsistency
strategies
* Special Situation strategies
_ _ _ _ _ _ _ _ _ _
Recommendation 3. Diagnostic or Functional Assessment Needs to
Clarify Screening Test Results (at Tier 1)
Many state RtI guidebooks,
adopting the flawed approaches of the U.S. Department of Education's RtI
technical assistance centers, advocate for a "wait to fail, then
assess" strategy. That is, when students are not succeeding academically
(for example) at Tier I, they recommend 30 minutes of largely unspecified group
interventions at Tier II. Then, if the students are still having problems, they
recommend a diagnostic (or, for behavior, functional) assessment as the entry
point to Tier III.
And yet, critically, I don't
know many doctors, electricians, car mechanics, or other professionals who would
not do a diagnostic assessment at the beginning of the problem solving process.
. . to ensure that their first recommendations are their last recommendations
(because the problem is solved).
And so. . . Why would we, in
good conscience, "allow" a student to struggle for six to ten or more
weeks in the classroom, and in a Tier II intervention, so that we can get to
the point where we finally do a diagnostic assessment to figure out what really
is wrong?
And why would we do this knowing
that, after these multiple and prolonged periods of failure, the problem may be
worse (or compounded), the student might be more confused or frustrated, and we
might need even more intensive interventions because we did not identify and
analyze the problem right from the beginning?
ALL
of these practices and issues were confirmed by the RtI Report.
_ _ _ _ _ _ _ _ _ _
Recommendation 4. Assessment Must Link to Intervention.
Many state RtI guidebooks and
systems do not delineate the different types of assessment (e.g., screening
versus progress monitoring versus diagnostic versus implementation integrity
versus high stakes/proficiency versus program evaluation assessments).
This often occurs because state departments of education write their guidebooks
to meet a statutory requirement. . . rather than to educate their
practitioners.
Relative to RtI processes that
will effectively help students with academic or behavioral difficulties, state
guidebooks and systems typically do not emphasize the importance of linking
diagnostic assessment results with the instructional or interventions
approaches that have the highest probability of success.
Critically, when school
practitioners do not strategically choose their student-focused instructional
or intervention approaches based on reliable and valid diagnostic assessment
results, they are playing a game of "intervention roulette."
And, as in Vegas, the
"house" usually wins. But, in the classroom, the loss here is the
student's loss.
Every time we do an intervention
that does not work, we potentially make the problem worse, and the student more
resistant to the next intervention.
Said a different way:
Intervention is not a benign act. . . it is a strategic act. We should
not be satisfied, professionally, because we are implementing interventions. We
should be satisfied when we are implementing the right interventions that have
the highest probability of success for an accurately identified and analyzed
problem.
_ _ _ _ _ _ _ _ _ _
Recommendation 5. Progress Monitoring is NOT an Intervention. .
. When Needed, Focus on Strategic
Instruction and/or Intervention
Many state RtI guidebooks and
systems overemphasize progress monitoring. . . and then, they compound the
problem by overemphasizing curriculum-based measurement (CBM) to the exclusion
of other curriculum-based assessment (CBA) approaches.
In addition, most of the
progress monitoring examples-- in the state guidebooks that I have reviewed--
are in the area of reading decoding and fluency (where the progress monitoring
research and writing has been most prevalent).
Rarely do you see state
guidebooks discuss progress monitoring for vocabulary and comprehension. . .
not to mention the lack of progress monitoring examples in the different areas
of math, written expression, spelling, and oral expression. This is
because progress monitoring for these outcomes does not work well with
CBM.
Finally, most state guidebooks
do not explain how to effectively create (or evaluate the acceptability of) a
progress monitoring probe. That is, they do not emphasize that progress
monitoring approaches must be connected to the instructional or intervention
goals, outcomes, and implementation strategies.
Said a different way:
* If the instructional or
intervention target for a student is increasing his/her understanding and
receptive/expressive use of a specific list of grade-level vocabulary words,
then the assessment protocol must be designed to sensitively measure these
explicit outcomes.
* If the instructional outcome
is a certain format of expressive writing, then reliable and valid scoring
rubrics need to be created to guide not just progress monitoring, but
instruction and student self-evaluation.
As noted earlier, progress
monitoring is an evaluation approach. Thus, for students with academic or
behavioral problems, it follows the (a) identification and (b) analysis of the
problem, and the (c) intervention preparation and implementation stages.
Unfortunately, some educators still believe that progress monitoring is the
intervention. Or, they believe that the intervention must fit the progress
monitoring tool adopted by the district- -rather than the tool being fit to the
intervention outcomes desired.
_ _ _ _ _ _ _ _ _ _
Recommendation 6. The Intensity of Student Interventions and
Services are based on the Intensity of Student Needs
I have no problem with a state RtI
guidebook providing a blueprint on the typical sequences and decision rules
that a teacher needs to follow to "move" a student from Tier I to
Tier II to Tier III. However, I do have a problem when the sequence must
be followed in a rigid, fixed way.
Simplistically, there are two
types of students with academic or behavioral problems: students with
progressive, longstanding, or chronic problems; and students with significant,
severe, or acute problems.
For the latter students
especially, they often need immediate and intensive (Tier III, if you will)
services, supports, strategies, and/or programs. They (and their teachers)
should not have to go through a series of intervention layers so that they
eventually "qualify" for the services that they need.
I "get" that many
worry about an influx of inappropriate referrals to the Building-level RtI
Team. But, if you break your leg, you need to go to the emergency room. If you
try to fix it yourself, you may get an infection and lose the whole leg.
The "trick is in the
training." In the schools where I work, we create a collaborative system
where everyone in the school is trained on the data-based problem-solving
process. We also create an early warning "problem solving, consultation,
intervention" culture, along with a "check and balance" approach
to minimize the number of capricious referrals to the Building-level team.
It works. But more important is
the fact that more students are receiving earlier and more successful
instructional and intervention approaches. And, the teachers are leading the
entire process. . . with greater enthusiasm, involvement, self-direction, and
success.
Isn't this the true goal of RtI?
Parenthetically,
based on the RtI Report, two critical questions in this area emerge:
1. Why didn’t the Report identify the specific
instructional or skill-based interventions being implemented at the Tier II and
Tier III levels?
Small group instruction or one-on-one tutoring are NOT
interventions. The interventions are the
strategies that are implemented in these instructional formats.
_ _ _ _ _
2.
Are we comfortable with the fact that between 22 (Grade 3) and 37%
(Grade 1) of our most needy students were receiving their “intervention”
services from paraprofessionals and not certified teachers or reading
or other specialists?
With all due respect to our hard-working
paraprofessionals, and even assuming that they were being supervised, can we
expect these individuals to have the expertise and sensitivity to know how to
deliver, monitor, evaluate, and adapt (as needed) specific interventions for
students who (if correctly identified) have diverse and sometimes complex
needs?
_ _ _ _ _ _ _ _ _ _
Recommendation 7. Students Receive Multidisciplinary Attention
when they NEED Multidisciplinary Attention
To concretize the ultimate
point in Recommendation 6 above: If a student needs to be immediately
considered by the Building-level RtI Team, then this should occur without the
need for a certain number of interventions implemented for a certain number of
weeks, under a certain level of conditions.
In other words, get on with it.
. .
But I want to extend this
point: If a teacher needs a
consultation with a colleague in order to better understand and work with a
student, there should not be restrictions on what colleagues are available.
To be explicit: Too
often, I hear that general education teachers cannot consult with special
education personnel (teachers, OTs, PTs, speech pathologists, etc.) until a
student needs "Tier III" attention. This makes no sense if this
earlier consultation will result in "Tier I" success. . . thereby
eliminating the need for more intensive Tier II or Tier III attention.
Sometimes, the reason for
restricting the consultation is that the "special education teacher is
paid through federal special education funds that don't allow the consultation
to occur earlier."
This is simply not true.
Even in the most extreme
interpretation, the federal special education law (IDEA) encourages early
intervening services, and it allows districts to use up to 15% of their special
education funding for services and supports that are not directed to students
with a disability.
_ _ _ _ _ _ _ _ _ _
Summary
This week’s RtI Report is a wake-up call.
Educators need to be “good consumers” of research and practice. We should not jump on the newest bandwagon,
and we cannot assume that a framework will work for students, staff, and
schools just because the federal or a state department of education has
recommended (or near-mandated) it.
In
fact, as soon as the RtI Report came out this week, I e-mailed one of the
Report’s authors (who I know) and commended him/her on getting it published- - even in the face of what I expected would be
U.S. Department of Education pressure to bury it.
This
individual assured me (in a return e-mail) that there was no such
pressure.
But
understand, the U.S. Department of Education has a documented history of using
its bully-pulpit to advance its own (largely in-house and less-than-validated)
agenda. . . for example, in school improvement, PBIS, and RtI.
While I have written about this assertion elsewhere, why else do you
think that Congress is trying to minimize the role of the Secretary of Education
in dictating policy in the current draft of the Elementary and Secondary
Education Act’s (ESEA) reauthorization?
_ _ _ _ _
As
always, we need to use common sense and focus our decision making on student
outcomes.
We need to use RtI processes
that are supported by sound (not self-selected) research. . . that are based on
extensive, real, and representative (not limited trial) field studies. . . and
that result in demonstrable (not hypothetical, or even meaningless but
statistically-significant) student outcomes.
Rigid,
one-size-fits-all approaches do not work. Schools need be given the
flexibility, within the federal and state blueprints provided, to implement the
best problem solving, progress monitoring, and services and supports to
academically struggling and behaviorally challenging students. And
educational practitioners who are working directly with those students are in
the best positions to do this.
_ _ _ _ _
I hope that some of the ideas
above are thought-provoking, and motivate you to look at how you are providing
services and supports to all of your students.
If
these ideas validate what you are doing. . . excellent !!! If they
uncover areas of improvement. . . I appreciate your willingness and dedication
to the change and improvement process.
PLEASE read the Report. But also,
PLEASE feel free to look at the following free resources:
Our TA Paper: National Concerns about RtI and PBIS: A Review of Policy and Practice
Recommendations Not Based on Research or Effective Practice
_ _ _ _ _
Our Model RtI Implementation Guidebook: A Model Response-to-Instruction and
Intervention School and District Implementation Guidebook
_ _ _ _ _
Meanwhile, thanks for everything
that you do to support your students, staff, and schools. Let me know if
I can be of help.
Best,
Howie
