When Education Keeps Adopting the Same Shaky Stuff, It Will Keep Getting Repeated Rocky Results
[CLICK HERE to read this Blog
on the Project ACHIEVE Webpage]
Dear Colleagues,
Introduction
A few weeks ago, I was interviewed by two entrepreneurs who had developed an online program that they were selling to schools to teach students social, cooperative, and leadership skills. The program was game-based. . . meaning that students worked together on a series of online games that required them to virtually communicate, cooperate, and make consensual decisions.
Over time, I learned that (a) they seemed to be well-meaning and enthusiastic—if not idealistic—professionals who—it appeared—were trying to help students, staff, and schools in the areas above; (b) they had some educational background—but were really businessmen who also admittedly were interested in making money; and (c) they had a psychologist—albeit not a school or developmental psychologist—on their payroll to “ensure” the pedagogical “integrity” of their “instructional process.”
The “success” that they currently touted on their website was based on the fact that (a) virtually all of the students looked forward to their “game time” each week; (b) the students stayed “engaged” on the platform after each game began; and (c) the students (and some teachers) reported that the students were “more cooperative” and exhibited “more leadership behavior” after playing the games.
Ever the scientist-practitioner, I suggested to my colleagues that these outcomes did not objectively and scientifically prove that the program taught the participating students anything because—to cite just four examples:
· They did no assessments of the students’ pre-existing social, cooperative, or leadership skills. . . and some students may have already had these skills—even if they never demonstrated them at school;
· Any behavioral improvements might have been a placebo and/or sponsorship or social desirability bias effect because the schools chose and purchased the program, the students and teachers knew the expected outcomes, and everyone was getting “special attention” from the game developers;
· The “outcomes” were only self-reported outcomes. . . that is, there were no objective behavioral observations of the students’ leadership behavior in their classrooms. . . during academic tasks that required the same skills as during the game simulations; and
· The skills being “taught” by the game were hopelessly confounded as the games were simultaneously “teaching” such diverse skills as listening, following directions, addressing disagreements and conflicts, making joint decisions, and dealing with frustration and disappointment.
These are not criticisms. They are, instead, scientific and methodological critiques.
And yet. . . generalizing from my brief interactions with these entrepreneurs. . . it is my experience that:
· Their approach to evaluating and “validating” their product is strikingly similar to how many educators (and other entrepreneurs) assess their instruction, programs, interventions, and purchased “innovations”;
· Many schools do not evaluate their curricula, programs, interventions, student information systems, etc. in methodologically, empirically, and functionally-sound ways before spending money and time, training and resources, and staff commitment and faith on less-than-sound approaches; and
·
These faulty patterns repeat as schools often move from one
unsuccessful “solution” to the next one. . . without understanding that they
need to change how they are selecting their solutions, rather than which
solutions they have selected.
_ _ _ _ _ _ _ _ _ _
Putting a “Spotlight” on Fad or Flawed Approaches
Every time you do an intervention with a struggling student that doesn’t work,you potentially make the student more resistant to the next intervention.
Every time you do a school-wide intervention with a struggling staff that doesn’t work, you potentially make the staff more resistant to the next school-wide intervention.
Howie Knoff
A recent (August 6, 2024) EdSurge article cogently analyzed the implications when educators fall prey to the “entrepreneurial glitter” that leads to the implementation of an educational fad in their schools.
In past Blogs, we have extensively covered this phenomenon, exposing many past and still-present fads through objective data and sound, independent research.
We have also “spotlighted” (metaphorically; see film clip below) the fact that some flawed approaches—like the “upper-case” PBIS, MTSS, SEL, and RtI frameworks. . . and the Restorative Justice, Trauma-Sensitive, and Mindfulness school programs—are promoted by the U.S. Department of Education (or its many tax-payer-funded National Technical Assistance Centers), private political organizations (like CASEL), or private entrepreneurial organizations (like the International Institute for Restorative Practices—IIRP).
While these organizations make their approaches appear legitimate and sustainable, when they are analyzed in-depth, they are actually “Emperors without empirically-based clothes.” And many people know. . .
The point here is that, as in the quote above, when these flawed frameworks or programs predictably fail or fade away, there are student and staff losses.
· The First Loss is the loss of time, resources, and other investments on approaches that, once again—predictably, are not successful. . . and that take the place of other truly evidence-based approaches that have much higher probabilities of student success.
· The Second Loss is the time and effort needed to “undo the damage” to the original problem or challenge. . . that is now more complex, confounded, or extreme than it was before the flawed framework or program was tried.
· The Third Loss is to the students and staff who are now more resistance to change and/or the “next” approach because everyone trusted and invested their energy into the flawed framework or program. . . that did not (predictably) work.
Critically, this Loss is the most insidious of all as it poisons students’ and staff’s willingness to try the “next” intervention. . . even when the next intervention may actually be the “right” intervention.
So. . . there are losses here.
Whether to the individual student in-need (and those working with her or him),
or to the groups of students in-need (and an entire school community working
with them). . . there are losses here.
_ _ _ _ _
The EdSurge article asserts that education doesn’t seem to be learning from its past mistakes. Instead, education seems to move from fad to fad.
EdSurge quoted James Stigler, a distinguished professor of psychology at the University of California, Los Angeles:
It's
not that specific reform ideas are fads. It's that schools seem susceptible to
fads because people don't understand what it means to take an idea seriously.
In
reality, many ideas out there haven’t been properly tried out, because that
would mean focusing largely on how they are put into practice in classrooms.
There are probably a lot of ideas out there that are effective, but nobody
knows what they are.
_ _ _ _ _
The article then quoted Adrian Simpson, principal of St. Mary's College at Durham University in England and professor of mathematics education:
Evidence
is the magic word. It’s also the source of part of the problem.
Those
questing for evidence-based education approaches tend to rely on randomized
controlled trials, a robust form of study widely used in medicine to establish
causation. In education, that can mean field experiments that show a practice
worked in a particular context or laboratory experiments in cognitive science.
But
what [these] tell you is very powerful, but very narrow. These studies are
taken to show that certain approaches work. But, they only really establish
that the sum of all the differences in interventions caused learning for some
participants. Which specific intervention worked, and whether it would work for
other students, is hard to determine.
So
it’s tricky to translate the lessons of these experiments into learning. Researchers
also understand less about the mechanisms of how people think about, say,
fractions than how kidneys function. So the evidence provided by experiments
about specific practices in education is weaker than in other areas like
medicine where it tends to be similar from person to person. You can’t
establish laws of the classroom that will apply everywhere.
Ultimately,
there’s no quick fix for the reform cycle. Teachers should bring together
insights from a number of sources—from research about memory capacity to tips
from the teacher next door—to inform how they unlock learning for their students.
Rather than asking what they can do to make a student better with fractions, a
teacher might ask: ‘What’s causing this child to handle fractions poorly?’ That
could provide an insight that isn’t solely focused on teacher interventions
which could, nonetheless, help the student learn.
_ _ _ _ _
The necessary conclusion here is that educators need to be wary about inadvertently implementing flawed frameworks, programs, quick-fix solutions, and fads in their schools and classrooms.
As advocated by Simpson above, educators—like medical doctors—need to utilize data-based problem-solving approaches that are anchored by evidence-based blueprints.
These blueprints identify the components that contribute to the fullest understanding of a specific educational area.
An analysis of these components are used to frame the root cause analysis needed to determine the underlying reasons for students’ challenges in these specific areas. And the results of this analysis are then linked to the services, supports, and interventions needed to address the existing challenges.
It is during the Problem Identification stage that Simpson’s question above (“What’s causing this child to handle fractions poorly?”) is particularly important as it frames the root cause analyses that follow.
When moving from problem analysis to intervention, Simpson’s advice above (“Teachers should bring together insights from a number of sources—from research about memory capacity to tips from the teacher next door—to inform how they unlock learning for their students”) is particularly prescient.
Here, he is emphasizing the importance of a science-to-practice perspective when interventions are being developed. This occurs as experienced teachers utilize the evidence-based blueprint, relevant to the problem at-hand, to determine which research-based interventions have the highest probability of succeeding with an individual—and sometimes, unique—student.
Significantly, we have discussed a number of these blueprints in this Blog Series:
July 13, 2024 Part I:
The Seven Sure Solutions for Continuous Student and School Success: “If You Don’t Know Where You’re Going, Any Road Will Get You There”
[CLICK HERE to LINK to this BLOG]
_ _ _ _ _
July 27, 2024 Part II:
Are Schools Really Prepared to Address Educators’ Biggest Behavioral Student Concerns Right Now? “We’ve Got Serious Problems and We Need Serious People”
[CLICK HERE to LINK to this BLOG]
_ _ _ _ _
August 10, 2024 Part III:
Will Your School “Win the Gold” for Your Students This Year? Why the U.S. Women’s Gold Medal Olympic Gymnastics Team is a Model for All Schools (Part III)
[CLICK HERE to LINK to this BLOG]
_ _ _ _ _
August 24, 2024 Part IV:
Students' Health, Mental Health, and Well-Being Worsens Over the Past 10 Years: Summary of the August 2024 Centers for Disease Control and Prevention Report
[CLICK HERE to LINK to this BLOG]
_ _ _ _ _
We encourage you to (re)read one or more of these Blogs in the context of this Blog’s discussion.
But we also want to make one more
connection before completing this Blog Series.
_ _ _ _ _ _ _ _ _ _
Teacher Morale and Fad or Flawed Frameworks and Programs
An August 12, 2024 Education Week article reported on a survey of 1,487 public school teachers and 131 private school teachers between January and March of this year.
The article reported:
After
an uptick in morale last year, teachers nationally are saying that their mental
health has worsened and that they are less satisfied with their careers than
they were a year ago.
Teacher
job satisfaction appeared to reach an all-time low in 2022, with 12% of public
school teachers saying they were very satisfied with their jobs as they
grappled with the fallout from disruption caused by the COVID-19
pandemic. Though job satisfaction climbed last year, it slipped
slightly this year to 18% of public school teachers saying they were very
satisfied. This is still much lower than it was decades ago.
The
results come as classrooms are facing increased scrutiny
and politicization, students are grappling with mental health
concerns and increasing behavioral challenges, and teachers’
pay remains low. Even as educators report more difficult work
environments, though, they report having minimal or nonexistent programming to
support their mental well-being.
Here
are key takeaways from the report.
1.
Teachers want better working conditions and higher pay
Past
surveys have found that teachers are experiencing poor mental health.
However, teachers often point to changes in working conditions—and
compensation—as things that would improve their mental health, according to the
report.
_
_ _ _ _
2.
Teachers are increasingly concerned with student discipline
Student
behavior since the pandemic has increasingly become a concern for educators. More
public school teachers wanted to see more support for dealing with student
discipline issues. Elementary (74%) and middle school (71%) teachers, and
teachers in suburban districts, are more likely to say that more support for
dealing with discipline would help improve their mental health.
In
the survey, teachers wrote in open responses that they felt they weren’t able
to discipline students. Even when misbehaving students are sent out of class,
they come back with snacks and don’t change their behavior.
_
_ _ _ _
3.
Administrator support has an impact on well-being
Only
a small amount of educators said that mental health programming for teachers is
extensive. A larger amount—22% of public school teachers, and 24% of private
school teachers—said their schools don’t offer any type of mental health
programming.
Staff
were more likely to say their principals provided a lot of concrete support
this year compared to last year, but still, more than a quarter of the
respondents said the administration did not provide any support at all.
Survey
results also suggested that positive support from building leadership was not
the norm. 11% of public school teachers said their principals provided “a lot”
of support of their mental health, whereas 1 in 3 teachers said their
principals offered no support at all. Multiple teachers reported in open
responses that their principals had negative impacts on their well-being.
_
_ _ _ _
4.
Student and teacher mental well-being is connected
Student
mental health has become a more pressing concern following the pandemic’s
disruptions, with schools feeling less equipped to support students’
needs. Teachers who say their own mental health is negatively affecting their
work are more likely to say that students’ mental health can also have a
detrimental impact on learning.
There
are some positives: Compared to last year, a smaller percentage of teachers
reported that students’ mental health concerns are negatively affecting
learning and behavior. But a larger share of teachers also reported that
students’ mental health declined over the course of the school year, rather
than improving.
A
majority of teachers recommended at least one student receive intervention or
counseling services last school year, and roughly half of teachers said their
schools needed more counselors, psychologists, and social workers.
In
an open-ended question, a majority of teachers—83%—said they should be expected
to support students’ mental health and refer students to professionals when
needed, but the most common response was that teachers should not replace
mental health professionals.
_ _ _ _ _
Significantly, after reviewing the entire Report and its 32-question survey, it was noted that:
· Some of the survey questions had “forced-choice” answers where respondents were given:
(a) a specific metric (e.g., “Worse,” “The Same,” or “Better”; or “Negative,” “Positive,” or “Neutral”) to select for their answer; or
(b) lists of items to select from as
tbeir “answers” (e.g., “Choose from the list below those additional steps that
your district or school could take to better support student mental
well-being”; or “What steps could your district or school take to support your
mental well-being? Select all that apply”).
_ _ _ _ _
· Other questions were open-ended or free-response questions that were then coded to fit the responses from the previous year’s survey so they could be compared.
This “forced-coding” approach may have sacrificed
some of the nuances in the 2024 data relative to new issues that have emerged
only this past year.
_ _ _ _ _
Notably: The survey did not directly ask its teachers to specify, for example, the different reasons why their job satisfaction was so low, or why students’ mental health was “worse when compared to the beginning of the 2023-24 school year.”
And yet, when reading the Report, some district or school staff might incorrectly connect certain highlighted questions as “the reasons” underlying teachers’ current dissatisfaction or mental health issues. . . and then begin initiatives to “address the problem.”
Hence, this Report might inadvertently create a search resulting in the implementation of fad or flawed programs. . . as discussed earlier in this Blog.
For example, the Report provided data on “the percent of teachers saying that more/better support for student discipline-related issues is a step their districts/schools could take to improve their mental health.”
And while this is an important question, it is likely that student discipline is not the only (or even, the predominant) reason or area to target relative to interventions to improve teacher satisfaction or mental health.
And yet, if a school were to draw
this conclusion, and needlessly re-vamp its school-wide discipline approach
with a “fad or flawed” program or framework, this might result in more
discipline problems, decreased teacher mental health and wellness, and
increased teacher dissatisfaction.
_ _ _ _ _
As another example, the Report referenced “Social-Emotional Learning” in a number of questions. In fact, “Social-Emotional Learning” was the Number Two-rated “service your district or school currently offers to support the mental well-being of your students.”
And yet, the original survey never defined or described what it meant by “Social-Emotional Learning.”
But. . . if a school jumped on
this reference in this Report, and introduce a “fad or flawed” SEL program or
framework, this might result in more student problems, decreased teacher
mental health and wellness, and increased teacher dissatisfaction.
_ _ _ _ _
The ultimate point here, my friends, is:
When districts and
schools do not soundly and objectively analyze the root causes of their teacher
morale and student behavior/mental health challenges. . . and they adopt fad or
flawed programs or interventions. . . and these programs predictably do not
work. . . the district and/or school may inadvertently contribute to both a
worsening of the original problems and an increase in staff and student
resistance to try any more “recommended” approaches.
_ _ _ _ _ _ _ _ _ _
Summary
The
only Real Success in Education is when you can Explain Your Progress, and
Extend and Sustain it. One or two years of progress is accomplishment, not
success.
The
only Real Failure in Education is when you cannot explain why you have not
Progressed. . . so that you can Change it.
Howie Knoff
This is the fifth and last entry in my “Late Summer Blog Series” that highlighted some of the quotes in my June “spotlight” session at The Model Schools Conference in Orlando.
I have used these quotes to address some current issues and events in education. . . with a focus on helping schools and educational settings to really think about what they want to do—on behalf of their students—during this new school year.
While there is a lot to unpack in this Blog. . . we discussed the following:
· Districts and schools need to objectively evaluate the services, supports, interventions, and programs—using sound and accepted methodological, research, and evaluation principles and practices—that they are investigating for student, staff, and school implementation. . . before adopting, purchasing, and beginning the training and implementation process.
Many districts and schools still do not
complete (or effectively complete) this necessary due diligence process. . .
resulting in too many adopted fads, unproven practices, or flawed frameworks.
_ _ _ _ _
· Districts and schools need to closely read the original research and reports that are cited or described in the popular (education) press.
Some of these articles are written by authors who have not done, as immediately above, their methodological “due diligence”. . . some are written—given their limited editorial space—in shortened form as the authors assume that the reader will do their own “due diligence”. . . and some are written to highlight only certain elements of the original study (or those elements that the author favors) to make a specific (sometimes biased) point.
Regardless, if districts or schools
draw the wrong conclusions or take improper actions based on a popular press
article, they are nonetheless responsible.
_ _ _ _ _
·
Finally, and most importantly: districts and schools,
obviously, do need to attend to both student and staff health, mental health,
and wellness needs. But they need to, first, fully understand the root causes
of these needs so that they can address them with the most evidence-based and
proven, and highest-probability-of-success approaches.
_ _ _ _ _
As in the first quote that prefaced this Blog, educators need to avoid implementing poorly-selected approaches that do not succeed, and that both exacerbate the original problems or challenges, and create student and staff resistance to the next approach. . . (which may actually be the “correct” approach).
As in the quote prefacing this Summary Section, it is important to note that, methodologically, a sustained school-wide approach or initiative cannot be deemed “successful” until it has shown at least three years of objective, data-based progress or accomplishment. . . and until the data causally connect the approach or initiative to the progress or accomplishment.
Relatedly, there are times when well-selected approaches still do not work.
While these are certainly “set-backs,” they should not be considered “failures.”
As above, the only real failure is when educators cannot explain why the set-back occurred.
Often, once the set-back is
understood, staff can make needed “mid-course” corrections—even using the original
approach—that produce long-term, sustained success.
_ _ _ _ _
Now that virtually everyone across the country has begun their school year, we hope that this Blog and the entire Blog Series has been helpful and relevant. We encourage district and school leaders to use one or more of the Blogs in this Series as advanced reading for a staff or committee discussion.
If you would like me to join that discussion, please drop me an e-mail (howieknoff1@projectachieve.info) so we can set up a free Zoom call to look at your needs and how to close your gaps or attain the outcomes that you want for students and staff.
Together, I know that we can make this the school year that you want and that every student needs.
Best,
Howie
[CLICK HERE to read this Blog
on the Project ACHIEVE Webpage]
