Why are Schools Wasting
their Time and Resources on Fads with Poor Research and Unrealistic Results?
Dear Colleagues,
Over the past year or more, the educational
community has become obsessed with a number of “mind-related” programs
with the hope that they will improve students’ engagement and academic
proficiency at school.
After reviewing our most-popular educational
websites, list-servs, social media platforms, and media outlets, four
predominant “mind-related” programs emerged.
Some of these programs are often confused with each other, and most of
them have not been critically analyzed. . . and yet, they are already being
used in large-scale implementations.
For example, schools in New York City, Los
Angeles, Baltimore, and Louisville have begun large-scale implementations of a Mindfulness
program. And yet, given the research (see below), these implementations
represent sociological experiments rather than scientific initiatives.
The four predominant programs are:
1.
Dr. Ellen Langer’s Mindful Learning
2.
Dr. John Hattie’s Mind Frames
3.
Dr. Carol Dweck’s Growth Mindset
4.
Dr. John Kabat-Zinn’s Mindfulness
_ _ _ _ _
In Part I of this two-part Blog message, Langer’s
Mindful Learning and Hattie’s Mind Frames were reviewed.
In this Part II Blog, Dweck’s Growth
Mindset and Kabat-Zinn’s Mindfulness are reviewed.
Across both Blogs, there is one
critical theme:
Districts
and schools need to selectively do their own reviews of the curricula,
programs, or interventions that they are considering for implementation.
At the very least, they should consult with
professionals who can provide objective, independent evaluations of these
curricula, programs, or interventions.
Conversely, schools and districts should
not make selection and implementation decisions based on testimonials, and
they should not necessarily trust research- - unless it is sound, has been
replicated, is published in a refereed professional journal, and has direct
applicability to the students and staff with whom it will be used.
Said a different ways, schools and
districts should not invest time, money, professional development, supervision,
or other resources in programs that have not been fully validated for use with
their students and/or staff.
Such
investments are not fair to anyone- - especially when they do not result in the
desired outcomes, and they create staff resistance to “the next program” which
may actually be the “right” program.
_ _ _ _ _ _ _ _ _ _
Students’ Growth Mindsets
Dr.
Carol Dweck’s Growth Mindset (Mindset: The New Psychology of
Success, 2006) is based on her long-standing and well-established research
at Stanford University that investigates how students’ cognitive self-beliefs
and attributions affect their motivation and achievement.
At its core, her research asserts that
students tend to achieve better when they have a “mindset” where they regard
their intelligence and achievement not as fixed traits (that they either have
or do not have)- - but as attributes that
can be improved through effort.
According to Dweck:
“Individuals with a fixed mindset
believe that their intelligence is simply an inborn trait- - they have a
certain amount, and that's that. In
contrast, individuals with a growth mindset believe that they can
develop their intelligence over time.”
Dweck’s research has demonstrated that
students who believe that their intelligence can be developed (a growth
mindset) outperform those who believe that their intelligence is fixed (a fixed
mindset). It also suggests that student
achievement increases when students are successfully involved in a structured
program that changes their cognitive attributions toward a growth mindset.
Finally, her research has found that
children who focus on the processes underlying learning (e.g., hard work or trying
new strategies) can improve their growth mindsets and the related benefits.
_ _ _ _ _
Dweck used a recent Education Week
article to clarify some of the functional implications of her work in the
classroom. Among her comments, she
noted:
“A growth mindset isn’t just about effort. Perhaps the most common misconception
is simply equating the growth mindset with effort. Certainly, effort is key for
students’ achievement, but it’s not the only thing. Students need to try new
strategies and seek input from others when they’re stuck. They need this
repertoire of approaches- - not just sheer effort- - to learn and improve.”
[CLICK HERE for Dweck Article]
She closed her piece
stating:
“My colleagues and I are taking a
growth-mindset stance toward our message to educators. Maybe we originally put
too much emphasis on sheer effort. Maybe we made the development of a growth
mindset sound too easy. Maybe we talked too much about people having one
mindset or the other, rather than portraying people as mixtures. We are on a
growth-mindset journey, too.”
_ _ _ _ _
In a follow-up Education Week
article, Peter Dewitt cites Hattie’s meta-analytic work (see above) that found
an effect size of .19 relative to research correlating growth mindset
approaches with student achievement.
Critically, when meta-analytic research is conducted, an effect size
of .40 is typically used as the “cut-score” where an effect results in
meaningful outcomes.
Continuing to cite Hattie’s research, DeWitt
notes that teachers would be better served by teaching students to use
meta-cognitive strategies (with an effect size of .69), and by providing
students with specific, instructive feedback on their classroom work (with an
effect size of .73).
DeWitt notes:
“Unfortunately, as important as Dweck's
research is, it is at risk of following in a long line of other important
research, like Howard Gardner's Multiple Intelligence. . . that seems to be
misused by schools.”
[CLICK HERE for DeWitt Article]
_ _ _ _ _
I discussed information related to Dweck’s
work in a previous Blog message:
[CLICK HERE to
View Earlier Blog]
In that message,
I noted that- - in order to motivate students- - teachers and students need to
consciously create consistently positive, trusting, supportive, and
collaborative classroom climates. At the
foundation of a positive classroom climate is a teacher who is caring and
supportive, and who also presents classroom materials in captivating ways.
I then described
the most effective ways that teachers can show support and caring to their
students (see the Blog for its additional commentary and explanations):
* Listen to
students with your full attention.
* Acknowledge and label students’ feelings,
while teaching and reinforcing their emotional control skills. Help students to recognize how emotions link
to interpersonal, social problem-solving, conflict prevention and resolution,
and emotional coping skills.
* Talk with your students using a
problem-solving approach, and teach and model effective problem-solving in
different situations.
* Talk with students using an appropriate
volume, tone of voice, and level of respect—even under “emotional” conditions.
* Give students time to process their
feelings, thoughts, issues, and responses.
In other words, when needed, be patient, don’t talk too much, and give
your students a chance to work things out on their own.
* Remember to reinforce your students for
Good Choices, while teaching and prompting them to self-management and
self-reinforce themselves.
* Finally, give students hope.
_ _ _ _ _
Relating this to a growth mindset, I noted:
“Students need encouragement
for their growth, progress, and effort—even if they are not always
‘perfect.’ Help them expect and belief
that they can improve and succeed over time.
Give them opportunities to see different situations in different
ways. Critically: give them a
chance to see themselves as positive, productive, valued, and valuable
individuals.”
_ _ _ _ _ _ _ _ _ _
Mindfulness: A Fad without Facts
Mindfulness has been popularized by
Dr. John Kabat-Zinn (Mindful Meditation, 1995), an Emeritus Professor of
Medicine at the University of Massachusetts Medical School and creator of the
Stress Reduction Clinic and the Center for Mindfulness in Medicine, Health
Care, and Society.
With a goal of helping people to cope with
stress, anxiety, pain, and illness, Kabat-Zinn integrated meditational
practices from the Buddhist tradition with yoga and medical science into a mindfulness-based
stress reduction (MBSR) program.
A number of national and international
groups have adapted his work to schools and students- - among them The Inner
Kids Program (Los Angeles), MindUP (The Goldie Hawn Foundation), Mindful
Moment (The Holistic Life Foundation), Rise-Up (Mindful Schools),
and Mindfulness in Schools Program (Great Britain).
The critical question about all of these
programs (or others) involve whether they have been:
* Independently validated across. . .
* Multiple randomly-selected
communities and school sites, involving. . .
* Students who are representative of
students in communities nationwide;
and whether the
participating schools and students have been:
* Compared with randomly-selected comparison
schools that received the same amount of time and training relative to
students’ attention and emotional control (just not through a Mindfulness
curriculum). . .
* Evaluated on outcomes using objective,
reliable, and valid measures completed by different observers- - including the
students themselves. . . where
* These measures were given at least twice before
the program was begun, multiple times as the program was implemented, and at
least twice (after at least 6 months, and then 12 months) after the program
was over?
That is, have these programs been
implemented with fidelity and objectively evaluated in ways that demonstrate
that they produce meaningful results, and that the results are directly related
to the programs (and not due to attention, the individual doing the
implementation, or the school or students selected).
Indeed, as with any medical or psychological
intervention, it is essential to field-test and demonstrate that the
Mindfulness programs above are clearly effective- - and that they are able to
sustain their effects- - before any large-scale adoption occurs.
But this has largely not happened. Indeed, as noted earlier in this Blog,
Mindfulness (and its roots in meditation) have become a media-fed freight train
across this country. . . largely on the strength of adoptions by schools in New
York City, Los Angeles, Baltimore, Louisville, and others.
But, beyond this, let’s think ahead and use
some common sense.
One of the potential problems with Mindfulness
is that- - even if it works- - any improvements in self-control,
self-awareness, or attention do not necessarily translate into student
improvements in demonstrating social, behavioral, emotional coping and control
skills.
And these are the outcomes that educators
are interested in.
Thus, just because students are able to be
more attentive and focused on the present, this does not mean that they have
learned, mastered, and are able to apply the interpersonal, social
problem-solving, conflict prevention and resolution, or emotional coping skills
needed to deal with that present.
If we are going to invest the money, time,
and training that these Mindfulness programs require, why are we not,
instead, investing in the evidence-based social and emotional skills programs
proven to actually produce behavioral
change?
_ _ _ _ _
And so. . . speaking of evidence-based
programs, what does the recent Mindfulness research look like?
A review of some of the most-recent
Mindfulness studies, identified the following four published journal
articles:
Black, D. S. &
Fernando, R. (2013). Mindfulness training and classroom behavior among
lower-income and ethnic minority elementary school children. Journal of
Child and Family Studies.
Klatt, M., et al. (2013).
Feasibility and preliminary outcomes for Move-into-Learning: An arts-based
mindfulness classroom intervention. Journal of Positive Psychology,
8(3), 233-241.
Wisner, B. L. (2013). An
exploratory study of mindfulness meditation for alternative school students:
Perceived benefits for improving school climate and student functioning. Mindfulness.
Kuyken, W., et al.
(2013). Effectiveness of the Mindfulness in Schools Programme: Non-randomised
controlled feasibility study. British Journal of Psychiatry.
Here is a summary of these four studies:
Students
* Two
of these studies involved elementary school students (one with 400 students;
one in two classrooms)
* The
other two studies were at the secondary level (one in an alternative high
school; the other involving 500 12 to 16-year old students)
COMMENT:
Are we really ready to conclude, from four studies in four specific
locations (one in Great Britain), that any Mindfulness curriculum or
intervention has actually worked?
_ _ _ _ _
Student
Selection and Comparison Groups
* NONE of the student samples in these four
studies were randomly selected
* Only one of the four studies used a
Comparison Group (that did not receive the curriculum or intervention). Critically, the setting where this Comparison
Group went to school was not randomly selected.
COMMENT:
Thus, these samples were “samples of convenience.” Moreover, biases in selecting these samples (for
example, choosing students with the highest probability of behavioral change,
who were most motivated to produce results for those implementing the program,
who had specific personality characteristics that would most respond to the
intervention) may have made any positive Mindfulness results more probable.
Even with the study using a Comparison Group,
there are questions - - as to whether the Comparison and Intervention groups
were similar, and whether Comparison Group students (again) were representative
of typical students in the population.
Indeed, overall, were the student
participants in all of the studies representative of the students in other
schools now considering a Mindfulness program?
_ _ _ _ _
Study Length
* The four studies, respectively, taught
their Mindfulness curriculum for 5 weeks, 8 weeks, 8 weeks, and 9 lessons.
_ _ _ _ _
Data Collection Methods Used
* All of the studies used only pre-post
measures. The measures involved
interviews, surveys, and self-reports. No objective assessment tools or classroom
observations were included.
COMMENT:
Virtually all of the data collected to determine the impact of the
Mindfulness programs were subjective in nature, and did not control for
“positive result biases”- - either student or teacher expectations that the
curriculum or intervention would work.
Moreover, pre/post studies do not accurately evaluate positive trends that may have pre-dated the intervention. Indeed, to accurately assess trends, you need at least two pre-test data points so that trends occurring prior to implementation are picked up.
Overall:
given the absence of any tools or approaches that collected objective
and unbiased data, the validity of the four studies’ results cannot be assumed.
_ _ _ _ _
Study Results
COMMENT:
While all four studies reported improved behavioral outcomes (one study
reported that stress and depression symptoms decreased)- - as noted above, these
results have not been demonstrated to be reliable, valid, or directly related
to the Mindfulness curriculum or intervention.
_ _ _ _ _
Follow-Up of Results
*
After completing their Mindfulness implementation, two of the studies
(respectfully) collected follow-up data after 7 weeks and 3 months. Two of the studies did not collect
follow-up data at all.
COMMENT:
Most psychological interventions need to show that their behavioral
effects persist (in the absence of more or continued adult or therapist
guidance) for at least 6 months. . . if not 12 months. None of the studies collected these
data.
Thus, we have no idea if any of the results
(that might actually have occurred) continued.
If an intervention’s positive results do not
continue, then we need to question whether the time and effort invested in any
initial results either was worth it, or was due to the presence of the
individuals helping the students to learn and implement the intervention.
_ _ _ _ _
The Mindfulness BOTTOM LINE: Without reviewing data from the original
development of the Mindfulness curricula or interventions used, these studies
suggest that- - if anything- - “the jury is still out.”
None of these results objectively
proved that the Mindfulness approaches had any short- or long-term effect on
student behavior.
And, none of these results come close
to making a compelling argument for adopting any of these Mindfulness
approaches in any other classroom, school, or district.
_ _ _ _ _ _ _ _ _ _
Summary
The reason why the four mind-related
programs or approaches reviewed in these two Blogs have generated such interest
is that teachers and other educators are truly hoping that they will help them
address the following “universal outcomes of education”:
* To teach students social, emotional, and
behavioral awareness, control, and self-management skills; so that. . .
* They have learned, mastered, and can
independently use these skills in the classroom and in other school-related
social/interactional situations; so that. . .
* Students’ academic engagement and success
is enhanced, and the needs of students demonstrating academic struggles and/or
behavioral challenges are addressed.
And yet, the brief review of the research
and practice in this and my last Blog has concluded that:
* Langer never intended to use her work
in the schools. Thus, her work needs
to be adapted and objectively validated for classrooms and school-aged
students.
* Hattie’s Mind Frames are useful in
thinking about the teacher and/or teaching mindsets that relate to student
achievement. But his constructs and
generalizations still need to be implemented through specific, effective, and
field-tested step-by-step strategies- - that, again, are independently validated.
* Dweck’s Growth Mindset research is
impressive, but its application has spawned a number of different, largely
unvalidated, implementation approaches with different components, methods, and strategies;
many of these approaches do not provide the specificity such that they can be
independently replicated; and Growth Mindset research has yet to
demonstrate a significant effect size on student achievement (especially in
large-scale studies).
* Kabat-Zinn’s Mindfulness- - at
least, at present- - is simply an educational bandwagon that needs to be
derailed. None of these results of the recent research reviewed
objectively demonstrate that the Mindfulness approaches have any short- or
long-term effect on student behavior. And,
none of these results come close to making a compelling argument for any
classroom, school, or district adoption.
_ _ _ _ _
Critically- - relative to the “universal
outcome” above, the underlying science and practice is well-established.
The primary scientifically-based components needed
are:
* Staff,
Student, and Parent Relationships that establish Positive School and Classroom
Climates
* Explicit Classroom and Common School Area Expectations supported by Social, Emotional, and Behavioral Skill/Self-Management Instruction (that are embedded in preschool through high school "Health, Mental Health, and Wellness" activities)
* School-wide and Classroom Behavioral Accountability systems that include Motivational Approaches that encourage and reinforce students’ "Good Choice" behavior
* Consistency- - in the classroom, across classrooms, and across staff, time, settings, and situations
* Applications of the above four components across all Settings in the school, and relative to Peer Group interactions (specifically targeting teasing, taunting, bullying, harassment, hazing, and physical aggression)
This is the already-existing science. This science is universal and indisputable. Thus, educators need to use this blueprint
when “new” approaches emerge- - either to validate their utility, or to reject
them as unsound, not worthy of our time, and potentially dangerous.
In the end, we do not have time to
experiment on students, staff, and schools.
Approaches need to be field-tested and empirically demonstrated to work across
many different students, staff, and schools before large-scale
implementation.
To do anything less is wasteful at best. . .
and indefensible at worst.
_ _ _ _ _
As the school year continues, I
thank you for the time, dedication, care, and support that you give every day
to your students and their families, and to your colleagues and collaborators.
I
look forward to your reactions and comments.
Let me know if I can help you, your students, or your school/district in
any way.
Best,
Howie
No comments:
Post a Comment