Finding, Sorting, Analyzing, and Synthesizing the (Right) Data
[CLICK HERE to read this Blog on the Project ACHIEVE Webpage]
Dear Colleagues,
Introduction
Students across the country are demonstrating more social, emotional, and behavioral challenges the past two years than ever before.
The problems are so significant that, this past October, the American Academy of Pediatrics, the American Academy of Child and Adolescent Psychiatry, and the Children’s Hospital Association formally declared our country’s child and adolescent mental health status “a national emergency.”
Adding to the considerable Pandemic-related statistics, a JAMA Pediatrics study published just this week looked at the adolescent suicide rates (ages 10 to 19) in 14 states from 2015 to 2020 (which included Year 1 of the Pandemic).
The results indicated that adolescent suicide in these states increased by 10% in 2020 compared with the average rate over the pre-Pandemic period from 2015 to 2019.
There was some variability, however. For example, California, Georgia, Indiana, New Jersey, Oklahoma, and Virginia all saw statistically significant suicide increases, while Montana saw significant decreases.
Concurrently, even more extensive social, emotional, behavioral, and mental health challenges have been well-documented over the past year for all school-aged students. . . beyond the suicide statistics above.
The implications of these increases were discussed in Part I of this two-part Blog Series:
Schools Must Use Effective Practices to Screen and then Validate Students’ Mental Health Status (Part I)
YES: Teachers Should Help Screen Students for Social, Emotional, and Behavioral Challenges. . . NO: That’s NOT Where the Screening Process Ends
_ _ _ _ _
Part I addressed the implications that, because of the increase in students’ social-emotional challenges, many schools have re-invested in the informal and formal screening processes that help to identify these students—determining which ones need strategic or intensive multi-tiered services, supports, strategies, or interventions.
The “thesis” of Part I was that, while general education teachers are an important “early warning” layer to the student screening process, they are only the beginning and not the end of the process.
To this end, most of Part I described ten essential practices needed in an effective mental health screening-to-services process. The state goal of this process is to differentiate between (a) students having minor, moderate, or significant social, emotional, and behavioral challenges versus (b) students demonstrating concerns that are more momentary, transient, developmentally-expected, or situational in nature.
Part I emphasized that this process also involves the use of (a) multiple assessment approaches or tools; (b) completed by multiple raters (including the student him or herself); and (c) that assess student behavior across multiple settings.
In this Part II of this Blog
series (below), we will describe the six ways to collect social, emotional, and
behavioral student data in the most reliable and valid ways.
_ _ _ _ _ _ _ _ _ _
How Do You Collect Screening-to-Service Information and Data?
Across the screening-to-services continuum, there are many ways to collect important social, emotional, and behavioral (SEB) student data. Ultimately, to be useful and usable, these data must be reliable and valid (this is a required psychometric principle), and they need to identify and differentiate among specific, legitimate student concerns.
The different ways to collect social, emotional, and behavioral screening data can be summarized in the acronym RIOTS: Review, Interview, Observe, Test, and Self-report. This acronym is operationalized below with a few specific examples of how to use them in the screening-to-service process.
(R) Review
Every school has records, files, electronic spreadsheets, and other documents on every student in their Student Information (or Data Management) Systems or their physical file cabinets.
There are (a) “official” records—like electronic or paper Cumulative Folders; (b) instructional folders— like work sample folders collected by teachers; (c) clinical files—like case notes written and kept by counselors or school psychologists when students are attending therapy groups or are being evaluated through the school’s pre-referral, multi-tiered services process; and (d) plan-related documents—like Academic or Behavioral Intervention, 504, and Individual Education Plans.
At the beginning of any social, emotional, behavioral (SEB) screening-to-services process, the Multi-Tiered Services and Support (MTSS) Team (see below) should review these records to identify students who are, were receiving, or may have benefitted from SEB and/or mental health services. These records also document (a) the past psychoeducational status and progress of a student-of-concern; (b) administrators, teachers, and support staff of record; and (c) family, medical, developmental, or situational (e.g., school or district moves or transitions) information of note.
All of this record-related information is potentially useful if the screening process transitions into a root cause analysis, and then service, support, and intervention activities.
For some schools, all students’ electronic records (and status) are reviewed annually or (even better) when quarterly report cards or triannual interim academic assessments are completed. These reviews represent the school’s formal SEB (and academic) screening process.
For other schools, these reviews are supplemented by teacher-completed SEB screeners (see Part I of this Blog series).
Regardless, the Review of Student Records is an often-missed activity in the screening-to-service process. It sometimes is amazing what essential current and cumulative information is in these (especially electronic) records that can help teachers to more fully understand their students and students-of-concern.
Indeed, in some cases, previous successful
interventions for specific students are well-documented in their records. These
past interventions are the same, current interventions that will
help a student now to be successful... they simply were not
communicated from one teacher to another, for example, from one school year to
the next.
_ _ _ _ _
(I) Interview
Because the experiences with and perceptions of students may differ, when educators have “common” concerns about the social, emotional, and behavioral (SEB) status of different students, these concerns should be formally communicated to and recognized by a school’s MTSS Team for action.
An important “next step” in the screening-to-service process involves interviews with the different educators (as well as, for example, previous teachers, intervention specialists, and parents/guardian) to (a) specifically and operationally clarify the concerns; (b) check the similarities and differences across different people’s student-related perceptions and experiences; and (c) begin the information-gathering process to help determine the root cause(s) of the problem(s).
A significant gap here is that some schools do not have a duly constituted MTSS Team (consisting of the best academic and SEB assessment and intervention experts in or available to the school), and this Team often meets on a “case-by-case,” rather than a regularly-scheduled weekly or bimonthly, basis.
In the absence of regularly-scheduled meetings, information on specific students-of-concern with SEB challenges may not reach the MTSS Team until the student is exhibiting significant or prolonged challenges. In the absence of a multi-disciplinary MTSS Team, the screening-to-service process may involve only selected professionals (typically, the administrator, counselor, and general education teacher(s) currently teaching the student-of-concern)... and this process may lack the important perspective of a missing multi-disciplinary professional.
Moving on, there are many different ways to interview teachers, other educators, and parents/guardians. Depending on when the interview is occurring across the screening-to-service continuum, the interview may focus on (a) clarifying concerns, (b) gathering or eventually validating student background or current status information, (c) identifying hypotheses or validating these hypotheses to determine the root cause(s) of a student’s challenges, or (d) evaluating a staff member’s commitment and ability to implement strategic or intensive interventions.
As such, interviews can be open-ended, semi-structured, or structured—the latter often based on an evidence-based diagnostic protocol that guides the interviewer through a decision-tree that results in the clarification, validation, and analysis of a specific student challenge.
One of the more specialized interviews is the Social-Developmental History. This is typically completed with a student’s parents/guardians, and it results in information regarding the historical, familial, biological/ physiological/neurological, developmental, fine and gross motor, speech and language, and social-behavioral past and present of the student.
The Social-Developmental History often provides an important baseline regarding the student-of-concern, and it can both generate and dismiss specific hypotheses about the root cause(s) of a student’s challenge(s).
Other diagnostic interviews can
screen for or analyze specific internalizing (e.g., anxiety or depression) or
externalizing (e.g., anger or explosiveness) SEB or mental health conditions or
circumstances.
_ _ _ _
(O) Observe
An independent and objective observation of a student-of-concern is an essential (early) part of the screening-to-service process. These observations should occur both in the settings where the challenges seem to exist, as well as settings that are not problematic or where the student is “successful.”
These observations are particularly important both to validate concerns that may arise from a screening instrument or process, as well as to (a) identify any common triggers or sequences of the challenging behavior; and (b) determine if other students have the same challenges (but have not been referred), or are involved either as triggers or reinforcers of the student-of-concern’s challenging behavior.
Observations also are important because teachers are often processing hundreds of micro-events in their classrooms at any point in time, and they may inadvertently miss important interactions involving the challenging student, his or her peers, or even the curriculum and instruction or classroom management process. Thus, having another set of “eyes and ears” for a time in the classroom can help to close any observational gaps.
Given all of this, MTSS Team
members should be trained and available to do classroom observations as needed
across the screening-to-service timeframe. Collectively, the Team should develop
reliable, valid, and systematic observational protocols and processes for the
most commonly referred SEB student challenges. Moreover, the Team should develop
consistent and valid ways to collect, combine, and analyze all observational
data to maximize their contributions to the screening-to-service process.
_ _ _ _ _
(T) Test
Testing usually involves a student’s face-to-face (or virtual) response to specific structured SEB questions that are asked by a teacher, support educator, or (especially) related services professional (e.g., counselor, school psychologist, or social worker). The testing could involve a screening, SEB diagnostic or personality, or progress monitoring/evaluative assessment, respectfully. Thus, some of the testing will focus on a broad range of possible SEB challenges, and some will address more narrowly specific concerns (e.g., attention deficit hyperactivity, autism spectrum disorders, stress and trauma).
As discussed in Blog Part I, any SEB or personality testing should be part of a broader MTSS Team process, and involve psychometrically-sound instruments that are part of a multi-instrument, multi-respondent, multi-setting, multiple-gated process.
Critically, SEB testing should use tests, tools, inventories, surveys, or other instruments that (a) differentiate between students with SEB skill gaps and those with motivational problems; (b) assess students’ strengths and competencies, rather than just challenges or deficits; and (c) are administered in ways that minimize subjectivity, bias, and fatigue—the latter occurring, for example, when practitioners attempt to complete too many assessments with a student because of limited time.
When different MTSS professionals
complete different tests with the same student, the results should be
integrated into a cohesive whole. That is, each professional should not write
or report on their own, individual test results. Instead, all of the results
should be interdependently synthesized and reconciled to reflect “the whole
student” and his/her comprehensive SEB status.
_ _ _ _ _
(S) Self-Report
The social, emotional, and behavioral screening-to-services process should include opportunities where students can formally or informally discuss or “report” their own concerns, self-analyses, or needs for themselves. Self-report opportunities often are completed with a teacher during the screening process, or a mental health professional during the diagnostic or root cause analysis phase of the process.
When a planned activity, Self-reports typically occur in the context of a screening or diagnostic (clinical) interview.
Relative to screening, teachers should receive training (and, perhaps, supervision) in how to conduct a student interview so that the quality and objectivity of the information received is maximized. In addition, teachers need to learn how to avoid triggering undue student emotionality. . . thereby exacerbating or accelerating the student’s concerns.
Relative to clinical interviews, mental health practitioners need to balance open-ended listening with questions that help students to share their major concerns, SEB reflections, and hypothesized reasons for the concerns. Like interviews, self-report protocols can be structured, semi-structured, or open-ended. Clinical interviews need to be age- and developmentally-appropriate, and students may be provided a sample of the interview questions ahead of time if strategically helpful.
Another type of self-report involves social-emotional or behavior rating scales that are completed, typically, by teachers and/or parents. These scales have demonstrated internal reliability and external validity, and they often are normed using specific, representative student populations. This allows a teacher or parent self-report to be compared to a larger sample of students so that the severity or intensity of an individual student’s SEB challenge(s) can be quantified.
Some self-report scales—for
example, the Behavioral Assessment System for Children (the BASC)—have
protocols that are independently completed by teachers, parents, and the
student-of-concern him- or herself. By cross-walking the results of these three
self-reports, there is more “confidence” in the existence of specific student
concerns, and the self-report information can be used in follow-up interviews
to get a more personal perspective of each individual’s (teachers, parents, and
the student-of-concern) challenge-related perceptions or experiences.
_ _ _ _ _ _ _ _ _ _
Summary
One of the goals of this Blog Part II is to help educators and MTSS Team members recognize the many sources of information and data within the social, emotional, and behavioral screening-to-service process. In discussing this process, we have emphasized the importance of:
· Using psychometrically-sound procedures and instruments that are part of a multi-instrument, multi-respondent, multi-setting, multiple-gated process; and
· Connecting SEB screening activities with diagnostic root cause analysis activities that then link to strategic or intensive services, supports, strategies, or interventions (as needed).
We have also emphasized that schools do not necessarily need to conduct formal screenings using, for example, behavior ratings scales where every student in a school is assessed. If reliable and valid, teacher reports, cumulative record reviews, and other sources of information may be all that is needed for schools to successfully complete a sound screening process.
Given the information and discussion in both Blog Parts I and II, it is recommended that districts and (their) schools:
· Write (or update) their comprehensive Multi-Tiered Mental Health Needs Assessment, Screening, Implementation, and Evaluation Plan;
· Recognize that the development, implementation, and evaluation of this Plan should involve the ongoing participation of the District’s mental health staff, as well as community-based mental health agency and service representatives;
· Include needed resources and the training of teachers, administrators, and related service professionals in the Plan; and
· Include “Case Study” practice, ongoing consultation and coaching, and periodic formative and summative evaluations of the process so that students in need are accurately identified and served in objective, data-based, and timely ways.
All screening-to-service implementation processes should be overseen by each school’s Multi-Tiered Services (Child Study, or Student Assistance) Team which (a) includes the best trained academic and social, emotional, behavioral assessment and intervention specialists in or available to the school; and (b) meets on a regular basis to address the needs of students in the school who are exhibiting academic and/or behavioral challenges.
In the end, this process should focus on the academic and social,
emotional, and behavioral independence, self-management, progress, and
proficiency of all students—especially those who are struggling or presenting
with specific challenges.
_ _ _ _ _ _ _ _ _ _
I hope that the two Blogs in this Series resonate with you and motivate you to think—especially toward the end of this school year—about how you want to begin the next school year relative to effectively identifying and serving your students.
We all have had many Pandemic-related experiences over the past two-plus years, and our students do not always have the awareness, insight, or resources to understand and address their social, emotional, or behavioral needs.
As always, if I can help you in any of the areas discussed in this Series, I am always happy to provide a free one-hour consultation conference call to help clarify your needs and directions on behalf of your students and colleagues.
I am currently completing a number of Needs Assessments and Resource Analyses for different school districts in the areas of (a) school improvement, (b) social-emotional learning/positive behavioral discipline and classroom management systems, and (c) multi-tiered (special education) services and supports.
The results are a research-to-practice Action Plan and implementation blueprint that helps many districts to reach both their short-term and long-term student, staff, and school goals and outcomes.
Please feel free to reach out if you would like to begin this process. . . or just throw some ideas around.
Best,
Howie
[CLICK
HERE to read this Blog on the
Project ACHIEVE Webpage]
