May 29, 2019

Progress Monitoring for Internalizing Symptoms:
Anxiety, Depression, and Social Skills

---

From: Pent
Sent: Wednesday, May 29, 2019 11:45 AM
Subject: Progress Monitoring for Internalizing Symptoms: Anxiety, Depression, and Social Skills

Hello PENT Cadre members, SELPA Directors, and supporters,

I hope the end of the school year is wrapping up nicely for you! It's tough to complain when summer is just around the corner, but over here at UC Riverside we still have four weeks left before things wrap up, campus quiets down, and all the good sitting-under-a-tree-and-flipping-through-your-phone spots are finally up for grabs.

For my second cadre-wide email, I decided to focus on something that a lot of educational professionals struggle with: progress monitoring internalizing symptoms like anxiety and depression. As the person who teaches behavioral and social/emotional assessment in our program, after about week three in class, I start getting questions like: "Well, what if I can't see the behavior? Like, what if I feel sad but I look happy? How would someone measure that?". And at that point, I know it's time to start chatting about rating scales. Whether it's a big rating scale like the BASC 3 or the SSIS, or a relatively-smaller rating scale like the Children's Depression Inventory (CDI) or the Revised Children's Manifest Anxiety Scale (RCMAS), rating scales are our go-to for measuring internalizing behaviors for a lot of reasons. One big one is that they rely on information from people who know the student really well (like the student themselves!) telling us their perceptions of the student. These rating scales also often have normative samples that we can compare our student's responses to, in order to determine to what extent our kiddo's internalizing behaviors are like or unlike those of other kids (thanks t-scores!). However, they're also usually really long, which makes asking a teacher, parent, or kiddo to complete one every day or every week a pretty tough thing to ask. But it's really important to have that many items in a rating scale, because having lots of items is one really great way to make the information that comes out of your measure more reliable.

Forgive me, but I'm going to get real wonky real fast before I launch into discussing specific measures. We're all familiar with the term "reliability," which basically means "consistency". So, if I ask whether a measure is "reliable," I'm basically asking: are the numbers that come out of this measure consistent along some dimension of interest? When I have lots of items in a rating scale and they generally behave similarly (e.g., if I score Item 1 as a 5, I'm likely to score Item 2 as a 5 too), I can say that my measure is "internally consistent" because the information I'm collecting is pretty consistent across lots of items. This is more or less what "alpha" is measuring. Interobserver agreement is another measure of reliability, but here, we're asking about consistency across observers, not items. So, it's complicated, but it's not. Reliability just means consistency, so when I ask if something is "reliable", what I really should be asking is: reliable for what?

So, to get back to our main question: why don't we have more and better options for short measures that can be used to progress monitor internalizing outcomes? One potential reason is that it's pretty tough to develop a measure that gives you the information you want and is consistent across the dimension of interest that you care about (i.e., is reliable), while also managing to do both of those things without a lot of items (i.e., is actually usable). Bizarrely, before this year, no one had done a solid review of the measures that are out there for the school-based progress monitoring of internalizing behaviors. But thanks to a team of people with some very diligent spreadsheet skills and some serious research cred, we finally have a systematic review to help guide us. Which brings us to: Evan Dart, Prerna Arora, Tai Collins, and Beth Doll's paper entitled "Progress monitoring measures for internalizing symptoms: A systematic review of the peer‑reviewed literature ".

The authors identified progress monitoring measures for this paper that were "able to detect small changes in functioning and that [were] designed for frequent (i.e., at least weekly) administration (i.e., resistant to practice effects, practical, and feasible) to provide information on treatment progress and inform treatment decisions."

The authors do an excellent job of outlining the measures they found, but I want to go one step further and summarize a few options that folks may want to consider, provide a brief description of each, and most importantly, link to the specific place you can find the measure (or share the version that I made when I couldn't find the original measure).

ANXIETY AND/OR DEPRESSION

PANAS-C (anxiety and depression). Completed by student. Thirty items, but different versions have different numbers of items (there's a PANAS-C Short Form with 10 items, but I haven't been able to find a copy online). Some published psychometric evidence .

SAAI-C (anxiety). Completed by self or caregiver only. Twelve items. Strong published psychometric evidence . A student and I made the Word version of the student form that's linked in the title, which was developed from the original article .

CASI-PM (anxiety and depression). Completed by parent or teacher. Thirty items. Strong published psychometric evidence . Not available for free, but it's really affordable. A complete kit is $63 and includes the manual and 50 each of the parent and teacher checklists as well as score sheets.

SOCIAL SKILLS OR BROAD INTERNALIZING CHARACTERISTICS

BBRS based on SSRS (social skills). Completed by teacher. Twelve items. Strong published psychometric evidence . It's important to note that a "BBRS" is just a brief behavior rating scale, which is a term for when folks make a measure shorter by examining which items in a longer rating scale are most effective; some people also call this an Abbreviated Rating Scale . Since this is based on the SSRS, which is a copyrighted measure, I've opted not to make a version of this on my own. BUT! If you look at pages 375 and 376 of this article , the authors say exactly which items they used, so it'd be pretty easy to replicate it yourself.

BPC (broad characteristics). Completed by student or caregiver. Twelve items. Strong published psychometric evidence . Here's a link to the student form and the caregiver form .

CUSTOMIZABLE

Don't forget that rating scales are great for unobservable behaviors, but if you have observable behaviors, that's even better! Think about using daily point sheets , Direct Behavior Rating (used in this article for anxiety ), or systematic direct observation methods to track data on how kiddos are doing.

I hope that you find these resources useful, and as always, please don't hesitate to get in touch with any questions or suggestions for future topics to cover or materials to provide!

All the best,
Dr. Austin Johnson

---

Austin H. Johnson, PhD, BCBA
Licensed Psychologist (CA#29540)
Assistant Professor, School Psychology
Graduate School of Education
University of California, Riverside
Email: austin.johnson@ucr.edu
Phone: 951-827-5958
Web: ahjohnson.com