ABA Journal Club: A Response from Dana Reinecke

Welcome back to ABA Journal Club! One of the tenets of ABA is to provide evidence-based practice. The best way to help us do this is to keep up with the literature! Each month, Sam Blanco, PhD, LBA, BCBA will select one journal article and provide discussion questions for professionals working within the ABA community. The following week another ABA professional will respond to Sam’s questions and provide further insight and a different perspective on the piece.

Check out last week’s discussion questions here!

Behavior analysts engage in many different professional activities, many of which are more or less likely given specific clinical or research settings. For example, some behavior analysts who work with individuals with disabilities are likely to conduct preference assessments and use token economies, while those who work with organizations are less likely to use these technologies. One part of the behavior analyst’s repertoire that is always important, however, is the careful and accurate collection of data. This skill set is necessary for understanding and assessing behavior, as well as for ongoing monitoring of the effectiveness of behavioral interventions. 

It is important to understand not just how to measure behavior, but when to use each type of measure.  LeBlanc, Raetz, Sellers, and Carr (2016) describe some of the critical questions that should be considered when choosing a measurement procedure and offer a clinical decision-making model to guide behavior analysts in making these choices.  This article is useful for helping trainees to practice choosing measurement procedures, and reminding more experienced behavior analysts about the considerations involved in measurement. 

LeBlanc, L. A., Raetz, P. B., Sellers, T. P., & Carr, J. E. (2016). A proposed model for selecting measurement procedures for the assessment and treatment of problem behavior. Behavior analysis in practice, 9(1), 77-83.

Why is this article important for practitioners to read?

Measurement of behavior is one of the most important activities that a behavior analyst engages in.  Without accurate, meaningful measurement, assessment of both behavior and intervention effectiveness is impossible.  Trainees should read this article to learn about the important variables involved in choosing appropriate measurement systems, and more seasoned behavior analysts should read it to remind themselves about those variables.  Even though the article is focused on the measurement of problem behavior, the same principles can be applied to the measurement of behavior targeted for increase.

The model proposed by the authors incorporates several variables (such as observability of behavior and personnel resources.) Are there any other variables you might consider when selecting a measurement procedure?

Length of observation period might be a relevant factor in choosing a measurement procedure.  Consideration of the availability of resources may be influenced by the goal for how much observation is desired.  To use the case example provided by the authors, Joey’s teacher and aide might not be able to continuously record his work engagement throughout the day, but one of them might be able to do so for a limited sample of each day.  They could choose to conduct continuous measurement during a sample interval, and compare it to the longer period of discontinuous measurement to ensure that the discontinuous measure does not result in an over- or under-estimate of behavior. 

Table 1 clearly outlines each form of measurement along with strengths and limitations. Discuss the forms of measurement you frequently use and the limitations to incorporating other forms into your current practice.

As a consultant, I need to measure behavior based on limited samples when I can observe, and I also need to design data collection plans for the staff who are there for the rest of the week.  Staff are often responsible for more than one student, and may not have the resources to conduct continuous event recording.  Behavior is also often not discrete (e.g., crying) or occurs too frequently to count (e.g., stereotypy).  I often use partial-interval recording when I consult in school programs.  This allows for a very easy, non-intrusive overview of the pattern of behavior across the school day.  Another common measure is duration of behavior, because it is also relatively easy to start a timer when behavior begins, and stop it when it ends.  Frequency data are pretty rare in my practice, and reserved for low-frequency behavior that only occurs under specific circumstances.

In Figure 1, the authors provide a flow chart for easily selecting the most appropriate form of measurement. Many of the questions are directly related to observer resources. In this article, the term “resources” relates directly to the ability of personnel to continuously monitor the behavior. Are there any other factors you would consider in relation to personnel? If yes, how do you typically address those factors?

When training staff to collect data, it’s important to acknowledge any unintended bias.  Depending on the staff member’s level of experience, I will conduct more or less frequent IOA to reduce the risk of observer drift, and will also regularly review behavioral definitions to ensure that we are still talking about the same thing.

In discussing the behavior being measured, the authors write: “If the behavior can occur at any time, consider all dimensions of the response and select the ones that are most critically important to fully capture the important features of the behavior and the potential change in the behavior that may occur due to intervention” (p. 81).  How do you determine which dimensions of the response are the most critically important? Can you think of an example?

The importance of each dimension of the behavior will depend on the situation, the behavior, and the target or goal for the behavior.  For example, if a student is able to answer social questions but only does so after a delay, we would want to target, and therefore measure, latency to respond instead of frequency.  Or, a learner might engage in several very brief tantrums throughout the day.  In that case, I would expect that duration would be less important, and frequency a more meaningful measure.  By contrast, if a learner engages in one or two very long tantrums per week, we would want to measure duration and possibly intensity, rather than highlighting frequency.

One of the limitations of this paper is that the model it presents has not been empirically tested. What might such an empirical study look like?

One possible way to validate this model would be to provide several experienced behavior analysts with some case studies, and ask them to use the model to recommend measurement procedures for each case study.  High levels of agreement between the behavior analysts might indicate some validity for the model.  Further validity could be achieved by using the model to select measures, and then conducting those measures and comparing them to true values (e.g., permanent products or continuously-collected event recording).

About The Author

Dana Reinecke, Ph.D., BCBA-D is a doctoral level Board-Certified Behavior Analyst (BCBA-D) and a New York State Licensed Behavior Analyst (LBA).   Dana is a Core Faculty member in the Applied Behavior Analysis department at Capella University.  She is also co-owner of SupervisorABA, an online platform for BACB supervision curriculum, forms, and hours tracking.  Dana provides training and consultation to school districts, private schools, agencies, and families for individuals with disabilities. She has published her research in peer-reviewed journals, written chapters in published books, and co-edited books on ABA and autism.  Current areas of research include use of technology to support students with and without disabilities, self-management training of college students with disabilities, and online teaching strategies for effective college and graduate education.  Dana is actively involved in the New York State Association for Behavior Analysis (NYSABA).