Byrne K, Lord C (2023) The Brief Observation of Social Communication Change (BOSCC): Procedures, Strengths, Limitations, and Future Directions. Arch Paediatr Dev Pathol 6(1): 1026.

The Brief Observation of Social Communication Change (BOSCC) was developed as a treatment response measure of broad social communicative behaviors and other behaviors associated with ASD. It has been validated for use with minimally verbal individuals, those who speak in phrases, and young children under the age of 8 years who are verbally fluent. The validation study for older children, adolescents, and adults who are fluent speakers is currently underway. The current review discusses the process by which the BOSCC is implemented in research, as well as its strengths, limitations, and future directions.

The field of intervention research for individuals diagnosed with autism spectrum disorder (ASD) has historically struggled to identify and implement a shared battery of measures by which to measure social communication skill improvement in response to intervention. Commonly used outcome measures include caregiver or clinician report questionnaires, which provide indispensable information about a child, but can be biased due to “unblinding” and should be collected in addition to observational measures [1]. Others were not designed to measure subtle changes in response to treatment; rather, they were designed to diagnose or measure symptom severity, such as the Autism Diagnostic Observation Schedule (ADOS; [2]). The immense number of disparate outcome measures used in intervention research begets a lack of consistency in measurement across studies, making comparisons of treatment effects difficult. The Brief Observation of Social Communication Change (BOSCC) provides a novel method to measuring subtle but meaningful changes in social communication skills and other behaviors for individuals diagnosed with autism spectrum disorder (ASD) in a way that can be used across diverse sites and research studies.

The BOSCC was developed as a treatment response measure of broad social communicative behaviors and other behaviors associated with ASD. The BOSCC is split into four versions based on the age and language level of the child. The Minimally Verbal (MV) BOSCC is suitable for individuals of any age who are nonspeaking, use single words, or who use practiced, rote, nonflexible phrases. The Phrase Speech/Young Fluent (PSYF) BOSCC is suitable for individuals of any age who speak in flexible, generative phrases, or young children under the age of 8 years who are verbally fluent (i.e., speak in complex sentences). The Fluent 1 (F1) BOSCC is suitable for children ages 6 through 10 years who are verbally fluent. The Fluent 2 (F2) BOSCC is suitable for individual’s ages 11 years through adulthood who are verbally fluent.

The BOSCC MV and PSYF show convergent validity with other cognitive and adaptive measures such as the Mullen Scales of Early Learning [3] and Vineland Adaptive Behavior Scales, Third Edition (VABS-III; [4]). The BOSCC has been validated as being sensitive to changes in a short period of time, while ADOS CSS scores [5] and VABS Socialization and Communication scores [6] in the same period of time was not. These studies suggest that the BOSCC may be more sensitive to changes in a short period of time as compared to other commonly used outcome measures. For full psychometric properties of the BOSCC MV and PSYF, see [5] and [6]. The validation studies for the BOSCC F1 and F2 are currently underway.

Across all versions of the BOSCC, the assessment and scoring are done separately. The assessment consists of a standardized set of play materials and can be easily administered by anyone including caregivers, undergraduate research assistants, trained clinicians, etc. The standardized materials differ depending on the version of the BOSCC that is used. The MV and PSYF assessment kits are comprised of toys for self-directed play (which differ depending on the age of the child) while the F1 and F2 assessment kits are comprised of games and question cards. The assessment is videotaped and later scored by a trained and reliable coder on approximately 20 distinct items. Items are scored on a 0-5 scale, with higher scores indicating greater degree of difficulty on a given item. While it is not required to have prior research experience to reliably code a BOSCC, it is helpful to have knowledge of general child development and core characteristics of ASD.

The items differ depending on the version of the BOSCC being used. Nonetheless, coding procedures are the same. The BOSCC is split into two equivalent segments: Segment A and Segment B. Segment A is watched once, then all items are coded. Coders follow a series of yes/no questions through decision trees until they arrive at a final score for each item. Segment A is then watched a second time. Each code is subsequently changed or confirmed based on the second (and final) watch. This procedure is repeated for Segment B. Final scores are comprised of the average of Segment A and B.

The BOSCC has many strengths as an outcome measure of treatment response, one of which being its user-friendly and flexible nature. The BOSCC can be collected remotely via videoconferencing using caregivers as the play partner. If administering the BOSCC is impossible for a given sample, the BOSCC coding scheme can be retroactively applied to parent child interaction videos [7], videotaped administrations of the ADOS [8-10] or other structured observations consisting of a participant and play partner [11].

Furthermore, the BOSCC is completed in 12- to 14-minutes. Some studies have applied the BOSCC coding scheme to videotaped parent child interactions as short as 10 minutes in length [12]. The brevity of this assessment and its potential utility with other structured observations makes it relatively easy to add to assessment batteries or research protocols. Yet, it is important to note that differences in standardized vs nonstandardized BOSCC administrations have not been thoroughly examined, so it is recommended that researchers use the standardized administration whenever possible.

Another strength of the BOSCC is its relatively low degree of bias. The BOSCC must be coded by someone who is blind to the child, time point, and treatment group to ensure that codes are not influenced, even unintentionally, by knowledge of these characteristics. This significantly limits the possibility of bias as compared to other measures of treatment response, such as parent report questionnaires. While this quality of the measure presents as a major strength for intervention research, it also comes with its challenges.

Using the BOSCC for research requires two separate “teams”: assessors and coders. Because coders must be blind, it is essential that these individuals not contribute to any assessment of the participant. Furthermore, coding teams must be comprised of at least two trained coders, and oftentimes more, depending on the number of videos to be coded for a given project. As such, it can be challenging for some research groups, particularly those that are smaller in size, to gather the personnel required for independent assessors and coders. As an increasing number of research groups obtain reliability in BOSCC coding, it may become possible for studies to recruit independent sites to perform BOSCC coding, while primary sites oversee assessment.

An additional limitation researcher should be aware of when implementing the BOSCC for research is the inability to compare scores across BOSCC versions. To analyze BOSCC data, domain totals are summed and compared across time to evaluate effect sizes of changes in social communication behaviors. The various versions (i.e., MV, PSYF, F1, F2) are comprised of distinct items (some of which overlap, some of which don’t) that load on to their own factors. As such, raw totals from one version of the BOSCC are not comparable to raw totals from a different version of the BOSCC. This means that if a participant received the BOSCC -MV at study start, that participant should continue to receive the BOSCC-MV across all subsequent time points, even if this participant’s language has progressed to flexible phrases.

Future directions of the BOSCC will include creating comparison scores across versions, similar to the Calibrated Severity Score (CSS) obtained in an ADOS [13]. This will allow researchers to administer the version of the BOSCC appropriate to a participant’s current language level, regardless of language level at study entry. Researchers will also be able to more accurately evaluate effect sizes of social communication improvement without the possibility of obtaining a ceiling score.

We are currently developing an online platform by which BOSCC coding can be completed. This platform would be made available to reliable coders and would improve the accessibility and ease of BOSCC coding, in addition to storing large amounts of coded data. It is our goal that the BOSCC will eventually utilize automatic coding procedures through machine learning and acoustic and visual engineering techniques. Limited research has already been conducted on this topic which demonstrated that machine learning approaches used within the context of the BOSCC can provide evidence of speaker diarization (i.e., “who speaks when” within an audio segment; [14]) and decreases in intra-topic latency as a result of intervention (i.e., length of time between conversational turns; [15]).

Machine learning and automatic coding present a potentially exciting process by which the BOSCC can bridge a gap between research and clinical practice. While the BOSCC is currently only available for research, we hope that the BOSCC can be used in the future as one component of measurement-based care for individuals diagnosed with ASD. It’s user-friendly and relatively flexible nature, brief duration, and low degree of bias make it a feasible and compelling option for measuring social communication changes in response to treatment. Furthermore, we are hopeful that this measure will prove to be more sensitive to changes in social communicative behaviors than existing observational measures, thus filling a gap in the currently available measures for intervention researchers.

1. Wolery M, Garfinkle AN. Measures in intervention research with young children who have autism. J Autism Dev Disord. 2002; 32: 463- 478.

2. Lord C, Rutter M, DiLavore PS, Risi S, Gotham K, Bishop S. Autism diagnostic observation schedule (2nd edition). Western Psychological Services (WPS). 2012.

3. Mullen EM. Mullen Scales of Early Learning (MSEL). American Guidance Service Inc. 1995.

4. Sparrow SS, Cicchetti DV, Saulnier CA. Vineland adaptive behavior scales: Third edition (Vineland-3). Bloomington: NCS Pearson. 2016.

5. Grzadzinski R, Carr T, Colombi C, McGuire K, Dufek S, Pickles A, et al. Measuring Changes in Social Communication Behaviors: Preliminary Development of the Brief Observation of Social Communication Change (BOSCC). J Autism Dev Disord. 2016; 46: 2464-2479.

6. Byrne K, Sterrett K, Holbrook A, Kim SH, Grzadzinski R, Lord C. Extending the Usefulness of the Brief Observation of Social Communication Change (BOSCC): Validating the Phrase Speech and Young Fluent Version. J Autism Dev Disord. 2022; 1-15.

7. Nordahl-Hansen A, Fletcher-Watson S, McConachie H, Kaale A. Relations between specific and global outcome measures in a socialcommunication intervention for children with autism spectrum disorder. Res Autism Spectr Disord. 2016; 29-30: 19-29.

8. Grzadzinski R, Jatkar A, Donovan K, Truong K, Holbrook A, Lord C, et al. Examining Treatment Outcomes Across Contexts: How Do Child Baseline Characteristics Impact Measurement of Treatment Response? J Autism Dev Disord. 2023; 53: 4121-4131.

9. Kim SH, Grzadzinski R, Martinez K, Lord C. Measuring Treatment Response in Children with Autism Spectrum Disorder (ASD): Applications of the Brief Observation of Social Communication Change (BOSCC) to the Autism Diagnostic Observation Schedule (ADOS). Autism. 2019; 23: 1176-1185.

10. Kitzerow J, Teufel K, Wilker C, Freitag CM. Using the brief observation of social communication change (BOSCC) to measure autism-specific development. Autism Research. 2016; 9: 940-950.

11. Gengoux GW, Abrams DA, Schuck R, Millan ME, Libove R, Ardel CM, et al. A pivotal response treatment package for children with autism spectrum disorder: An RCT. Pediatrics. 2019; 144: e20190178.

12. Green J, Leadbitter K, Ellis C, Taylor L, Moore HL, Carruthers S, et al. Combined social communication therapy at home and in education for young autistic children in England (PACT-G): A parallel, singleblind, randomised controlled trial. Lancet Psychiatry. 2022; 9: 307- 320.

13. Gotham K, Pickles A, Lord C. Standardizing ADOS Scores for a Measure of Severity in Autism Spectrum Disorders. J Autism Dev Disord. 2009; 39: 693-705.

14. Kumar M, Kim SH, Lord C, Narayanan S. Improving speaker diarization for naturalistic child-adult conversational interactions using contextual information. J Acoust Soc Am. 2020; 147: EL196.

15. McKernan EP, Kumar M, Di Martino A, Shulman L, Kolevzon A, Lord C, et al. Intra-topic latency as an automated behavioral marker of treatment response in autism spectrum disorder. Sci Rep. 2022; 12: 3255.