Professional Documents
Culture Documents
Psychiatry Research
journal homepage: www.elsevier.com/locate/psychres
A R T I C L E I N F O A BS T RAC T
Keywords: The aims of this paper are to present the Autism Behaviour Coding System (ABCS), a novel, video-based
Early intervention observational instrument for assessing core autism symptoms during intensive early interventions in autism
Video coding system spectrum disorder (ASD), to provide preliminary data on its psychometric characteristics and to discuss its
Observational methods clinical utility. Video recordings of child-therapist interactions during the 'Frhintervention bei autistischen
Child-therapist interaction
Strungen' (FIAS) were coded by treatment-independent raters who were blind with respect to the temporal
Therapy evaluation
order of the sequences. We assessed inter-rater reliability using intra-class correlations (ICCs). Mean ICCs
FIAS
ranged from 0.85 to 0.90. We analysed the sensitivity of the ABCS to change by comparing the change in ABCS
scores with the change in a validated external measure of level of functioning (Developmental Disorder-Child-
Global Assessment of functioning Scale, DD-C-GAS) in a sample of 15 children who received intensive
treatment. Both the ABCS and DD-C-GAS indicated that the intervention improved symptoms. The ABCS has
promise as a research instrument and has good to excellent inter-rater agreement and sensitivity to
intervention-related changes. This pilot study suggests that the ABCS may be useful as an objective method
of assessing the proximal eects of therapy in young children with ASD.
Correspondence to: University Department of Child and Adolescent Psychiatry, Schahauserrheinweg 55, CH-4058 Basel, Switzerland.
E-mail address: evelyn.herbrecht@upkbs.ch (E. Herbrecht).
http://dx.doi.org/10.1016/j.psychres.2016.11.006
Received 27 April 2016; Received in revised form 26 September 2016; Accepted 6 November 2016
Available online 10 November 2016
0165-1781/ 2016 Elsevier Ireland Ltd. All rights reserved.
D. Dima et al. Psychiatry Research 247 (2017) 6367
1) The social interaction domain comprises behaviours reecting 1.1.2. Coding procedure
autism-specic decits in core aspects of social development. The technology underpinning the ABCS is Interact (Mangold
Some qualier variables provide qualitative information about International GmbH, 14th version), a professional software package
target behaviours (appropriate; inappropriate) or indicate the level that provides multiple functionalities including the ability to watch a
of social development (verbal or nonverbal expression of desire, child's behaviour in slow motion, automatic recording of the frequency
types of joint attention). Play situations are coded using qualier and duration of specic behaviours, graphical displays of data and the
variables which capture the level of a child's participation and the option to view behaviour from two perspectives (e.g. recorded from
extent of his or her social awareness. opposite corners of the room) simultaneously. The ABCS is an event
64
D. Dima et al. Psychiatry Research 247 (2017) 6367
record-based coding system and coding is continuous, consisting of a the FIAS intervention. Eligibility for the intervention was based on the
record of the presence or absence of a particular behaviour and its total following criteria: a) diagnosis of autism using a standardised assess-
duration (in seconds) or frequency within a given observation period. ment instrument; b) aged below 5 years at the start of the intervention;
Coding for treatment evaluation purposes should be performed by c) absence of severe neurological disorder. Parents gave written,
raters who are blind to the temporal order of video sequences from a informed consent to the anonymised use of the data.
given treatment period. The ABCS manual describes the variable
categories and codes, and provides general guidance about the 2.2.2. Statistical analysis
Interact functions which can be used to support ABCS coding. Repeated measures (RM) MANOVA was used to analyse the pattern
of changes in core autism-related symptoms, as measured using the
1.1.3. Analysis options ABCS, over three dierent time-points (days 4, 10 and 17 of the 18-d
The behavioural categories gaze, initiation of joint attention, intervention). We chose the lunch situation for this analysis as it
response to joint attention, expression of wishes and verbal stereo- constitutes a relatively invariant setting and was thus the situation in
type are reported as frequencies. The remaining categories, namely which data from a heterogeneous patient sample would be most
social positive aect, functional play, repetitive behaviour and comparable. Other interaction settings such as play situations in which
shared attention, are described in terms of total duration of the toys are available or routine events could also be chosen to suit the
behaviour. Other meaningful variables such as the combined codes specic research question being addressed (more detailed information
gaze with positive aect and joint attention with positive aect are about dierent settings is provided in the ABCS manual (Dima et al., in
reported as frequencies. preparation). We analysed changes in the following categories of
behaviour: gaze, joint attention, imitation, expression of wishes and
2. Method repetitive behaviour. Coding of the video sequences was done by the
rst and second author of this article using the same procedure as in
2.1. Inter-rater reliability the inter-rater reliability study (see 2.1.2).
Data on level of functioning were also analysed to enable us to
2.1.1. Participants compare the patterns of change suggested by the ABCS and another
The sample consisted of three boys treated at the FIAS centre. They tool. The DD-C-GAS (Developmental Disorder-Child-Global
were aged between 39 and 49 months at the start of the intervention Assessment of functioning Scale; Wagner et al., 2007, German transla-
and had been diagnosed with ASD using standard instruments, i.e. the tion), an adaptation of the Children's Global Assessment Scale
Autism Diagnostic Observation Schedule (ADOS, module 1; German (Schaer et al., 1983) developed for use with children with ASD, was
version by Rhl et al., 2004) and the Autism Diagnostic Interview- used as the reference indicator of level of functioning. The DD-C-GAS
Revised, diagnostic algorithm: 2; 03.11 years old (ADI-R; German ratings were made immediately before and after the intensive inter-
version by Blte et al., 2006). None of the children had additional vention (days 0 and 18) by the lead FIAS therapist working with the
neurological or somatic problems. Their parents gave written, informed child in question. The DD-C-GAS describes ten categories of function-
consent to the anonymised use of the data. ing using a 0100 scale; scores below 70 indicate that a child has
important special needs. Level of functioning data were analysed in the
2.1.2. Rating method same way as the ABCS data. Changes in everyday functioning,
Twenty-seven video sequences were coded (three video sequences intellectual performance, communication and social behaviour were
of therapist-child interaction per child at each of three dierent time- analysed statistically using RM MANOVA. Variance over time is
points during the intervention). We chose to record children's beha- represented by the partial eta squared (2p) statistic for eect size;
viour in three situations which occur routinely as part of the FIAS values of 2p > 0.06 indicate a medium eect and values of 2p > 0.14
intervention (Herbrecht et al., 2015): functional play, symbolic play (a a large eect. We also calculated the standard signicance parameter
standard set of toys was available in both play situations) and (p).
lunchtime. Each video sequence lasted 5 min
The frequency and/or duration of the following variables were 3. Results
recorded for all three situations: expression of wishes, gaze, imitation,
joint attention, functional play, repetitive behaviour and social positive 3.1. Inter-rater reliability
aect. Coding was performed by three dierent raters who were blind
to the children's intervention status. None of the raters was involved in IRR (see Table 2) was high for six out of seven variables in the
FIAS treatment. Raters 1 and 2 were the rst and second authors of context of functional play; ICCs ranged from 0.82 to 0.99 (M=0.88), the
this article and were heavily involved in the development of the only variable with a lower IRR was frequency of joint attention
instrument. Both are clinical psychologists experienced in the treat- (ICC=0.56). IRR was also high for six out of seven variables in the
ment of children with autistic disorder. Rater 3 is a medical student
familiar with ASD clinical symptoms and was trained in use of the Table 2
ABCS by Rater 1. This training took approximately one working day. Intra-Class Correlation Coefficients (ICCs).
65
D. Dima et al. Psychiatry Research 247 (2017) 6367
Table 3 with ASD during interactions with an adult. We evaluated the IRR of
MANOVA results for ABCS assessment of core autistic symptoms in 15 patients at three the ABCS and found very good to excellent agreement between raters
time points (days 4, 10, 17 from a total of 18 days of intensive intervention).
for most of the ABCS variables. There were a few variables with less
Variable Parameter Day 4 Day 10 Day 17 p n2p than satisfactory ICCs; for example, in the lunch situation frequency of
m (SD) m (SD) m (SD) expression of wishes had an ICC below the threshold of acceptability
(0.80). It is possible that because the food is already on the table in this
Gaze Frequency 4.7 (6.5) 7.5 (9.0) 8.7 (9.5) 0.109 0.147
situation it oers children little opportunity to express their wishes. For
Joint attention Frequency 1.5 (2.6) 1.0 (1.9) 2.0 (3.2) 0.119 0.141
Limitation Frequency 0.1 (0.4) 0.2 (0.4) 0.3 (0.8) 0.042 0.052 this reason, and because frequency of expression of wishes had good
Expression of Frequency 0.1 (0.4) 0.4 (0.6) 0.5 (1.4) 0.297 0.081 IRR in the contexts of functional play (ICC=0.94) and symbolic play
wishes (ICC=0.96), we decided that coding this variable during the lunch
Repetitive Duration 55.6 38.9 34.8 0.321 0.078 situation should be optional. In general raters found it more dicult to
behaviour (72.6) (42.4) (53.5)
score expression of wishes than other variables, especially in the case
Mean (m), standard deviation (SD), p-value, eect size (2p > 0.140: large eect, > of nonverbal children.
0.060: medium eect) The variable joint attention had lower ICCs than expected in the
contexts of functional play (ICC=0.56) and symbolic play (ICC=0.75).
context of symbolic play; ICCs ranged from 0.86 to 0.99 (M=0.90), This could be because the less structured play settings makes it more
once again only frequency of joint attention had a lower value dicult to observe subtle features of behaviour, such as joint attention.
(ICC=0.75). In the lunch situation (mean ICC=0.85), three out of ve Joint attention had a higher ICC in the lunch situation (see Table 2),
variables had ICCs between 0.95 and 0.98, representing very good IRR. which is more structured: the child and therapist sit next to each other
The behaviours represented by two variables were not observed in any at a small table and interact whilst having lunch. The play situation is
of the sequences and ICC values were lower for frequency of expression mostly object-focused, but as either the therapist or the child tend to be
of wishes (ICC=0.67) and duration of social positive aect (ICC=0.68). moving short periods of joint attention may be dicult to observe.
The ABCS was designed for use in semi-structured settings where
3.2. Sensitivity to change strict control over stimuli is not possible. Behaviour in these situations
tends to be somewhat unpredictable as children are encouraged to
Eect sizes were large for two ABCS variables, gaze and joint initiate social contact rather than just responding to the demands of an
attention, both of which are clinically signicant behaviours, but in adult. This means that a given interaction situation will be similar but
neither case was dierence between pre- and post-intervention scores. not identical at dierent time-points, and this may introduce some bias
statistically signicant. Eects sizes were medium for improvement in into comparisons of results from dierent time-points e.g. pre- and
expression of wishes and repetitive behaviour during the course of post-intervention. The denitions of variables in the ABCS manual are
the intervention. There was least improvement in imitation, for which designed to minimise this kind of bias. For example, children's requests
the eect size was small (see Table 3). In summary, scores on all ABCS tend to increase in frequency if an initial request is refused; if simple
variables indicated that the intensive intervention had produced some frequency data were used this might be misinterpreted as an improve-
improvement in autistic symptoms. Analysis of DD-C-GAS data ment in ability to express wishes, so the ABCS scores only requests
indicated that there were very large, highly signicant improvements expressed before a refusal.
in several symptom domains over time (see Table 4). In our sample scores on the ABCS variables indicated that there was
The variance in ABCS results was high, indicating that there were an improvement in symptoms, especially towards the end of the
considerable dierences between patients. This reects the heteroge- intervention. This nding corresponds well with our clinical impression
neity of symptom expression in this patient group. It is also notable that children's development during the FIAS intervention is discontin-
that improvements in behaviour were not apparent until the end of the uous. Improvements in social interaction and communication skills
intervention (third time-point, day 17); at the second time-point (day and a reduction in repetitive behaviours seem to occur as part of a
10) there was no substantial improvement in behaviour and in some discontinuous maturation process. The ABCS seems to be capable of
cases behaviour had actually deteriorated. capturing even subtle changes in behaviour during the intervention.
The ABCS was developed to provide a more nely grained assess-
4. Discussion ment of the eects of intensive therapeutic interventions for children
with ASD than other ASD assessment tools. However, although eect
Designing reliable, video-based instruments for coding the beha- sizes were large for changes in some ABCS variables the improvements
viour of children with ASD is a challenge, because of the heterogeneity did not reach statistical signicance, whereas the improvements in DD-
in expression of the disorder. The frequency of changes in the quantity C-GAS scores, which are based on assessments made by the therapists
and quality of the behaviour of young children with ASD presents a working with the children rather than by an external rater, were greater
further challenge. in absolute terms and were statistically signicant. Thus, contrary to
We attempted to address these challenges by developing an the claim made by Hawkes et al. (2013), the ABCS - which captures
instrument, the ABCS, for capturing behaviours produced by children behaviour at the micro-analytical level - proved to be less sensitive to
changes in behaviour than the DD-C-GAS, which captures behaviour in
Table 4
broader terms. It remains to be determined whether and to what extent
MANOVA results of level of functioning (DD-C-GAS) in 15 children before and after the the apparently superior sensitivity of the DD-C-GAS is genuine rather
intensive intervention. than a product of therapist bias.
It takes approximately ve to six hours to code 90 min (30 minx3
Variable Day 0 Day 18 p n2p
time-points) of recordings of a child. Because of this the ABCS is
m (SD) m (SD)
primarily intended for use in research, and is not considered suitable
Self-care (score 1100) 34.9 (16.1) 44.3 (17.0) 0.006*** 0.429 for routine use in clinical settings. In research contexts the ABCS can
Communication (score 1100) 34.8 (15.3) 43.6 (17.7) 0.002*** 0.492 be used as OC tool to provide objective, precise assessments of
Social behaviour (score 1100) 27.0 (11.0) 37.7 (13.1) 0.001*** 0.553
clinically relevant changes in core autistic symptoms during and after
Intellectual skills (score 1100) 43.8 (17.5) 51.6 (17.9) 0.002*** 0.499
intensive therapeutic interventions (Herbrecht et al., 2015). The ABCS
Mean (m), standard deviation (SD), p-value (*p < 0.05; ***p < 0.01), eect size (2p > diers from other video-based approaches such as the BOSCC
0.140: large eect, > 0.060 medium eect) (Kitzerow et al., 2015; Grzadzinski et al., 2016) in that it does not
66
D. Dima et al. Psychiatry Research 247 (2017) 6367
produce global scores representing children's behaviour over a given Disorders. American Psychiatric Association, Washington, DC.
Baranek, G.T., 1999. Autism during infancy: a retrospective video analysis of sensory-
period of time; instead it provides a detailed, continuous protocol of a motor and social behaviours at 9-12 months of age. J. Autism Dev. Disord. 29 (3),
child's behaviour in a given situation. Such protocols may enable the 213224.
identication of turning points in intensive therapeutic interventions Barkley, R.A., 1997. Attention-decit/ hyperactivity disorder. In: Marsh, E.J., Terdal,
L.G. (Eds.), Assessment of Childhood Disordersthird ed.. Guilford Press, New York,
such as the FIAS; however the ABCS is essentially an instrument for 71129.
measuring quantitative rather than qualitative changes in behaviour. Blte, S., Rhl, D., Schmtzer, G., Poustka, F., 2006. Diagnostisches Interview fr
This is an important limitation, given that over longer periods of time Autismus-Revidiert (ADI-R). Huber, Bern.
Carter, A.S., Messinger, D.S., Stone, W.L., Celimli, S., Nahmias, A.S., Yoder, P., 2011. A
qualitative changes in behaviour (e.g. the social modulation of eye randomized controlled trial of Hanen's More Than Words' in toddlers with early
contact) may be better indicators of the course of autistic symptoms autism symptoms. J. Child Psychol. Psychiatry. http://dx.doi.org/10.1111/j.1469-
than quantitative indices (e.g. frequency of eye contact). It means that 7610.2011.02395.x.
Casenhiser, D.M., Shanker, S.G., Stieben, J., 2011. Learning through interaction in
the ABCS may be most suited to short-term evaluations.
children with autism: preliminary data from asocial-communication-based
The most important limitations of this pilot investigation are (1) the intervention. Autism. http://dx.doi.org/10.1177/1362361311422052.
small sample size in the IRR study and (2) the sensitivity-to-change Cliord, M.S., Dissanayake, C., 2008. The early development of joint attention in infants
analysis must be regarded as preliminary, as we only calculated IRR for with autistic disorder using home video observations and parental interview. J.
Autism Dev. Disord.. http://dx.doi.org/10.1007/s10803-007-0444-7.
the main variables and other properties of the instrument, such as its Cliord, S., Young, R., Williamson, P., 2007. Assessing early characteristics of autistic
validity have not been investigated. disorder using video analysis. J. Autism Dev. Disord.. http://dx.doi.org/10.1007/
s10803-006-0160-8.
Dima, D., Lazari, O., Schmeck, K., Herbrecht, E. The Autism Behaviour Coding System
5. Conclusion Manual. Department of Child and Adolescent Psychiatry, Psychiatric University
Clinics (UPK) Basel, 2016 (in preparation).
Although they are relatively time-consuming, observational assess- Grzadzinski, R., Carr, T., Colombi, C., McGuire, K., Dufek, S., Pickles, A., Lord, C., 2016.
Measuring changes in social communication behaviors: preliminary development of
ments of children undergoing treatment for ASD by independent the Brief Observation of Social Communication Change (BOSCC). J. Autism Dev.
observers have important advantages over retrospective assessments Disord. 46, 24642479.
by therapists or parents. They can provide detailed information about Guralnick, M.J., Groom, J.M., 1985. Correlates of peer-related social competences of
developmentally delayed preschool children. Am. J. Ment. Dec. 90 (2), 140150.
changes during the course of treatment that is not captured by other
Hawkes, D.J., Dadds, M.R., Pasalich, D.S., 2013. Observational coding strategies. In:
assessment methods. The ABCS was designed for monitoring changes Colmer, J.S., Kendall, P.C. (Eds.), The Oxford Handbook of Research Strategies for
in core ASD symptom domains during and after therapy, at a micro- Clinical Psychology. Oxford University Press, Oxford, 120141.
Herbrecht, E., Kievit, E., Spiegel, R., Dima, D., Goth, K., Schmeck, K., 2015. Become
analytical level. Documented changes over time indicate that the ABCS
related: FIAS, an intensive early intervention for young children with autism
is sensitive to FIAS intervention-associated changes in symptoms; spectrum disorders. Psychopathology. http://dx.doi.org/10.1159/000375504.
however the changes detected using the ABCS were not signicant. Kasari, C., Gulsrud, A., Freeman, S., Paparella, T., Hellemann, G., 2012. Longitudinal
This preliminary analysis suggests that the ABCS is a reliable tool and follow-Up of children with autism receiving targeted interventions on joint attention
and play. J. Am. Acad. Child Adolesc. Psychiatry 51 (5), 487495.
could be used to evaluate early interventions for ASD. Evaluation of Kitzerow, J., Teufel, K., Wilker, Ch, Freitag, Ch.M., 2015. Using the Brief Observation of
videotapes using the ABCS can add value to standard assessments of Social Communication Change (BOSCC) to measure autism-specic development.
psychosocial functioning and intellectual performance, as the ABCS Autism Res. 9 (9), 940950.
Leekam, S.R., Prior, M.R., Uljarevic, M., 2011. Restricted and repetitive behaviours in
focuses on basic, autism-sensitive aspects of interaction and commu- autism spectrum disorders: a review of research in the last decade. Psychol. Bull. 137
nication. The ABCS is particularly recommended for assessing thera- (4), 562593.
pist- or parent-child interactions when proximal treatment eects are Lord, C., Wagner, A., Rogers, S., Szatmari, P., Aman, M., Charman, T., Dawson, G.,
Durand, V.M., Grossman, L., Guthrie, D., Harris, S., Kasari, C., Marcus, L., Murphy,
the main object of interest. The clinical utility of this new instrument S., Odom, S., Pickles, A., Scahill, L., Shaw, E., Siegel, B., Sigman, M., Stone, W.,
will be evaluated in future research. Smith, T., Yoder, P., 2005. Challenges in evaluating psychosocial interventions for
autistic spectrum disorders. J. Autism Dev. Disord. 35 (6), 695708.
McMahon, C.M., Vismara, L.A., Solomon, M., 2013. Measuring changes in social
Conict of interest behavior during a social skills intervention for higher-functioning children and
adolescents with autism spectrum disorder. J. Autism Dev. Disord. 43 (8),
Diana Dima was founded by the Botnar foundation. No other 18431856.
Osterling, J., Dawson, G., 1994. Early recognition of children with autism: a study of rst
potential conict of interest was reported.
birthday home videotapes. J. Autism Dev. Disord. 23 (3), 247257.
Rhl, D., Blte, S., Feineis-Matthews, S., Poustka, F., 2004. ADOS, Diagnostische
Acknowledgments Beobachtungsskala fr Autistische Strungen. Huber, Bern.
Schaer, D., Gould, M.S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., Aluwahlia, S.,
1983. A children's global assessment scale (CGAS). Arch. Gen. Psychiatry 40,
We thank Nadine Fink for her important initial input into the 12281231.
design of the ABCS and Julia Frh for her contribution to rating the Vismara, L.A., Rogers, S.J., 2010. Behavioural treatments in autism spectrum disorders:
child participants in this study. what do we know? Annu. Rev. Clin. Psychol. 6, 447468.
Vorgraft, Y., Farbstein, I., Spiegel, R., Apter, A., 2007. Retrospective evaluation of an
intensive method of treatment for children with pervasive developmental disorder.
References Autism 11, 413. http://dx.doi.org/10.1177/1362361307079605.
Wagner, A., Lecavalier, L., Arnold, L.E., Aman, M.G., Scahill, L., Stigler, K.A., Johnson,
C.R., McDougle, C.J., Vitiello, B., 2007. Developmental disabilities modication of
Aldred, C., Green, J., Adams, C., 2004. A new social communication intervention for
the children's global assessment scale. Biol. Psychiatry 61, 504511.
children with autism: pilot randomized controlled treatment study suggesting
Williams White, S., Keonig, K., Scahill, L., 2007. Social skills development in children
eectiveness. J. Child Psychol. Psychiatry. http://dx.doi.org/10.1111/j.1469-
with autism spectrum disorders: a review of the intervention research. J. Autism
7610.2004.00338.x.
Dev. Disord.. http://dx.doi.org/10.1007/s10803-006-0320-x.
American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental
67