Comparison of Assessment Instrument s

CAFAS CANS YOQ ATOM

Child and Adolescent Child and Adolescent Youth Outcome Adolescent Treatment
Functional Assessment Needs and Questionnaire Outcomes Module

1
Purpose of Defining Assessment Tracking Designed to
Data eligibility for of type & outcomes over assess multiple
Risk intensity of severity of shorter periods outcomes for
assessment services – clinical and of time, youth ages 11 to
Resource assess psychosocia sensitive to 18, also collects
eligibility functional l factors change-can be prognostic data
Tracking skills of that may utilized on a that can be used
outcomes youth with a impact session-to- for case-mix
Quality psychiatric treatment session basis adjustment and
improvement diagnosis decisions: or on a longer assist with
Provider (especially for Useful in 3 basis. Also can treatment
monitoring SED services), areas be used as a planning.
has also been decision screening Domains include:
used to track support, instrument Clinical status
outcomes. quality There is a parent (diagnosis &
The CAFAS has improvement and youth Symptom severity)
8 subscales and outcome version with 6 Prognostic
that assess the Areas questions and a variables—factors
youth’s include: shorter that have been
behavior and 2 Problem questionnaire found to influence
subscales that Presentation with 30 outcomes.
assess Risk questions. Functional
caregivers. Behaviors Questions are Impairment—
A trained rater Functioning based on home, school, legal
rates a youth Care Intensity observed problems,
by reviewing & behavior friendships
each item in a Organization Subscales dysfunctional
subscale; rates Caregiver include: friends
rate the youth Needs & Intrapersonal Sentinel Indicators
by level of Strengths distress —events that are
severity—rater Strengths Somatic indicative of high
describes the Functions as Interpersonal risk
youth’s most an relationships Consumer
impaired information Somatic perspective of
function during integration Social Problems change-BAS and
a 3-month tool and a Behavioral severity of 3
period. prospective Dysfunction primary problems
There is an assessment Critical Items Family impact and
optional set of for planning Total Scores, focal problems
positive services plus scores for (Burden
behaviors for Can be used each subscales Assessment Scale)
each subscales in treatment are generated Satisfaction with
which can be planning and can be used care
used for without to track 150 item Parent
strengths and scoring—can outcomes Baseline, 93-item
create even be Short version has Adolescent
treatment modified by a Total score and Baseline, 24-item
goals. adding items 4 subscales. clinician baseline
2
Approach Mostly deficit Deficit and Mostly deficit Mostly deficit
Strengths based strength based- 7 health based
based Strengths are based items, which
Deficits based optional could be
considered
strength based
Level of Trained Training of Any staff can Any staff can
clinical staff professional or any staff administer the administer
Trained paraprofession through a questionnaire,
Para- al—training sequenced paraprofessional
professional provided only training s can be trained
Case by HODGES or model—takes to read and
Manager specified about 4 hours understand the
Clinician trainers— –requires test reports
Master’s periodic to be reliable generated by the
Level retraining is at .70 computer system
necessary
Time taken to 10 minutes to Depends on 5 to 15 minutes, 15 to 25 minutes
administer rate-however, the extent of less for shorter for parent and
it takes information version adolescent
additional time gathered 1 ½ minutes for
to collect clinician
information by
the youth or
caregiver
Costs of Cost per rating Free Instruments are Free (paper &
administering tool $1?- also However, low cost—one pencil version)
and has a computer training is time fee to use,
purchasing scoring involved have rates for
tool program , also though less clinics, state
cost of training, structured versions, about
retraining, and than CAFAS $1,500 to
tracking the $3,000. Only
training of with
raters, use of computerized
clinical staff is versions does
also very cost compute per
expensive employee or per
clinic
Cultural Unknown Unknown Spanish, French, Unknown
Competency German, Laotion
versions—
normed for
French and
German
Reliability Reliability: Reliability: Reliability: Reliability:
Validity Internal Internal Internal Internal

3
Consistency: consistency Consistency Consistency
Range from .63 was not was .94-strong Functioning, .71
to .78 rated, as single factor to .81
depending on authors claim Subscale Consumer
the time of that CANS is consistency Perspectives- .82
measure and a ranged from .51 to .90 (parents)
study (Ft. Brag communicati to .90 (lowest Test-retest
versus CMHS on were critical Clinical Status .89
study) perspective items and for parents, .53 for
Inter-rater Inter-rater somatic). Short adolescents
reliability: Reliabili version was .92 Functioning, .78
Trained lay ty Test-Retest for parents, .36 to .
raters using Using case Total- .83, 71 adolescents
vignettes vignettes subscales .56 to . Sentinel Indicators,
ranged from . average .76 82 For Short .76 for parents, .83
92 to .96 Using case version was .80 for adolescents
Validity records, to .91. Consumer
Strong face average .83 Validity Perspectives ?
validity for 19 trained Concurrent Validity:
Concurrent by author validity Concurrent
Validity— In Anderson Correlated with Clinical Status-
different scores study, CBCL .78 Parent’s scale
for outpatient caseworkers Ohio used the correlated with
versus day reviewed YOQ as a validity CBCL Internalizing
treatment, etc. chart, measure &, scale at .65 and
Predictive average was . found a .70 Child health
validity— 85 correlation. Questionnaire
predict that Individual Construct (CHQ) at -.71, and
more severe items ranged Validity— Behavior Problems
kids receive 7from .57 for reliability scale at .72.
more services school different Functioning:
functioning to community and -correlated with
.90 mental health the Columbia
Validity samples, impairment Scale
Concurrent: Sensitivity and CHQ Scale and
Correlation Specificity: 82% CBCL (differing
between to 94% of sample scales)
CAFAS and correctly Predictive
CANS, .63, classified as Validity of
between clinical groups. prognostic
School Short version variables—
Functioning also correctly prognostic
was .59, classified clinical variables have
between self- and community some correlation
harm was . 84% of time. with symptoms
61, etc. and functional
Predictive: outcomes.

4
 Cans most
accurate at
identifying
children at
the highest
level of care
(80%), but
also
predicted low
level of care
at (65%).

Appropriatene
ss to Identified
Population

Acceptability Rapid feedback Face validity Face valid Face valid

to Staff if use computer Very Rapid Rapid Feedback if
Face program and feedback with entered into
validity enter scores computer NETOUTCOMES
Brief program
Clinical
utility
Rapid
feedback

Training Training for Training is 4 Training not Training not

Requirements CAFAS fairly hours, required required
Time extensive unknown
Cost whether it ,
Impact on can be done
Localities online—
seems like it
can be.
However,
authors
emphasize
on-hands
training.
Singularity of
Instrument
(i.e., can only
this one
instrument be
used for
needed
purposes?)

5
Does Current Current Current Current and some
instrument Behavior behavior behaviors historical
weigh current
behavior of
child as well
as history?

Assessment of Mostly child Child Child Child and some

family as well family (mainly
as child burden of child)

Utility in CAFAS form Can be used The 1st 1st- Critical Items
Service and computer in both administration might influence
Planning program establishing provides a type and intensity
contains a appropriate baselines level: of intervention,
treatment plan level of care 1st Identification 2nd Levels of care
that can be and assisting of primary and influenced by
completed by in service secondary critical items and
rater. planning problems sentinel indicators
The CAFAS 2nd Identification 3rd- Symptoms can
profile and of critical items be used to develop
treatment plan 3rd Levels of Care primary treatment
provide a way (Comparison of plan—depression,
to organize total score to irritability, ADHD
target normative group symptoms, etc.
behaviors and averages for 4th-Functional
goals for youth residential, assessment looks
and family. inpatient, CMHC, at problems in
Level of outpatient, EAP, various
severity helps community relationships and
the clinician to populations) settings allow
decide what 4th –look at targeting of
services are individual items treatment in these
best for client that may need settings.
further 5th –screens for 5
questioning or disorders
key off treatment (depression,
plan ADHD, anxiety,
5th-Tracking ODD, and Conduct
system for YOQ disorder)
from one session
to next is highly
developed;
authors have
developed
algorithms for

6
 when clients are
on target for
improvement or
who are at risk
for poorer
outcomes early
in treatment.
Parents and Neither except Neither-but Both are Both are included
Youth through rater- can be included
rater is informants
instructed to
seek
information
from all
important
informants-
Caregiver
instrument
does ask
parent
Components

Contains
Required
Outcomes
Domains

Range of Age 6 thru 17 for Ages 4-18 Ages 4-17 Age 11-18, version
CAFAS Another Youth ages 12- for 5 to 9 years old
Younger for version ages 18- being develop?
PECAFAS 0-3
Data
Ownership

Computerized There is a None known, The OQ Analyst Web-based scoring

computerized but can has developed a system,
version which probably be electronic Unknown whether
generates a computerized administration test can be
report and with author’s and scoring scanned or
treatment plan permission system for long entered directly by
(if done), plus 2 and short clients.
administrative versions of YOQ,
reports utilizing Wireless
(severity at PDA, Docked
intake) and PDA, Scanner,
outcome manual data,
results Kiosk Terminal,

7
 and Internet
using Home PC
Information is
stored on a local
computer, but
can be accessed
by Web
(password,
clinicians have
access to only
their data) If
used regularly;
outcomes can be
tracked per
client and can be
shared with
parent.
Sensitivity to CAFAS – Outcome Reliable Change Sensitive to
change clinically measurement Index of 13 for change in clinical
significant was used in parents and 18 status, functioning,
change is Intensive for adolescents, and family impact.
defined as a 20 services— subscales also Not known if a RCI
point change in shows effect have reliable has been
total CAFAS sizes of change indices.. calculated.
score small to The majority of Symptom severity
(approximated moderate questions are may be more
the Reliable with largest sensitive to sensitive to change
Change Index) effect size on change over in internalizing
Outcome problems for time; the short domain versus
indicators: MH and YOQ-30 items externalizing
Change In functioning were chosen that domain.
Mean Total for Juvenile were most Designed to
CAFAS Justice. sensitive to measure quality of
Change in According to change. care across
Subscales— authors, the Numerous patients.
suggest CANS is likely published studies
monitoring less sensitive are conducted
over 3-months, to change, using the YOQ as
limited particularly an outcome tool
application for over short
short-term periods, than
monitoring other
measures.
Note: Dr. Ogles
who is
responsible for
OHIO graduated

8
 from Bingham
Young and works
closely with YOQ
researcher (Dr.
Lambert)
References:
Ohio Scales: Ogles, B., Dowell, K., Hatfield, D., Melendez, Gregorio, & Carlston, D. (2004) The Ohio Scales in Maruish, In M.E (Ed.) The Use of Psychological Testing for Treatment Planning
and Outcomes Assessment, 3rd edition, Volume 2, pp. 275-304
Ogles, B, Melendez, G., Davis, D, and Lunnen, K. Ohio Youth Problem, Functioning, and Satisfaction Scales-Technical Manual
Ogles, B., Melendez. G., Davis, D., and Lunnen, K. (2001) The Ohio Scales: Practical Outcome assessment, Journal of Family Studies, 10, pp 199-212
Turchik, J.A., Karpenko, V, and Ogles, B. (2007) Further Evidence of the Utility and Validity of a Measure of Outcomes for Children and Adolescents—Journal of Emotional and
Behavioral Disorders, summer, pp 119-128
CAFAS: Hodges, Kay. The Child and Adolescent Functional Assessment Scale (CAFAS) (2004) In M.E (Ed.) The Use of Psychological Testing for Treatment Planning and Outcomes
Assessment, 3rd edition, Volume 2, pp. 405-411.
CBCL:
Achenbach, T.M., Empirically Based Taxonomy: How to Use the Syndromes and Profile types derived from the CBCL/4-18, TRF, and YSR
Achenbach, T.M., & Rescorla, L.A. (2004), The Achenbach System of Empirically Based Assessment (ASEBA) for Ages 1.5 to 18 years, In M.E (Ed.) The Use of Psychological Testing for
Treatment Planning and Outcomes Assessment, 3rd edition, Volume 2, pp. 179-213
CANS-MH:
Anderson, R.L., Lyons, J.S., Giles, D.M., Price, J.A., & Estle, G. (2003) Reliability of the Child and Adolescent Needs and Strengths-Mental Health (CANS-MH), Journal of Child and Family
Studies, 12 (3) pp. 279-289
Lyons, J. S., Weiner, D. A., Lyons, M. B. (2004) Measurement as communication in outcomes management: The child and adolescent needs and strengths (CANS). In M.E (Ed.) The Use of
Psychological Testing for Treatment Planning and Outcomes Assessment, 3rd edition, Volume 2 , pp.461-476.
ATOM
Kramer, T. L., & Robbins, J.M. The Adolescent Treatment Outcomes (2004) In M.E (Ed.) The Use of Psychological Testing for Treatment Planning and Outcomes Assessment, 3rd edition,
Volume 2, pp 355-370
YOQ:
Burlingame, M., Wells, M.G., Hoag, M.J, Hope, C.A., Nebeker, R.S., Konkel, K., McCollam, P., Peterson, G, & Lambert, M.J. (1996). Administration and Scoring Manual for the YOQ. American
Professional Credentialing Services, LLC
Burlingame, M., Wells, M.G., Lambert, M.J. Youth Outcome Questionnaire (Y-OQ) (2004). In M.E (Ed.) The Use of Psychological Testing for Treatment Planning and Outcomes Assessment, 3rd
edition, Volume 2 , pp 235-273.
Dunn, T.W., Burlingame, G.M., Walbridge, Mi, Smith, J, & Crum, M.J (2005). Outcome Assessment for children and adolescents: Psychometric validation of the youth outcome
questionnaire 30.1, Clinical Psychology and Psychotherapy, 12, 388-401.
CASII/CALOCUS: Fallon, T, Jr., Pumariega, A., Sowers, W. Klaehn, R, Huffine, C. Vaughn, T., Winters, N., Chenven, M, Marx, L, Zachik, A, Heffron, W, Grimes, K. (2006) A level of care
instrument for children’s systems of care: Construction, reliability and validity. Journal of Child and Family Studies, 15(2), pp.143-155.
PTPB: (Treatment Manual)
http://www.masspartnership.com/provider/index.aspx?lnkID=outcomesmanagement/factsheetsummary.ascx--several instruments