Translating Evidence-based Interventions for ASD: Multi-Level Implementation Strategy

Overview of Collaborative R01. The investigators propose to conduct two, coordinated studies
testing the impact of the "Translating Evidence-based Interventions for ASD: A Multi-Level
Implementation Strategy" (TEAMS). TEAMS focuses on improving implementation leadership,
organizational climate (Teams Leadership Institute; TLI), and provider attitudes and
engagement (TEAMS Individualized Provider Strategy for Training; TIPS) in order to improve
two key implementation outcomes - ASD evidence-based intervention (EBI) fidelity, and
subsequent child outcomes. The TLI module applies the LOCI ("Leadership and Organizational
Change for Implementation") strategies, and the TIPS module applies MI (Motivational
Interviewing) strategies to facilitate individual leader and provider level behavior change.
These studies will use a randomized Hybrid implementation/effectiveness, Type 3, trial. Study
#1 (PI: L Brookman-Frazee/UCSD) will test the TEAMS model with An Individualized Mental
Health Intervention for ASD (AIM HI) in publicly-funded mental health services. Study #2 (PI:
A Stahmer/UC Davis) will test TEAMS with Classroom Pivotal Response Teaching (CPRT) in school
settings.

The Collaborative R01 mechanism will advance implementation science by allowing the research
team to: 1) obtain a sufficient sample size to isolate the impact of individual and combined
modules targeting different change mechanisms (implementation leadership/climate, attitudes);
examine change mechanisms as mediators of outcomes; and provider background and
organizational structure as moderators of outcomes; 2) enhance generalizability by testing
TEAMS in combination with two clinical EBI in two public service systems critical for
children with ASD; and maximize the diversity of the target population. Each site has unique
expertise in one of the two EBI to be tested. The PIs have a strong history of collaboration
and a clear management plan.

The Centers for Disease Control (CDC) estimates that 1 in 68 children have ASD. Long term
outcomes for this populations are poor and the annual cost in the US is estimated to be $268
billion. Research on the effectiveness of methods to scale up EBI in routine care is critical
to meet this growing public health need. The efficacy of a growing number of ASD EBI has been
established. Emerging data from AIM HI and CPRT studies support the overall effectiveness of
ASD EBI for improving child outcomes only when providers complete training and deliver
interventions with fidelity. Unfortunately, adoption and provider training outcomes,
considered key implementation outcomes, are variable (e.g., up to 35% of providers in our
studies either do not complete training or have poor fidelity). These findings are especially
concerning given the link between fidelity and child outcomes and the rapid increase of
large-scale usual care implementation of EBI with little attention to training completion or
fidelity, even with well-established training and consultation methods. Therefore, testing
methods of improving implementation outcomes is key to ensuring positive child-level outcomes
when EBI are implemented in routine care.

AIM HI and CPRT data indicate that (1) implementation leadership/climate and (2) provider
attitudes towards EBI are promising targets of implementation interventions. The roles of
both factors have been indicated for broader patient populations and also in current AIM HI
and CPRT projects. As such, the project will apply two, established interventions (LOCI, MI)
in the TEAMS model to target these specific mechanisms of change. This study will test the
impact of combining standard, EBI-specific training with the two TEAMS modules individually
and together on multiple implementation outcomes. A dismantling design will be used to
understand the effectiveness of TEAMS and the mechanisms of change across settings and
participants. The specific aims and hypotheses are:

1. Test the effectiveness of the TEAMS modules individually and in combination on
implementation outcomes when paired with two ASD EBI.

a) Compared to standard ASD EBI training (control) and individual TEAMS modules (LEAD or
PROV), the full TEAMS model will lead to more positive implementation outcomes for
providers (training completion, fidelity), and children (improvements in targeted
symptoms).

2. Test the impact of TEAMS modules on organization and provider level mechanisms of
change.

a) TEAMS-LEAD will increase use of implementation leadership strategies and TEAMS-PROV
will lead to greater changes in provider attitudes and engagement in EBI training.

3. Identify moderators and mediators of implementation outcomes.

1. Identify provider and organization characteristics that moderate implementation
outcomes; and

2. Identify provider and leader level mechanisms of change that mediate implementation
outcomes.

Impact: This implementation intervention has the potential to increase quality of care for
ASD by improving effectiveness of EBI implementation. The process and modules will be
generalizable to multiple service systems, providers, and interventions, providing broad
impact in mental health, educational and community services.

The combined multi-level sample for both studies will include 74 programs/districts, 148
agency/district leaders, 590 providers (average of 8 per program/district) and 590 parents
(1 per provider). It is estimated that an additional 590 participants will complete the 360
Organizational Assessment. Providers are expected to be approximately 85% female and 35%
Hispanic. Parent participants are expected to be approximately 80% female and 60% Hispanic.

Mental health programs will be those providing publicly funded psychotherapy services to
children in San Diego, Sacramento and LA Counties. Districts will be those providing public
education services to elementary school children with ASD in San Diego, Sacramento and LA
Counties.

Inclusion Criteria for Leaders

(1) Identified as Program Managers at an enrolled site or identified as Program Specialist
in an enrolled program/district

Inclusion Criteria for Providers

1. Employed at a participating program/district

2. Employed for at least the next 7 months

3. Has an eligible child on current caseload/classroom (see below)

4. Did not participate in the AIM HI or CPRT effectiveness studies

Inclusion Criteria for Parent Participants (enrolled in a dyad with participating provider)

1. Has a child age 3-13 years.

2. Has a child with a current ASD diagnosis on record or a primary educational
classification of autism as indicated in school records

Inclusion Criteria for 360 degree Organizational Assessment

1. Identified as a leader or provider at enrolled site

2. Linked to a participant leader (either as a supervisor or direct report)