TOP Implementation Project

Background

The Veterans Health Administration (VHA) provides care to 3.3 million Veterans living in
rural areas, comprising 36% of all VHA enrollees. In 1995, VHA began expanding its system of
Community Based Outpatient Clinics (CBOCs) in order to improve access for the geographically
dispersed Veteran population. There are now approximately 900 CBOCs delivering a range of
services to approximately 64% of VHA enrollees. While these CBOCs have dramatically improved
access to first class primary care services, it has been more challenging to deliver
specialty mental health care to rural Veterans. Evidence based specialty mental care
practices developed for large VA Medical Centers are often not feasible to deploy in small
CBOCs and thus not accessible to rural Veterans. Rural Veterans with posttraumatic stress
disorder (PTSD) treated at CBOCs experience little to no improvement in their symptoms over
time. A major contributor of poor PTSD outcomes is that trauma-focused evidence-based
psychotherapy is not being provided to Veterans in the CBOC setting. Moreover, travel
barriers prevent most rural Veterans from receiving trauma-focused evidence-based
psychotherapy at large VHA Medical Centers (VAMC). Telemedicine Outreach for PTSD (TOP) is a
technology-facilitated virtual care clinical intervention that is designed to enhance access
to evidence based psychotherapy and pharmacotherapy. The VHA Office of Rural Health and
Office of Connected Health and Telehealth Services intend to deploy the TOP intervention
nationally. This project will lay the ground work for this national implementation
initiative.

Specific Aims

Impact Goal - The goal of this proposed Type II Hybrid effectiveness-implementation project
is to support the national deployment of the TOP intervention and evaluate its clinical
effectiveness in routine care.

To achieve this goal, the following three specific aims will be conducted.

SPECIFIC AIM 1 - Compare the effectiveness of a standard VA implementation strategy to an
enhanced implementation strategy in promoting uptake of TOP.

Hypothesis 1 - For sites not meeting the TOP performance metric benchmark with a standard
implementation strategy, those randomized to the enhanced implementation strategy will have
better reach and engagement outcomes than those randomized to continued standard VA
implementation. Reach is defined using two variables: 1) the likelihood of having a care
manager encounter and 2) the likelihood of having an evidence based psychotherapy encounter.
Engagement is defined using two variables: 1) the likelihood of having 8 care manager
encounters and 2) the likelihood of having 8 evidence based psychotherapy encounters.

SPECIFIC AIM 2 - Determine if implementation of TOP in routine care improves PTSD outcomes
for rural Veterans.

Hypothesis 2 -Clinical outcomes at sites with higher reach and engagement rates for care
manager and evidence based psychotherapy encounters will have better PTSD outcomes and
perceived access compared to sites with lower reach and engagement rates and compared to the
pre-implementation period.

Hypothesis 3 - Perceived access at sites implementing TOP successfully will be improved
compared to sites not implementing TOP successfully and compared to the pre-implementation
period.

SPECIFIC AIM 3 - Calculate the cost of the standard and enhanced implementation strategies
and estimate the population level cost effectiveness of the enhanced implementation.

Methodological Approach

Setting - The project will be conducted at 6 VAMCs. CBOCs served by these 6 VAMCs without
on-site psychologists trained in trauma-focused evidence-based psychotherapy will be
identified and a CBOC or combination of CBOCs with 300-400 eligible patients will be selected
as the implementation site(s).

Population (for evaluation) - Using administrative data from VHA's Clinical Data Warehouse
(CDW), the investigators will identify all patients at participating CBOCs whose most recent
PC-PTSD screen was positive, had a VHA encounter in the past 12 months with a PTSD diagnosis,
and had no specialty mental health encounters at the VAMC in the past 6 months. Eligible
patients will be sampled for the evaluation from the selected CBOCs and 100 will be recruited
to participate from each VAMC via opt-out letters. The total anticipated sample size will be
600. Participating patients will complete a baseline survey and 3 follow-up surveys every 9
months. Local clinicians at implementation sites will have the flexibility to decide who is
eligible to receive the TOP clinical intervention, but the investigators expect there will be
considerable overlap between those who are eligible for the evaluation and those considered
to be eligible for the TOP intervention by local clinicians.

Telemedicine Outreach for PTSD (TOP) Intervention - TOP is delivered by a virtual care team
comprising a CBOC provider, and a telephone care manager, telepsychologist and
telepsychiatrist located at the VAMC. The telephone care manager activities include:
education, activation, barrier assessment/resolution, medication adherence and side-effect
monitoring, therapy/homework adherence monitoring and symptom monitoring. The
telepsychologist activities include the delivery of trauma-focused evidence-based therapy
(e.g., Cognitive Processing Therapy [CPT] or Prolonged Exposure Therapy [PE]). The
telepsychiatrist activities include conducting case reviews and psychiatric consultations.
The core element of the TOP intervention is a case review by the virtual care team (telephone
care manager, telepsychologist and telepsychiatrist) of all patients newly enrolled in care
management and all those not responding to treatment, along with documentation of the case
review in the Electronic Health Record. Given the dynamic nature of technology and the high
degree of variability in context across clinics, the investigators will employ methods that
enable the new clinical intervention to be adapted from setting to setting and to be refined
over time rather than focusing on fidelity to a manualized protocol. The telepsychiatry and
telepsychology can either be delivered via interactive video to the CBOC or via clinical
video telehealth to home. The Veteran and the telepsychologist can choose to use smartphone
apps (CPT Coach or PE Coach) designed to augment these two trauma-focused evidence-based
therapies. Care managers can also use secure messaging with patients in addition to telephone
calls.

Standard VA Implementation Strategy - Standard VA implementation strategies will include
disseminating a TOP clinical intervention manual, a TOP local champion guide, care manager
training materials, PTSD case-finder tool (with the same inclusion/exclusion criteria used
for evaluation sample), and technical support from the facility level telehealth technician.
Internal facilitation will be conducted by the designated local champion. In addition, each
VAMC will receive funds from the Office of Rural Health to hire a full time telephone care
manager for three years devoted to managing CBOC patients with mental health disorders
including PTSD.

Enhanced VA Implementation Strategy - The enhanced implementation strategy will add external
facilitation to the standard VA implementation strategies. External facilitation will begin
with an assessment of the current workflow at the VHA Medical Center and the affiliated CBOCs
using System Redesign methods. A member of the external facilitation team will conduct site
visits at each VAMC and conduct telephone interviews with CBOC staff. The member of the
external facilitation team will also collaborate with the local System Redesign staff member.
To assess clinical workflow, the investigators will examine the administrative structure of
the clinical units, staffing patterns, scopes of practice, service mix, standard operating
procedures (e.g., patient check-in, screening, referrals, coding), job descriptions and
annual evaluation criteria, and use of technology including telehealth, eHealth, mHealth, and
the electronic health record. For the TOP intervention, key clinical workflow elements
include: 1) how patients are referred to the care managers, 2) whether care managers are
located at the VISN, VAMC or CBOC and in what administrative unit they are housed, 3) what
type of psychotherapy is provided at the CBOC (e.g., anger management, PTSD groups, etc.), 4)
formulary restrictions and prescribing patterns, 5) how psychiatric consultations are
arranged, 6) how frequently patients are seen in mental health and primary care, caseloads,
and wait times, 7) appointment scheduling procedures for mental health and primary care, 8)
no show rates, 9) care manager software availability, 10) use of interactive video, clinical
video telehealth to home, SmartPhone apps, 10) use of psychiatric rating scales, and 11) how
workload credit is distributed. The external facilitation team will then generate a clinical
workflow chart that describes the current process of care. With advice from the external
facilitation team, the local champion will then incorporate the clinical process of the TOP
intervention (including use of technology) into the current clinical workflow chart, making
changes to the TOP intervention and/or current clinical workflow as needed. The local
champion will also meet monthly with external facilitators to troubleshoot and make
refinements.

Project Design - A stepped wedge Sequential Multiple Assignment Randomized Trial (SMART)
design will be used for this Type II Hybrid Effectiveness-Implementation project. The project
will compare implementation outcomes (patient reach) of the standard VA implementation
strategy to the implementation outcomes of the enhanced implementation strategy. This design
will also allow us to document improvements in perceived access and PTSD outcomes for
patients at sites that successfully implement the TOP intervention. The investigators will
begin by collecting pre-implementation access and outcomes data from a sample of patients. In
the first implementation step, all six VAMCs will conduct the standard VA implementation
strategy at the selected CBOCs for nine months. At the end of the first implementation step,
will collect follow-up data from the sample of patients to assess perceived access,
intervention reach, and clinical effectiveness. To conduct the SMART trial design it will be
necessary to specify a performance measure benchmark (i.e., tailoring variable) that will be
used to determine whether the standard implementation effort is successful. The investigators
will define implementation success based on the capacity of the full time care manager to
enroll a prespecified number of patients. Specifically, the care manager will report the
number of patients enrolled in the care management protocol to the evaluation team and
implementation success will be defined as 50 enrolled patients over the 9 month period. If
this performance measure benchmark is attained, the VAMC will discontinue the standard VA
implementation efforts, but the investigators will continue to collect evaluation data. If
the performance measure benchmark is not attained, the VAMC will be randomized to either
continued standard VA implementation strategy or the enhanced implementation strategy in the
next step. In the second implementation step, these VAMCs will conduct either the standard or
enhanced implementation strategy at selected CBOCs for nine months. At the end of the second
implementation step, the investigators will again collect follow-up data from the sample of
patients to assess perceived access, intervention reach, and clinical effectiveness. If the
performance measure benchmark is attained in the second step at standard or enhanced
implementation sites, the VAMC will discontinue implementation efforts. If the performance
measure benchmark is not attained at standard implementation sites, the VAMC will receive the
enhanced implementation strategy in the next step. If the performance measure benchmark is
not attained at enhanced implementation sites, the VAMC will continue to receive enhanced
implementation. At the end of the third implementation step, the investigators will again
collect follow-up data from the sample of patients to assess perceived access, intervention
reach, and clinical effectiveness.

Outcome Measures - The evaluation of the implementation strategies will be based on the
RE-AIM framework. Data will be collected from patient survey and chart review for all
patients sampled for the evaluation. The reach implementation outcome measure will be
specified as the proportion of sampled patients who received the core element of the TOP
intervention (documentation in the electronic health record of a case review of the patient
by the virtual care team). This will be assessed by chart review at three time points: 1) 9
month follow-up, 2) 20 month follow-up, and 3) 31 month follow-up. For each patient sampled,
perceived access and clinical effectiveness data will be collected at four different time
points: 1) pre-implementation baseline, 2) 9 month follow-up, 3) 20 month follow-up, and 4)
31 month follow-up. PTSD outcomes will be specified as a continuous change in the PCL-5 score
between baseline and follow-up. Perceived access will be measured using items specifically
developed for the project. Provider adoption will be assessed with qualitative interviews of
all CBOC clinicians treating a sampled patient as well as members of the TOP intervention
team (i.e., care manager, telepsychiatrists and telepsychologists) and the local champion.
During the qualitative interviews with these CBOC and VAMC providers, the investigators will
also assess barriers and facilitators to adoption, including their perceived benefits and
weaknesses of the TOP intervention.

Costs - The investigators will measure the cost of both implementation strategies both
prospectively and retrospectively. The investigators will collect data on implementation
activities during both the standard VA and enhanced implementation strategies. These costs
and activities will be assessed for the investigators' operational partners, local champions,
and external facilitators, as well as personnel from the implementation sites at the VISN,
VAMC, and CBOCs. The investigators will first examine the components of each implementation
strategy and then identify activities for each component. These activities will include
clinical workflow mapping, development of training materials, training sessions, meetings,
and conference calls. The investigators will develop project logs to document these
activities and time spent on each activity. The external facilitators, local champions and
frontline providers on the implementation team will record the logs on a regular basis. The
investigators will review meeting and conference call minutes to document time spent and
attendance by the facilitation team, operational partners and clinical leaders and frontline
providers. TOP intervention costs will be assessed from CDW (using provider IDs to identify
TOP encounters).

Data Analysis - The stepped wedge SMART design allows us to estimate the effectiveness of the
implementation strategy by comparing patient reach between groups of VAMCs randomized to
standard or enhanced implementation and by comparing reach outcomes over time within the same
VAMC as it is transitioned from standard implementation to enhanced implementation. Likewise,
by collecting PCL-5 scores pre-baseline and again throughout each implementation step, the
investigators will be able to compare access and PTSD outcomes of patients between VAMCs
successfully implementing TOP to those not successfully implementing TOP and by comparing
access and PTSD outcomes over time within the same VAMC as it is transitions from
pre-implementation to successful implementation. This will allow us to estimate the clinical
effectiveness of the TOP promising practices as implemented in routine care. Following intent
to treat principals, the investigators will estimate the clinical effectiveness of TOP for
the entire sample regardless of whether they were reached to generate a measure of population
level effectiveness. Generalized linear models will be used to account for the clustering of
patients and providers within VAMCs and CBOCs. VAMCs and CBOCs will specified as random
effects and time period will be specified as a fixed effect. The reach, access and PTSD
outcome regressions will be estimated using patients as the unit of analysis and data from
all VAMCs will be included regardless of whether the VAMC attained the performance measure
benchmark. For hypothesis 1 (reach regression), the explanatory variable will be whether the
VAMC was randomized to standard VA implementation or enhanced implementation during each
step. For hypothesis 2 (PTSD outcomes regression) and 3 (access regression), the explanatory
variable will be whether the VAMC attained the performance measure benchmark during each time
period. Using implementation and intervention cost data, the investigators will also
calculate the population level cost-effectiveness of using the enhanced implementation
strategy to deploy the TOP promising practice relative to the standard VA implementation
strategy.

Inclusion Criteria:

- PTSD Diagnosis

- Positive PTSD Screen

Exclusion Criteria:

- No specialty mental health encounters at the VAMC