Developing an Online Therapeutic Intervention for Chronic Pain in Veterans

The research study for this career development plan involves the development, iterative user
testing, and evaluation of an online ACT treatment program for Veterans with chronic pain.
This study will determine whether a larger-scale, more definitive VACT-CP full scale efficacy
trial is indicated.

Phase 1: Development Phase (15 months): I will develop a detailed VACT-CP protocol for
individual online treatment. Treatment techniques will be based on previously-developed ACT
treatments for chronic pain, as well as additional ACT training I will obtain during the
early phase of the CDA-2. During this phase, mental health and other clinical staff providing
pain care at the Bedford VAMC (n = 10) will provide feedback and input on the intervention
content.

Phase 2: Pilot Evaluation of Intervention Components in Field-Based Iterative Usability
Testing (21 months): The investigators will recruit Veterans (n=15) with chronic pain for
usability testing, to assess the delivery mode, and utilize an Integrated Technology
Acceptance Model to iteratively assess feasibility and usability of the VACT-CP online
treatment. This portion of the project will focus on refining the delivery mode, and revising
chronic pain intervention content as necessary. With my mentors, I will review the adequacy,
utility of specific procedures, and intervention foci to revise the online treatment
accordingly.

Phase 3: Feasibility Pilot Randomized Controlled Trial (30 months): The investigators will
assess the feasibility and acceptability using a Stage 1b RCT pilot test comparing VACT-CP (n
= 20) to a wait-list control group (n = 20), including ease of recruitment, retention in each
condition, treatment receptivity, attrition and retention in each condition, sustained
participation, and the assessment process. The investigators will also describe preliminary
data on the impact of VACT-CP vs. the control condition on pain-related functioning related
to social, occupational, and physical functioning, emotional functioning, mental health, and
quality of life.

PHASE 1: INTERVENTION DEVELOPMENT

The goal of this phase will be to 1) adapt ACT manualized treatment of chronic pain in
Veterans into an online platform using the ECA as the intervention coach, and 2) gather
qualitative feedback from chronic pain providers and content experts (n = 10 - 12) on the
VACT-CP system for intervention refinement and potential benefits, barriers, referral
concerns, and overall thoughts on the VACT-CP system.

Initial Development of the VACT-CP System

Procedure. The development of the intervention and its revision will be an iterative process
over Phases 1 (Intervention Development) and 2 (Feasibility and Usability Testing)
(specifications for intervention revision for Phase 2 are below) and will include: 1)
reviewing existing ACT CP treatment manuals with mentors; 2) developing a preliminary online
version of the intervention, with mentor input, 3) revising the intervention based on mentor
and consultant feedback.

Creation of the initial VACT-CP platform will include collaboration with mentors , as well as
the Northeastern Personal Health Informatics team at Northeastern University led by Dr.
Timothy Bickmore. Using input from this team of experts, as well as my own experience in
educational technology and online course creation, I will create a basic prototype of the
intervention including the main VACT-CP content components, alignment of session goals and
outcomes, and the creation of a corresponding patient workbook with information on the
system. This process will also include close mentorship between Drs. Kelly and Heapy to
assess current ACT for CP manualized treatment, and adaption of this content to the online
system.

VACT-CP Treatment Content and Structure. The following components will be emphasized in
VACT-CP based on previous manualized ACT-chronic pain treatment by Vowles and Sorrell Life
with Chronic Pain: An Acceptance-based Approach Therapist Guide and Patient Workbook and
utilizing ECA technology to create a personalized, therapeutic experience. Creation of the
intervention will entail modification of the ECA image, creation of the intervention scripts
and programming for the online user-experience. Participants will receive eight online
modules, provided as weekly sessions that will feature an ECA (virtual therapist) as a
treatment guide.

Adaption to Online Platform. Treatment content for VACT-CP will be integrated into the
Northeastern University platform created and maintained by the relational agent research
group managed by Dr. Timothy Bickmore, which has already programmed and run several
VA-specific, IRB-approved interventions. Dr. Bickmore has confirmed that both the budget and
timeline for this project are appropriate, and thus feasible. This website will be available
to Veterans for free for this project, and Veterans will be able to re-access the any module
where they left off, or restart a module if preferred after logging out using their
de-identified username and password provided by research staff. For the purposes of pilot
testing with providers and Veterans, the user interface will also first be mocked-up using
PowerPoint slides, and following feedback from providers will then be adapted to the current
ECA online platform (Phases 1 and 2). This platform will include elements such as
personalized conversations with the embodied conversation agent, starting with a mindfulness
exercise at the beginning of each module, and moving through major session components. Each
session will incorporate different interactive psychoeducation experiences for each session,
goal setting and tracking, and finally end with weekly homework aligned with the goals of the
next session.

Planning for and addressing potential safety concerns online is a major concern, and
consultation for the inclusion of safety procedures and information has been provided by Drs.
Kelly, Heapy, and Greene. Consequently, in addition to the modules, information regarding the
Veteran Crisis Hotline will be incorporated into the website with the addition of a button
providing phone number information and the PI's number will be included within the website as
well should a Veteran need to access clinical staff. The investigators will also assess with
Veterans whether they were able to locate these resources early in the usability and pilot
testing.

Qualitative Provider Interviews

Procedure. A sample of 10 current mental health providers (including social workers,
physicians, PACT nurses, and psychologists) who provide treatment for chronic pain at the
ENRM VA will be interviewed for formative assessment of the potential benefits, concerns, and
institutional VHA dissemination issues for the VACT-CP intervention, using a semi-structured
interview guide and a "think-aloud" strategy for intervention review.

Sample. Providers will be recruited from presentations and emails to Primary Care/Behavioral
Staff and the Mental Health Service Line at the Bedford VAMC, and scheduled for a time for a
45 minute to one hour interview during regular work hours, though potential participants will
also have the option to interview over the phone. Participants will be consented for
participating in the interview and having their participation audio-recorded.

Interview Guide. This interview will include open-ended questions related to palatability of
the intervention, feasibility of engagement with Veterans with chronic pain, interest or
potential concerns with referring Veterans to such a program, and reactions to a short
walk-through of the intervention. The purpose of these questions is to gather information
from valuable stakeholders and potential referral-providers on VACT-CP system to address
refinement of the program, possible benefits, barriers, referral concerns, and overall
thoughts on the VACT-CP system Questions for palatability are based on the "organizational
perspective" components of the Practical, Robust Implementation and Sustainability Model
(PRISM). Interviews will be audio recorded and transcribed.

Provider Interview Analyses Qualitative data analysis will use a thematic approach for the
data related to the goal of ascertaining intervention palatability and feasibility and
referral issues, and a narrative approach for data collected with the goal of intervention
refinement. Two independent reviewers will code Section 2 of the interview transcriptions
using open-ended coding. Each element will be summarized separately, and overall summary
themes will be generated. Summary tables of key points will be generated, similar to previous
formative assessments performed. These tables will ensure data saturation has been reached
for both provider input. Two reviewers will analyze qualitative data using a modified
grounded coding to identify themes using audio recordings after each interview. Intervention
areas will be modified as feedback is given. Each set of interviews will take place with 3 -
4 participants, to allow for iterative adjustment of the intervention based on feedback.
Number of changes needed for each intervention area will be tracked, and less intervention
refinement will be anticipated by the end of the interview cycle. However, if intervention
palatability is not achieved, further testing using other providers will be utilized. The
generated information and themes will be utilized for development and refinement of the
VACT-CP intervention, to learn from major stakeholders important areas for potential revision
during intervention development.

PHASE 2: FEASIBILITY AND USABILITY TESTING The goals of Phase 2 are to: 1) gain further
feedback on the VACT-CP treatment by gathering information from Veterans via usability
testing to; 2) assess Veteran interest, satisfaction, and usability of the online VACT-CP
treatment; 3) iteratively address any technological concerns. This Phase will allow for
feedback and preliminary support that the Phase 3 small-pilot RCT is merited.

Given past research suggesting that similar interventions are usable and beneficial, the
investigators anticipate that usability findings suggesting concerns or places for revision
will not be such a strong barrier that the intervention will not be at all useful in this
population. However, should Phase 2 data indicate that a small-pilot RCT is not merited, the
usability information from this Phase will be utilized to inform further VACT-CP refinement
until the usability bugs have been worked out and the system is ready for testing in a
feasibility pilot RCT. This information will also be disseminated to inform future online
chronic pain treatment development.

Sample Information

Sample Size: No single participant reviewer can identify all the usability problems in a
system; typically 3-5 reviewers can identify 80% of the usability problems, with diminishing
returns from additional reviewers per round of testing. The investigators will conduct 3
rounds of usability testing with 4-5 Veterans per round to assess and iteratively address
usability concerns (n=12-15 total).

Recruitment. Subjects will be 15 patients primarily recruited by referral from the Primary
Care Behavioral Health at the ENRM VA Hospital. Eligible participants will have a current
diagnosis of non-cancer chronic pain, defined as related to either Musculoskeletal pain or
Joint Problems/Osteoarthritis or presence of chronic pain of at least mild to moderate
severity. The investigators have decided to target joint and musculoskeletal pain as these
have been shown to best respond to psychological, behavioral interventions, whereas
neuropathic pain and headaches having less research supporting this. However, the
investigators will not exclude for other diagnoses of pain, as this is typical and would not
necessarily preclude Veterans from benefiting from the treatment.

The investigators have verified that there are 2,605 Veterans who were seen over the past
year (i.e., having had at least two appointments for pain) at the Bedford VA with chronic,
non-cancer pain concerns, suggesting that the recruitment goal of 15 Veterans within 3 months
to do a single-session usability test for Phase 2 is achievable. In addition, I will give
presentations to primary care providers, social workers, and mental health clinicians at the
Bedford VA to enhance recruitment. Using CPRS, Veterans meeting eligibility criteria for
chronic pain seen within 12 months at the Bedford VA will also be sent letters inviting them
to participate.

Screening Procedures. Callers will be screened by phone. Eligible participants will be
scheduled for an in-person assessment with the PI, who will confirm study eligibility and
obtain informed consent. A research associate affiliated with the research study will then
conduct clinician-rated assessments, and subjects will complete self-report measures.

Feasibility and Usability Assessment Procedures Eligible Veterans will come to the Bedford VA
to complete baseline demographic and health measures. Following baseline assessment, Veterans
will be provided with an iPad that will walk them through different portions of the user
interface and treatment set-up, during which time they will be asked questions and recorded
as a usability interview. Specifically, different images for options of the virtual therapy
guide, as well as a PowerPoint with information about the user interface will be shown. Next,
Veterans will be provided with written instructions for use, and do a behavioral walk-through
of how to use the online system. Patients will be given instructions on interacting with the
system, and will then walk through a portion of the treatment using PowerPoint slides that
will simulate the VACT-CP treatment and usability interface, and then complete the Usability
Survey and SUS (see measures below). During this time, the Veterans will take part in a
contextual interview wherein Veterans move through the system unguided and are observed and
asked questions about their experience. After this, additional, open-ended questions will be
asked regarding Veteran preferences.

For purposes of feedback and iterative program development, the Veteran will only be provided
with the part of the first VACT-CP session during this time. The participant will then be
left in a secure, confidential room to walk through the VACT-CP system on their own, during
which time their use of the system will be recorded. Approximately 4 - 5 Veterans will be
recruited and participate in this usability testing at a time, with 3 different usability
testing arms, in order to allow for time to iteratively address any functionality issues or
concerns with the user interface. Following this walkthrough, participants will be provided
with measures of demographics, usability and health information technology use, including a
retrospectively worded version of the Usability Survey items (e.g., I enjoyed using the
VACT-CP program) and a semi-structured qualitative exit interview. Participants will receive
compensation for their in-person visit, which should take approximately 2 - 3 hours.

Assessment Measures A broad range of reliable and valid measures will be used to provide an
assessment of chronic pain difficulties, demographic information, and technology use,
technology interest, and comfort. In addition, Veterans will be observed while interfacing
with the VACT-CP system, recorded, and asked questions by the interviewer. This will consist
of specific questions related to the different parts of the system the veteran interacts with
(e.g., the ECA design, structure of the modules, feedback system) as well as more open-ended
when not provided with specific cues (e.g., what are you trying to get to from this page,
what could be better about how this page is set up, etc.) Questions will be open-ended in
nature, and address the extent of any problems with accessing and using the VACT-CP treatment
online, how they might perceive the usefulness or importance of VACT-CP focal areas of
treatment, whether they would recommend this treatment to a fellow Veteran, and any problems
or concerns they may have with using the VACT-CP program.

Using this additional, open-ended interview protocol, the investigators will also inquire as
whether they would be interested in using the VACT-CP treatment at home and the role of their
virtual therapist in the patient's interest in the treatment, and about past strategies or
tools the patient used to address their chronic pain issues and whether the VACT-CP treatment
might seem more useful than prior options or strategies. Finally, participants will be asked
to indicate what, if any, changes they would make to the treatment to assist in
self-management of their chronic pain. This will also include assessing their interest in it
complementary and alternative methods of pain management (stretching, yoga, progressive
muscle relaxation, etc.) to assess if it would be useful and feasible to add these options to
the platform later if desired. The interview will be video recorded and coded to look for
similar themes, difficulties, revisions, and strengths of the system.

Intervention Revisions During Phase 2, in which Veterans will interact with the VACT-CP
online treatment, the investigators will: 1) gather feedback from Veterans regarding
acceptability and usefulness of the treatment, 3) gather and analyze complementary data from
SUS-8, 4) address any technological issues as they arise, and 5) examine preliminary data
from this phase of the study. Modifications to the treatment will be based upon: 1)
acceptability and feasibility of the online treatment, and 3) qualitative feedback obtained
from patients, therapist, and mentors. Individual feedback sessions at the end of phase 2
will be held to gather information for refining the treatment manual and make it acceptable
to patients.

The investigators anticipate that the Veteran intervention reviewers might miss bugs or other
user interface problems, and anticipate that some revisions to the online system and program
may need to be made for Phase 3. For instance, one potential issue that may arise might be an
inability for the Veteran to read the text on the tablet screen. After receiving this
information during an initial feedback round, the researchers will amend the program with a
potential solutions i.e. text size or text color being altered, with the revised version then
being piloted in the next round of Veteran user testing. Some usability challenges will be
addressed with introductory handouts/tutorial materials/orientation to the program at the
beginning of the intervention session. In addition, large organizational issues will be
discussed before moving on to platform refinement.

Data Management Patient files will be kept in a locked cabinet and password protected files
where data will be encoded with ID numbers. IDs will be unrelated to any identifying
information. Video recordings will be stored on a secure, VA approved server. The majority of
data analysis will consist of qualitative and observational methods, including coding
qualitative interview data and coded video files. Data will be checked for inconsistencies,
omissions and errors. As only 15 subjects will participate in Phase 2, only summary
quantitative statistics will be computed to describe the sample and to determine outcome on
the study measures. Data gathered through qualitative interviews and quantitative measures
will be housed on a VA-approved cloud-based data storage server.

Data Collection and Analysis Qualitative Combined Usability and Contextual Interview. The
investigators will focus on usability and feasibility to address Aim 2. During 60-75-minute
usability sessions held in private, secure rooms SoCRR rooms within the Bedford VAMC,
participants will assess the usability of the VACT-CP intervention using a "think-aloud
protocol", verbalizing their thoughts as they navigate the website. After a period of
exploration with the site, the research staff will use a pre-created protocol to ask Veterans
to perform a series of tasks on the website (i.e., entering therapy goals, accessing the
different modules, completing pre-session assessment, interacting with the virtual
therapist). The usability testing space is already equipped with a computer, tablet, and a
web camera. As a participant navigates the intervention website on a tablet or computer,
their actions will be recorded through webcam electronically, and Dr. Reilly will observe the
actions in real-time within the room, and a blinded observer/note taker. Drs. Waring and
Quigley will consult on usability testing procedures.

The video recordings and notes made by Dr. Reilly during user testing will be transcribed and
analyzed using NVivo software available through the Bedford VAMC. A modified consensual
qualitative research (CQR) approach will be utilized to code the transcribed videos, which
Dr. Reilly has been trained in and utilized in past peer-reviewed publications. CQR can be
particularly useful in allowing for a team of coders to identify themes arising in both
structured and more open-ended interview moments. Dr. Reilly will develop an initial codebook
for using in coding based on the key activities and questions in the user-testing protocol.
Dr. Reilly and the research associate will then employ the codebook in identifying
usability-themes within the videos, with an additional research staff identified to act as a
code auditor. According to CQR, the following processes occur to allow for depth and breadth
in describing the shared experiences of participants: (b) two coders utilize the initial
codebook throughout the data analysis process to foster multiple perspectives; (c) consensus
is researched between coders about the meaning of the data; (d) at least one auditor must
check the work of the primary team of judges and minimize potential bias; and (e) domains and
themes are reported in terms of the frequency of arising themes.

Themes will then be categorized according to the following CQR groupings: general (include
all or all but one of the cases), typical (include more than half of the cases up to the
cutoff for general), and variant (at least two cases up to the cutoff for typical).
Specifically, the themes generated by this qualitative data will be used to identify user
performance measures, errors and difficulties, and tasks that do not meet acceptable
criteria, and general trends and categories of users' behavior such as how well users can
complete an assigned task on the VACT-CP system and where they are encountering problems.
Using this method, the investigators will classify all general comments regarding usability
as necessary to include in iterative development, while variant and typical will be discussed
within the research group to decide upon potential ways to incorporate this feedback into
intervention refinement.

In addition, Veterans will complete a series of brief usability questionnaires following user
testing and the qualitative interview. Usability Survey and SUS data will be input into
REDCap, reported descriptively using SPSS software, and used to describe Veteran's general
perceptions and opinions of the technology's usability, both within the controlled testing
setting and for future use within their home. This feedback will then be triangulated with
qualitative data, and used to iteratively refine the VACT-CP program between the three waves
of user testing. The investigators will complete this process with 4-5 Veterans per round;
Dr. Reilly then summarize suggested changes, obtain input from the full mentoring team, then
meet with Dr. Bickmore's staff to implement potential changes. Results will be published to
inform other researchers about usability findings related to format feedback, user interface,
interest, and perceptions of behavioral intervention content.

PHASE 3: FEASIBILITY PILOT RCT

The goals of the feasibility pilot RCT study phase are to conduct a Stage 1b RCT pilot trial
to: 3a) explore relationships between Veteran traits, usability beliefs, and technology
perceptions on intended and actual use of the VACT-CP system; 3b) describe differences in
VACT-CP versus a wait-list control group on pre- and post-test measures of pain acceptance,
pain level, pain-related functioning, and quality of life, treatment satisfaction with the
VACT-CP system and ACT-related outcomes including pain acceptance, valued living and
experiential avoidance; 3c) evaluative the relative feasibility and acceptability of the
VACT-CP intervention procedure and waitlist control, including ease of recruitment, retention
in each condition, treatment receptivity, attrition and retention in each condition,
sustained participation, and the assessment process. A total of 40 Veterans will be enrolled
to test the VACT-CP system within their home.

Participant Selection and Recruitment

Sample Size: Based on the previous recruitment of Veterans with chronic pain at the Bedford
VAMC, the investigators estimate that approximately 65% of Veterans screened for
participation by research staff, recruitment letters, flyer recruitment, and referrals from
PCBH staff (N=60 enrolled participants) will meet the inclusion/exclusion criteria and be
eligible for the pilot trial. Similar to Phase 2, the investigators will not exclude for
other diagnoses of pain. Given this, the investigators will aim to recruit a total of 60
participants over fifteen months total to allow for the desired final 40 participants
screened as eligible and randomized into the pilot RCT study, specifically aiming for
recruiting 3-4 participants per month. Subjects for the proposed study will be recruited over
the course of approximately 15 months and be screened according to their presence of chronic
pain and other important.

Subjects will be recruited from the Bedford VAMC, where 2,605 Veterans are currently (as of
November, 2017) being seen at the Bedford VAMC for chronic, non-cancer pain concerns,
suggesting that the investigators' final recruitment goal of 40 persons total enrolled (1.5%
of the total Bedford population) is feasible. In addition, I will give presentations to
primary care providers, social workers, and mental health clinicians at the Bedford VA to
enhance recruitment. Using CPRS, Veterans meeting eligibility criteria for chronic pain seen
within one month at the Bedford VA will also be sent letters inviting them to participate.

Study Procedures and Assessment

Treatment Procedures. Following recruitment, a research associate not otherwise involved in
the study will screen participants prior to enrollment. Veterans will be randomized in a 1:1
ratio, with a research staff member utilizing and keeping the schedule. Veterans randomized
to VACT-CP will receive 8 online-module based weekly sessions of treatment via personal
computer or provided tablet with wireless accessibility at the Bedford VAMC. Veterans in the
waitlist control will be provided with a list of common pain resources at the Bedford VAMC.
The VACT-CP program will be housed on a secure web server through Northeastern University,
with no Veteran private health information collected by the system.

If during Phase 2 it becomes clear that more or less sessions are necessary (based on
indicators previously discussed for usability and feasibility testing), the number of
sessions for Phase 3 will be changed accordingly. Veterans in both conditions will complete
the assessment battery (including all measures below, except for the treatment satisfaction
and working alliance inventory) at baseline. In addition, the full battery will be taken at
mid-point of the intervention phase (4 weeks) and at the end, immediately following the
treatment (8 weeks). Veterans will be tracked in terms of their post-referral treatment
engagement (e.g., enrolling in any of the pain-resource options on the handout). In addition,
a research associate will call the Veteran at weeks 3 and 6 and conduct a brief safety
assessment and qualitative interview with Veterans assess for potential suicidality,
usability issues, and participant health and well-being.

Inclusion Criteria:

- Current diagnosis of non-cancer chronic pain, defined as:

- at least one pain-related diagnosis indicated by an ICD-9 or -10 code related to
either Musculoskeletal pain or Joint Problems/Osteoarthritis

- or presence of chronic pain of at least mild to moderate severity as indicated by
two or more NRS pain scores of 4 at three separate VA outpatient visits in past
year based on a CPRS record review;

- Has a working, high-speed wireless internet connection at home, or is willing to
access sessions either at the Bedford VAMC by using a tablet in a secure space

- Competent to provide written informed consent

Exclusion Criteria:

- Any current or lifetime DSM-5 psychotic disorder

- Current or recent (within 1 month of study entry) DSM-5 alcohol or drug use disorder

- [Current use of any other chronic pain-related behavioral or psychological treatment]

- Any cognitive impairment that would interfere with study participation

- Clinically significant suicidality within the past year

- Presence of any clinical features requiring a higher level of care (inpatient or
partial hospital treatment)

- Any cognitive or physical impairment that would interfere with study participation of
using a tablet/computer and providing feedback