Efficacy of a Web‐Based Tool in Reducing Burnout Among Behavioral Health Clinicians: Results From the PTSD Clinicians Exchange

Military populations exposed to combat are at risk for developing posttraumatic stress disorder (PTSD) (1, 2, 3). Behavioral health clinicians treating these populations face a number of challenges, including a high prevalence of PTSD among patients, increased caseloads, and long hours (4, 5, 6). Research has shown that these types of challenges can lead to burnout, a prevalent issue among the broader population of behavioral health clinicians (7). Estimates of burnout among behavioral health clinicians range from 21% to as high as 67% (8). Symptoms of burnout include feelings of hopelessness and difficulties in dealing with or doing work effectively (9). These symptoms have been found over time to degrade clinicians’ physical and mental well‐being, which can in turn affect clinicians’ personal lives (10), the quality of care provided (11), and the organizations in which they are employed (e.g., greater turnover and reduced productivity) (8, 12).

Given the consequences of clinician burnout, research has been conducted to evaluate factors that may decrease risk for the condition. Ballenger‐Browning and colleagues (4) found that having more confidants at work was associated with reduced risk of burnout. Additionally, greater transformational leadership, organizational support, and specialized clinician training are thought to be protective (13, 14, 15). Use of evidence‐based practices may also affect burnout by improving clinicians’ perceived self‐efficacy in treating PTSD (16); however, studies have found conflicting results (16, 17). Finally, compassion satisfaction, characterized by appreciation of the positive aspects of providing care, has also been found to protect against burnout (9, 18).

In addition to identifying protective factors, studies have evaluated the effectiveness of interventions designed to mitigate burnout (8, 19, 20, 21, 22). Most of these interventions consisted of trainings or workshops and have been found to have limited effectiveness over time (8, 19). Other studies focused on examining the utility of Web‐based interventions (21, 22) have reported mixed results. Given the range of effectiveness among interventions for burnout (19), research into more diverse approaches is needed. Thus, a Web‐based tool, the PTSD Clinicians Exchange, was designed to increase clinician familiarity with, perceived benefits of, and implementation of evidence‐based practices for the treatment of PTSD and to address clinician burnout. The Exchange built upon previous intervention efforts by using a Web‐based format and by linking clinicians with resources to enhance access to specialized training, which has been associated with reduced levels of burnout (15). Similar to previous interventions (22), the Exchange also provides clinicians with immediate feedback regarding their current level of burnout and suggests possible self‐care strategies to address burnout symptoms. Unlike previous efforts, the Exchange targets social support, an important protective factor against burnout (14, 23), by providing clinicians with the opportunity to connect with other colleagues and to receive feedback from experts. Given the number of protective factors offered to clinicians through use of the Exchange, this study sought to determine whether the Web site was effective in reducing clinician burnout. Secondary to this aim, levels of burnout across clinician settings were compared.

Methods

Intervention

The PTSD Clinicians Exchange is composed of three sections: “Engage,” which focuses on 26 key practices for PTSD; “Connect,” which includes a number of interactive features aimed at connecting clinicians with each other; and “Inspire,” a self‐care section (of primary interest in the present study) that provides resources aimed at managing stress, burnout, and secondary traumatic stress (STS). The Exchange also includes a self‐assessment component consisting of the Professional Quality of Life Scale–5 (ProQOL‐5) (9). By completing this self‐assessment, clinicians can ascertain their current level of burnout, STS, and compassion satisfaction.

Sample

Data were collected as part of the PTSD provider registry study, a 12‐month randomized controlled trial of clinicians in the U.S. Department of Veterans Affairs (VA), Department of Defense (DOD), and general community beginning in May 2016, after approval by the institutional review boards of Stanford University and the New England Research Institutes. Of the 1,453 clinicians who responded to recruitment invitations sent via email and posted to online message boards for the behavioral health provider community, 792 consented to participate (55%). A total of 605 clinicians completed the baseline survey (76.4%) and were randomly assigned, with the use of a 3:1 randomization scheme, to either the intervention group with access to the Exchange or to the newsletter‐only control group. Participants in the intervention group were also sent biweekly email reminders featuring PTSD treatment practices included on the Web site. All participants were surveyed again at 6 and 12 months after baseline. The surveys consisted of questions to assess participants’ demographic characteristics, burnout status, and other covariates (e.g., years of experience treating veterans, compassion satisfaction, STS, practice patterns, and organizational support).

Measures

Burnout, STS, and compassion satisfaction were measured with three 10‐item subscales within the ProQOL‐5 (9). This scale is the most commonly used measure of the positive and negative effects of working with individuals who have experienced traumatic events and has demonstrated exceptional construct validity (9, 14, 15, 16, 22, 24, 25, 26). Items were scored on a 5‐point scale (1, never, to 5, very often) and summed. Higher subscale scores indicate greater risk of burnout and STS or higher levels of compassion satisfaction. Cronbach's alpha ranged from 0.88 to 0.91, 0.81 to 0.84, and 0.82 to 0.84 across time points for the compassion satisfaction, burnout, and STS subscales, respectively.

To assess practice patterns, clinicians were asked about the total number of patients they treated during the past week and the number of those with PTSD. Clinicians also reported the number of hours over the past week spent on direct patient care and administrative activities and the total number of hours worked. The use of evidence‐based practices for PTSD was assessed by asking clinicians whether they had ever used prolonged exposure treatment, cognitive processing therapy, stress inoculation training, or eye movement desensitization and reprocessing.

Overall attitudes toward evidence‐based practices were measured using the Evidence‐Based Practice Attitudes Scale (EBPAS) (27). All fifteen items were rated on a 5‐point Likert scale (0, not at all, to 4, to a very great extent) and averaged (Cronbach's alpha over time points ranged from 0.82 to 0.84).

Organizational support was measured using an item developed for this study that assessed whether the clinician's primary supervisor was supportive of using “treatments supported by research.” This item was scored by using a 7‐point scale ranging from extremely unsupportive to extremely supportive; higher scores indicated greater support. An additional category of “NA, no primary supervisor” (not part of the 7‐point scale) was treated as the highest level of support for this analysis because of the lack of barriers to practice implementation when there is no supervisor. Specifically, a clinician who has no supervisor would be able to implement research‐supported treatments free of any supervisory barriers, as even a supportive supervisor may explicitly or implicitly limit which techniques the clinician uses.

Web site tracking metrics were integrated into the Exchange. Each participant in the intervention group was assigned a unique identifier to access the Web site, which was used to determine whether the participant used the Exchange. Participants’ Web site usage was categorized as having no access (control group), having access but never visiting the site, and having access and visiting the site once or more.

Data Analysis

We completed all statistical analyses by using SAS, version 9.4, and we tested statistical significance at a level of 0.05. We used one maximum likelihood iteration (28) to impute missing data for the ProQOL‐5 and EBPAS scores, age, years of experience, number of clients, and hours, although these factors all had few missing data (range 2–18 missing items from 605 participants). Descriptive statistics were then calculated for all variables of interest. Associations between demographic characteristics and Exchange participation were examined by using chi‐square tests for categorical variables and t‐tests or Mann‐Whitney U tests for continuous variables. We used t‐tests to examine the associations between burnout and Exchange participation at baseline and 12 months. Because the primary outcome of interest was burnout scores at 12 months after baseline, we examined the associations between these scores and the variables of interest by using unadjusted linear regression models and then by using a multivariable analysis with backward elimination (step 1 below). We then constructed a stepwise linear regression model, with the first step based on the presence of significant predictors of burnout at 12 months found during the initial analysis above. Further steps were used to add a variety of clinical characteristics. In the last steps, the model examined group assignment, Web site usage, and use of evidence‐based practices.

Results

Sample Characteristics and Preintervention Equivalence

Characteristics of the participants are shown in Table 1. Most participants were Caucasian (75%), female (68%), and psychologists or social workers (76%), and participants had a mean age of 47. Forty‐three percent of participants were from the VA, 18% were from the DOD, and 39% were from the community. Participants reported an average of 24 clients per week, with 11 of those having a diagnosis of PTSD. The clinicians also reported working an average of 40 hours per week, with 25 of those hours spent on direct client care and 12 on administrative activities. Most of the sample (84%) reported using at least one of the four evidence‐based practices for PTSD listed on the Exchange. No significant differences were found among the intervention and control groups for demographic or other baseline measures. Among the 605 participants, 379 (63%) completed the assessment at 6 months, 395 (65%) completed the assessment at 12 months, and 311 subjects (51%) completed both assessments. Because completion of assessments varied by practice setting, we controlled for setting in all analyses.

Relationship Between Burnout and Study Variables

For the primary outcome of interest, burnout scores over time, we found no significant differences between treatment arms (Figure 1), even when we controlled for other factors in the regression analyses. At baseline, mean±SD scores of burnout for the intervention and control groups, respectively, were 19.9±5.1 and 20.2±5.4; at 6 months, mean burnout scores were 20.7±5.5 and 20.5±5.4; and at 12 months mean scores were 20.3±5.4 and 20.4±5.6.

The results from our unadjusted regression models (Table 2) showed that greater age (β=–0.28, p<0.001), more experience treating mental illness (β=–0.17, p<0.01), more organizational support for using treatment supported by research (β=–0.24, p<0.001), and a higher score on compassion satisfaction at 12 months (β=–0.75, p<0.001) were inversely related to 12‐month burnout scores. On the other hand, having a larger caseload of clients with PTSD (β=0.21, p<0.001), more administrative hours (β=0.12, p<0.05), a higher baseline burnout score (β=0.75, p<0.001), and a higher STS score at 12 months (β=0.56, p<0.001) were predictive of having significantly higher 12‐month burnout scores. Clinicians in the DOD (β=–0.16, p<0.01) and community (β=–0.34, p<0.001) sectors had significantly lower burnout scores compared with participants in the VA.

A stepwise linear regression model was developed to further examine the relationship between participant characteristics and treatment group assignment with burnout. As shown in Table 3, baseline burnout score, 12‐month compassion satisfaction score, 12‐month STS score, and age were entered in step 1. Each of the variables entered in step 1 were significant predictors of burnout score at 12 months. Practice setting, years of experience treating mental illness, hours spent on administrative work, organizational support for using evidence‐based practices, number of clients with PTSD, and EBPAS scores were entered in steps 2–7, respectively. None of these variables were found to significantly predict burnout score. We entered treatment arm in step 8 and use of evidence‐based practice in step 9, neither of which significantly predicted burnout at 12 months (Table 3, Model 1). Because only a third of those randomized to the intervention group accessed the Web site, Web site usage was entered in the model (Table 3, Model 2) to further investigate its effect on burnout. Similar to treatment group assignment, Web site usage was not associated with burnout at 12 months.

Discussion

The PTSD Clinicians Exchange, an innovative Web resource, was developed with the primary goal of increasing behavioral health clinicians’ familiarity with evidence‐based practices for the treatment of PTSD and increasing the perceived benefits and implementation of these practices. In addition to providing treatment resources, the Web site was designed to enable clinicians to connect with one another via the Clinicians’ Corner and to provide self‐care resources, including ways to self‐assess for burnout. While reducing clinician burnout was not the main goal of the Exchange, several features target a number of protective factors that could mitigate burnout (4, 16). Although it was anticipated that clinicians’ main focus in using the Web site would be to identify resources for clinical best practices for the treatment of PTSD, we hypothesized that the use of the additional self‐care resources would reduce burnout. However, Exchange participation did not significantly reduce burnout during the study, even after accounting for clinician use (some or none) of the Web site.

Although the Exchange uses a unique approach to address burnout, previous burnout interventions have had varying levels of success, so these results were not completely unexpected. The Web‐based self‐care program developed by Shoji et al. (21), whose only aim was to reduce burnout and STS symptoms, did not significantly reduce burnout for the Web‐only group, nor did an online intervention designed for physicians (29). A mobile app for reducing burnout designed by Wood et al. (22), however, was found to significantly reduce burnout, suggesting that mobile technologies may be more effective for motivating busy clinicians to access self‐care resources, particularly during nonwork hours. Adapting our tool so these resources could be used at the times and places chosen by the clinicians via an app may have led to greater use of the Web site; only a third of the clinicians in the intervention group accessed the Exchange. Although baseline burnout was not found to be associated with use of the Web site, in a study of how the Exchange was used by participants, we found that lower burnout was predictive of more visits to the Web site and more pages viewed, indicating that burnout may have played a role in how the clinicians interacted with the Web site. Furthermore, as compared with national norms (9), we may not have seen an effect of the Web site because of the high average level of burnout among clinicians in our sample.

Although the intervention did not reduce burnout, we identified baseline predictors of burnout at 12 months that may help optimize future interventions. Our results were fairly consistent with previous research: younger age, less experience and organizational support, lower compassion satisfaction score, higher baseline burnout and STS scores, increased caseloads of patients with PTSD, and more administrative hours were all associated with higher burnout scores (4, 6, 9, 13, 14, 24, 25, 30). The findings also suggested differences in levels of burnout across service sectors, with VA clinicians reporting significantly higher burnout scores compared with DOD and community clinicians. This finding is surprising given that the VA has a number of targeted self‐care resources designed to reduce burnout, which are less available among community clinicians. However, other organizational factors, which are less prevalent in other sectors, may predispose VA clinicians to burnout (e.g., less diagnostic diversity in their caseloads and more administrative burden on clinicians) (31). We also found that use of evidence‐based practices did not reduce burnout, a finding that was somewhat consistent with previous equivocal findings (16, 17). Additionally, these null findings may have been due to the large percentage of clinicians in our sample who reported already using evidence‐based practices to treat patients with PTSD.

The results from our multivariable models, showing that the only significant predictors of 12‐month burnout across all steps were baseline burnout, 12‐month STS score, 12‐month compassion satisfaction score, and age, suggest that baseline burnout and other factors may contribute to the maintenance of burnout over time. Compassion satisfaction at 12 months was also shown to have the largest effect size on burnout score at 12 months. This finding confirmed previous research, which found that increased compassion satisfaction scores are associated with reduced burnout, because work is seen as a source of personal gratification rather than a source of fatigue or emotional exhaustion (9, 18).

One noteworthy limitation to this study is the low number of participants in the intervention group who accessed the Exchange. While this finding is typical of Web site engagement when no incentive to participate is offered, it limited our ability to assess whether Exchange features effectively reduced burnout. However, this pattern of Web site engagement is likely to reflect how clinicians would use this tool in the real world. Also, while participants were randomly selected for the Exchange group or the newsletter‐only control group, the overall sample consisted of self‐selected volunteers, which may have introduced unmeasured response biases. Additionally, self‐care resources were a relatively small part of the Exchange overall.

Conclusions

In this study, we found average levels of burnout across clinicians in the VA, DOD, and community sectors treating military populations that had experienced trauma, and we found significantly higher levels of burnout among clinicians in the VA compared with those in the DOD and community sectors. Although the PTSD Clinicians Exchange did not reduce burnout, it may be a valuable tool for clinicians looking for resources to improve their practice. One possible explanation for our findings is that the self‐care content provided on the Web site was not perceived as relevant for mitigating burnout. Instead, clinicians may perceive content promoting organizational support or work satisfaction as more useful. The results were striking in how few clinicians accessed the Web site. Given burnout's far‐reaching effects, from implications for the individual patient to the organization as a whole, organizations should explore ways to engage clinicians in targeted burnout interventions and provide resources to promote self‐care (32). Findings from the present study indicate that a Web‐based dissemination tool focused on self‐care is unlikely to be used by those who need it most. Busy clinicians who are at risk for burnout are unlikely to add additional tasks to their to‐do list. Although it was not tested in this study, a different dissemination method, such as an app that can be accessed in a variety of settings (22), may be more likely to be used by busy clinicians. Future research should build on these findings by investigating the relative importance of dissemination methods versus intervention content for reducing burnout.