Validation of Two Diagnostic Assessments for Opioid and Stimulant Use Disorder for Use by Non‐Clinicians

Aggressive efforts are needed to identify those who would likely benefit from substance use disorder treatment. Current tools to determine if a patient meets criteria for a moderate to severe opioid and/or stimulant use disorder per the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM‐5) have limitations. Most DSM‐5 diagnostic tools such as the Structured Clinical Interview for DSM Disorders (SCID) (1), the Composite International Diagnostic Interview (CIDI), and DSM‐5 Checklist (2) require intensive trainings, are time consuming, and are costly; many also required to be administered by a clinician (3, 4, 5, 6, 7).

In response to the need for diagnostic tools that can be quickly administered by non‐clinicians as well as clinicians, the Rapid Opioid Dependence Screen (RODS) was developed in 2015. Despite its name, the RODS was designed as a diagnostic tool to identify those with opioid dependence based on DSM‐IV criteria with eight items (8, 9) and is currently used in non‐medical settings (e.g., courtrooms) by non‐clinical staff (10, 11, 12) as well as clinical settings (13) to identify patients eligible to initiate MOUD. The RODS is also used in emergency departments to rapidly assess for MOUD eligibility (14). Since the development of the RODS, DSM criteria have been updated from a hierarchal diagnosis of substance abuse or dependance in the 4th version based on two sets of symptoms criteria, to a single diagnosis of a substance use disorder with three levels of severity (mild, moderate, and severe) based on the number of criteria the individual met from one set of symptoms in the 5th version. The DSM‐5 provides additional guidance for being in remission: long‐term remission, someone who met DSM‐5 criteria for a substance use disorder over 12 months ago, but does not meet criteria for the past 12 months other than the symptom of craving; and short‐term remission, someone who met DSM‐5 criteria for a substance use disorder within the past 12 months, but does not meet criteria within the past 3 months other than the symptom of craving (15). A notable change from the DSM‐IV for those who use stimulants, is the change in the DSM‐5 from segregating cocaine abuse/dependence and amphetamine and similar sympathomimetics abuse/dependence to a single stimulant use disorder diagnosis. The DSM‐5 also removed the symptom of legal problems due to use, and added the symptom of craving (15). Previous researchers have found a high level of agreement between the DSM‐IV substance dependence and DSM‐5 moderate to severe substance use disorder diagnoses, therefore minimal updates to diagnostic tools are needed (4, 16, 17, 18).

This study updates the RODS (now the Rapid Opioid Use Disorder Assessment [ROUDA]) for DSM‐5 criteria and includes fentanyl as a commonly misused opioid. Due to the increase in co‐occurring use, the Rapid Stimulant Use Disorder Assessment (RSUDA) was also developed to diagnose moderate to severe stimulant use disorder to assess for additional treatment and harm reduction needs. This study evaluated the performance of the ROUDA and RSUDA in community settings by non‐clinical interviewers.

METHODS

In this cross‐sectional study, 150 adults were recruited who had a history of opioid and/or stimulant misuse (e.g., used a stimulant or opioid to get high or not as prescribed). Enrollment eligibility included being 18 years of age or older, able to speak and read English, and able to provide informed consent. Participants were paid $20 to complete a 20‐min one‐time study interview by non‐clinical Research Assistants in various community settings, including a soup kitchen, city park, and our research office in the state of Connecticut, United States. Community partners provided a private space to speak with potential participants, they also helped identify those that may be eligible and want to participate in the study. After completing written informed consent, participants were asked to complete items regarding their demographic characteristics and substance use. Participants were then administered the new ROUDA and RSUDA and the substance use module of the Mini International Neuropsychiatric Interview (MINI) version 7 (19, 20). Data were collected between February and September 2022. Research Assistants completed the MINI training prior to conducting this project, but no additional training for the ROUDA and RSUDA. All project protocols were approved by Yale University's Institutional Review Board (IRB 000031439).

Sample Size Considerations

Study sample size estimations for these scale validation were based on a 10:1 ratio of participants to measure items (21). There are eight items in the ROUDA and RSUDA, therefore a minimum of 80 people would be needed to complete the analyses.

Statistical Analysis

Consistent with the RODS, ROUDA and RSUDA scores of 3 or greater indicated the presence of a moderate to severe opioid or stimulant use disorder (8, 9). MINI diagnostic criteria for an opioid, cocaine, or stimulant use disorder were based on scoring instructions (19, 20). Descriptive analysis, concordance, sensitive, specificity and predictive values were calculated using Stata version 17 (22).

Measures

MINI

Participants were administered the substance use disorder diagnostic module of the MINI version 7 that was created based on DSM‐5 criteria. The MINI has been validated to the SCID‐P and CIDI (23, 24) and is considered one of the gold standard diagnostic tools for substance use disorders as it can be administered, for a fee, by trained non‐clinicians typically used in research settings; it is therefore ideal for comparing with another measures administered by non‐clinical staff in non‐medical settings. It consists of 13 yes/no items based on reported substance use and provides a substance use disorder diagnosis (19, 20). It should be noted that distinct reporting windows for remission are not an option in the MINI. Instruction for determining if someone is in remission is provided in the fee‐based training. This current project included the substance use disorder module, specifically concerning opioids, cocaine, and stimulants. The majority of the participants in this study reported lifetime cocaine use (98.6%) therefore cocaine and other stimulant use were combined in the MINI for direct comparison to the RSUDA.

Rapid opioid use disorder assessment

The ROUDA consists of eight items updated from the previous version of the RODS (created by author S. Springer) (8, 9). Updates include the inclusion of fentanyl listed as a commonly misused opioid. It also updates the timepoints of each item to capture long‐term (more than 12 months) and short‐term (3 months) remission.

Rapid stimulant use disorder assessment

The DSM‐IV and DSM‐5 criteria for different substance use disorders are similar across all substances, the notable difference being the withdrawal symptoms. Therefore, all items from the ROUDA were modified to refer to stimulant use instead of opioid use to create the RSUDA. To correspond to the DSM‐5 criteria of stimulant withdrawal, Item 4 of the RSUDA, was modified from the opioid “In the morning, did you ever use opioids to keep from feeling ‘dope sick’ or did you ever feel ‘dope sick’” to stimulant “In the morning, did you ever use stimulants to keep from feeling ‘the crash’ or did you ever feel ‘the crash’” based on the authors' experience in the field. During the interview process participants noted they would describe their morning stimulant withdrawal symptoms to be better described as a depressed state than a crash. Therefore, wording in the final version of the RSUDA was updated to include “crash/depressed.”

Demographic characteristics captured

Limited self‐reported current gender identify, age, race, ethnicity, highest educational level achieved, and years of substance use were collected.

RESULTS

Participants

Project participants had a mean age of 46.3 years; 59.3% identified as male, 28.7% Hispanic, 49.3% white, and 72.0% had a high school diploma or GED equivalent (Table 1). Of the 150 people who completed the study interview, 122 (81.3%) reported using an opioid not as prescribed or to get high in their lifetime for an average of 20.1 years. When administered the ROUDA, 120 reported lifetime use (2 did not report use when asked to specify substance), 100 used in the past 12 months, and 85 used in the past 3 months (Table 2). The most reported opioids used in their lifetime were heroin (93.3%) and fentanyl (74.2%). One hundred and forty (93.3%) people reported stimulant use, for an average of 19.4 years. The most reported stimulant used was cocaine (98.6%, Table 3).

Rapid Opioid Use Disorder Assessment

Concordance analysis between the MINI and the ROUDA for current opioid use disorder (within the past 12 months) showed a Cohen's Κ level of agreement was 0.66, indicating substantial agreement. Sensitivity (82.5%; 95% confidence interval [CI]: 75.7, 89.2) was strong, and specificity (100%; 95% CI: 100, 100) was excellent, positive predictive value was also excellent (PPV = 100%; 95% CI: 100, 100) and negative predictive value (NPV = 59.5%; 95% CI: 50.8, 68.2) was moderate. Long‐term remission had a Cohen's Κ agreement of 0.33, indicating fair agreement; sensitivity of 36.8% (95% CI: 27.4, 46.3), specificity of 92.7% (95% CI: 87.6, 97.8), PPV 53.8% (95% CI: 44.1, 63.6), and NPV 86.4% (95% CI: 79.7, 93.1). Short‐term remission had a moderate Cohen's Κ agreement of 0.41, sensitivity of 42.9% (95% CI: 33.2, 52.5), specificity of 93.2% (95% CI: 88.2, 98.1), PPV 70.6% (95% CI: 61.7, 79.5), and NPV 81.0% (95% CI: 73.3, 88.6). Internal consistency for items related to lifetime opioid use disorder was good (α = 0.77), excellent internal consistency was found for the past 12 months (α = 0.94) and within the past 3 months (α = 0.94).

Rapid Stimulant Use Disorder Assessment

Concordance analysis between the RSUDA and the MINI for current stimulant use disorder had a substantial Cohen's Κ agreement of 0.66. Sensitivity and specificity were strong (83.8%; 95% CI: 77.7, 89.9 and 91.4%; 95% CI: 86.8, 96.1, respectively), PPV was also excellent (96.7%; 95% CI: 93.8, 99.7) and NPV was moderate (65.3%, 95% CI:57.4, 73.2). Long‐term remission had a no agreement with a Cohen's Κ of −0.03, sensitivity of 6.3% (95% CI: 1.7, 10.8), specificity of 91.5% (95% CI: 86.3, 96.7), PPV 11.1% (95% CI: 5.2, 17.0), and NPV 85.2% (95% CI: 78.5, 91.8). Short‐term remission had a Cohen's Κ agreement of 0.20, indicating slight agreement; sensitivity of 30.0% (95% CI: 20.3, 39.7), specificity of 87.9% (95% CI: 81.0, 94.8), PPV 42.9% (95% CI: 32.4, 53.3), and NPV 80.6% (95% CI: 72.2, 88.9). Internal consistency was strong for items related to lifetime stimulant use disorder (α = 0.86), past 12 months (α = 0.87), and within the past 3 months (α = 0.88).

DISCUSSION

This exploratory study demonstrates the validity and reliability of two new rapid diagnostic tools for DSM‐5 opioid and stimulant use disorders. The ROUDA is an updated version of the RODS (8, 9) that is currently being used to safely diagnose opioid use disorder in order to initiate people on MOUD in various settings including non‐clinical settings (10, 11, 12, 13). The RSUDA was developed to identify those with moderate to severe stimulant use disorder in order to assist in referral to behavioral treatment programs, such as contingency management which is to date the most successful form of treatment for stimulant use disorder (25, 26). Both diagnostic tools can be used in non‐clinical as well as clinical settings administered by non‐clinical staff without requiring special training or any cost (authors created tools to be free of charge).

When compared against the MINI version 7 (19, 20, 23, 24), the ROUDA and RSUDA were both able to identify those with current moderate to severe opioid and/or stimulant use disorder well. The ROUDA and RSUDA differ from the MINI in the way they capture information regarding remission. The MINI has a screening question built into the module for faster administration based on current substance use. If there was no substance use reported within the past 12 months, the items regarding each symptom are skipped. Remission is then determined by the interviewer, based on trainings, rather than responses to symptom‐based questions and as such the interviewer alone determines if the individual is in remission or not. The ROUDA and RSUDA specifically ask substance use disorder symptoms for within the individual's lifetime, past 12 months, and past 3 months. Based on the responses to these timeframes and scoring instructions, the person administering the measures can determine if the individual undergoing assessment meets criteria for long‐term or short‐term remission. Therefore, the ROUDA and RSUDA provide a structured approach for assessing whether an individual is in remission from an opioid or stimulant use disorder which cannot be assessed with the current DSM‐5 version of the MINI.

The ROUDA and RSUDA scoring are based on the number of symptoms reported by the participant, therefore the list of substances listed in question one can be updated as new opioids are used recreationally. Given this, minor adjustments were made to the final tool (see Supporting Information S1: Appendix) based on feedback from Research Assistants and participants.

Although this project has many strengths, there are some limitations that should be addressed in subsequent research. All non‐clinician Research Assistants who administered the study interviews have between 10 years and 4 months experience with persons who use substances and were trained in administering the MINI. Thus, results obtained from this study may not generalize to settings where staff have less experience with the target population. The study design was also cross‐sectional; therefore it did not allow for the assessment of test‐retest reliability. In addition, we did not randomize the order of administration of these assessments, so there may be order effects. The high rate of persons with moderate to severe opioid and/or stimulant use disorder may be reflected in the lower‐than‐expected NPV. Lastly, this study did not address the larger issue of substance use disorder treatment availability in non‐clinical settings. Identifying those who might benefit from substance use disorder treatment is only useful if effective treatments are available, which is often not the case. For example, only about one third of U.S. jails offer any form of MOUD (27).

Despite these limitations, the results of this initial validation study suggest that the ROUDA and RSUDA are viable alternatives to their more costly and time‐consuming counterparts. This finding carries important public health implications, as the removal of friction points in diagnosing opioid and stimulant use disorders—especially in non‐clinical settings—will likely improve access to appropriate treatments and save lives.

Rapid diagnosis of opioid or stimulant use disorder is critical for linking people to lifesaving treatments in the age of worsening substance use related overdose deaths. Being diagnosed with an opioid or stimulant use disorder by a non‐clinician in non‐clinical settings provides a reachable moment and provides an opportunity to have a clinician rapidly initiate treatment. These tools could be lifesaving in the hands of community outreach workers who work where people live and spend time in the community, such as parks. These diagnostic tools allow people working in the community to provide an opioid and/or stimulant use disorder diagnosis and to provide faster linkage to clinician‐provided treatment options, as well as education, and critical harm reduction services (e.g., naloxone, syringe services, drug testing, etc).

The ROUDA and RSUDA were effectively administered by non‐clinicians in community settings to identify those with moderate‐to‐severe opioid and/or stimulant use disorders. The measures can be administered quickly and capture the criteria needed for DSM‐5 diagnoses and specificities. In the context of the current worsening opioid and stimulant epidemics, the ROUDA and RSUDA show promise in their ability to quickly identify those who are eligible for life saving interventions including MOUD as well as effective behavioral treatments for stimulant use disorders. The potential of these rapid diagnostic measures is especially critical for non‐clinical settings where time and staff training are limited.