0
Original Article |

A Randomized Clinical Trial of a New Behavioral Treatment for Drug Abuse in People With Severe and Persistent Mental Illness FREE

Alan S. Bellack, PhD; Melanie E. Bennett, PhD; Jean S. Gearon, PhD; Clayton H. Brown, PhD; Ye Yang, MS
[+] Author Affiliations

Author Affiliations: University of Maryland School of Medicine and Veterans Affairs Capitol Health Care Network Mental Illness Research, Education, and Clinical Centers, Baltimore, Md.


Arch Gen Psychiatry. 2006;63(4):426-432. doi:10.1001/archpsyc.63.4.426.
Text Size: A A A
Published online

Context  Drug abuse by people with severe mental disorder is a significant public health problem for which there is no empirically validated treatment.

Objective  To evaluate the efficacy of a new behavioral treatment for drug abuse in this population: Behavioral Treatment for Substance Abuse in Severe and Persistent Mental Illness (BTSAS).

Design  Participants were randomly assigned to 6 months of treatment in either BTSAS or a manualized control condition: Supportive Treatment for Addiction Recovery (STAR).

Setting  Treatment was conducted in community-based outpatient clinics and a Veterans Affairs medical center in Baltimore, Md.

Participants  Participants were 129 stabilized outpatients meeting DSM criteria for drug dependence (cocaine, heroin, or cannabis) and serious mental illness: 39.5% met DSM-IV criteria for schizophrenia or schizoaffective disorder; 55.8%, for major affective disorders; and the remainder met criteria for severe and persistent mental illness and other Axis I disorders.

Interventions  Both treatments were administered by trained health care professionals in small groups, twice a week for 6 months. The BTSAS program is a social learning intervention that includes motivational interviewing, a urinalysis contingency, and social skills training. The control condition, STAR, is a supportive group discussion treatment.

Main Outcome Measure  The primary outcome measure was urinalysis results from twice-weekly treatment sessions.

Results  The BTSAS program was significantly more effective than STAR in percentage of clean urine test results, survival in treatment, and attendance at sessions. The BTSAS program also had significant effects on important community-functioning variables, including hospitalization; money available for living expenses; and quality of life.

Conclusions  The BTSAS program is an efficacious treatment. Further work needs to be done to increase the proportion of eligible patients who are able to become engaged in treatment.

Figures in this Article

Drug and alcohol abuse by people with severe and persistent mental illness (SPMI) is one of the most significant problems facing the public mental health system. These patients pose major problems for themselves, their families, health care professionals, and the mental health system. Lifetime prevalence of substance abuse was assessed at 48% for schizophrenia and 56% for bipolar disorder in the Epidemiologic Catchment Area Study,1 and estimates of current abuse for the SPMI population range as high as 65%.2,3 Substance use disorders in patients with SPMI begin early in the course of illness4 and have a profound impact on almost every area of patient functioning and clinical care. Patients with SPMI with substance use disorders show more severe symptoms of mental illness, more frequent hospitalizations, more frequent relapses, and a poorer course of illness than patients with a single diagnosis, as well as higher rates of violence, suicide, and homelessness.5,6 They also have higher rates of incarceration,7 greater rates of service use and cost of health care,8,9 poorer treatment adherence, and treatment outcome.10,11 Substance use also impairs information processing, which is particularly problematic for people with SPMI.12

There is an extensive literature on the treatment of patients with dual-disordered SPMI,13,14 and there is a broad consensus on a number of elements required for effective treatment, including: (1) integration of both psychiatric and substance abuse treatment1519; (2) conceptualizing treatment as an ongoing process in which motivation to reduce substance use waxes and wanes2022; and (3) a harm-reduction model is more appropriate than an abstinence model, especially during the early stages of treatment when the patient has uncertain motivation to change.23,24 Conversely, there is a dearth of empirical data on effective techniques for producing change. This literature has been surveyed in 3 recent reviews, each of which used somewhat different criteria for identifying and evaluating trials. Drake et al were generally positive about the effectiveness of available treatments but concluded that “As yet there is little evidence for any specific approach to treatment. . . . ”14(p368) Dumaine25 reported that the largest effect size in studies covered by his review was 0.35 for intensive case management without a specific psychoeducational component. The largest effect size for a specific psychosocial treatment procedure was only 0.25. In the least optimistic view in the literature, Ley et al concluded that “There is no clear evidence supporting an advantage of any type of substance misuse program for those with serious mental illness over the value of standard care. No one program is clearly superior to another.”26(p1)

In response to this dearth of empirically sound interventions, we developed a new, multifaceted treatment for substance abuse in patients with dual disorders that addresses the specific problems and needs of this population: Behavioral Treatment for Substance Abuse in SPMI (BTSAS).13,20 The 6-month, small-group treatment includes 6 integrated components: (1) motivational interviewing to increase motivation to reduce use; (2) a urinalysis contingency applied in each session to enhance motivation to change and increase the salience of goals; (3) structured goal setting to identify realistic, short-term goals for decreased substance use; (4) social skills and drug refusal skills to enable patients to refuse social pressure to use substances and to provide success experiences that can increase self-efficacy for change; (5) education about the reasons for substance use and the particular dangers of substance use for people with SPMI to shift the decisional balance toward decreased use; and (6) relapse-prevention training that focuses on behavioral skills for coping with urges and dealing with high-risk situations and lapses. This article reports the results of a randomized trial in which BTSAS was compared with Supportive Treatment for Addiction Recovery (STAR), a manualized group comparison treatment, in 126 outpatients meeting criteria for SPMI and current dependence on cocaine, heroin, or cannabis. The study was approved and monitored by the University of Maryland School of Medicine (Baltimore) institutional review board.

PARTICIPANTS

A total of 293 individuals provided informed consent, of whom 175 met DSM-IV criteria for current dependence on cocaine, heroin, or marijuana; had SPMI27; and completed baseline assessments. These 175 subjects were randomized to either BTSAS or STAR using an adaptive urn randomization procedure28 that adjusted for sex, psychiatric diagnosis, drug of choice, and number of substance use disorders. A separate randomization was conducted for participants recruited from community clinics and a Veterans Affairs medical center. This sample was predominantly male (63.4%), African American (75.4%), and had never been married (42.3%). Mean (SD) age was 42.7 (7.10) years, with 11.2 (2.28) years of education. Diagnostically, 38.3% met DSM-IV criteria for schizophrenia or schizoaffective disorder; 54.9%, for major affective disorders; and the remainder met criteria for SPMI and other Axis I disorders. The mean (SD) number of past psychiatric hospitalizations for the sample was 5.28 (7.97), and the mean (SD) age at onset of psychiatric disorder was 26.7 (10.8) years. The predominant drug of abuse was cocaine (68.6%), followed by opiates (24.6%) and cannabis (6.86%). Participants reported a mean (SD) of 5.73 (8.76) years of heroin use, 10.2 (8.21) years of cocaine use, 10.2 (10.4) years of marijuana use, and 12.1 (10.7) years of polydrug use. Participants were outpatients recruited from community clinics (58.9%) and a Veterans Affairs medical center in Baltimore. There were no differences between the groups on any psychiatric, drug preference, or demographic variables.

TREATMENTS

Both treatments followed detailed manuals (available on request) and were administered to small groups (4-6 participants) twice per week for 6 months by trained therapists. Participants were entered into ongoing groups on completion of baseline assessments to minimize the delay between recruitment and treatment initiation. This rolling admission procedure also minimized the group effect (nonindependence) on individual participant data.

Behavioral Treatment for Substance Abuse in SPMI

The BTSAS program is a highly structured social learning program that was developed by us specifically for people with SPMI. In creating the BTSAS program, we identified techniques that have been successfully used to treat substance abuse in less impaired primary substance–abusing populations (eg, contingency contracts, motivational interviewing) and modified the procedures to be applicable to people with SPMI. We also included treatment techniques to address issues that are particularly germane to drug abuse by people with SPMI. Recognizing that abstinence is a desirable goal but not one that can be easily achieved by people with SPMI, BTSAS uses a harm-reduction model in which small gains are reinforced and intermittent drug use is not punished. Given that motivation to reduce drug use waxes and wanes in this population, BTSAS attempts to increase motivation by conducting individual motivational interviewing29 sessions at baseline, 3 months, and 6 months. In addition, a urinalysis contingency is applied in each session to serve as a motivational prosthesis. Subjects receive between $1.50 and $3.50 per session, increasing in $0.50 increments for successive sessions with clean urine test results. The amount is reset to $1.50 in the session following a dirty sample or an absence. To minimize failure experiences, achievable short-term goals are established in each session, and coping skills training is provided to prevent lapses from spiraling into full-blown relapses. A major factor contributing to drug use among people with SPMI is social pressure and the desire to seem normal. Hence, a considerable amount of time is devoted to social skills training30 to teach participants how to refuse drugs, engage in alternative social activities, and develop non–drug using social contacts.

Each session adheres to the same basic structure: (1) A urine sample is secured and the results are announced to the group. Participants with negative samples receive social reinforcement from the therapists and group members and financial reinforcement. Positive samples are followed by a nonaccusatory discussion of situational factors that contributed to use and rehearsal of coping strategies to increase the likelihood of abstaining in the situation in the future. (2) Each participant is then assisted in setting realistic goals for decreased drug use until the next session and signs a goal contract. (3) The remainder of each session follows a structured curriculum for drug abuse education, skills training, and relapse prevention.

Supportive Treatment for Addiction Recovery

The STAR program is based on usual treatment at the university-run community mental health center from which many of our subjects are recruited. The groups are designed to be supportive and encouraging and to provide a safe and nonjudgmental place for participants to talk about substance use and their ideas and feelings about it. Some didactic education is provided about the effects of drugs and factors involved in reducing drug use when it fits into the discussion, but there is no formal curriculum or session-by-session plan regarding these issues. The group sets its own pace and determines its own topic, and the therapists encourage, but do not require, patient interaction. A urine sample is taken before each session, but no systematic feedback is provided.

The BTSAS and STAR programs were each administered by trained therapists (primarily masters degree–level health care professionals) who were supervised on a weekly basis throughout the project. All sessions were videotaped and fidelity of treatment administration was evaluated by blinded raters who independently rated randomly selected videotapes to assess adherence (22 yes or no items) and competence (14 items rated on 5-point Likert scales). The mean (SD) adherence ratings (proportion of adherent items) were 0.9771 (0.05) (range, 0.82-1.00) for BTSAS and 0.9722 (0.08) (range, 0.75-1.00) for STAR. The mean (SD) competence rating for BTSAS therapists (on a 5-point Likert scale) was 4.52 (0.33) (range, 3.80-5.00) and for STAR, 4.51 (0.41) (range, 3.60-5.00). The treatments were thus administered effectively and as dictated by the respective manuals.

OUTCOME MEASURES

A primary outcome measure was urinalysis results secured at each treatment session attended. Urine samples were collected from all subjects at every session beginning in session 3, providing an objective measure of drug use throughout the 6 months of the trial. The second primary outcome was time until dropping out of treatment (dropout defined as missing 8 consecutive sessions). Secondary outcomes included achievement of 4 and 8 weeks of continuous abstinence and number of treatment sessions attended. Participants also completed a number of self-report measures at baseline and posttreatment, including the Addiction Severity Index (ASI),31,32 the Substance Use Event Survey for Schizophrenia,33 and the Brief Quality of Life Scale.34

STATISTICAL ANALYSIS

Consistent with the literature,35 we differentiated between individuals who failed to engage in treatment (attended ≤2 treatment sessions) and those who engaged in treatment and either graduated or dropped out. All analyses described later include all available data for the subjects who became engaged in treatment. The 2 primary hypotheses tested were that across the 6-month treatment period participants in BTSAS vs STAR would (1) produce a higher proportion of negative (clean) urine test results and (2) survive in treatment longer. The power analysis conducted prior to initiating the study indicated that a sample of 110 would yield a power greater than 0.80 to detect a medium effect size at a conservative α level of P<.01. After multiple-test Bonferroni adjustment, statistical significance is achieved with P value <.025.

To compare the mean proportion of clean urine test results provided by the 2 treatment groups, we used a logistic mixed-effects model (SAS macro GLIMMIX; SAS Institute Inc, Cary, NC), which allows different numbers of repeated observations (negative urine test results = yes/no) across subjects. A random-intercepts model was used to account for within-subject correlation among urine samples. A standard practice in analyzing urine data in substance abuse trials is to impute missing samples as positive (dirty), based partly on the assumption that absences are associated with drug use. This approach is consistent with an intent-to-treat analysis, but it can be misleading if there is high and/or unequal attrition. We therefore imputed (positive) urine values for missed visits in 2 ways, resulting in 2 analyses: (1) intermittent unexcused absences were assumed positive test results up until dropout; (2) all unexcused absences were assumed positive test results for the duration of the trial. This second method is more heavily weighted by attendance. In each case, we used the model to test for a treatment effect and estimated the relative odds of having a clean urine test result with a 95% confidence interval. Treatment groups had open enrollment, group size was small, and group dynamics were minimized by the structured training curriculum. Consequently, the effects of group membership were minimized and we did not include group as a factor in the analysis. In secondary analyses, we used the urinalysis data further to compare the 2 groups on 3 summary abstinence outcomes: percentage achieving 4 weeks, 8 weeks, and multiple 4-week blocks of continuous abstinence. We used 1-df χ2 tests for these analyses.

To compare the treatment groups on length of time in treatment until dropping out, we used a simple Cox proportional hazards regression model. With this model, we tested for a difference in survival curves and computed the relative risk of dropout (hazard ratio). In a secondary analysis, we compared the 2 groups on number of days attended with a t test.

The ASI data was collected at baseline and posttreatment. We analyzed 2 key variables: number of drug days and number of days with drug problems. These variables were significantly zero-inflated; hence, we used a 2-part mixed-effects model specifically designed for zero-inflated distributions (SAS macro MIXCORR; SAS Institute Inc)36 and that allowed use of all available observations. This model uses a 2-tier approach, simultaneously fitting a logistic regression model for change in proportion of nonzero drug days (among all participants) and a linear regression for change in mean number of drug or problem days (only among participants with any nonzero days.) With this analysis, we tested both within-group change and between-group differential change for both proportion of nonzero days and mean number of drug or problem days.

We conducted parallel analyses as for the ASI data for all other outcomes assessed at baseline and posttreatment using generalized estimating equations37 for dichotomous outcomes and mixed-effects models for continuous and Likert-type scaled variables. These models were used so that all available data could be analyzed and to account for within-subject correlation over time.

RETENTION OF PARTICIPANTS

As shown in Figure 1, 293 subjects provided informed consent for the trial, of whom 175 completed baseline assessments, met study inclusion criteria, and were randomly assigned to BTSAS or STAR. Attrition is widely recognized as a critical problem in substance abuse treatment programs, especially for participants with dual disorders.16,35 Of the 175 participants who were randomized, 129 attended at least 1 session and 110 became engaged in treatment (BTSAS, n = 61; STAR, n = 49), 63% of those who completed baseline. Subsequent analyses are based on the engaged sample of 110. Subjects randomized to STAR were slightly more likely to become engaged in treatment than those in BTSAS (92.4% vs 80.3%; χ21 = 3.69; P = .055), but most of this difference reflects subjects who never began treatment and is therefore most likely a random effect. Comparison of the engaged BTSAS group and the engaged STAR group using χ2 and t tests found no statistical differences between the 2 on demographic characteristics, diagnosis, or primary drug of abuse (Table 1).

Place holder to copy figure label and caption
Figure 1.

Subject flow.

Graphic Jump Location
Table Graphic Jump LocationTable 1. Baseline Demographic and Clinical Characteristics of Participants Engaged in Treatment

As shown in Figure 2, BTSAS was significantly more effective than STAR in retaining subjects who became engaged in treatment (log-rank test for difference in survival curves, χ21 = 6.88; P = .009). The relative risk of dropout (hazard ratio) for BTSAS was about half that for STAR (hazard ratio, 0.51 [95% confidence interval (CI), 0.30-0.85]). Subjects in BTSAS also attended significantly more sessions: 29.0 vs 19.0 (t108 = 3.50; P<.001; effect size, 0.64 [95% CI, 0.25-1.03].) That is noteworthy in this difficult-to-treat population because patients who attend drug treatment generally do better than those who do not.38,39

Place holder to copy figure label and caption
Figure 2.

Survival in treatment. BTSAS indicates Behavioral Treatment for Substance Abuse in Severe and Persistent Mental Illness; STAR, Supportive Treatment for Addiction Recovery.

Graphic Jump Location

We further conducted a series of analyses to see if we could identify factors that discriminated subjects who became engaged in treatment from those who did not and dropouts, from subjects who completed treatment. There were no differences between those who became engaged in treatment and those who did not or between dropouts and those who completed treatment on any demographic or diagnostic factors or on primary drug of abuse. Subjects who became engaged in treatment reported fewer days of drug use on the ASI at baseline than subjects who failed to become engaged in treatment (mean [SD], 5.61 [9.63] days vs 9.57 [12.5] days; Wilcoxon z = 2.59; P<.001), but, as indicated later, the reliability of these self-report data are suspect. Moreover, subjects who completed treatment had a greater number of positive (dirty) drug screens at baseline than dropouts (mean, 79.3% vs 57.1% for dropouts; P = .06), indicating that the trial was not limited to subjects with less severe drug problems. Overall, the data do not identify predictors of treatment participation or retention.

DRUG USE OUTCOMES

Urinalysis results are presented in Table 2. Subjects in BTSAS had a significantly higher proportion of clean urine test results during the 6 months of treatment than subjects in STAR, regardless of imputation strategy: mixed-model estimates, mean, BTSAS, 0.589 vs STAR, 0.247; F1,3671 = 16.05; P<.001; odds ratio, 4.4 (95% CI, 2.1-9.0) for imputation with missing urine test results assumed positive until time of dropout and mean, BTSAS, 0.466 vs STAR, 0.142; F1,5146 = 15.38; P < .001; odds ratio, 5.3 (95% CI, 2.3-12.1) for imputation with missing urine test results assumed positive for the duration of the trial. Treatment sessions were always separated by at least 2 to 3 days (eg, Monday-Wednesday, Tuesday-Friday), and the urinalysis tests used in this trial detect metabolites of cocaine and opiates during a 2- to 3-day period. Consequently, successive sessions attended with clean urine test results provide a rough estimate of periods of continuous abstinence. We examined 4-week and 8-week periods during which subjects attended every session, and these data show a pronounced advantage for BTSAS. Significantly more subjects in BTSAS had at least one 4-week block of continuous abstinence (54.1% vs 16.3%; χ21 = 16.6; P<.001), as well as multiple 4-week blocks (44.3% vs 10.2%; χ21 = 15.3; P<.001). More BTSAS subjects also had at least one 8-week block of continuous abstinence (32.8% vs 8.16%; χ21 = 9.66; P=.002). There were a small number of subjects whose primary drug of abuse was marijuana. Given that cannabinoid metabolites can be detected for up to 28 days, the overall rate of clean samples probably underestimates the amount of time not taking drugs.

We examined 2 ASI variables: number of drug days (in the prior month) and days with drug problems. Both variables were characterized by a large number of zero scores at both baseline and follow-up assessments and were analyzed as described in the “Statistical Analysis” subsection. There was no significant effect to suggest that one treatment had a more pronounced effect than the other on either variable. However, these self-report data must be viewed cautiously because they were often in conflict with other sources of data on drug use. For example, of those participants who denied any drug use in the past 30 days on the ASI at baseline, 32.9% met DSM-IV dependence criteria on the Structured Clinical Interview for DSM-IV40 and 9.33% had positive toxicology screens, suggesting that actual drug use during the prior month was considerably higher than was reported.

OTHER OUTCOMES

We conducted post hoc, exploratory analyses on a number of ancillary clinical dimensions that are germane to outcomes and quality of life in patients with dual disorders. Given the low base rates of several of these variables (eg, inpatient admissions), we report within-group changes that might be informative, as well as between-group analyses. Participants in BTSAS had better outcomes than participants in STAR on a number of important clinical dimensions in addition to drug use. Inpatient admissions (psychiatric or substance abuse) declined from 29.5% in the 90 days prior to baseline to 6.5% in the 90 days prior to the posttreatment assessment for subjects in BTSAS (χ21 = 9.39; P = .002), compared with 20.4% and 16.2%, respectively, for STAR subjects (P=.30). Prior to treatment, 46.6% of BTSAS subjects reported having enough money for food, clothing, housing, and transportation (Brief Quality of Life Scale) compared with 69.2% at the end of treatment (χ21 = 6.99; P = .008). This could reflect reduced expenditures on drugs. There was no change for subjects in STAR (43.8% prior to treatment and 51.5% afterward). Subjects in BTSAS also reported a small but significant increase in general life satisfaction (on the Brief Quality of Life Scale) from pretreatment to posttreatment (mean [SD], 4.25 [1.65] to 4.79 [1.66]; t106 = 2.14; P = .04), and there was a significant increase in the overall quality of life (mean [SD], 3.97 [1.74] to 4.46 [1.68]; t106 = 2.38; P = .02). Subjects in STAR failed to register a significant increase on either of these variables. There was also a significant reduction in the number of BTSAS subjects who reported being arrested (31% at baseline vs 12.8% at posttreatment; χ21 = 5.55; P = .02), compared with 22.9% and 27.3%, respectively, for STAR subjects (P = .51). Finally, BTSAS subjects reported a significant increase in ability to independently perform activities of daily living on the Social Functioning Scale from baseline to posttreatment (mean [SD], 27.2 [7.43] to 31.5 [5.69]; t107 = 3.73; P<.001). There was no comparable change for STAR subjects (baseline mean [SD], 27.3 [6.92]; posttreatment mean [SD], 28.1 [5.62]; P=.51). The interaction effect for this variable was significant (t107= 1.98; P = .05).

The cost of administering the urinalysis contingency was relatively modest. Total payments to subjects were $3676.50. The mean amount per subject was $60.27, the median was $51.50, and the range was $0 to $168.50. The average payment per session was $2.29. Total cost of the urinalysis tests and associated supplies was (approximately) $19 000.

These data provide considerable support for the efficacy of BTSAS. Subjects in BTSAS achieved a mean of almost 59% clean urine test results, compared with 25% for subjects in STAR. The biweekly urinalysis data provide an indication of cocaine and opiate use during the prior 2 to 3 days. Given that BTSAS subjects attended an average of almost 56% of sessions, that reflects a considerable amount of sobriety during the 6-month trial. While few participants achieved total abstinence, every day without drug use is associated with increased physical safety, increased likelihood of medication adherence, and decreased vulnerability to relapse in this population. In that regard, the data indicate that participants in BTSAS had decreased hospitalizations and arrests, more money available for daily expenses, and improved quality of life. The costs of the program, approximately $372 per subject for the urinalysis contingency plus therapist time, are quite modest given that the benefits include a 23% reduction in inpatient admissions.

The comparator treatment, STAR, was not a diluted treatment as usual or a pseudotreatment designed to provide an easy target for an experimental intervention. The STAR program is a manualized version of a real treatment that is representative of quality care in a university-based clinic. Moreover, it was administered by trained health care professionals who videotaped sessions and received weekly supervision. Subjects were drawn from 3 different clinical settings and are representative of patients with serious psychiatric and substance abuse problems. Hence, these findings have meaningful public health significance.

The BTSAS program contains a number of different elements that each have potential clinical impact: motivational interviewing, goal setting, urinalysis contingency, social skills training, and relapse prevention. These elements were each included because they have had some reported positive effect with other populations and/or address particular problems associated with drug use by people with SPMI (eg, need to cope with social pressure). The current trial was designed as an efficacy study for the package, and it is not possible to empirically determine the relative contributions of each element. Our anecdotal observations suggest that each component contributed to overall outcomes, with differential importance for different participants. From a public health perspective, it would be desirable to evaluate the effects of the urinalysis contingency because the average cost ranged from $7 to $10 for the urinalysis and financial reinforcer. However, given the potential savings achieved from reductions in hospitalization, medical illness, unstable housing, and unemployment, this could be viewed as a modest expense.

The BTSAS program was effective in retaining participants who became engaged in treatment (attended ≥3 sessions). In contrast, 76 subjects (25.94%) who provided consent failed to complete baseline assessments, and 37.7% of those who completed baseline assessments failed to become engaged in treatment. This finding reflects a significant problem with any drug treatment. For example, of 1777 eligible persons screened by telephone for the National Institute on Drug Abuse Cocaine Collaborative Study, 937 (52.7%) attended an initial orientation session and 487 (52.7%) of those still eligible completed assessments and began the trial, 27% of the initial pool.41 In a large naturalistic study of veterans with dual diagnoses referred for outpatient treatment, 53% failed to become engaged in treatment.42 Similar results have been reported for a state hospital sample43 (42% attended their first outpatient visit), for patients discharged from a community hospital44 (35% attended their first outpatient treatment session), and for patients entering a therapeutic community45 (49% were lost to follow-up within the first 2 weeks).

Engaging patients with dual disorders in treatment is particularly difficult because most are ambivalent about the need to reduce their drug use and their desire for treatment waxes and wanes over time. Moreover, they frequently have difficult living situations, residual psychiatric symptoms, cognitive impairments, and financial constraints (eg, no money for car fare) that make it difficult for them to attend treatment. Several studies have examined the effects of structural procedures to increase engagement in treatment, such as providing mail and telephone prompts before sessions, providing an initial orientation session, scheduling sessions quickly after initial contact, and providing tangible benefits such as child care and car fare.4648 While many of these techniques should be used as part of good clinical practice, they have not proven to be sufficient to produce engagement in treatment. A more sophisticated approach, based on the Transtheoretical Model of Change,20,49 hypothesizes that patients unengaged in treatment need encouragement to move from the persuasion (precontemplation) or contemplation stage to the action stage of change. This has led to a number of trials that use versions of motivational enhancement therapy50 to foster engagement by increasing motivation to change.23,51,52 The published results for these approaches to date have been modest, at best. Moreover, our data and several published studies have failed to find a relationship between motivation to change and treatment participation,44,52,53 raising questions about the hypothesis that increasing motivation to change is critical in this population.

We are currently conducting a trial using a 2-pronged intervention to increase engagement in treatment and thereby widen the applicability of BTSAS. The approach involves (1) a time-limited case management technique referred to as a critical time intervention to help develop a relationship between the treatment team and the patient and to help him or her overcome structural obstacles to treatment and (2) a time-limited psychoeducational intervention for concerned significant others (family members, friends) that aims to enlist them as partners to help connect the patient with treatment. Preliminary data suggest this approach may be highly effective. This trial will also provide further information in regard to the trend for early termination from BTSAS that was detected in the current study. Participants in BTSAS are faded into treatment to maximize engagement. They are not required to provide a urine sample for the first 2 sessions, and they are invited to observe the skills-training and goal-setting activities for 2 sessions before participating. Consequently, there is no obvious reason why patients would elect to terminate treatment in the first week. The new trial will help to determine whether this was a chance finding or if modifications to the protocol are warranted.

Correspondence: Alan S. Bellack, PhD, Veterans Affairs Maryland Health Care System, Mental Illness Research, Education and Clinical Centers 6A, 10 N Greene St, Baltimore, MD 21201 (alan.bellack@med.va.gov).

Submitted for Publication: May 10, 2005; final revision received September 22, 2005; accepted September 29, 2005.

Funding/Support: This research was supported by grant RO1 DA 012265 from the National Institute on Drug Abuse, Baltimore, Md (Dr Bellack).

Previous Presentations: Portions of this study were presented at the annual meeting of the Association for Advancement of Behavior Therapy; November 19, 2004; New Orleans, La; and the International Congress of Schizophrenia Research, April 5, 2005; Savannah, Ga.

Regier  DAFarmer  MERae  DSLocke  BZKeith  SJJudd  LLGoodwin  FK Comorbidity of mental disorders with alcohol and other drug abuse. JAMA 1990;2642511- 2518
PubMed
Mueser  KTYarnold  PRBellack  AS Diagnostic and demographic correlates of substance abuse in schizophrenia and major affective disorder. Acta Psychiatr Scand 1992;8548- 55
PubMed
Mueser  KTBennett  MKushner  MG Epidemiology of substance use disorders among persons with chronic mental illnesses. In:Lehman  AFDixon  LBeds.Double Jeopardy: Chronic Mental Illness and Substance Use Disorders. 3 Longhorne, Pa Harwood Academic Publishers1995;9- 25
Van Mastrigt  SAddington  JAddington  D Substance misuse at presentation to an early psychosis program. Soc Psychiatry Psychiatr Epidemiol 2004;3969- 72
PubMed
Bennett  MEBarnett  B Adult psychopathology and diagnosis: dual-diagnosis. In:Hersen  MTurner  SMeds.Adult Psychopathology and Diagnosis 4th New York, NY John Wiley & Sons2003;
Dixon  L Dual diagnosis of substance use in schizophrenia: prevalence and impact on outcomes. Schizophr Res 1999;35S93- S100
PubMed
Wallace  CMullen  PEBurgess  P Criminal offending in schizophrenia over a 25-year period marked by deinstitutionalization and increasing prevalence of comorbid substance use disorders. Am J Psychiatry 2004;161716- 727
PubMed
Miles  HJohnson  SAmponsah-Afuwape  EFinch  MThornicroft  G Characteristics of subgroups of individuals with psychotic illness and a comorbid substance use disorder. Psychiatr Serv 2003;54554- 561
PubMed
Ross  HELin  ECunningham  J Mental health service use: a comparison of treated and untreated individuals with substance use disorders in Ontario. Can J Psychiatry 1999;44570- 577
PubMed
Owen  RRFischer  EPBooth  BMCuffel  BJ Medication noncompliance and substance abuse among persons with schizophrenia. Psychiatr Serv 1996;47853- 858
PubMed
Pages  KPRusso  JEWingerson  DKRies  RKRoy-Byrne  PPCowley  DS Predictors and outcome of discharge against medical advice from the psychiatric units of a general hospital. Psychiatr Serv 1998;491187- 1192
PubMed
Tracy  JIJosiassen  RCBellack  AS The neuropsychology of dual diagnosis: understanding the combined effects of schizophrenia and substance use disorders. Clin Psychol Rev 1995;1567- 97
Bellack  ASGearon  JS Substance abuse treatment for people with schizophrenia. Addict Behav 1998;23749- 766
PubMed
Drake  REMueser  KTBrunette  MFMcHugo  GJ A review of treatments for people with severe mental illnesses and co-occurring substance use disorders. Psychiatr Rehabil J 2004;27360- 374
PubMed
Carey  KB Substance use reduction in the context of outpatient psychiatric treatment: a collaborative, motivational, harm reduction approach. Community Ment Health J 1996;32291- 306
PubMed
Drake  REMueser  KT Psychosocial approaches to dual diagnosis. Schizophr Bull 2000;26105- 118
PubMed
Lehman  AFedDixon  LBed Double Jeopardy: Chronic Mental Illness and Substance Use Disorders.  Longhorne, Pa Harwood Academic Publishers1995;
Ziedonis  DMFisher  W Assessment and treatment of comorbid substance abuse in individuals with schizophrenia. Psychiatr Ann 1994;24447- 493
Mueser  KTNoordsy  DLDrake  REFox  L Integrated Treatment for Dual Disorders: A Guide to Effective Practice.  New York, NY The Guilford Press2003;
Bellack  ASDiClemente  CC Treating substance abuse among patients with schizophrenia. Psychiatr Serv 1999;5075- 80
PubMed
Minkoff  K An integrated model for the management of co-occurring psychiatric and substance disorders in managed-care systems. Dis Manage Health Outcomes 2000;5251- 257
Osher  FCKofoed  LL Treatment of patients with psychiatric and psychoactive substance abuse disorder. Hosp Community Psychiatry 1989;401025- 1030
PubMed
Carey  KBCarey  MPMaisto  SAPurnine  DM The feasibility of enhancing psychiatric outpatients' readiness to change their substance use. Psychiatr Serv 2002;53602- 608
PubMed
Ziedonis  DMWilliams  JCorrigan  PSmelson  D Management of substance abuse in schizophrenia. Psychiatr Ann 2000;3067- 75
Dumaine  ML Meta-analysis of interventions with co-occurring disorders of severe mental illness and substance abuse: implications for social work practice. Res Soc Work Pract 2003;13142- 165
Ley  AJeffery  DPMcLaren  SSiegfried  N Psychosocial treatment programs for people with both severe mental illness and substance misuse.  Oxford, England The Cochrane Library2003; (3)
Lehman  AFDixon  LBKernan  EDeForge  BRPostrado  LT A randomized trial of assertive community treatment for homeless persons with severe mental illness. Arch Gen Psychiatry 1997;541038- 1043
PubMed
Stout  RLWirtz  PWCarbonari  JPDel Boca  FK Ensuring balanced distribution of prognostic factors in treatment outcome research. J Stud Alcohol Suppl 1994;1270- 75
PubMed
Miller  WRZweben  ADiClemente  CCRychtarik  RG Motivational Enhancement Therapy Manual: A Clinical Research Guide for Therapists and Individuals With Alcohol Abuse and Dependence. Project MATCH Monograph Series, 2, SHHA Pub No. (ADM) 92-1894.  Rockville, MD National Institute on Alcohol Abuse and Alcoholism1992;
Bellack  ASMueser  KTGingerich  SAgresta  J Social Skills Training for Schizophrenia: A Step-By-Step Guide. 2nd New York, NY Guilford2004;
McLellan  ATAlterman  AICacciola  JMetzger  DO’Brien  CP A new measure of substance abuse treatment: initial studies of the Treatment Services Review. J Nerv Ment Dis 1992;180101- 110
PubMed
McLellan  ATKushner  HMetzger  DPeters  RSmith  IGrissom  GPettinati  HArgeriou  M The fifth edition of the Addiction Severity Index. J Subst Abuse Treat 1992;9199- 213
PubMed
Bennett  MEBellack  ASGearon  JS Assessing substance use and treatment utilization in patients with severe and persistent mental illness.  Paper presented at: 38th Annual Meeting of the Association for Advancement of Behavior Therapy November 19, 2004 New Orleans, La
Lehman  AF Evaluating Quality of Life for Persons with Severe Mental Illness: Assessment Toolkit.  Cambridge, Mass The Evaluation Center at Health Services Research Institute1995;
Miner  CRRosenthal  MDHellerstein  DJMuenz  LR Prediction of compliance with outpatient referral in patients with schizophrenia and psychoactive substance use disorders. Arch Gen Psychiatry 1997;54706- 712
PubMed
Tooze  JAGrunwald  GKJones  RH Analysis of repeated measures data with clumping at zero. Stat Methods Med Res 2002;11341- 355
PubMed
Diggle  PJHeagerty  PLiang  K-YZeger  SL Analysis of Longitudinal Data. 2nd Oxford, England Oxford University Press2002;
Crits-Christoph  PSiqueland  L Psychosocial treatment for drug abuse—selected review and recommendations for national health care. Arch Gen Psychiatry 1996;53749- 756
PubMed
Timko  CMoos  RH Symptom severity, amount of treatment, and 1-year outcomes among dual diagnosis patients. Adm Policy Ment Health 2002;3035- 54
PubMed
Spitzer  RLWilliams  JBWGibbon  MWilliams  JBW Structured Clinical Interview for DSM-IV (SCID).  New York Biometrics Research Department, New York State Psychiatric Institute1994;
Crits-Christoph  PSiqueland  LBlaine  JFrank  ALuborsky  LOnken  LSMuenz  LRThase  MEWeiss  RDGastfriend  DRWoody  GEBarber  JPButler  SFDaley  DSalloum  IBishop  SNajavits  LMLis  JMercer  DGriffin  MLMoras  KBeck  AT Psychosocial treatments for cocaine dependence. Arch Gen Psychiatry 1999;56493- 502
PubMed
Sekerka  RGoldsmith  RJBrandewie  LSomoza  E Treatment outcome of an outpatient treatment program for dually-diagnosed veterans: the Cincinnati Veterans Affairs Medical Center. J Psychoactive Drugs 1999;3185- 94
PubMed
Appleby  LLuchins  DJDyson  V Predischarge linkage and aftercare contact among dually-diagnosed public psychiatric patients. J Nerv Ment Dis 2001;189265- 267
PubMed
Pantalon  MVSwanson  AJ Use of the University of Rhode Island Change Assessment to measure motivational readiness to change in psychiatric and dually diagnosed individuals. Psychol Addict Behav 2003;1791- 97
PubMed
Guydish  JWerdegar  DSorensen  JL Drug abuse day treatment: a randomized clinical trial comparing day and residential treatment programs. J Consult Clin Psychol 1998;66280- 289
PubMed
Comfort  MLoverro  JKaltenbach  K A search for strategies to engage women in substance abuse treatment. Soc Work Health Care 2000;3159- 70
PubMed
Lash  SJ Increasing participation in substance abuse aftercare treatment. Am J Drug Alcohol Abuse 1998;2431- 36
PubMed
Lash  SJBlosser  SL Increasing adherence to substance abuse aftercare group therapy. J Subst Abuse Treat 1999;1655- 60
PubMed
Prochaska  JODiClemente  CCNorcross  JC In search of how people change: applications to addictive behaviors. Am Psychol 1992;471102- 1114
PubMed
Miller  WRRollnick  S Motivational Interviewing: Preparing People to Change Addictive Behaviors.  New York, NY Guilford Press1991;
Swanson  AJPantalon  MVCohen  KR Motivational interviewing and treatment adherence among psychiatric and dually diagnosed patients. J Nerv Ment Dis 1999;187630- 635
PubMed
Ziedonis  DMTrudeau  K Motivation to quit using substances among individual with schizophrenia: implications for a motivation-based treatment model. Schizophr Bull 1997;23229- 238
PubMed
Carpenter  KMMiele  GMHasin  DS Does motivation to change mediate the effect of DSM-IV substance use disorders on treatment utilization and substance use? Addict Behav 2002;27207- 225
PubMed

Figures

Place holder to copy figure label and caption
Figure 1.

Subject flow.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.

Survival in treatment. BTSAS indicates Behavioral Treatment for Substance Abuse in Severe and Persistent Mental Illness; STAR, Supportive Treatment for Addiction Recovery.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable 1. Baseline Demographic and Clinical Characteristics of Participants Engaged in Treatment

References

Regier  DAFarmer  MERae  DSLocke  BZKeith  SJJudd  LLGoodwin  FK Comorbidity of mental disorders with alcohol and other drug abuse. JAMA 1990;2642511- 2518
PubMed
Mueser  KTYarnold  PRBellack  AS Diagnostic and demographic correlates of substance abuse in schizophrenia and major affective disorder. Acta Psychiatr Scand 1992;8548- 55
PubMed
Mueser  KTBennett  MKushner  MG Epidemiology of substance use disorders among persons with chronic mental illnesses. In:Lehman  AFDixon  LBeds.Double Jeopardy: Chronic Mental Illness and Substance Use Disorders. 3 Longhorne, Pa Harwood Academic Publishers1995;9- 25
Van Mastrigt  SAddington  JAddington  D Substance misuse at presentation to an early psychosis program. Soc Psychiatry Psychiatr Epidemiol 2004;3969- 72
PubMed
Bennett  MEBarnett  B Adult psychopathology and diagnosis: dual-diagnosis. In:Hersen  MTurner  SMeds.Adult Psychopathology and Diagnosis 4th New York, NY John Wiley & Sons2003;
Dixon  L Dual diagnosis of substance use in schizophrenia: prevalence and impact on outcomes. Schizophr Res 1999;35S93- S100
PubMed
Wallace  CMullen  PEBurgess  P Criminal offending in schizophrenia over a 25-year period marked by deinstitutionalization and increasing prevalence of comorbid substance use disorders. Am J Psychiatry 2004;161716- 727
PubMed
Miles  HJohnson  SAmponsah-Afuwape  EFinch  MThornicroft  G Characteristics of subgroups of individuals with psychotic illness and a comorbid substance use disorder. Psychiatr Serv 2003;54554- 561
PubMed
Ross  HELin  ECunningham  J Mental health service use: a comparison of treated and untreated individuals with substance use disorders in Ontario. Can J Psychiatry 1999;44570- 577
PubMed
Owen  RRFischer  EPBooth  BMCuffel  BJ Medication noncompliance and substance abuse among persons with schizophrenia. Psychiatr Serv 1996;47853- 858
PubMed
Pages  KPRusso  JEWingerson  DKRies  RKRoy-Byrne  PPCowley  DS Predictors and outcome of discharge against medical advice from the psychiatric units of a general hospital. Psychiatr Serv 1998;491187- 1192
PubMed
Tracy  JIJosiassen  RCBellack  AS The neuropsychology of dual diagnosis: understanding the combined effects of schizophrenia and substance use disorders. Clin Psychol Rev 1995;1567- 97
Bellack  ASGearon  JS Substance abuse treatment for people with schizophrenia. Addict Behav 1998;23749- 766
PubMed
Drake  REMueser  KTBrunette  MFMcHugo  GJ A review of treatments for people with severe mental illnesses and co-occurring substance use disorders. Psychiatr Rehabil J 2004;27360- 374
PubMed
Carey  KB Substance use reduction in the context of outpatient psychiatric treatment: a collaborative, motivational, harm reduction approach. Community Ment Health J 1996;32291- 306
PubMed
Drake  REMueser  KT Psychosocial approaches to dual diagnosis. Schizophr Bull 2000;26105- 118
PubMed
Lehman  AFedDixon  LBed Double Jeopardy: Chronic Mental Illness and Substance Use Disorders.  Longhorne, Pa Harwood Academic Publishers1995;
Ziedonis  DMFisher  W Assessment and treatment of comorbid substance abuse in individuals with schizophrenia. Psychiatr Ann 1994;24447- 493
Mueser  KTNoordsy  DLDrake  REFox  L Integrated Treatment for Dual Disorders: A Guide to Effective Practice.  New York, NY The Guilford Press2003;
Bellack  ASDiClemente  CC Treating substance abuse among patients with schizophrenia. Psychiatr Serv 1999;5075- 80
PubMed
Minkoff  K An integrated model for the management of co-occurring psychiatric and substance disorders in managed-care systems. Dis Manage Health Outcomes 2000;5251- 257
Osher  FCKofoed  LL Treatment of patients with psychiatric and psychoactive substance abuse disorder. Hosp Community Psychiatry 1989;401025- 1030
PubMed
Carey  KBCarey  MPMaisto  SAPurnine  DM The feasibility of enhancing psychiatric outpatients' readiness to change their substance use. Psychiatr Serv 2002;53602- 608
PubMed
Ziedonis  DMWilliams  JCorrigan  PSmelson  D Management of substance abuse in schizophrenia. Psychiatr Ann 2000;3067- 75
Dumaine  ML Meta-analysis of interventions with co-occurring disorders of severe mental illness and substance abuse: implications for social work practice. Res Soc Work Pract 2003;13142- 165
Ley  AJeffery  DPMcLaren  SSiegfried  N Psychosocial treatment programs for people with both severe mental illness and substance misuse.  Oxford, England The Cochrane Library2003; (3)
Lehman  AFDixon  LBKernan  EDeForge  BRPostrado  LT A randomized trial of assertive community treatment for homeless persons with severe mental illness. Arch Gen Psychiatry 1997;541038- 1043
PubMed
Stout  RLWirtz  PWCarbonari  JPDel Boca  FK Ensuring balanced distribution of prognostic factors in treatment outcome research. J Stud Alcohol Suppl 1994;1270- 75
PubMed
Miller  WRZweben  ADiClemente  CCRychtarik  RG Motivational Enhancement Therapy Manual: A Clinical Research Guide for Therapists and Individuals With Alcohol Abuse and Dependence. Project MATCH Monograph Series, 2, SHHA Pub No. (ADM) 92-1894.  Rockville, MD National Institute on Alcohol Abuse and Alcoholism1992;
Bellack  ASMueser  KTGingerich  SAgresta  J Social Skills Training for Schizophrenia: A Step-By-Step Guide. 2nd New York, NY Guilford2004;
McLellan  ATAlterman  AICacciola  JMetzger  DO’Brien  CP A new measure of substance abuse treatment: initial studies of the Treatment Services Review. J Nerv Ment Dis 1992;180101- 110
PubMed
McLellan  ATKushner  HMetzger  DPeters  RSmith  IGrissom  GPettinati  HArgeriou  M The fifth edition of the Addiction Severity Index. J Subst Abuse Treat 1992;9199- 213
PubMed
Bennett  MEBellack  ASGearon  JS Assessing substance use and treatment utilization in patients with severe and persistent mental illness.  Paper presented at: 38th Annual Meeting of the Association for Advancement of Behavior Therapy November 19, 2004 New Orleans, La
Lehman  AF Evaluating Quality of Life for Persons with Severe Mental Illness: Assessment Toolkit.  Cambridge, Mass The Evaluation Center at Health Services Research Institute1995;
Miner  CRRosenthal  MDHellerstein  DJMuenz  LR Prediction of compliance with outpatient referral in patients with schizophrenia and psychoactive substance use disorders. Arch Gen Psychiatry 1997;54706- 712
PubMed
Tooze  JAGrunwald  GKJones  RH Analysis of repeated measures data with clumping at zero. Stat Methods Med Res 2002;11341- 355
PubMed
Diggle  PJHeagerty  PLiang  K-YZeger  SL Analysis of Longitudinal Data. 2nd Oxford, England Oxford University Press2002;
Crits-Christoph  PSiqueland  L Psychosocial treatment for drug abuse—selected review and recommendations for national health care. Arch Gen Psychiatry 1996;53749- 756
PubMed
Timko  CMoos  RH Symptom severity, amount of treatment, and 1-year outcomes among dual diagnosis patients. Adm Policy Ment Health 2002;3035- 54
PubMed
Spitzer  RLWilliams  JBWGibbon  MWilliams  JBW Structured Clinical Interview for DSM-IV (SCID).  New York Biometrics Research Department, New York State Psychiatric Institute1994;
Crits-Christoph  PSiqueland  LBlaine  JFrank  ALuborsky  LOnken  LSMuenz  LRThase  MEWeiss  RDGastfriend  DRWoody  GEBarber  JPButler  SFDaley  DSalloum  IBishop  SNajavits  LMLis  JMercer  DGriffin  MLMoras  KBeck  AT Psychosocial treatments for cocaine dependence. Arch Gen Psychiatry 1999;56493- 502
PubMed
Sekerka  RGoldsmith  RJBrandewie  LSomoza  E Treatment outcome of an outpatient treatment program for dually-diagnosed veterans: the Cincinnati Veterans Affairs Medical Center. J Psychoactive Drugs 1999;3185- 94
PubMed
Appleby  LLuchins  DJDyson  V Predischarge linkage and aftercare contact among dually-diagnosed public psychiatric patients. J Nerv Ment Dis 2001;189265- 267
PubMed
Pantalon  MVSwanson  AJ Use of the University of Rhode Island Change Assessment to measure motivational readiness to change in psychiatric and dually diagnosed individuals. Psychol Addict Behav 2003;1791- 97
PubMed
Guydish  JWerdegar  DSorensen  JL Drug abuse day treatment: a randomized clinical trial comparing day and residential treatment programs. J Consult Clin Psychol 1998;66280- 289
PubMed
Comfort  MLoverro  JKaltenbach  K A search for strategies to engage women in substance abuse treatment. Soc Work Health Care 2000;3159- 70
PubMed
Lash  SJ Increasing participation in substance abuse aftercare treatment. Am J Drug Alcohol Abuse 1998;2431- 36
PubMed
Lash  SJBlosser  SL Increasing adherence to substance abuse aftercare group therapy. J Subst Abuse Treat 1999;1655- 60
PubMed
Prochaska  JODiClemente  CCNorcross  JC In search of how people change: applications to addictive behaviors. Am Psychol 1992;471102- 1114
PubMed
Miller  WRRollnick  S Motivational Interviewing: Preparing People to Change Addictive Behaviors.  New York, NY Guilford Press1991;
Swanson  AJPantalon  MVCohen  KR Motivational interviewing and treatment adherence among psychiatric and dually diagnosed patients. J Nerv Ment Dis 1999;187630- 635
PubMed
Ziedonis  DMTrudeau  K Motivation to quit using substances among individual with schizophrenia: implications for a motivation-based treatment model. Schizophr Bull 1997;23229- 238
PubMed
Carpenter  KMMiele  GMHasin  DS Does motivation to change mediate the effect of DSM-IV substance use disorders on treatment utilization and substance use? Addict Behav 2002;27207- 225
PubMed

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics
PubMed Articles