Testing Standard and Modular Designs for Psychotherapy Treating Depression, Anxiety, and Conduct Problems in Youth:  A Randomized Effectiveness Trial FREE

Depression, anxiety, and conduct-related disorders and problems in youth are among the priority conditions identified by the World Health Organization,¹ which reports that mental health problems affect up to 20% of all youths worldwide. Fortunately, conducting randomized trials over the decades, intervention researchers have produced numerous manual-guided, evidence-based treatments (EBTs) for depression, anxiety, and conduct in youth.² Unfortunately, these treatments have not been incorporated into most everyday clinical practice.^3- 5 A common view is that the complexity and comorbidity of many clinically referred youths, whose problems and treatment needs can shift during treatment, may pose problems for EBT protocols, which are typically designed for single or homogeneous clusters of disorders, developed and tested with recruited youths who differ from patients seen in everyday clinical practice, and involve a predetermined sequence of prescribed session contents, limiting their flexibility.^3- 8 Indeed, trials testing these protocols against usual care for young patients in clinical practice have produced mixed findings, with EBTs often failing to outperform usual care.^7,9

The Modular Approach to Therapy for Children with Anxiety, Depression, or Conduct Problems (MATCH)¹⁰ addresses these concerns through treatment redesign, informed by experience in clinical practice settings.^11- 15 In MATCH, treatment procedures from EBTs for anxiety (cognitive behavioral therapy), depression (cognitive behavioral therapy), and disruptive conduct (Behavioral Parent Training) are structured as free-standing modules (eg, modules for self-calming, modifying negative cognitions, and increasing compliance with parents' instructions). The modules form a menu of options for clinicians. Decision flowcharts guide module selection and sequencing,¹⁶ with a default module sequence suggested, but with changes in the sequence specified to address treatment difficulties. For example, the anxiety flowchart includes a core“practicing” module involving graduated exposure to feared situations; if the youth's motivation is low for“practicing,” the therapist may use optional modules for“praise” and“reward,” to boost motivation and thus increase practice.

We designed a trial to test both modular and standard treatment approaches. We addressed some of the prior criticisms of EBT research by ensuring that (1) participants and study context were clinically representative, (2) there were no systematic differences in clinician competence across conditions (ie, all clinicians were randomly assigned), and (3) the sample would include the ethnic diversity that critics have found insufficient in the randomized controlled trial literature.^17- 20 Accordingly, we obtained samples from outpatient treatment programs that served the general public across a broad demographic and income range, we included only youths whose families sought treatment (ie, no recruiting or advertising), all treatment was provided by professional clinicians employed in the participating programs, and all treatment was provided in those programs (ie, not in university laboratory clinics).

The youths in our sample had all sought outpatient care and had primary disorders or referral problems involving anxiety, depression, or disruptive conduct. The practitioners were randomly assigned to 3 treatment conditions: modular (ie, MATCH), standard (ie, the use of 3 established EBTs for anxiety, depression, and conduct problems), or usual care. Measures included weekly problem assessments to measure change throughout treatment, plus diagnostic assessments before and after treatment. Analyses tested whether outcomes of usual care were improved on by the use of standard EBT manuals, MATCH, or both.

All study procedures were approved by the institutional review boards of Judge Baker Children's Center (affiliated with Harvard Medical School) and the University of Hawaii at Manoa, and all participants signed informed consent/assent documents approved by these institutional review boards.

In our trial, there were a total of 174 youths aged 7 through 13 years (mean [SD] age, 10.59 [1.76] years). Of these youths, 121 (70%) were boys; 45% were white, 32% were multiethnic, 9% were African American, 6% were Latino or Latina, 4% were Asian American or Pacific Islander, 2% were classified as“other,” and 2% did not choose to identify their ethnicity. Annual family income was less than $40 000 for 55% of the sample of youths, $40 000 to $79 000 for 28% of the sample, $80 000 to $119 000 for 12% of the sample, and $120 000 or more for 6% of the sample; 53% of the youths lived in single-parent households. Figure 1 shows the number of youths screened for eligibility, assessed at pretreatment, and allocated and treated within each treatment condition.

We sought youths with DSM-IV²¹ disorders or clinically elevated problem levels in the areas of anxiety, depression, and/or disruptive conduct. Diagnoses were obtained via the Children's Interview for Psychiatric Syndromes,^22- 23 and elevated problem levels (ie, T scores of≥65) were identified through relevant scales of the Child Behavior Checklist and the Youth Self-Report.²⁴ Youths were excluded for mental retardation (n = 1), pervasive developmental disorder or psychotic symptoms (n = 1), primary bipolar disorder (n = 2), or primary inattention or hyperactivity (n = 3). Diagnoses, scale scores from the Child Behavior Checklist and the Youth Self-Report, and youth- and parent-identified top problems were used to identify the primary disorder and clinical problem in each case. The mean (SD) number of DSM-IV disorders was 2.74 (1.52). Table 1 shows the primary disorders and all disorders for the full sample. The 3 study conditions did not differ significantly in number of disorders (P = .76).

Our study included 84 therapists who worked in 10 different outpatient clinical service organizations in Massachusetts and Hawaii, providing treatment in clinic office and school settings. Of these therapists, 80% were women; 56% were white, 23% were Asian American, 6% were African American, and 6% were Pacific Islander. The mean age was 40.6 years, and the mean number of years of clinical experience was 7.6; 40% were social workers, 24% were psychologists, and 36% were classified as“other” (eg, licensed mental health counselor). There were no significant differences across conditions on any of the therapist characteristics. Therapists used individual (not group) treatment in our study, with family members often included for parts of sessions.

Trajectories of change during treatment were tracked by blinded assessors using weekly measures selected to be sufficiently brief that youths and parents would complete them frequently, and to include (1) standardized measures reflecting widely recognized dimensions of youth psychopathology and (2) an assessment of the specific problems youths and parents identified as most important to them at the outset of treatment.

The Brief Problem Checklist (BPC),²⁵ administered by telephone, is a 12-item measure of internalizing (6 items; scores can range from 0 to 12), externalizing (6 items; score range, 0-12), and total problems (12 items; score range, 0-24), developed through application of item response theory and factor analysis to data from the Youth Self-Report and the Child Behavior Checklist (2 very widely used youth symptom measures). The reliability and validity evidence is strong, and the BPC significantly predicts change in youth symptoms during treatment. In the original BPC clinical sample²⁵ (N = 184), the mean (SD) scores were 2.79 (2.62), 2.90 (2.40), and 5.68 (4.14) for youth-reported internalizing, externalizing, and total problems and 4.41 (3.11), 5.14 (3.04), and 9.55 (4.90) for parent-reported internalizing, externalizing, and total problems, respectively.

The Top Problems Assessment (TPA),²⁶ also administered by telephone, obtains youth and parent severity ratings (on a scale of 0 to 10) of the top 3 problems the youth and parent independently identified as most important to them in separate structured pretreatment interviews. Psychometric analyses of the TPA have shown strong reliability, validity, and sensitivity to change during treatment. In the original TPA clinical sample²⁶ (N = 178), the mean (SD) youth rating was 4.96 (2.96), and the mean (SD) parent rating was 6.70 (2.33).

Blinded interviewers administered this structured interview to assess DSM-IV diagnoses. The reliability and validity of this structured interview are well documented in studies of outpatient and inpatient samples,²⁷ and 5 psychometric studies have shown mean sensitivities of 0.66 to 0.83 and mean specificities of 0.78 to 0.88.^22- 23 Combined diagnoses were generated via the Silverman-Nelles²⁸ procedure for integrating youth and parent reports, in which all diagnoses generated by both informants are accepted: all diagnoses generated by the child's report are accepted if internalizing (and thus potentially more evident to youths than adults [eg, anxiety or depressive disorders]), and all diagnoses generated by the parent's report are accepted if externalizing (eg, oppositional defiant disorder). See the eAppendix for interviewer training and diagnostic reliability. Because our study focused on youths across a broad spectrum (Table 1) and not just on a single target diagnosis (as in most randomized controlled trials), we used the total number of diagnoses as an outcome measure, to reflect that spectrum.

The Services for Children and Adolescents–Parent Interview has shown psychometric integrity, with a particular strength in medication assessment, including start and end dates, and dosage (intraclass correlation coefficient, 0.99).^29- 30 It was administered to parents before treatment and in weekly calls thereafter to assess psychotropic medication use.

We used a cluster randomization design^31- 33 with therapists assigned to a condition (usual care, standard, or modular) using blocked randomization³³ stratified by the educational level of the therapist (doctoral vs master's degree). A computerized random number generator produced an unpredictable sequence of numbers representing condition; these numbers were assigned to therapists. Block size was the entire cohort of therapists within each educational level within each site, and the allocation ratio for each block was 1:1:1. Allocation concealment was maintained through the use of therapist identification numbers. Youths and caregivers knew they were receiving treatment and that randomization was involved, but they did not know the type of treatment they received.

The initial treatment focus for the modular and standard conditions was determined by using symptom and diagnostic information plus the TPA. For example, if the Children's Interview for Psychiatric Syndromes and Child Behavior Checklist/Youth Self-Report assessments identified both depression and conduct as relevant treatment foci, then the rank ordering of client-identified problems on the TPA was used to determine whether treatment began with a focus on depression or conduct.

Clinicians randomly assigned to usual care agreed to use the treatment procedures that they used regularly and believed to be effective. Clinical supervision followed usual practices in their setting, and therapy continued until a normal end of treatment for the client.

Clinicians randomly assigned to the standard condition were trained to use 3 treatment protocols, with manualized instructions and prescribed order of treatment sessions:

Therapists in the modular condition used MATCH,¹⁰ a collection of modules designed to correspond to the treatment procedures included in Coping Cat, Primary and Secondary Control Enhancement Training, and Defiant Children protocols. With MATCH, the therapist focuses on the initial problem area identified as most important, based on the standardized measures and the patient priorities identified in the TPA. The flowchart for the problem area selected (eg, depression) specifies a default sequence of modules. If interference arises (eg, if a comorbid condition or stressor impedes use of the default sequence), the sequence is altered, with other modules used systematically to address the interference. For example, if treatment begins with a focus on depression but disruptive behavior interferes, the therapist may use modules from the conduct section of the protocol to help parents manage the disruptive behavior, returning to depression treatment when the interference is resolved.

Clinicians randomly assigned to the standard or modular treatment condition were trained together; all of these clinicians had 2 days of training on treatment for each problem area, for a total of 6 days. Subsequently, both standard- and modular-assigned clinicians received weekly consultations on study cases from project supervisors; these supervisors were informed by participating in consultant-guided discussions of measurement feedback on client progress and practice history.⁴⁰ Clinicians randomly assigned to usual care received the usual supervision procedures in their settings, with no intervention from project personnel, to ensure that usual care would not be altered.

The mean treatment duration was 275.49 days in usual care, 196.24 days in the standard treatment condition, and 210.15 days in the modular treatment condition; a fixed-effects analysis of variance showed that the groups were significantly different from each other (F_2,171 = 4.66, P = .011). The usual care condition showed a significantly longer duration than the standard (P = .011) and modular conditions (P = .04); the standard and modular conditions did not differ significantly. For total number of sessions, we have information on only the standard and modular conditions owing to the total separation of study personnel from usual care. Based on the therapist reports for this 70% of the sample, the mean (SD) number of treatment sessions was 16.17 (9.95); 34% of the sessions included only the child, and 41% of the sessions included the child plus 1 or more family members. The mean (SD) time between sessions was 11.96 days (4.65 days).

We obtained therapist reports on the contents of sessions to determine how often modular and standard cases included treatment procedures from multiple problem areas (eg, including a depression procedure in a treatment episode for conduct problems or having anxiety treatment followed by treatment for depression). Exactly half of the 62 modular cases met this criterion, whereas only 2% of the standard cases met this criterion (χ²₁ = 36.26, P < .001). Coding of a sample of 309 individual treatment sessions showed that 13% of modular treatment sessions used content from more than 1 problem area protocol, whereas no standard sessions did so (χ²₂ = 34.48, P < .001). Observational coding of recorded session content showed adherence to the condition in all 3 groups (eAppendix). In the standard condition, 93% of the contents of sessions fit the treatment elements of Coping Cat, Primary and Secondary Control Enhancement Training, and Defiant Children protocols. In the modular condition, 83% of the contents of sessions fit the MATCH protocol. In usual care, only 8% of the contents of sessions were consistent with either the standard or modular manuals. The modular condition contained more“other” (nonmanual) content than did the standard condition (means, 17% and 7%, respectively) (t₃₀₇ = −3.54, P < .001). Thus, multiple measures suggested that treatment in the modular condition entailed more use of treatment content from multiple problem domains (ie, anxiety, depression, and conduct) and more flexibility than treatment in the standard condition.

Because none of the 3 study conditions had a fixed duration and because the usual care condition had no constraints on content or duration of treatment, a comparison of groups only after treatment would have left condition confounded with treatment duration/dose. To address this concern, we focused planned analyses on the question of whether there were treatment group differences in trajectories of change across time on (1) the mean BPC overall score (youth and parent reports included in the same model, with informant [youth or parent] treated as a random effect) and (2) the mean TPA overall score. For each outcome variable, we estimated mixed effects regression models with outcome = a₀(intercept) + a₁(informant) + a₂(treatment group) + a₃(time) + a₄(condition × time) with intercept, informant, and time (log day) treated as random effects. To assess whether our cluster-randomized design was associated with substantial therapist-level variance, we evaluated 3-level models that included nesting of youths within therapists. The estimated therapist variances were all near zero, and comparisons of model fits between 2-level (no therapist effect) and 3-level models were not statistically significant. We also found no significant level-3 effects for models with organization (the 10 outpatient programs) included as a third level of nesting. Tests of the condition × time interaction were virtually identical with and without nesting of youths within therapists or organizations.

Based on the current data set, we determined the effect size that would be required at 1 year to achieve 80% power assuming a sample size of 58 subjects per group (174 total), with time measured in log days, a type I error rate of 5%, and use of a 2-sided test. This effect size estimate was then translated back to the original score metric (ie, difference in the original scale score units). Based on these assumptions, 80% power is achieved at 1 year for an effect size of 0.56 SD units for the BPC total score, 0.57 SD units for the BPC internalizing score, 0.58 SD units for the BPC externalizing score, and 0.51 SD units for the TPA score.

For these mixed effects regression analyses, we focused on the BPC total score and the TPA score, but we also examined the BPC internalizing and externalizing scores, the 2 components of the BPC total score. This provided the most complete look at the constructs, combining across both informants. To explore whether we should also report parent- and youth-reported measures separately, we fitted a model that included informant × time, informant × treatment, and informant × treatment × time interactions for these outcome measures; informant × treatment × time interactions were significant (all had P <. 05), so we also reported parent- and youth-reported measures separately. To reduce the risk of chance findings, we began with omnibus tests comparing the 3 treatment groups on overall BPC total score and on TPA score, applying a familywise Bonferroni adjustment to correct for the 2 tests. Omnibus effects that were significant after a Bonferroni adjustment were followed up by conducting each possible 2-group comparison among the standard, modular, and usual care groups, again applying a familywise Bonferroni correction. For 2-group comparisons that survived the Bonferroni correction, we proceeded to significance tests on the individual variables, including combined and separate youth- and parent-reported measures.

A second set of planned analyses involved comparison of the treatment groups on number of posttreatment diagnoses,^22- 23 controlling for pretreatment diagnoses. This included a test of the overall treatment group difference using a fixed-effects analysis of covariance model applied to all youths for whom we had both pretreatment and posttreatment data, with analyses using type III sum of squares and controlling for number of pretreatment diagnoses. Two-group comparisons were to be conducted if the overall treatment group effect was significant.

Statistical power was calculated for direct comparisons of treatment groups on the number of posttreatment Children's Interview for Psychiatric Syndromes–based diagnoses, controlling for pretreatment diagnoses. This analysis assumed a sample size of 58 per group and a type I error rate of .05. We found power of 0.80 to detect an effect size of f = 0.26 (corresponding to d = 0.52), a medium effect. In none of the planned analyses was power adequate for more fine-grained analyses, such as tests of moderation by sex or age.

Analyses of baseline scores on the BPC internalizing, externalizing, and total problem scores, on the TPA score, and on number of Children's Interview for Psychiatric Syndromes–based diagnoses showed no significant treatment group differences. Analyses of Services for Children and Adolescents–Parent Interview data showed that, prior to treatment, 25% of study youths were taking some psychotropic medication, and there was no significant treatment group difference. During treatment, 27% of youths used some psychotropic(s) for at least 1 day. To determine whether effects of treatment condition were moderated by medication effects, a binary variable was added to the BPC and TPA analyses, and medication use was controlled in the diagnostic analyses. The significant findings that we report later, involving treatment group × time interactions on the BPC and TPA, and involving the significant difference between modular and usual care in posttreatment diagnoses, all remained statistically significant after adjusting for medication use.

For the mixed effects regression analyses, our planned analyses (eAppendix) involved examining the overall scores of parents plus youths combined, with informant included as a random effect, and we focused on the BPC total and TPA scores; these scores are shown in Table 2, together with the component BPC internalizing and externalizing scores. As noted previously, we also include parent- and youth-reported measures separately in Table 2. Omnibus tests comparing the 3 groups on the overall BPC total and TPA scores were significant, even after Bonferroni adjustment, so we tested each possible 2-group comparison among the standard, modular, and usual care groups, again with a focus on overall BPC total and TPA scores. These tests were not significant for standard vs usual care following the Bonferroni adjustment, so no further tests within the standard vs usual care columns are considered significant (although all the comparisons are shown in Table 2, for full presentation of study data). In subsequent tests, comparing modular with usual care and modular with standard, for any overall score that was significant, we tested the corresponding youth- and parent-reported measures separately for significance.

In the modular vs usual care columns in Table 2, the group comparisons on the overall BPC total and TPA scores were significant following the Bonferroni adjustment, so we examined these effects further via tests on the internalizing and externalizing subscales of the BPC, and on the parent and youth reports on all measures. The findings showed significantly steeper trajectories of improvement in the modular treatment condition than in the usual care treatment condition on the BPC total overall score and the parent report, on the TPA overall score and the parent report, on the BPC internalizing overall score (the BPC internalizing parent report was marginal), and on the BPC externalizing overall score and the parent report.

In the modular vs standard columns, the group comparisons on the overall BPC total and TPA scores were significant following the Bonferroni adjustment, so we examined these effects further via tests on the internalizing and externalizing subscales of the BPC, and on the parent and youth reports on all measures. The findings showed significantly steeper trajectories of improvement in the modular treatment than in the standard treatment on the BPC total overall score and the youth and parent reports, on the TPA overall score and the youth report, on the BPC internalizing overall score and the youth report, and on the BPC externalizing overall score and the youth and parent reports.

In general, modular treatment outperformed usual care treatment on overall (P = .004 and .011 and effect sizes of 0.59 and 0.54, for BPC total and TPA scores) and parent-reported measures (P = .003 and .001 and effect sizes of 0.62 and 0.72, for BPC total and TPA scores), and modular treatment outperformed standard treatment on overall (P = .001 and .012 and effect sizes of 0.71 and 0.61, for BPC total and TPA scores) and youth-reported measures (P = .014 and .009 and effect sizes of 0.50 and 0.53, for BPC total and TPA scores) as well as on 2 parent-reported measures. In all comparisons of the modular condition with usual care and with standard treatment, the direction of the effects consistently indicated more rapid improvement in the modular group. In the modular vs usual care comparisons, 7 of the 12 comparisons were statistically significant (one was marginal). In the modular vs standard comparisons, 10 of the 12 comparisons were significant.

Effect sizes for log-linear rates of change are displayed in Table 2. These effect sizes are the ratio of the difference in estimated time trends divided by the square root of the estimated random time effect variance. Statistically significant condition × time interactions were associated with effect sizes ranging from 0.41 to 0.72. Table 3 shows slopes and 1-year change estimates for each of the 3 treatment groups.

Before treatment, there was no significant overall condition difference in number of diagnoses (F_2,146 = 0.541, P = .58). After treatment, however, the overall condition difference was significant (F_2,145 = 3.49, P = .03). After treatment, the youths in the modular treatment group had significantly fewer diagnoses than did the youths in the usual care group (mean [SD], 1.23 [1.01] vs 1.86 [1.52] diagnoses) (F_1,96 = 6.83, P = .01) (Figure 2). No significant differences were found between youths in the standard treatment group (mean [SD], 1.54 [1.30] diagnoses) and youths in the usual care group (F_1,90 = 2.023, P = .16) or between youths in the standard treatment group and youths in the modular treatment group (F_1,103 = 1.232, P = .27).

The findings support the effectiveness of a modular approach to the treatment of youth, an approach designed to address (1) the needs of clinicians who carry diagnostically diverse caseloads and (2) the comorbidity and flux that are common among youths referred for mental health treatment. In our analyses of change trajectories measured via weekly assessments, and with an initial Bonferroni correction applied, youths in modular treatment showed significantly faster improvement than youths in usual care, on overall and parent-reported BPC total and TPA Problems measures, and modular treatment also outperformed standard treatment, on overall and youth-reported BPC total and TPA measures as well as on the parent-reported BPC total score. By contrast, with the same analytic procedures applied, outcomes in the standard manual condition did not differ significantly from outcomes in usual care.

Our analyses of diagnostic outcomes showed a similar pattern. Youths receiving modular treatment showed significantly fewer posttreatment diagnoses than youths receiving usual care (after controlling for pretreatment diagnoses). By contrast, there was no significant difference between the standard condition and usual care on number of disorders after treatment. Interestingly, we found superior outcomes of modular treatment relative to usual care despite the fact that youths in usual care were in treatment a mean 75 days longer than youths in modular treatment (P < .008).

The findings suggested that the modular design allowed a balanced flexibility: modular sessions included much more evidence-based content than did usual care sessions (83% vs 8%) and also contained more“other” content than standard sessions (17% vs 7%). The results may reflect the greater flexibility of modular treatment, which may have facilitated the coverage of more problems compared with standard treatment. Indeed, this was a key goal in designing the modular approach (ie, to enhance the potency of standard evidence-based practices through a modular arrangement that supports the flexible application of those practices).

The study limitations included the constraints on the level of analysis imposed by sample size. Although our sample provided adequate power to test the primary questions of the study, the power was not adequate for testing the potential moderating effects of such variables as sex, age, and ethnicity, each of which would have been of interest. In addition, our emphasis on trajectories of change across the weekly assessments (the BPC and the TPA) as primary outcome measures of the study required that we exclude some youths who had been randomly assigned but whose lack of participation in treatment made it impossible to calculate a trajectory. Finally, our interest in clinical representativeness led us to include only those who sought treatment on their own, to include a broad array of diagnoses (Table 1), and to include substantial comorbidity; one effect is that the population to whom our findings apply is not so precisely defined as in an efficacy trial focused on a single disorder.

If the findings of our study are replicated in future work, the implications for the use of evidence-based practice within clinical care settings could be significant. The modular approach might also fit well into pediatric primary care, the initial point of entry for many youths referred for anxiety, depression, and disruptive conduct. Our measurement model, too, may have potential value for multiple kinds of intervention (in mental health and other domains); for example, it may be wise to learn the priorities of patients and their families and to focus on these when developing and adjusting treatment plans. For youth mental health in particular, the findings suggest that intervention procedures developed and tested over the decades in randomized controlled trials do have value for clinical practice but that a systematic restructuring of those procedures may enhance their benefits for clinically referred youths who are treated by practitioners in everyday treatment settings.