A Meta-analysis of the Efficacy of Second-Generation Antipsychotics

ONE OF THE most important clinical decisions is which antipsychotic agent to prescribe. Consensus panel recommendations differ markedly. Geddes and his collaborators (2000) in the UK National Schizophrenia Guideline Development Group conducted a meta-analysis and concluded^{1 (p1371)}: "There is no clear evidence that atypical antipsychotics are more effective or are better tolerated than conventional antipsychotics." Other researchers^{2 - 4} share their view. In contrast, some algorithms recommend second-generation antipsychotics (FGAs) as first-line treatment based on adverse effect advantages but equivocal efficacy differences.^{5 - 8} The American Psychiatric Association treatment guidelines,⁹ the Schizophrenia Patient Outcomes Research Team funded by the Agency for Healthcare Research and Quality (formerly the Agency for Health Care Policy and Research),¹⁰ the US National Institute of Mental Health,¹¹ and others^{12 - 13} have issued equivocating guidelines recommending all antipsychotics and leaving the clinician without an effective guide.

Our primary aim was to perform a meta-analysis of randomized trials on the efficacy of second-generation antipsychotics (SGAs) vs FGAs and trials comparing SGAs. Our meta-analysis, which included more drugs and more recent trials, reviewed 142 controlled studies (124 studies of SGAs vs FGAs [18 272 patients] and 18 studies of SGAs [2748 patients]), about 4 times the number of studies included in previous meta-analyses. To carry out the primary analysis correctly, it was necessary to analyze the SGA efficacy dose-response curve, as some meta-analyses used suboptimal doses in their estimates,^{14 - 15} and to perform a meta-analysis of 24 comparisons of FGAs in which patients were randomized to receive a medium dose vs a high dose. Geddes et al¹ assert that overly high doses of a comparator are less efficacious than medium doses, and, therefore, the observed better efficacy of some SGAs might be an artifact of a negative effect on efficacy of the overly high dose of FGA comparator. We conducted meta-regressions to explore the effect of comparator dose on our data, on data from the 30 studies reviewed by Geddes et al¹ (our analysis of the raw data of Geddes et al), and on the comparable pooled Cochrane dataset of 33 trials (6464 patients) of SGA vs FGA efficacy.^{15 - 19}

Additional methods, search strategies, tables, figures, discussions, citations, and sensitivity analyses can be accessed on our Web site (herein referred to as "Web") (http://www.psych.uic.edu/faculty/davis/risperidone/index.htm) (University of Illinois at Chicago, Department of Psychiatry, 2003) or by requesting a copy from the authors. We plan to update our meta-analysis on the Web quarterly.

We selected random-assignment, controlled clinical trials of patients with schizophrenia or schizoaffective disorder with no restriction on publication date, language, or sample size of 10 SGAs (amisulpride, aripiprazole, clozapine, olanzapine, quetiapine fumarate, remoxipride hydrochloride, risperidone, sertindole, ziprasidone hydrochloride, and zotepine) compared with either FGAs or another SGA, and a dose-response comparison of FGAs and SGAs. Previous research²⁰ has established that all FGAs are equally efficacious. We analyzed the medical literature in its original language. We performed a sensitivity analysis and generated funnel plots to assess the possibility of publication bias.

Modeled after the search strategy of Cochrane reviews, we searched the following databases: MEDLINE (January 1, 1966, to May 31, 2002), International Pharmaceutical Abstracts (January 1, 1970, to March 31, 2002), CINAHL (January 1, 1982, to April 30, 2002), PsychINFO (January 1, 1987, to January 31, 2002). We also searched the Cochrane Database of Systematic Reviews (Issue 2, 2002) and reference lists in journal articles. The Quality of Reporting of Meta-Analyses statement²¹ and the empiric study by McAuley and coworkers²² indicate that exclusion of unpublished studies produces a systematic positive bias, so we included data from the US Food and Drug Administration (FDA) Web site, data obtained through the Freedom of Information Act, poster presentations, and unpublished data from Cochrane reviews or other meta-analyses, conference abstracts, and manuscripts submitted for publication. We queried investigators to locate additional studies, and we contacted manufacturers to obtain company monographs.

Effect sizes were calculated from the Positive and Negative Syndrome Scale (PANSS)²³ or, when that was not available, from the Brief Psychiatric Rating Scale (BPRS).²⁴ When neither the PANSS nor the BPRS was available, the Clinical Global Rating was used, using change scores adjusted for baseline (analysis of covariance) or, when not available, change scores (baseline minus end point score) and, when both were unavailable, end point scores. Effect size is essentially the improvement score of SGA minus FGA divided by their pooled standard deviation. Normal quantile plots were generated to ensure that the outcome variable was reasonably normally distributed (Web, Figure 11).

We based our meta-analysis, as far as possible, on the intent-to-treat sample using the last-observation-carried-forward method. The mean, sample size, and standard deviation data of all studies were extracted by one of us (J.M.D.); one of us (I.D.G. or N.C.) performed independent data extractions.

We conducted extensive sensitivity analyses to determine whether results were altered by excluding certain studies or by meta-regression. We explored the effects of study design, report completeness (qualitatively), peer-reviewed publication vs non–peer-reviewed publication (including data from posters and the FDA Web site), quality of study, global rating vs PANSS/BPRS continuous scales, and exclusion of certain drugs (Web, "Sensitivity Analysis"). To evaluate data extraction, we compared our effect sizes with those we calculated from the sample size, mean, and standard deviation from the Cochrane reviews^{15 - 19} and Geddes et al¹ effect sizes.

Five Hedges-Olkin–based²⁵ software programs were used: Cochrane MetaView (version 4.1.1),²⁶ 2 SAS-based programs (version 8.2),^{27 - 28} MetaWin (version 2.0),²⁹ Comprehensive Meta-Analysis (version 1.0.25; Biostat, Englewood, NJ), and a DOS program to establish consistency across different meta-analytic techniques. We used fixed-effects models except when significant heterogeneity dictated the use of random-effects models.(Significant heterogeneity implies that effect sizes between the studies differ more than expected by chance.) Because our conclusions differed from those of Geddes et al,¹ we evaluated whether this was due to meta-analytic methods or interpretation (effect of comparator dose). Consequently, to hold dose constant, we constructed dose-response curves from fixed-dose, random-assignment, double-blind studies of SGAs (and FGAs using haloperidol equivalents) to identify the therapeutic dose range by inspection(Figure 1, point B).^{30 - 33} In the randomized, multiple fixed-dose studies, we pooled all doses greater than approximately 60% of the therapeutic dose, that is, medium olanzapine doses of approximately 11 mg or greater (Figure 1, point A), risperidone doses of 4 mg or greater, quetiapine doses of 150 mg (the most efficacious dose) or greater, and sertindole doses of 12 mg or greater. Risperidone at 2 mg was about 50% less effective than the pooled 6- to 16-mg dose and 60% less than the 6-mg dose.^{34 - 36} Low olanzapine doses (about 6 mg) constituted approximately 33% of the optimal dose (Figure 1; Web, "Dose-response Analyses and the Pooling of Doses in Fixed-Dose Studies"). Similarly, the meta-analysis by Geddes et al¹ includes data from the therapeutic dose (the dose used in practice) only, and our dose determination corresponded exactly to theirs. Geddes et al¹ argued that higher comparator doses produce less efficacy, and consequently we tested this with a meta-analysis of a high dose vs medium dose of randomized, double-blind, fixed-dose FGA studies (Figure 1, points C and B, respectively). If true, the higher dose should be less efficacious. We also analyzed randomized, double-blind clozapine dose-response and plasma level studies.

The haloperidol dose, chlorpromazine hydrochloride dose, and other comparator doses (converted to haloperidol equivalents)³⁷ were investigated as continuous and dichotomous variables (based on the haloperidol cutoff point of ≤12 vs >12 mg/d of Geddes et al¹ ) by using MetaWin and Comprehensive Meta-Analysis across all drugs and then for each drug individually (Web, Table 6 and Table 7). Second, meta-analysis based on a 2-factor analysis of variance was conducted to analyze the effect of the dichotomized haloperidol (or all drug) dose for 3 drug groups (Table 1) using the method of Wang and Bushman.²⁸

The effect sizes (95% confidence intervals [CIs]) from our meta-analysis of clozapine,^{38 - 66} amisulpride,^{67 - 76} risperidone,^{17 ,35 ,66 ,77 - 96} and olanzapine^{15 ,66 ,91 - 92 ,97 - 106} were 0.49 (0.32-0.67), 0.29 (0.16-0.41), 0.25 (0.18-0.33), and 0.21 (0.14-0.28), respectively, and each was highly statistically significant—the best evidence of difference (P = 10⁻⁷− 10⁻¹²) (Figure 2, Table 2, and Web, Tables 1-3). Clozapine produced a better response than FGAs with effect size d = 0.49, whereas amisulpride, risperidone, and olanzapine clustered around 0.25 effect size units (corresponding to 4-6 PANSS points or 3-4 BPRS points). For perspective, based on 7 studies performed contemporaneously with recently released SGAs, the mean haloperidol–placebo effect size was 0.60 (95% CI, 0.44-0.76) (corresponding to 12.6 PANSS points or 7.8 BPRS points; Web, Figure 4).^{30 ,35 ,81 ,107 - 111} Thus, the effects of amisulpride, risperidone, and olanzapine vs FGAs are somewhat less than half the effect size of FGAs over placebo or clozapine over FGAs. The large risperidone and olanzapine studies found consistent differences vs FGAs. The outliers were small exploratory studies. Examination of funnel plots for publication bias showed no gross asymmetry, except for those of clozapine and risperidone, which indicate that smaller studies reported better efficacy (greater effect sizes) for SGAs (Web, Figure 10). Sensitivity analyses omitting open-label randomized or non–peer-reviewed studies, including studies with low-dose conditions, and meta-regression with study quality, or PANSS/BPRS vs global rating, study duration, use of global vs continuous measures, etc, as moderator variables showed essentially identical results (Web, "Sensitivity Analysis").

We rejected the assertion of Geddes et al¹ that the SGAs were equally efficacious as a homogeneous group because the amount of variance attributable to the different SGAs was large (Q₉ =58.8; P = 10 ⁻⁸ random-effects model). Aripiprazole,^{112 - 114} quetiapine,^{108 ,115 - 118} remoxipride,^{119 - 135} sertindole,^{109 - 111 ,136 - 138} and ziprasidone^{107 ,139 - 142} show similar efficacy to FGAs in the sense that the improvement scores produced by these SGAs were not statistically significantly better than those of FGAs(Table 2). Failure to find a statistically significant difference does not prove that these drugs are equal to FGAs because there is a possibility that further studies could demonstrate this. We place substantial weight on the ziprasidone data from the FDA.¹⁰⁷ Because data for 3 of 4 ziprasidone studies (1341 patients) and all 3 aripiprazole studies (560 patients) were poster data, although sufficient data exist to warrant inclusion, definitive judgment regarding differential efficacy must await publication of poster or FDA data. The 12 studies comparing zotepine with FGAs showed no clear evidence of superiority. There is some variability between studies (with most studies clustering at an efficacy similar to that of FGA, with 2 outliers); thus, conclusions are limited.^{143 - 152} Zotepine's effect size of 0.15 computed using MetaWin (95% CI, −0.01 to 0.30) and MetaView (95% CI, −0.02 to 0.45) just missed significance, whereas the effect size computed using Comprehensive Meta-Analysis was significant (95% CI, 0.01 to 0.28; P = .03; Web, Tables 1-3). Most of the studies were short-term studies, but data were available on a few long-term studies (n = 16), suggesting that long-term studies produce the same differential efficacy (Web, "Reviews of Amisulpride, Risperidone, and Zotepine").

We examined double-blind trials with patients randomly assigned to medium/high or very high doses of FGAs in a reanalysis of the data of Bollini et al¹⁵³ and also of 24 trials (Web, "Dose-response Analyses and the Pooling of Doses in Fixed-Dose Studies"). Neither the analysis by Bollini et al¹⁵³ nor our analysis of the average efficacy between the high/very high dose FGA and the medium/high doses was statistically significant. Indeed, the trend was opposite to that postulated by Geddes et al.¹

One clozapine dose-response study¹⁵⁴ found that 600 mg/d was somewhat superior to 300 mg/d, which in turn was superior to 100 mg/d in a small sample study, and some patients clinically needed 900 mg/d. Plasma level studies of 400 mg of clozapine (and one with a high clozapine dose)^{155 - 157} showed that patients with higher clozapine plasma levels had an excellent response, whereas those with lower clozapine plasma levels had a poor response, suggesting that many patients require doses greater than 400 mg. When the dose of the poor responders was increased, most patients' responses increased. The doses of clozapine used in risperidone or olanzapine comparisons were generally 400 mg or less (sometimes much less).^{158 - 164}

Meta-analyses of olanzapine vs clozapine^{66 ,158 - 159} and risperidone vs clozapine^{66 ,160 - 163 ,165 - 166} showed no significant differences (Table 3). Meta-regression of risperidone vs clozapine showed that clozapine dose was a statistically significant moderator variable (P =.007). Clozapine tended to be more efficacious than risperidone in studies that used a higher dose of clozapine (Web, Figure 1). Our clozapine dose-response study and plasma level studies suggest that overly low clozapine doses were used in most comparisons of SGAs. Consequently, our meta-analysis does not exclude the possibility that adequate doses of clozapine could be superior to other SGAs.

Six olanzapine vs risperidone studies^{92 ,167 - 171} yielded a nonsignificant effect size (effect size d = 0.10; 95% CI, −0.06 to 0.26) (Table 3). Two studies^{172 - 173} showed amisulpride to be similar to risperidone (effect size d = −0.10; 95% CI, −1.27 to 1.07), and single studies of clozapine vs zotepine,¹⁷⁴ olanzapine vs amisulpride,¹⁷⁵ olanzapine vs ziprasidone,¹⁷⁶ remoxipride vs clozapine,¹²⁵ and risperidone vs aripiprazole¹⁷⁷ did not display significant differences (Web, Figure 3).

The correlations between the effect sizes of the Cochrane reviews^{15 - 19} and that of Geddes et al¹ (r = 0.95) and between each of these and our effect sizes (r = 0.92 and 0.93, respectively) show a high level of agreement in data extraction (Web, "Reliability of Data Extraction").

We performed meta-analyses using data from the Cochrane reviews^{15 - 19} and Geddes et al¹ (our data synthesis of their effect size data). Table 2 gives the results of our meta-analysis of all 3 datasets. Our calculation of the overall effect sizes was similar to those of the Cochrane reviews and that of Geddes et al¹ using 5 different software programs (Web, Tables 1-5). Differences in conclusions are not a result of different statistical methods of data synthesis per se, as our results were virtually identical. Geddes et al¹ found that the same 4 SGAs (amisulpride, clozapine, olanzapine, and risperidone) were more efficacious than FGAs. Our P values and CIs are smaller owing to a much larger total sample size.

Geddes et al¹ arrived at the opposite conclusion by meta-regression; they suggested that this efficacy difference was caused by an overly high dose of the comparator haloperidol, which reduced its efficacy. Using their data, we replicated the results of Geddes et al¹ using our meta-analysis programs (Table 4). Our results show a small effect of comparator (P = .02), but the test for heterogeneity was highly significant (P<.001) (Table 5). Our meta-regression of the discontinuous haloperidol dose did not find a significant effect of haloperidol dose even with the data of Geddes et al¹ (Q₁ = 2.34; P = .13; Web, Table 6).

To explain the results of Geddes et al,¹ note that clozapine is used in treatment-resistant patients for whom a high dose of haloperidol comparator was often used. Seven of 9 studies of their 2 most effective SGAs (clozapine and amisulpride) used high haloperidol doses, whereas only 1 of 5 studies of quetiapine and sertindole ("similarly" effective SGAs) used high haloperidol doses (Figure 3B). We believe that the superiority of clozapine and some of the other SGAs are an important finding and that the effect of dose of comparator is an artifact because most studies with high comparator doses were clozapine or amisulpride studies. This is a "Which came first, the chicken or the egg?" problem. Geddes et al¹ suggest that the effect of haloperidol dose explains the better effect of clozapine and some SGAs.

In deciding between 2 alternatives, we first tested the effect of haloperidol dose on efficacy for each SGA considered separately. Dose of haloperidol comparator, as a continuous (Table 5) or dichotomous (Web, Tables 6 and 7) variable, did not reliably affect differential efficacy of any SGA using data from Geddes et al,¹ Cochrane, or us. We also examined all FGA comparators converted to haloperidol-equivalent doses (Web, Table 9). All P values were nonsignificant(P>.05). If the identity of the drug is held constant, effect of comparator dose disappears, a finding consistent with our interpretation. The overall effect of continuous dose of haloperidol comparator was not significant using Cochrane data (coefficient for dose effect = 0.005; P = .65) or our data (coefficient for dose effect = 0.003; P = .59). The effect of the dichotomous haloperidol dose also did not significantly affect efficacy (Figure 3 and Figure 4): present study—Q₁ =2.5; P = .11, Cochrane—Q₁ = 0.63; P = .43, Geddes et al¹ —Q₁ = 2.3; P = .13 (Q evaluated the significance of the categorical dose of comparator). We believe that the finding of Geddes et al¹ may be an artifact stemming from the fact that the more effective SGAs used higher doses of haloperidol comparator and the less effective SGAs used lower doses.

As a second test, analysis of variance models with 2 categorical factors simultaneously tested the effect of high vs low haloperidol dose for 3 groups of drugs: (1) clozapine; (2) amisulpride, risperidone, and olanzapine; and(3) sertindole, quetiapine, aripiprazole, zotepine, remoxipride, and ziprasidone. The haloperidol comparator dose did not have a significant effect on differential efficacy: our data—Q₁ = 0.28; P =.60, Cochrane data—Q₁ = 0.02; P =.89, Geddes et al¹ data—Q₁ =0.06; P = .81, and all FGAs converted to haloperidol-equivalent doses for our data—Q₁ = 3.4; P =.07 (Table 1). When drug group and comparator dose group are both included in the model, drug is significant and dose of comparator is not, even in the data from Geddes et al.¹ Figure 3 depicts the effect sizes for the 3 groups of drugs by high and low haloperidol dose for data from Geddes et al¹ and our data. Figure 4 shows the same for FGAs converted to haloperidol-equivalent doses. The effect sizes are not very different for trials using 12 mg or less of haloperidol vs those using greater than 12 mg of haloperidol or haloperidol equivalents of all SGAs. We replicated our finding in sensitivity analyses when we used a different meta-regression model or when we omitted studies, that is, 3 single-blind studies, various drugs, non–peer-reviewed studies, etc. Sensitivity analyses using meta-regression with outcome variable (Clinical Global Impressions or PANSS/BPRS) and study quality as the moderator variable also showed no difference (Web, "2-Way Meta-Regression" and "Sensitivity Analysis").

We found a robust correlation (approximately 0.93) among the effect sizes found by Cochrane, Geddes et al,¹ and us. The agreement on data extraction and the statistical methods (for each drug separately) supports the validity of meta-analysis and is itself an important finding. It is easier to "spin" a narrative review, which can quote select articles to support a position. The Cochrane reviews are particularly thorough, with many methodological safeguards, including evaluation of indirect measures of efficacy, such as dropouts due to failure to respond. Since our present meta-analysis focuses on overall differential efficacy, it supplements but does not substitute for the rigor of Cochrane reviews, such as, the classic clozapine meta-analysis.¹⁷⁸ One qualification is that almost all studies have been sponsored by the pharmaceutical industry. It is possible that bias from this source (or others) could be present despite randomized double-blind methods (Web, "Potential Sources of Bias in Meta-analysis"). Consequently, trials independent of the pharmaceutical industry are needed (ie, the National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness [CATIE] project).¹⁷⁹

Some SGAs (clozapine, amisulpride, risperidone, and olanzapine) are significantly more efficacious than FGAs, whereas others are not proven to be so. Some SGAs produce a better functional recovery than FGAs and are cost-effective because reduction of other costs (hospitalization, etc) offsets these much greater medication costs.^{180 - 190} If efficacy differences are a "myth," it is a myth that reduces costs. Because there are qualitative and quantitative adverse effect and efficacy differences among SGAs, we believe that most guidelines that group SGAs as a homogeneous class are imprecise. Some researchers suggest that the property of blocking serotonin receptors, characteristic of most SGAs, accounts for the improved efficacy. However, many SGAs (ziprazodone, quetiapine, sertindole, etc) seem to have about the same efficacy as FGAs despite being potent serotonin receptor blockers, and amisulpride, although not a serotonin receptor blocker, is more efficacious than FGAs. This questions serotonin receptor blockade as the primary cause of efficacy differences.

Our meta-analyses on the raw data of the registrational studies of olanzapine and risperidone^{36 ,191} revealed that both SGAs were slightly superior to FGAs on positive symptoms but moderately superior on negative symptoms, cognitive symptoms (thought disorder), mood, and impulse control/excitement, improving many symptoms that were untouched by FGAs. So that the disagreement is not merely semantic, those who argue that SGAs are as efficacious as FGAs on positive symptoms while recognizing that SGAs may be more efficacious on negative symptoms, cognition, or mood hold a somewhat similar position as ours.

There is good evidence that negative studies are more likely to go unpublished. One variant of failure to publish is incomplete publication where only favorable results such as a good effect on negative symptoms are published, but the unfavorable results on total score are omitted. We have made considerable efforts in obtaining complete data (from the Freedom of Information Act, FDA Web site, posters, etc).

Geddes et al^{1 (p1374)} argue,"when we controlled for the higher than recommended dose of conventional antipsychotics. . . the differences in efficacy and overall tolerability disappear." We disagree because their tolerability is based on the number of total dropouts. Because the less effective drug has substantially more dropouts due to lack of efficacy, this is a different phenomenon from dropouts due to adverse effects (total number of dropouts confounds 2 issues: adverse effects and efficacy). Furthermore, dropouts from a double-blind study often reflect concern about "unknown" toxicity in experimental drugs (Web, "Significance of Dropout Rate"). There is no one-to-one correspondence between meta-analyses and treatment recommendations. One limitation of meta-analysis is that it cannot balance qualitative differences (apples and oranges) such as between adverse effects. Clinicians need to weigh the medical seriousness and reversibility of rare but serious adverse effects (eg, agranulocytosis with clozapine and cardiac conduction disturbance changes with sertindole) vs the frequency and seriousness of more common adverse effects (eg, weight gain and diabetes mellitus found with olanzapine and clozapine, prolactin elevation with risperidone, etc) in the context of long-term use. Rare adverse effects cannot be accurately estimated from trials with small sample sizes. A few fixed-dose studies show that some SGAs (ie, risperidone and amisulpride) cause dose-related extrapyramidal symptoms (EPS). Other SGAs cause so few EPS that their incidence fades into that of placebo.

Substantially fewer EPS results in better acceptance and long-term risk-benefit ratios and is clinically more important than the efficacy differences. We do not believe that it is valid to infer efficacy differences between 2 or more SGAs from effect size comparisons between SGAs and FGAs. Head-to-head comparisons are necessary for proof. Nevertheless, if some SGAs were empirically more efficacious than others with equally few EPS, we believe that they should be recommended above other FGAs with just a low EPS advantage. Some SGAs are more efficacious than FGAs because they alleviate a greater variety of symptoms, resulting in more complete rehabilitation. Consequently, at this time efficacy and EPS advantages necessitate the consideration of olanzapine, risperidone, and amisulpride as first-line drugs. For further discussion on this article, please see the Web.

Corresponding author and reprints: John M. Davis, MD, the Psychiatric Institute (MC 912), University of Illinois at Chicago, 1601 W Taylor, Chicago, IL 60612 (e-mail: Jdavis@psych.uic.edu).

Submitted for publication May 29, 2002; final revision received August 13, 2002; accepted September 4, 2002.

Neither Dr Davis nor Ms Chen has received any direct or indirect support (in honorarium, travel funds, gifts to favorite charity) from the pharmaceutical industry. Dr Glick has received no support for the study from the pharmaceutical industry, but he has been supported on other projects from Eli Lilly & Co, Indianapolis, Ind; Janssen Pharmaceutic Products LP, Titusville, NJ; AstraZeneca Pharmaceuticals LP, Wilmington, Del; Otsuka America Phara Inc, Rockville, Md; and Pfizer Inc, New York, NY.

We thank Michael E. Bennett, BS, for assistance with references and manuscript preparation.