Congruence of Diagnoses 2 Years After a First-Admission Diagnosis of Psychosis FREE

SINCE THE introduction of DSM-III,¹ psychiatric clinicians and psychiatric epidemiologists have used a common set of diagnostic criteria, but application of these criteria differs. Clinicians observe patients' phenomenology longitudinally, integrating multiple sources of information before making a firm diagnosis. Epidemiologists usually rely on cross-sectional, retrospective assessments to formulate a differential diagnosis. Regardless of how diagnoses are formulated, they can shift over time because of changes in patients' clinical state, clarity stemming from treatment response, emergence of significant, previously unrevealed clinical information, or reinterpretation of previously gathered information.² However, in epidemiology, diagnostic changes are usually attributed to procedural unreliability rather than changes in the illness picture.

The stability of clinical diagnoses has been evaluated in several readmission populations, usually to the same facility.^3- 10 Such studies are limited by inadequate reliability of clinical diagnoses and the bias inherent in sampling rehospitalized patients.¹¹ We identified 9 prospective studies of first-admission psychotic patients describing diagnostic stability over 6 months,² 9 to 18 months,¹² 4 years,¹³ 7 years,¹⁴ 12 to 17 years,¹⁵ 13 years,¹⁶ 18 years,¹⁷ 25 years,¹⁸ and 30 to 40 years.¹⁹ These studies found schizophrenia to be the most stable initial diagnosis (about 90%), affective psychoses less stable (around 80%), and schizophreniform, schizoaffective disorder, and psychosis not otherwise specified (NOS) the least stable. Few studies have examined predictors of diagnostic instability.

This study's research diagnostic procedure was designed to partially mimic the clinical process.^2,20 Diagnoses were formulated at 6- and 24-month follow-up by integrating information gathered longitudinally from subjects, medical records, relatives, and when available, school records. Additionally, diagnoses were decided by consensus of several psychiatrists and DSM-IV²¹ criteria were strictly embraced. This is the first report to examine the congruence between 2 such longitudinal research diagnoses, and to investigate demographic, clinical history, and early course variables associated with shifting to a schizophrenia spectrum diagnosis.

The Suffolk County (Long Island, NY) Mental Health Project is an epidemiologic study of first-admission patients with psychotic disorders.²⁰ From 1989 through 1995, 695 eligible subjects were recruited primarily from the 12 psychiatric inpatient facilities in the county: six 20- to 30-bed community hospital units, a 30-bed university hospital unit, a Veterans Administration hospital, the state adult and child psychiatric centers, and 2 private hospitals (added in 1994). Except for the university and state adult facilities, in which project staff handled recruitment, the chief nurse, social worker, or psychologist at each unit identified appropriate subjects. The screening criteria were subjects who were between ages 15 and 60 years, residents of Suffolk County, and showed clinical evidence of definite or possible psychosis. Exclusion criteria were first psychiatric hospitalization more than 6 months before current admission, moderate or severe mental retardation, and inability to speak English.

Separate written consents were obtained for each interview, medical record review, contact with treating clinician, and request for birth and school records. For minors, a parent also gave written permission. At state facilities, a physician precertified that patients were mentally competent to provide informed consent. All procedures were approved by the State University of New York and the individual facilities.

The initial interview usually took place at the hospital. The response rate at baseline was 72%. Follow-up assessments took place 6 and 24 months later, with interim telephone contact every 3 months. Nearly 80% of follow-up assessments were conducted face-to-face. The loss to attrition was 10% at 6 months and 12% at 24 months. Interviewers were masters-level mental health professionals (primarily psychiatric social workers).

After the 6- and 24-month assessments, 2 project psychiatrists independently reviewed all longitudinal information, except the previous research diagnoses, and then met to resolve disagreements on DSM-III-R²² and DSM-IV diagnoses and supporting criteria. Information came from the Structured Clinical Interview for DSM-III-R (SCID) administered at baseline and at 6- and 24-month follow-up (to cover the subsequent intervals²³) plus medical records, significant others, interviewers' ratings and written narratives, and school records. Each case was then presented to a meeting of several project psychiatrists and investigators, during which the final consensus diagnosis was decided. Criteria sheets were completed to substantiate all diagnostic decisions. "Probable" diagnoses were assigned if a specific diagnosis was strongly suggested but the clinical information failed to support all diagnostic criteria. In a few cases, a probable diagnosis could not be formulated, and the research diagnosis was considered "unknown" (deferred). Interviewers were blind to research diagnoses until after the 24-month assessment, at which time their input was included. At the 24-month consensus conferences, ratings of illness course were obtained.

The diagnoses analyzed here are bipolar and major depressive disorders with psychotic features; schizophrenia spectrum disorders (schizophrenia, schizoaffective and schizophreniform disorders); other nonaffective psychoses (delusional disorder, brief psychosis, psychosis NOS, and organic psychosis); diagnosis deferred (but definite/possible psychotic symptoms); and a residual category of "not psychotic." Definite and probable diagnoses were combined. Because of sample attrition or inadequate information, 25 subjects were missing diagnoses at both 6 and 24 months, 45 were missing just the 6-month diagnosis, and 56 were missing the 24-month diagnosis. Another 22 subjects had no evidence of psychosis. This article examines the 547 subjects diagnosed at both 6- and 24-month follow-up and categorized as psychotic or possibly psychotic, on at least one occasion.

Six clusters of variables were used to predict changes in diagnosis.

Seven variables were analyzed: age at admission, sex, race (African American vs other); education (whether finished high school); marital status (never vs ever married); social class of household (white collar vs blue collar or public assistance); and low intellectual functioning (either full-scale IQ below 85 in school record or 2 of the following if testing not available: Quick Test²⁴ score below 85 at baseline and 6-month follow-up; in a special education program; school dropout or graduated on a nonacademic track; unable to add/subtract; or unable to hold a job for reasons other than mental illness).

Family history information was compiled from medical records and interviews, modeled on the Family History-Research Diagnostic Criteria,²⁵ with subjects and relatives at 6- and 24-month follow-up. Diagnoses of schizophrenia-like disorders, affective disorders, substance use disorders, or any Axis I disorder in first-degree relatives were considered.

Seven clinical history variables were analyzed: premorbid social adjustment (average social and school functioning at ages 5-11, 12-15, and 11-18 years²⁶); time from first psychotic symptom until first hospitalization (≥3 months vs <3 months); prior outpatient treatment (any vs none); prior antipsychotic medication treatment (any vs none); level of functioning during best month of the year before baseline (Global Assessment of Functioning [GAF]); lifetime suicide attempt; and lifetime DSM-III-R substance use disorder.²³

Four hospital variables were included: type of hospital where initially admitted (public vs community/private); facility diagnosis of schizophrenia; length of first hospitalization controlling for type of hospital and year of hospitalization (1989-1995); and whether given antipsychotic medication upon hospital discharge. Four clinical features were analyzed: current substance use disorder; negative symptoms (mean of global ratings on Scale for the Assessment of Negative Symptoms²⁷ [SANS]); positive symptoms (mean global score on Scale for the Assessment of Positive Symptoms²⁷ [SAPS]); and the Brief Psychiatric Rating Scale²⁸ (BPRS) clinical global rating. Ratings were completed by interviewers after the baseline interview, and interrater reliability was good to excellent.²⁰

From the 6-month follow-up, 9 variables were analyzed: rehospitalized in the interval; interviewer's rating of whether subject achieved partial or full remission (minimum duration of 1 month); receipt of antipsychotic medication during interval; GAF score for best month of the interval; GAF for worst week of the month preceding interview; current SANS; current SAPS; BPRS clinical global rating; and whether 6-month research diagnosis was probable vs definite.

Ten variables from the 24-month follow-up were analyzed: rehospitalized between the 6- and 24-month follow-up; consensus rating of return to best premorbid level of psychosocial functioning²⁹ (1, complete return, to 4, marked deterioration); consensus rating of occupational functioning during 24-month period (1, steady, full time more than 50% of interval, to 5, not working most of interval); consensus rating of 24-month illness course (World Health Organization scale³⁰ dichotomized according to whether full remission of at least 6 months was ever achieved); same ratings as at 6 months; and whether 24-month diagnosis was probable vs definite.

The analysis of diagnostic stability between the 6- and 24-month follow-ups is based on a cross-tabulation of 11 diagnostic categories. Two measures of stability are presented for each diagnosis. One, called "prospective consistency," equals the proportion of individuals in a category at 6 months who retain the same diagnosis at 24 months; this would correspond to positive predictive value if the 24-month diagnosis was the gold standard. The second measure, "retrospective consistency," equals the proportion of individuals in a 24-month category who previously received the same diagnosis; this is conceptually similar to sensitivity.

Background and clinical characteristics are summarized for 7 groups: schizophrenia spectrum diagnosis at both 6- and 24-month follow-up (n=171); schizophrenia spectrum diagnosis at 24-month follow-up only (n=45); bipolar disorder at both follow-ups (n=117); major depressive disorder at both follow-ups (n=76); the same residual nonaffective psychotic disorder at both follow-ups (n=50); change among nonschizophrenia spectrum diagnoses (n=70); and change from a 6-month schizophrenia spectrum diagnosis to a nonspectrum diagnosis at 24 months (n=18).

The analysis of predictors of diagnostic change focus on the 45 subjects whose diagnosis shifted into the schizophrenia spectrum. These subjects are first compared with those who received a spectrum diagnosis at both evaluations. This analysis looks backwards from the perspective of the 24-month diagnosis, identifying those factors that discriminate those who did and did not receive a schizophrenia spectrum diagnosis at 6 months. Group differences are analyzed using a t test of means for continuous measures and the χ² test of independence for categorical measures. Nominal levels of significance (without correction for multiple comparisons) are shown in the tables, but only results with P<.01 are described in the text.

A second analysis compares the same 45 subjects whose diagnosis shifted into the schizophrenia spectrum with those receiving nonschizophrenia spectrum diagnoses at both times. Both groups had diagnoses outside the schizophrenia spectrum at 6 months, and this prospective analysis identifies factors predicting who was subsequently reclassified as having a schizophrenia-like condition. Each factor was entered into a logistic regression analysis that controlled for differences among the 3 broad nonschizophrenia spectrum diagnoses—bipolar disorder, major depressive disorder, or residual nonaffective psychotic disorder—at 6 months. Controlling for 6-month diagnosis is prudent since: (1) diagnostic groups differ on several background and clinical variables; (2) a higher proportion of the residual nonaffective psychotic disorder group shifted to a schizophrenia spectrum diagnosis; and (3) the number within each broad diagnostic group that changed to schizophrenia is insufficient to permit separate analyses. For each background and clinical variable, we also examined its interaction with 6-month diagnosis; only 1 of 45 tests was significant at P<.05, consistent with chance expectation.

Those variables that were significant at P<.01 in the bivariate analyses were entered into a stepwise logistic regression analysis to determine which factors independently predict change from a nonschizophrenia spectrum to a schizophrenia spectrum diagnosis vs remaining outside the spectrum. Based on the results, we suggest an algorithm for identifying those patients whose diagnosis is most likely to subsequently change to the schizophrenia spectrum.

The 547 subjects were predominantly male (56.5%), non–African American (85.7%), never married (63.9%), and had a high school education (80.3% of those older than 18 years). The median age was 28 years (range, 15-58 years), and the primary breadwinner in 54.6% of subjects' households had a white collar occupation.

The prospective consistency of the 3 most prevalent 6-month diagnoses was 74% for major depressive disorder, 83% for bipolar disorder, and 92% for schizophrenia (Table 1). The retrospective consistency was 73% for schizophrenia, 82% for major depression, and 85% for bipolar psychosis. Generally, diagnostic categories with fewer subjects had much poorer consistency. The overall rate of consistency in Table 1 was 72%; 394 of the 547 subjects received the same 11-category diagnosis at both follow-ups. Restricting the analysis to those receiving a definite diagnosis at 6 and 24 months (n=386) increased the overall consistency rate to 80%, indicating that probable diagnoses are less stable than definite diagnoses.

One hundred seventy-one subjects (31%) received a schizophrenia spectrum diagnosis at both the 6- and 24-month follow-ups. This represents 170 (90%) of the 189 subjects within this broad category at 6 months, and 171 (79%) of the 216 in this category at 24 months. The shift into a schizophrenia spectrum diagnosis was the most frequent diagnostic change (n=45). Their most common 6-month diagnoses of these individuals were bipolar disorder (n=12), major depressive disorder (n=9), psychosis NOS (n=8), and diagnosis deferred (n=7). At the consensus diagnostic meetings, previous diagnoses were revealed after the current diagnosis was agreed on. When the current diagnosis differed from the previous one, psychiatrists rated the reason for the change: illness course, change in the symptom picture, new information from the subject or other sources, or new interpretation of previous information. The most commonly identified reasons were changes in the symptom picture or the course.

A similar analysis was also performed comparing the baseline with the 24-month diagnoses, although at baseline, far less information was available to the project psychiatrists and the extensive discussions characterizing the 6- and 24-month diagnosis meetings did not occur. In terms of prospective consistency, the percentages of those with schizophrenia, bipolar disorder, and depression at baseline who received the same diagnosis at 24 months were high (88%, 86%, and 70%, respectively), but other categories were low (eg, 23% for schizophreniform disorder, 39% for schizoaffective disorder, 38% for delusional disorder, 45% for substance-induced psychosis). The retrospective consistency levels were also mixed, with 55% of subjects with a 24-month diagnosis of schizophrenia receiving this diagnosis at baseline. The best retrospective consistency was found for schizophreniform disorder (86%), bipolar disorder (73%), and depression (74%). The figures were 26% for schizoaffective disorder, 50% for delusional disorder, and 30% for substance-induced psychosis. Overall, 57% of subjects had the same diagnosis at baseline and 24 months. This reduced consistency reflects greater instability because of the longer time interval and the poorer reliability of the cross-sectional baseline diagnoses.

The comparison of the 45 subjects who shifted into the schizophrenia spectrum with the 171 stably diagnosed schizophrenia spectrum subjects revealed no significant differences in demographic characteristics or family psychiatric history (Table 2, first 2 columns). With respect to clinical history, a smaller percentage of the change group had received antipsychotic medication prior to their first hospital admission, and the mean GAF rating for the best level of functioning during the year before baseline was better in the change group than the stably diagnosed. During the initial hospitalization (Table 3), the change group had lower SANS and SAPS scores. Their illness was also rated as less severe. At 6-month follow-up, the change group continued to have better GAF, SAPS, and BPRS ratings. However, by 24-month follow-up (Table 4), the 2 groups were indistinguishable, except that the certainty of the diagnosis was greater in the stably diagnosed than in the change group. This pattern of results is consistent with the finding that the most common reasons given for a change in diagnosis were the illness course and change in the symptom picture. In a stepwise logistic regression, only the 6-month BPRS severity rating and SAPS independently discriminated between the 2 groups.

We next compared the 45 who shifted to a schizophrenia spectrum diagnosis with those who remained outside the spectrum (Table 2, columns 3-6) (n=313). Controlling for 6-month diagnosis (bipolar, major depression, or other nonaffective disorder), those in the schizophrenia shift group had a lower rate of lifetime substance abuse, poorer psychosocial adjustment during adolescence, and a longer interval between onset of psychosis and hospital admission (Table 2). During their first hospitalization (Table 3), the change-to-schizophrenia group had a longer length of stay, worse negative symptoms, and were more likely to receive a facility diagnosis of schizophrenia and be given antipsychotic medication upon hospital discharge. At 6-month follow-up, this group continued to receive worse SANS, GAF, and BPRS ratings. At 24-month follow-up (Table 4), those whose diagnosis changed to schizophrenia functioned more poorly on all of the variables. However, their diagnosis was also less definitive.

Nine demographic and clinical variables ascertained at or prior to the 6-month follow-up significantly differentiated, at the P<.01 level, those who changed to a schizophrenia spectrum diagnosis from those with a nonschizophrenia spectrum diagnosis at both follow-ups. In a stepwise logistic regression analysis that controlled for broad 6-month diagnostic category, 7 of these predictors entered the final model (Table 5). The remaining 2 variables, baseline SANS and 6-month GAF score for highest functioning during the preceding 6 months, were both highly correlated with the 6-month SANS.

Based on the results of the logistic regression, and inspection of the distributions for the 2 groups, we created an 8-point index. After dichotomizing the 6-month SANS at a score of 1.5, the adolescent psychosocial adjustment scale at 0.28 (the median), and the adjusted duration of initial hospitalization at greater than 14 days, subjects could have between 0 and 7 risk factors. We then identified the following 4 subgroups: (1) those with fewer than 3 risk factors or a 6-month SANS score of 0 (no subject with this SANS score had his/her diagnosis changed to schizophrenia spectrum at 24 months [n=173]); (2) those with 3 risk factors (n=87); (3) those with 4 risk factors (n=58); and (4) those with at least 5 risk factors (n=40). This 4-group classification identifies individuals with markedly distinct levels of risk for having their diagnosis changed to a schizophrenia spectrum disorder. The first subgroup contains nearly half those with a nonschizophrenia spectrum diagnosis at 6 months, and almost no one (1.7%) in this subgroup changed to a schizophrenia spectrum diagnosis at 24 months. At the other extreme, 55% of those in the relatively small fourth subgroup received a schizophrenia spectrum diagnosis at 24 months. This subgroup contains nearly half of those whose diagnosis changed. The 2 intermediate groups, those with 3 or 4 risk factors, had intermediate levels of risk (11.5% and 17.2%, respectively) of changing to a schizophrenia spectrum diagnosis.

The majority (72%) of the 6-month longitudinal diagnoses were consistent with those formulated at 24 months. The diagnoses with the best prospective consistency were schizophrenia (92% of subjects receiving this diagnosis at 6 months retained this diagnosis at 24 months), bipolar disorder (83%), and major depression (74%). These high levels of stability, and marked differences in course, support the distinct nature of these disorders.³¹ The category receiving the largest influx of cases at 24 months was schizophrenia spectrum (45 new diagnoses). Compared with the stably diagnosed schizophrenia group, subjects who shifted into schizophrenia had better prehospital and 6-month GAF scores, lower 6-month SAPS scores, and lower 6-month BPRS scores, but their subsequent course between 6- and 24-month follow-ups was similarly poor. We note that there were no differences between these 2 groups on fixed characteristics, demographic, or family history of illness. Rather, it seems that those whose diagnosis changed were hospitalized at an earlier stage of illness and took longer for their true disorder to become evident.

The conceptually more intriguing comparison was between those who shifted into the schizophrenia spectrum and those whose diagnosis remained outside the spectrum. The multivariate analysis revealed that overall, the best predictors of a shift to schizophrenia spectrum were poorer premorbid adjustment in adolescence, lack of a lifetime substance use disorder, interval between onset of psychosis and hospitalization exceeding 3 months, being diagnosed by the initial facility as having schizophrenia, having a longer length of hospitalization, being given antipsychotic medications upon hospital discharge, and having more negative symptoms at 6-month follow-up. The presence of 5 or more of these risk factors conferred a risk of more than 50% that the 6-month research diagnosis would change to schizophrenia at 24 months. Thus, although a sizable subgroup of subjects did not meet criteria for a schizophrenia spectrum disorder until after the 6-month evaluation, they nonetheless differed from others receiving nonschizophrenia spectrum diagnoses in a variety of ways that foreshadowed their subsequent change in diagnosis. The predictive power of receiving a facility diagnosis of schizophrenia and being given antipsychotic medications upon hospital discharge implies that there were clinical indications early on that some subjects might have schizophrenia, even though they did not meet DSM-IV criteria at 6 months. Perhaps the DSM-IV criteria for schizophrenia are excessively conservative during the early phase of the illness, at least when used to generate research diagnoses. If true, this has the desirable effect of reducing the risk of falsely labeling a patient as schizophrenic.

The methods of this study enabled us to minimize measurement effects by basing the second interview on the first, focusing on the interval course of the symptoms, and clarifying contradictory information obtained over time and from multiple sources. The prospective epidemiological design and careful documentation of diagnostic decisions enhanced the study. In contrast, studies of changes in facility diagnoses of rehospitalized patients are limited by selection bias and unverifiable diagnoses. We note, however, that while the strengths of this study derive from the prospective nature of the study design, the Suffolk County Mental Health Project is a first-admission rather than a first-episode study. This means that some subjects had complicated histories that were difficult to reconstruct, and this may have contributed to some of the temporal instability in diagnosis. The other limitation of this study is that there were too few subjects available to permit analyses of the factors contributing to the instability of less prevalent diagnoses, such as psychosis NOS, delusional disorder, or brief psychosis.

The temporal consistency of schizophrenia disorders in our study is similar to that found in other longitudinal studies. The diagnostic system in part contributes to this high level of consistency because of the requirement that symptoms of schizophrenia be present for at least 6 months.

Among the affective psychoses, a 6-month diagnosis of bipolar disorder with psychosis was somewhat more stable than that of major depression with psychosis. The less frequent specific diagnoses, such as delusional disorder and substance-induced psychosis, were less stable. The unknown and nonspecific diagnoses, including psychosis NOS and brief psychosis, showed little stability. It should be noted that since our 24-month diagnoses were based, in part, on the same information used to formulate the 6-month diagnoses, the consistency is probably higher than would be obtained if the 2 diagnoses had been made from independently collected data.

Some of the shifts in diagnosis over time were conceptually plausible. For example, seven 6-month major depression cases shifted to bipolar disorder, seven 6-month schizoaffective disorder cases shifted to an affective disorder, and 7 subjects with psychosis NOS shifted to schizophrenia at 24 months. Others were unexpected, such as the three 6-month schizophrenia cases who shifted to affective disorder and the three 6-month bipolar cases who shifted to psychosis NOS. We rereviewed each of these unexpected shifts to verify that the psychiatrists had not overlooked information at 6 months and simply erred in their diagnosis.

Diagnostic changes over time may reflect the evolution of an illness, the emergence of new information, or unreliability of measurement. In the Suffolk County sample, diagnosed with a consensus procedure and nonblind reinterview information, the changes in diagnosis were mostly attributable to the evolution of the illness. The greatest instability occurred in the least frequent diagnostic categories. This is one of only a few studies to investigate factors associated with a change in diagnosis, finding that aspects of prehospital functioning, facility diagnosis, and treatment were strongly predictive a diagnostic shift to schizophrenia. Ironically, rigid adherence to DSM-IV criteria caused us to initially underdiagnose individuals whom the facilities diagnosed as having schizophrenia. These findings support the need for a longitudinally based diagnostic process in research that relies on existing nosological systems to diagnose first-admission patients.

Accepted for publication March 1, 2000.

This study was supported in part by grant MH-44801 from the National Institute of Mental Health, Bethesda, Md (Dr Bromet).

We thank Lina Jandorf, MA, for assistance with the data; Lynda Geller, PhD, for overseeing the field work during the first phase of the study; Susan Squires Malvese, BA, for organizing the diagnostic conferences; Joanne Caponi, MSW, Theresa Escobar, MSW, Elyse Everett, MSW, Wayne Gurnick, MSW, Jacqueline Lasher, MSW, and Maribeth McKeever, MSW, for conducting the interviews; Beatrice Kovasznay, MD, Noam Eitan, MD, Kinga Koreh, MD, Ramin Mojtabai, MD, Ranga Ram, MD, Charles Rich, MD, Joan Rubinstein, MD, and Gerard Tolentino, MD, for diagnosing the subjects; and the many mental health professionals, study participants, and their family members for the efforts they have given to this project.

Reprints: Joseph E. Schwartz, PhD, Department of Psychiatry, Putnam Hall-South Campus, SUNY-Stony Brook, Stony Brook, NY 11794-8790 (e-mail: jschwartz@mail.psychiatry.sunysb.edu).