Testing the Reliability and Validity of DSM-IV-TR and ICSD-2 Insomnia Diagnoses:  Results of a Multitrait-Multimethod Analysis FREE

There are several diagnostic nosologies for insomnia^1- 6 designed to systematize descriptions of patients, facilitate communication among health care practitioners, guide treatment choices, predict clinical course, and standardize research.⁷ These nosologies differ markedly in their complexity and reliance on information external to the clinical interview. The American Psychiatric Association's DSM (DSM-III-R, DSM-IV, and DSM-IV-TR)^3- 4 describes a few global insomnia diagnoses and relies primarily on clinical interview. In contrast, the International Classification of Sleep Disorders (ICSD) and the International Classification of Sleep Disorders, second edition (ICSD-2)^5,8 delineate numerous primary and secondary insomnia subtypes and incorporate findings from interview and laboratory tests.

These divergent classification schemes are products of discrepant views about how many subtypes are needed to describe all individuals with insomnia. Proponents of the DSM-IV-TR nosology argue that many ICSD-2 insomnia subtypes have little empirical substantiation and should be subsumed within broader categories.^9- 11 However, the DSM-IV-TR system allows for considerable heterogeneity within diagnostic categories and may not provide optimal discrimination among distinctive insomnia disorders.^12- 18 What is clear is that the 2 nosologies result in markedly discordant classifications when applied to the same sample of patients with insomnia.¹⁹ This state of affairs creates costly variability in assessment and management of patients with insomnia and needless disunity within the insomnia research literature.

Whether the DSM-IV-TR or ICSD-2 offers a more accurate scheme for insomnia classification and diagnosis remains unknown. The few studies^20- 24 that assessed reliability of DSM-IV-TR or ICSD-2 diagnostic categories have found only modest reliability for the insomnia subtypes they evaluated. Reliability data are unavailable for many of the diagnoses in each system. Moreover, available literature^21,25- 27 provides indirect and limited support for a selected subset of DSM-IV-TR and ICSD-2 diagnoses. For example, one study²⁶ showed that treatment recommendations of clinicians vary as a function of the DSM-IV and ICSD diagnoses they assign. Other studies^21,25 comparing patient groupings resulting from standard clinical classification with the groupings resulting from statistical clustering procedures have shown some (albeit minimal) congruence between these 2 classification approaches. Whereas such studies represent proxies for testing the validity of insomnia diagnoses, formal omnibus empirical tests of these nosologies have not yet been conducted to our knowledge.

Nonetheless, the DSM-IV-TR and ICSD-2 have continued to enjoy widespread clinical and research use. Given the existence of these 2 discordant nosologies, insomnia diagnosis and treatment remain a hit-and-miss process guided more by clinicians' instincts and beliefs about various insomnia subtypes than by a well-validated diagnostic system. Clearly, research to ascertain the most viable insomnia nosology is sorely needed. This dual-site study was conducted with the following aims: (1) to determine and compare the reliabilities of DSM-IV-TR and ICSD-2 insomnia subtypes; and (2) to derive and compare convergent validity (CV) and discriminant validity (DV) indices for the DSM-IV-TR and ICSD-2 insomnia diagnoses. Our overarching goal was to ascertain the optimal scheme for insomnia classification.

This study was conducted at Duke University Medical Center and Rush University Medical Center using a multitrait-multimethod²⁸ design. Multiple insomnia diagnoses were assessed in a research cohort using multiple assessment methods. Within each study site, 6 sleep specialists were grouped to form 3 clinician pairs. Each pair was then assigned 1 of 3 assessment approaches to use throughout the study for discerning the insomnia diagnosis (or diagnoses) of each participant they interviewed. One pair used solely a structured sleep interview for discerning diagnoses; the second pair used standard unstructured clinical interviews; and the third pair relied on unstructured clinical interviews combined with polysomnographic (PSG) data. The latter 2 pairs were also given access to information from sleep diaries and sleep history questionnaires completed by study participants. Each clinician formulated impressions independently without knowledge of the other clinicians' impressions. The multiple traits considered were 10 DSM-IV-TR and 37 ICSD-2 insomnia diagnoses, which represent all DSM-IV-TR and ICSD-2 insomnia diagnoses that can be ascertained via interview. Following their interviews, clinicians rated how well each of these 47 insomnia diagnoses fit each patient. A standard multitrait-multimethod²⁸ correlational analysis was then applied to these ratings to test the reliability and validity of the insomnia diagnoses considered. The institutional review boards of the collaborating medical centers reviewed and approved the study protocol. Participants provided written informed consent and received parking expenses plus a maximum $400 payment for participation.

Recruitment occurred between January 2004 and February 2009 through posted announcements and physician referrals. Included individuals (1) met Research Diagnostic Criteria²⁶ for insomnia disorder, (2) were aged 18 years or younger, and (3) spoke English fluently. Excluded individuals (1) had an unstable or life-threatening medical condition, (2) were imminently suicidal, (3) scored 24 or lower on the Mini-Mental State Examination, or (4) were previously evaluated by any study clinicians. Of the 425 individuals enrolled, 8 were removed because they did not meet study selection criteria, 50 withdrew before beginning any study interviews, and 15 failed to complete all interviews. The remaining 352 participants (201 from Duke University Medical Center and 151 from Rush University Medical Center) composed the final sample. Table 1 shows demographic characteristics for the sample.

Participants underwent 2 consecutive nights of PSG with a monitoring montage consisting of 2 channels of electroencephalography (C3-M2, Oz-Cz), 1 chin electromyography channel, 2 channels of electro-oculography (left eye to M1, right eye to M2), 1 channel of airflow (nasal-oral thermistor), 2 channels of respiratory effort (thoracic and abdominal impedance), 1 channel of pulse oximetry, 2 channels of anterior tibialis electromyography (right and left legs), and 1 channel of body position monitoring. Participants followed their customary bedtimes and rising times on PSG nights. Those who occasionally used hypnotics underwent PSG sessions without such medications, whereas those who used hypnotics 3 or more nights per week or were taking antidepressants and/or anxiolytics underwent PSG sessions with these medications.

All PSG sessions were scored using traditional scoring criteria for sleep stages, apneas and hypopneas, periodic limb movements, and related arousals.^29- 31 Summary data including respiratory parameters (apnea-hypopnea index, desaturation index, etc), periodic limb movement indices (number of movements with and without arousals), and types and dosages of medications taken on PSG nights were included in a report made available to 1 clinician pair at each study site.

Participants recorded sleep data for 2 weeks using a handheld computer. Those who had difficulty using this device completed paper sleep diaries. The computer presented questions about each night's bedtime, sleep onset latency, number and length of nocturnal awakenings, time of final awaking, rising time, and sleep medication and alcohol use. Also, respondents' ratings (10-point scale) of sleep quality and how rested they felt on arising were acquired. Diary data were downloaded (or hand-entered for paper diaries) into a PC computer, and a printout of daily values and 2-week averages was generated showing the following: bedtime, sleep onset latency, number of nocturnal awakenings, time awake after sleep onset and prior to final awakening, time of final awakening, time of rising out of bed, total sleep time, total time awake, time in bed, sleep efficiency (total sleep time/time in bed × 100), sleep quality, restedness on arising, medication and alcohol use, and the time when diary entries were made.

Participants completed a 10-page questionnaire. This solicited information about their demographic characteristics, sleep complaints, medical and psychiatric history, and treatment history.

The Duke Structured Interview for Sleep Disorders (DSISD) was used by 1 clinician pair at each site to derive participants' insomnia diagnoses. The DSISD incorporates criteria for ascertaining DSM-IV-TR and ICSD-2 sleep disorder diagnoses and is divided into 4 modules: insomnia-related disorders, excessive daytime sleepiness–related disorders, sleep/wake schedule disorders, and parasomnias. The insomnia module assesses insomnias related to other mental disorders, general medical disorders, substance abuse, circadian rhythm disorders, restless legs syndrome, inadequate sleep hygiene, etc. Screening questions allow sections within a module to be skipped contingent on definitive negative answers from a respondent. However, interviewers may continue sections when answers to screening questions are ambiguous. Previous studies show that the DSISD has acceptable reliability and validity for DSM-IV-TR and ICSD-2 insomnia diagnoses.^32- 33

The rating forms consisted of the series of 10 DSM-IV-TR and 37 ICSD-2 diagnoses presented on the screen of a specially programmed handheld computer. Each diagnosis appeared individually accompanied by a 100-pixel visual analog scale (VAS) labeled“doesn't fit at all” at its left extreme and“fits extremely well” at its right extreme. Clinicians considered each diagnosis separately and decided how well it fit the participant in question. Clinicians moved a pointer on the VAS to indicate the goodness of fit for each DSM-IV-TR and ICSD-2 insomnia diagnosis listed. These ratings were converted into numeric values reflecting their locations on the 100-pixel VAS and used as the primary data for our multitrait-multimethod analyses.

Six clinicians at each study site were stratified by sleep medicine experience (<10 years vs≥10 years) and by professional degree (MD vs PhD). They were then randomly paired within strata to form 3 pairs who were reasonably similar in their experience and mix of clinical specialties. They were then assigned their respective assessment method: (1) solely the DSISD; (2) a combination of an unstructured clinical interview, sleep history questionnaires, and sleep diaries; or (3) a combination of an unstructured clinical interview, sleep history questionnaires, sleep diaries, and PSG information. All clinicians received training in the use of the computerized diagnostic rating forms, and clinicians using the DSISD also were given training in its administration. Each participant underwent 4 interviews (2 structured and 2 unstructured interviews). Clinicians using the structured sleep interview method conducted separate interviews because the DSISD required independent administration and interpretation. The 2 remaining clinician pairs each conducted a joint interview with each participant. During joint interviews, one clinician interviewed the participant while the other clinician remained silent. When the initial interviewer gained sufficient information to formulate diagnostic impressions, he or she exited the interview room and the second clinician interviewed the participant further if desired. The pair using the unstructured clinical interview and PSG method also reviewed PSG results. A randomization procedure was used so that each clinician served as the initial interviewer for a randomly determined 50% of the participants interviewed.

Study candidates first underwent telephone screening with the site's project coordinator. Those passing this screen next met with the project coordinator to provide informed consent and undergo a Mini-Mental State Examination.³⁴ Those who passed the Mini-Mental State Examination screening were enrolled and completed the following: (1) the Structured Clinical Interview for DSM-IV Disorders; (2) the sleep history questionnaire and sleep diary monitoring; and (3) the PSG sessions. Participants then were stratified by sex and age group (aged 18-39, 40-59, and≥60 years) and randomized to 1 of the 6 possible orders of interviews within strata. After completing all interviews, participants chose an in-person or telephone debriefing with the principal investigator (J.D.E. at Duke University Medical Center) or co–principal investigator (E.J.S. or J.K.W. at Rush University Medical Center). During debriefing, the PSG-informed final insomnia diagnosis (or diagnoses) were shared and a treatment referral was made if desired.

We followed traditional analytic guidelines²⁸ for multitrait-multimethod research designs. Analyses entailed computing correlations among the clinicians' VAS ratings across methods and diagnoses within each diagnostic system (DSM-IV-TR and ICSD-2) separately. Resulting correlation matrices were used to evaluate reliability and validity for each diagnosis. Reliability connotes the degree of agreement between clinicians who use the same assessment method; this is commonly called interrater reliability. The correlations between ratings made by the 2 clinicians within each pair for each diagnosis served as reliability indices. With 3 clinician pairs at each of 2 study sites, a total of 6 reliability correlations were derived for each diagnosis. Convergent validity connotes how well clinicians using different assessment methods agree in their diagnoses. The CV indices were those correlations reflecting the level of agreement shown for each diagnosis between clinician pairs using differing assessment methods. Concordant with a method described by Campbell and Fiske,²⁸ the ratings of paired clinicians were first averaged for each diagnosis. We then computed correlations of the resultant averaged ratings of each diagnosis produced by the 3 distinctive clinician pairs. By using this method, we derived 3 CV correlations per diagnosis at each site for a total of 6 such indices for each diagnosis.

Discriminant validity implies that diagnoses are distinctive and can be discriminated. This construct connotes that agreement between clinicians rating the same diagnosis should be notably greater than the agreement observed between clinicians rating distinctive diagnosis. Hence, DV required consideration of correlations of the averaged diagnostic ratings for discrepant diagnoses (within and between clinician pairs). The DV was supported when there was greater correlation between ratings of the same diagnosis derived by different assessment methods than was found for (1) different diagnoses derived by different methods and (2) different diagnoses assessed by the same method. Because the data were not normally distributed, we used Spearman correlation coefficients in all of these analyses.

At Duke University Medical Center, the pair using the unstructured clinical interview with access to PSG information experienced staff turnover. At Rush University Medical Center, each clinician pair changed membership; however, 2 of the pairs (the pair using the structured sleep interview method and the pair using the unstructured clinical interview with access to PSG information) retained one of the clinicians for the entire study period. Correlation analyses ignored staffing changes for the following reasons: (1) the information available to the clinicians (eg, PSG or not) was unchanged; (2) one or both members of the pair remained unchanged for 2 clinician pairs at each site; and (3) clinician characteristics remained reasonably stable.

We first examined the percentage of cases wherein all 6 interviewers rated each diagnosis as a possible fit (rating >0) as well as the percentage of cases wherein all 6 clinicians viewed each diagnosis as nonapplicable (rating = 0). Primary insomnia, insomnia related to another mental disorder, breathing-related sleep disorder, and insomnia due to a general medical condition were the most frequently selected DSM-IV-TR diagnoses. The remaining DSM-IV-TR diagnoses were rated less frequently but most were assigned ratings higher than 0 by 1 or more clinicians for at least 20% of the cases. Only the diagnosis of no sleep disorder was so infrequent that it was dropped from our reliability and validity analyses.

Many ICSD-2 categories were rated infrequently and hence were excluded from analyses. Diagnoses retained were psychophysiological insomnia, paradoxical insomnia, idiopathic insomnia, inadequate sleep hygiene, insomnia due to a mental disorder, insomnia due to a medical condition, insomnia due to a drug or substance, obstructive sleep apnea, circadian rhythm sleep disorder–delayed sleep phase type, restless legs syndrome, periodic limb movement disorder, environmental sleep disorder, and other sleep disorder. These were all assigned a rating higher than 0 by 1 or more clinicians in more than 29% of the cases evaluated.

Table 2 and Table 3 show the reliability indices obtained. The DSM-IV-TR categories with the highest interrater reliability were the insomnias related to another mental disorder or due to a medical condition, breathing-related sleep disorder, and circadian rhythm sleep disorder. More modest reliability estimates were noted for alcohol-related sleep disorder and substance-induced sleep disorder. Results for primary insomnia, dyssomnia not otherwise specified, and other sleep disorder were mixed with lower reliability estimates found at Rush University Medical Center.

The ICSD-2 diagnoses showing the greatest interrater agreement included insomnia due to a mental disorder, insomnia due to a medical condition, periodic limb movement disorder, restless legs syndrome, obstructive sleep apnea, and circadian rhythm sleep disorder–delayed sleep phase type. More modest reliability indices were obtained for the diagnosis of insomnia due to a drug or substance. The Rush University Medical Center site showed lower reliability estimates than the Duke University Medical Center site for psychophysiological insomnia and idiopathic insomnia using the clinical interview method. The Duke University Medical Center site had lower reliability estimates than the Rush University Medical Center site for inadequate sleep hygiene within the clinician pair using the structured sleep interview method and the clinician pair using the unstructured clinical interview with access to PSG information. Interviewers across both sites showed better agreement for the paradoxical insomnia diagnosis when given sleep history questionnaire and diary data to review compared with the DSISD only. The category of other sleep disorder showed much lower reliability at Rush University Medical Center than at Duke University Medical Center.

For our validity analyses, we retained diagnoses that showed at least modest reliability (mean r > .20) and/or had high endorsement rates (rated >0 by≥1 clinician for≥50% of all cases across sites). Accordingly, we eliminated the category of other sleep disorder listed in the DSM-IV-TR and ICSD-2 because of poor reliability at the Rush University Medical Center site and infrequent use overall.

Table 4 and Table 5 show CV and DV indices derived for the DSM-IV-TR and ISCD-2 insomnia diagnoses examined. A diagnosis is considered valid when the CV values are statistically significant (with higher values connoting greater CV) and the DV values are consistently lower than the CV values and, preferably, nonsignificant. The CV and DV values are not reported when poor reliability was found for that diagnosis within a study site.

The best-supported DSM-IV-TR diagnosis was insomnia associated with another mental disorder (Table 4). Insomnia due to a medical condition, breathing-related sleep disorder, and circadian rhythm sleep disorder also showed reasonable validity. Alcohol-related sleep disorder showed more modest validity indices, whereas other substance-induced insomnia received less support. Primary insomnia and dyssomnia not otherwise specified were least supported. Their CV indices were in the low to medium range across methods at Duke University Medical Center; validity correlations for primary insomnia and dyssomnia not otherwise specified were not calculated at Rush University Medical Center owing to low reliability indices noted there.

The best-supported ICSD-2 diagnoses were insomnia due to a mental disorder and insomnia due to a medical condition (Table 5); their CV indices mainly fell in the large range and the DV indices were generally in the insignificant range. Obstructive sleep apnea was also reasonably supported, but its pattern of correlations suggested that clinicians who had access to PSG information differed from those who did not. Restless legs syndrome, circadian rhythm sleep disorder–delayed sleep phase type, and idiopathic insomnia received more modest support: the CV indices for these categories fell in the medium to large range and were generally larger than their related DV indices. Insomnia due to a drug or substance and environmental sleep disorder received less support, with the CV indices falling in the small to medium range. At Duke University Medical Center, the validity indices for psychophysiological insomnia ranged from small to large, but poor reliability for this diagnosis at Rush University Medical Center obviated its validity testing there. A comparison of CV and DV indices acquired for inadequate sleep hygiene suggested reasonable validity for this diagnosis at the Rush University Medical Center site, but its poor reliability at Duke University Medical Center prevented assessing its validity there. The validity indices were variable for periodic limb movement disorder across sites and methods, whereas paradoxical insomnia received little support overall.

Because the study design and data obtained may be novel to many readers, we conducted an additional classification analysis to summarize results and place them in a practical context. Since to our knowledge there are no standardized methods for classifying outcomes of multitrait-multimethod studies, we offer the rationally derived classification rules in Table 6. These rules consider the size and significance of the reliability and validity correlations obtained to gauge the acceptability of each diagnosis. The correlations themselves were appraised using Cohen's guidelines,³⁵ wherein correlation coefficients in the order of 0.10 are regarded as small, those of 0.30 are medium, and those of 0.50 or higher are large. These cutoffs could be considered arbitrary when applied to individual reliability or validity indices, but our classification approach arguably provides a practical manner for synthesizing our results.

Table 7 shows how the diagnoses rate when applying these classification rules. Within DSM-IV-TR, insomnia related to another mental disorder is rated as highly acceptable; breathing-related sleep disorder, insomnia due to a medical condition, and circadian rhythm sleep disorder are acceptable; alcohol-related sleep disorder is marginally acceptable; and the remaining diagnoses are unacceptable. Within ICSD-2, insomnia due to a mental disorder is highly acceptable; obstructive sleep apnea and insomnia due to a medical condition are acceptable; restless legs syndrome, circadian rhythm sleep disorder–delayed sleep phase type, and idiopathic insomnia are marginally acceptable; and the remaining diagnoses are unacceptable.

The DSM-IV-TR and ICSD-2 sleep disorders nosologies are used widely, but few studies have tested their reliability and validity. Results of this trial show that each system includes diagnoses with acceptable reliability and validity. Within DSM-IV-TR, insomnia related to another mental disorder, insomnia due to a general medical condition, breathing-related sleep disorder, and circadian rhythm sleep disorder were best supported. Alcohol-related sleep disorder was more marginally supported, whereas other substance-induced sleep disorder was not well supported. Least supported were dyssomnia not otherwise specified, other sleep disorder, and, surprisingly, primary insomnia. Although primary insomnia was frequently rated, our data call into question its reliability and validity.

Within ICSD-2, insomnia due to a mental disorder, insomnia due to a medical condition, and obstructive sleep apnea were well supported. Circadian rhythm sleep disorder–delayed sleep phase type and idiopathic insomnia also received reasonable support and fell in the marginally acceptable classification. Restless legs syndrome was also classified as marginally acceptable rather than acceptable largely owing to its overlap or correlation with periodic limb movement disorder. However, it is well recognized that periodic limb movements are highly prevalent among patients with restless legs syndrome.³⁶ Given this consideration, perhaps the restless legs syndrome diagnosis should be classified as acceptable. In contrast, insomnia due to a drug or substance and environmental sleep disorder appeared much more marginal. Support for the remaining ICSD-2 diagnoses was more variable and generally poor.

Table 7 suggests that an optimal insomnia nosology derives from a melding of DSM-IV-TR and ICSD-2 diagnoses. The categories of insomnia related or due to a mental disorder, insomnia due to a medical condition, and breathing-related sleep disorder or obstructive sleep apnea seemingly merit strong consideration for inclusion. These categories occur within both nosologies but have slightly different labels in each. Restless legs syndrome also seems to be a viable diagnosis, as does DSM-VI-TR circadian rhythm sleep disorder, which seems favored over the more specific ICSD-2 delayed sleep phase type. Finally, DSM-IV-TR alcohol-related sleep disorder along with ICSD-2 idiopathic insomnia merit consideration as well.

The DSM-IV-TR diagnosis of primary insomnia and most related ICSD-2 subtypes were rated frequently by study clinicians but garnered minimal support. The correlations obtained suggest that the addition of PSG data to interview findings seemingly complicates diagnostic ascertainment and reduces reliability and validity for most of these categories. Because cognitive and behavioral mechanisms are thought to be important perpetuating mechanisms in primary insomnia and its related ICSD-2 subtypes, assignment of these diagnoses is largely dependent on ascertaining the presence of such mechanisms as the likely cause of the insomnia disorder observed. However, patients presumed to have primary insomnia often have some symptoms of depression and/or anxiety, whereas those presumed to have comorbid forms of insomnia often manifest the cognitive-behavioral aberrations thought to perpetuate primary insomnia.³⁷ Hence, primary vs comorbid insomnia distinctions are often subtle and perhaps even arbitrary. Perhaps refinement of the definitional criteria for primary insomnia and its related ICSD-2 subtypes could improve their reliability and validity, but efforts to do so will be complicated by their degree of overlap with the comorbid insomnia subtypes. Ultimately, it may be necessary to adopt a different paradigm for insomnia diagnosis.

Perhaps the primary vs comorbid insomnia distinction could be abandoned in favor of a more inclusive diagnostic term such as insomnia disorder. This is the approach being taken in the upcoming DSM-5. Patients who meet the insomnia criteria outlined in the Research Diagnostic Criteria²⁶ for insomnia disorder are to be assigned this diagnosis regardless of any coincident sleep-disruptive comorbidities. However, diagnosticians are encouraged to also diagnose coexisting psychiatric and medical comorbidities. Such an approach should simplify clinicians' diagnostic task with patients who have insomnia. Of course, use of the global insomnia disorder diagnosis could also encourage a generic one-size-fits-all approach to insomnia treatment. Whether this concern is warranted will only be determined once the DSM-5 is placed into use.

We could also consider abandoning the sorting of patients into diagnostic“bins” but instead using dimensional measures of insomnia symptoms for patient characterization. This approach is advocated by the developers of Profile Analysis via Multidimensional Scaling.^38- 39 This method applies a multidimensional scaling analysis to patients' scores on syndrome-relevant questionnaires to identify core symptom profiles. Profile Analysis via Multidimensional Scaling then locates each patient's symptom pattern by assigning patient-specific weights for each profile. These weights connote the degree of match with each core profile and designate the patient's exact diagnostic location in relation to these profiles. Patients are not forced into single, often poorly fitting diagnostic categories but rather are characterized by their overall symptom arrays. This method may more accurately characterize each patient and support efforts toward the development of individualized therapies. The DSM-5 advocates use of dimensional measures, so perhaps future nosologies will incorporate this method.

In deciding on future changes to our insomnia diagnostic systems, it may be useful to consider this study's findings in conjunction with other projects such as the field trials for the DSM-5 sleep disorders nosology. Our data provide information about interrater reliability and the validity and credibility of the insomnia diagnoses examined in the eyes of our study clinicians. However, the DSM-5 trials will include many more study sites than used in this project and will produce the test-retest reliability data that were not provided with our method. Such additional findings in conjunction with our results should guide future improvements to our insomnia nosologies.

It is important to consider the potential effects of our study method on the findings obtained. Given the statistical demands of the multitrait-multimethod research design, it was necessary to have clinicians rate the goodness of fit of each insomnia diagnosis on VASs for each patient they evaluated. Such ratings may reveal clinicians' subjective decision-making processes when assigning diagnoses, but they do not replicate diagnostic outcomes in clinical situations wherein clinicians assign diagnoses in an all-or-none fashion. Cross-validation of our results using methods that replicate real-world diagnostic practices may therefore be warranted.

Admittedly, this study had several limitations. Our sample included mainly research volunteers. Whether similar results would have been obtained with clinical patients or those with insomnia who are selected from the community remains unknown. Furthermore, data were obtained from only 2 study sites. Replication of this study across additional sites would have been desirable, albeit quite costly. Because many of the subtypes considered were seldom or never rated by study clinicians, the study may have benefited from a much larger and diverse sample. Additionally, the greater turnover in our clinician raters at Rush University Medical Center may have contributed to greater variability in results noted there. Finally, our findings suggested notable differences in the reliability and validity indices obtained across sites for selected diagnoses. Whether such differences should be attributed to demographic differences in their study samples, general site-specific biases, and/or idiosyncratic diagnostic propensities of clinicians cannot be determined given our study design. Nonetheless, few previous studies have assessed reliability and validity of insomnia diagnoses routinely used in practice. Thus, results presented herein fill a void and provide guidance for revisions of our insomnia classification schemes.