0
December 1978, Vol 35, No. 12 >
Article | December 1978

Reliability and Validity in Binary Ratings: Title and subTitle BreakAreas of Common Misunderstanding in Diagnosis and Symptom Ratings

Gregory Carey, PhD; Irving I. Gottesman, PhD
[+] Author Affiliations

Accepted for publication March 29, 1978.

Reprint requests to Behavior Genetics Center, Department of Psychology, Elliott Hall, University of Minnesota, Minneapolis, MN 55455 (Dr Gottesman).


Arch Gen Psychiatry. 1978;35(12):1454-1459. doi:10.1001/archpsyc.1978.01770360058007
Text Size: A A A
Published online
Article
References
Comments

• Confusion may exist between the reliability of a binary rating (for example, schizophrenia versus not-schizophrenia) and its implications for validity. High reliability does not guarantee validity, but paradoxically, low reliability does not imply poor validity in all contexts. Changes in the base rate or in experimental design may indicate high validity even when the reliability was thought to be low. Attempts to improve the psychiatric nomanclature by increasing only reliability run the risk of the "attenuation paradox" where further increases in reliability will make the ratings less valid. Finally, the assumption of random error in making diagnoses does not always hold, so that statistical analyses must be adjusted accordingly. New statistical methods are needed to index only false-positive or false-negative rates in order to quantify the error that will reduce some validity coefficients.

REFERENCES

1 +
Wing JK, Cooper JE, Sartorius N: The Measurement and Classification of Psychiatric Symptoms . London, Cambridge University Press, 1974;.
2 +
Endicott J, Spitzer RL: A diagnostic interview: The Schedule for the Affective Disorders and Schizophrenia. Read before the annual meeting of the American Psychiatric Association, Toronto, May, 1977.
3 +
Fisher M:  Development and validity of a computer method for diagnosis of functional psychoses (DIAX) . Acta Psychiatr Scand 50:243-288, 1974;.
4 +
Spitzer RL, Endicott J, Cohen J, et al:  Constraints on the validity of computer diagnosis . Arch Gen Psychiatry 31:197-203, 1974;.
5 +
Feighner JP, Robins E, Guze SB, et al:  Diagnostic criteria for use in psychiatric research . Arch Gen Psychiatry 26:57-63, 1972;.
6 +
Spitzer RL, Endicott J, Robins E:  Clinical criteria for psychiatric diagnosis and DSM-III . Am J Psychiatry 132:1187-1192, 1975;.
7 +
Spitzer RL, Endicott J, Robins E: Research Diagnostic Criteria: Rationale and reliability. Read before the annual meeting of the American Psychiatric Association, Toronto, May 1977.
8 +
Cooper JE, Kendell RE, Gurland BJ, et al: Psychiatric Diagnosis in New York and London (U.S.-U.K. Diagnostic Project) . London, Oxford University Press, 1972;.
9 +
International Pilot Study of Schizophrenia . Geneva, World Health Organization, 1973;, vol 1.
10 +
Fleiss JL: Statistical Methods for Rates and Proportions . New York, John Wiley & Sons Inc, 1973;.
11 +
Fleiss JL:  Measuring agreement between two judges on the presence or absence of a trait . Biometrics 31:651-659, 1975;.
12 +
Krippendorff K:  Bivariate agreement coefficients for reliability of data , in Borgatta EF (ed): Sociological Methodology . San Francisco, Jossey-Bass, 1970;.
13 +
Cohen J:  A coefficient of agreement for nominal scales . Educ Psychol Measurement 20:37-46, 1960;.
14 +
Spitzer RL, Fleiss JL:  A reanalysis of the reliability of psychiatric diagnosis . Br J Psychiatry 125:341-347, 1974;.
15 +
Wiggins JS: Personality and Prediction: Principles of Personality Assessment . Reading, Mass, Addison-Wesley, 1973;.
16 +
Maxwell AE:  Coefficients of agreement between observers and their interpretation . Br J Psychiatry 130:79-83, 1977;.
17 +
Meehl PM, Rosen A:  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores . Psychol Bull 30:525-564, 1946;.
18 +
Loevinger J:  The attenuation paradox in test theory . Psychol Bull 51:493-504, 1954;.
19 +
Gottesman II, Shields J: Schizophrenia and Genetics: A Twin Study Vantage Point . New York, Academic Press Inc, 1972;.
20 +
Shields J, Gottesman II:  Cross-national diagnosis of schizophrenia in twins . Arch Gen Psychiatry 27:725-730, 1972;.
21 +
Arkonac O, Guze SB:  A family study of hysteria . N Engl J Med 268:239-242. 1963;.

First Page Preview

First page PDF preview

Figures

Tables

Interactive Graphics

Video

Country-Specific Mortality and Growth Failure in Infancy and Yound Children and Association With Material Stature

Use interactive graphics and maps to view and sort country-specific infant and early dhildhood mortality and growth failure data and their association with maternal

1 +
Wing JK, Cooper JE, Sartorius N: The Measurement and Classification of Psychiatric Symptoms . London, Cambridge University Press, 1974;.
2 +
Endicott J, Spitzer RL: A diagnostic interview: The Schedule for the Affective Disorders and Schizophrenia. Read before the annual meeting of the American Psychiatric Association, Toronto, May, 1977.
3 +
Fisher M:  Development and validity of a computer method for diagnosis of functional psychoses (DIAX) . Acta Psychiatr Scand 50:243-288, 1974;.
4 +
Spitzer RL, Endicott J, Cohen J, et al:  Constraints on the validity of computer diagnosis . Arch Gen Psychiatry 31:197-203, 1974;.
5 +
Feighner JP, Robins E, Guze SB, et al:  Diagnostic criteria for use in psychiatric research . Arch Gen Psychiatry 26:57-63, 1972;.
6 +
Spitzer RL, Endicott J, Robins E:  Clinical criteria for psychiatric diagnosis and DSM-III . Am J Psychiatry 132:1187-1192, 1975;.
7 +
Spitzer RL, Endicott J, Robins E: Research Diagnostic Criteria: Rationale and reliability. Read before the annual meeting of the American Psychiatric Association, Toronto, May 1977.
8 +
Cooper JE, Kendell RE, Gurland BJ, et al: Psychiatric Diagnosis in New York and London (U.S.-U.K. Diagnostic Project) . London, Oxford University Press, 1972;.
9 +
International Pilot Study of Schizophrenia . Geneva, World Health Organization, 1973;, vol 1.
10 +
Fleiss JL: Statistical Methods for Rates and Proportions . New York, John Wiley & Sons Inc, 1973;.
11 +
Fleiss JL:  Measuring agreement between two judges on the presence or absence of a trait . Biometrics 31:651-659, 1975;.
12 +
Krippendorff K:  Bivariate agreement coefficients for reliability of data , in Borgatta EF (ed): Sociological Methodology . San Francisco, Jossey-Bass, 1970;.
13 +
Cohen J:  A coefficient of agreement for nominal scales . Educ Psychol Measurement 20:37-46, 1960;.
14 +
Spitzer RL, Fleiss JL:  A reanalysis of the reliability of psychiatric diagnosis . Br J Psychiatry 125:341-347, 1974;.
15 +
Wiggins JS: Personality and Prediction: Principles of Personality Assessment . Reading, Mass, Addison-Wesley, 1973;.
16 +
Maxwell AE:  Coefficients of agreement between observers and their interpretation . Br J Psychiatry 130:79-83, 1977;.
17 +
Meehl PM, Rosen A:  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores . Psychol Bull 30:525-564, 1946;.
18 +
Loevinger J:  The attenuation paradox in test theory . Psychol Bull 51:493-504, 1954;.
19 +
Gottesman II, Shields J: Schizophrenia and Genetics: A Twin Study Vantage Point . New York, Academic Press Inc, 1972;.
20 +
Shields J, Gottesman II:  Cross-national diagnosis of schizophrenia in twins . Arch Gen Psychiatry 27:725-730, 1972;.
21 +
Arkonac O, Guze SB:  A family study of hysteria . N Engl J Med 268:239-242. 1963;.

Correspondence

Submit a Letter
CME Course for:


You need to register in order to view this quiz.


To understand the clinical management of acute heart failure syndromes.
Accreditation Information The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.
The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
To view and print your certificate and access a summary of your CME courses go to My CME.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s “Cited By” API will populate this tab (http://www.crossref.org/citedby.html).
Compression-Only CPR: Pushing the Science Forward
Cone
AAM 2010;304:1493-1495.
All you need to read in the other general journals
BMJ 2010;341:c5893-c1494.
Submit a Response

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

© 2012 American Medical Association. All Rights Reserved.
Powered bySilverchair Information Systems