0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Predicting Suicides After Psychiatric Hospitalization in US Army Soldiers The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) FREE

Ronald C. Kessler, PhD1; Christopher H. Warner, MD2; Christopher Ivany, MD3; Maria V. Petukhova, PhD1; Sherri Rose, PhD1; Evelyn J. Bromet, PhD4; Millard Brown III, MD, MB3; Tianxi Cai, ScD5; Lisa J. Colpe, PhD, MPH6; Kenneth L. Cox, MD, MPH7; Carol S. Fullerton, PhD8; Stephen E. Gilman, ScD9,10; Michael J. Gruber, MS1; Steven G. Heeringa, PhD11; Lisa Lewandowski-Romps, PhD11; Junlong Li, PhD5; Amy M. Millikan-Bell, MD, MPH7; James A. Naifeh, PhD8; Matthew K. Nock, PhD12; Anthony J. Rosellini, PhD1; Nancy A. Sampson, BA1; Michael Schoenbaum, PhD6; Murray B. Stein, MD, MPH13,14,15; Simon Wessely, PhD16; Alan M. Zaslavsky, PhD1; Robert J. Ursano, MD8 ; for the Army STARRS Collaborators
[+] Author Affiliations
1Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
2Department of Behavioral Medicine, Blanchfield Army Community Hospital, Fort Campbell, Kentucky
3US Army Office of the Surgeon General, Falls Church, Virginia
4Department of Psychiatry, Stony Brook University School of Medicine, Stony Brook, New York
5Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts
6National Institute of Mental Health, Bethesda, Maryland
7US Army Public Health Command, Aberdeen Proving Ground, Maryland
8Center for the Study of Traumatic Stress, Department of Psychiatry, Uniformed Services University of the Health Sciences, Bethesda, Maryland
9Department of Social and Behavioral Sciences, Harvard School of Public Health, Boston, Massachusetts
10Department of Epidemiology, Harvard School of Public Health, Boston, Massachusetts
11Institute for Social Research, University of Michigan, Ann Arbor
12Department of Psychology, Harvard University, Cambridge, Massachusetts
13Department of Psychiatry, University of California, San Diego, La Jolla
14Deapartment of Family and Preventive Medicine, University of California, San Diego, La Jolla
15Veterans Affairs San Diego Healthcare System, San Diego, California
16King’s Centre for Military Health Research, King’s College London, London, United Kingdom
JAMA Psychiatry. 2015;72(1):49-57. doi:10.1001/jamapsychiatry.2014.1754.
Text Size: A A A
Published online

Importance  The US Army experienced a sharp increase in soldier suicides beginning in 2004. Administrative data reveal that among those at highest risk are soldiers in the 12 months after inpatient treatment of a psychiatric disorder.

Objective  To develop an actuarial risk algorithm predicting suicide in the 12 months after US Army soldier inpatient treatment of a psychiatric disorder to target expanded posthospitalization care.

Design, Setting, and Participants  There were 53 769 hospitalizations of active duty soldiers from January 1, 2004, through December 31, 2009, with International Classification of Diseases, Ninth Revision, Clinical Modification psychiatric admission diagnoses. Administrative data available before hospital discharge abstracted from a wide range of data systems (sociodemographic, US Army career, criminal justice, and medical or pharmacy) were used to predict suicides in the subsequent 12 months using machine learning methods (regression trees and penalized regressions) designed to evaluate cross-validated linear, nonlinear, and interactive predictive associations.

Main Outcomes and Measures  Suicides of soldiers hospitalized with psychiatric disorders in the 12 months after hospital discharge.

Results  Sixty-eight soldiers died by suicide within 12 months of hospital discharge (12.0% of all US Army suicides), equivalent to 263.9 suicides per 100 000 person-years compared with 18.5 suicides per 100 000 person-years in the total US Army. The strongest predictors included sociodemographics (male sex [odds ratio (OR), 7.9; 95% CI, 1.9-32.6] and late age of enlistment [OR, 1.9; 95% CI, 1.0-3.5]), criminal offenses (verbal violence [OR, 2.2; 95% CI, 1.2-4.0] and weapons possession [OR, 5.6; 95% CI, 1.7-18.3]), prior suicidality [OR, 2.9; 95% CI, 1.7-4.9], aspects of prior psychiatric inpatient and outpatient treatment (eg, number of antidepressant prescriptions filled in the past 12 months [OR, 1.3; 95% CI, 1.1-1.7]), and disorders diagnosed during the focal hospitalizations (eg, nonaffective psychosis [OR, 2.9; 95% CI, 1.2-7.0]). A total of 52.9% of posthospitalization suicides occurred after the 5% of hospitalizations with highest predicted suicide risk (3824.1 suicides per 100 000 person-years). These highest-risk hospitalizations also accounted for significantly elevated proportions of several other adverse posthospitalization outcomes (unintentional injury deaths, suicide attempts, and subsequent hospitalizations).

Conclusions and Relevance  The high concentration of risk of suicide and other adverse outcomes might justify targeting expanded posthospitalization interventions to soldiers classified as having highest posthospitalization suicide risk, although final determination requires careful consideration of intervention costs, comparative effectiveness, and possible adverse effects.

Figures in this Article

The US Army suicide rate, although historically below the civilian rate, has increased since 20041 to exceed the civilian rate.2 Despite numerous efforts to address this problem, including universal interventions (eg, Ask/Care/Escort prevention education and depression, posttraumatic stress disorder, and suicide screening in all primary care encounters) and high-risk interventions (eg, postdeployment screening),3 the US Army suicide rate has continued to increase. One potentially important group for targeted interventions is soldiers recently discharged from inpatient psychiatric treatment. Such patients have long been known to have a high risk of suicide.4 US military administrative data document an 8-fold elevated suicide risk in the 3 months after psychiatric hospitalization and a 5-fold elevated risk for the remainder of the 12 months after hospitalization.5 A report6 on the similar patterns among civilians called for expansion of posthospitalization suicide preventive interventions, noting that such interventions in the United Kingdom (eg, required outpatient visits within 1 week of hospital discharge, assertive outreach for missed outpatient appointments, 24-hour community crisis teams, and intensive community support for patients difficult to engage in traditional services) were associated with significant before-after reductions in posthospitalization suicides.7

Suicide is a rare outcome even among recently discharged psychiatric inpatients8; therefore, the benefits of providing intensive posthospitalization suicide prevention interventions to all recently discharged inpatients are low. A more rational allocation of treatment resources would be to combine relatively inexpensive universal interventions9 with more intensively targeted high-risk interventions.4 However, this tiered approach would require developing a reliable risk stratification scheme. The US Department of Veterans Affairs (VA) and the US Department of Defense (DoD) called for this kind of differentiation in their Clinical Practice Guideline (CPG) entitled Assessment and Management of Patients at Risk for Suicide.10 However, the CPG provided little concrete guidance on how these assessments should be implemented. Research has consistently revealed that health care professionals are not accurate in making such assessments.1114

One potentially promising approach to assessing posthospitalization suicide risk would be to use administrative data available during hospitalization to generate an actuarial posthospitalization suicide risk algorithm. Previous research has revealed that actuarial suicide prediction is much more accurate than prediction based on clinical judgment.1114 An increasing number of computerized risk algorithms are being used as clinical decision support tools in other areas of medicine and have been found to improve clinical processes.15,16 Skepticism exists about developing such an algorithm for posthospitalization suicide interventions based on the relatively weak associations found in previous research17 on in-hospital predictors and subsequent suicides. However, a stronger risk algorithm might be developed in the US Army because of the availability of integrated administrative data for all US Army personnel. Absence of such data in the general population is widely recognized as an impediment to big data health care solutions.18 A number of empirical studies1923 have documented strong predictive associations between integrated US Army and DoD administrative data and subsequent US Army suicides, although none attempted to develop a risk algorithm for posthospitalization suicides. The objective of this study was to develop such an algorithm using administrative data from the Historical Administrative Data System (HADS) of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS).24

Sample

Creation and analysis of the consolidated and deidentified data system were approved by the Human Subjects Committees of the Uniformed Services University of the Health Sciences for the Henry M. Jackson Foundation (the primary grantee), the University of Michigan Institute for Social Research (site of the Army STARRS Data Enclave), and Harvard Medical School (site of data analysis). Obtaining informed consent from individual soldiers, most of whom were no longer in service at the time the HADS was constructed, was not required because the data were deidentified.

There were 53 769 regular US Army hospitalizations from January 1, 2004, through December 31, 2009, with any International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) psychiatric admission diagnosis exclusive of tobacco use disorders (eTable 1 at http://www.armystarrs.org/publications). These hospitalizations involved 40 820 soldiers (30 763 with 1 hospitalization, 6929 with 2, and 3128 with >2), representing 0.9% of all regular US Army soldiers in any 12-month period. We excluded the 13 936 additional hospitalizations in which nicotine dependence was the only psychiatric diagnosis because these were invariably for physical disorders and nicotine dependence was noted based on withdrawal during hospitalization. There was no elevated posthospitalization suicide risk among these soldiers. We also excluded the 406 additional hospitalizations that occurred through emergency departments because of a suicide attempt without an accompanying ICD-9-CM psychiatric diagnosis. Four of these 406 soldiers died in the hospital, whereas none of the others died by suicide in the next 12 months. On the basis of evidence from another study25 indicating that predictors of posthospitalization suicide vary with time since discharge and elevated risk persists 12 months after discharge, a discrete-time person-month survival file was created to examine suicides in the 12 months after hospital discharge, censoring all person-months at the beginning of new hospitalizations or terminations of active duty and allowing interactions between substantive predictors and time since hospital discharge. All person-months with suicide were coded 1 on the outcome, and all others were coded 0. This file contained 334 936 person-months for a mean of 6.2 months (334 936 per 53 760 months) after hospital discharge. This low mean reflects high rates of termination of service and subsequent hospitalization within 12 months of each hospitalization.

Measures

The HADS includes data from 38 US Army and DoD administrative data systems26 (eTable 2 at http://www.armystarrs.org/publications). In a comprehensive review of published studies of predictors of civilian posthospitalization suicides, Troister et al27 found 5 replicated classes of predictors: (1) sociodemographics (the most consistent being male sex and recent job loss), (2) history of prior suicidal behaviors, (3) quality of care (eg, low continuity of care), (4) time since hospital discharge (inversely related to suicide risk), and (5) other psychopathological risk factors (the most consistent being nonaffective psychosis, mood disorders, and multiple comorbid psychiatric disorders). Other studies17,28,29 found similar predictors. We extracted HADS variables operationalizing these predictors and added US Army career variables found to predict military suicides,1922 unit variables, criminal justice variables (violent crime victimization or perpetration), and measures of registered weapons. All predictors other than those that involved the hospitalization were defined as of the month before hospitalization, whereas predicted suicides were in the 12 months after hospital discharge.

We cast a wide net in extracting HADS measures of the predictor constructs. For example, we distinguished 23 categories of psychiatric diagnoses defined largely by aggregated ICD-9-CM codes (eg, attention-deficit/hyperactivity learning disorders [ICD-9-CM codes 314.0-315.9]), 8 additional categories of behavioral stressors (eg, marital problems, other stressors or adversities, suicidal ideation, and self-damaging behavior), and summary measures of any prior admission diagnoses, admission count variables, and parallel outpatient variables (eTable 1 at http://www.armystarrs.org/publications). We also included National Drug Code psychotropic medication codes collapsed into 15 categories (eg, antianxiety, antidepressant, and antipsychotic) and 25 subcategories (eg, selective serotonin reuptake inhibitor, serotonin-norepinephrine reuptake inhibitor, and tricyclic antidepressant) based on the First Databank Enhanced Therapeutic Classification System (http://www.fdbhealth.com) (eTable 3 at http://www.armystarrs.org/publications). A total of 421 individual variables were constructed (eTable 4 at http://www.armystarrs.org/publications).

Because the HADS data systems were not developed for research, more data were missing and inconsistent in some (eg, sociodemographic) component data sets than in research data sets. However, because the HADS data sets are updated monthly, missing values typically appeared in earlier and/or later months, allowing nearest neighbor imputations. Remaining missing values were resolved using randomly selected multiple imputations.30 Inconsistencies were reconciled using rational imputations (eg, a soldier classified female one month but male other months was recoded male).

Statistical Analysis

Discrete-time (person-month) survival analysis31 was used to predict suicides in the 12 months after hospitalization in 3 steps. First, functional forms of bivariate associations were examined and predictors transformed (usually sets of nested dichotomies but some collapsed-truncated continuous variables) to explore nonlinear multivariate associations. Second, all predictors were discretized and analyzed with 100 regression trees in distinct bootstrap pseudo-samples using the R package rpart program32 to prevent overfitting33 and allow detecting interactions among predictors.25,28 Third, predictors having significant bivariate associations and interactions emerging in 10% or more of regression trees were included as predictors in multivariate survival models.

A central challenge in the third step was multicollinearity among the 421 predictors. The classic way to address this problem is with stepwise analysis,34 but this approach overfits.35 Machine learning methods reduce overfitting.36,37 The machine learning method we used was the elastic net,38 a penalized regression method that provides stable and sparse estimates of model parameters by explicitly penalizing overfitting with a composite penalty λ{MPP × Plasso + (1 − MPP) × Pridge}, where MPP is a mixing parameter penalty with values between 0 and 1 that controls relative weighting between 2 types of penalties: the lasso penalty and the ridge penalty. The parameter λ controls the total amount of penalization.39 The ridge penalty handles multicollinearity by shrinking all coefficients smoothly toward 0 but retains all variables in the model.40 The lasso penalty allows simultaneous coefficient shrinkage and variable selection, tending to select at most one predictor in each strongly correlated set but at the expense of giving unstable estimates in the presence of high multicollinearity.41 The elastic net approach of combining the ridge and lasso penalties has the advantage of yielding more stable and accurate estimates than either the ridge or lasso alone while maintaining model parsimony.38

The 3-step approach of combining regression trees with penalized regression for variable selection enabled us to incorporate possible interactions and nonlinearities in a clinically meaningful way while controlling for possible overfitting. The R package glmnet program42 was used to estimate penalized models with MPPs of 0.1, 0.4, 0.7, and 1.0 (an MPP of 0.0 was not used because of multicollinearity in the full predictor set). Internal 10-fold cross-validation selected the coefficient in front of the penalty. Comparative fit across the 20 specifications (ie, 4 MPP values for each of 5 constraints on the number of predictors) was evaluated by inspecting the area under the receiver operating characteristic curve (AUC) and concentration of risk (CR). The CR is the proportion of observed suicides after hospitalizations in each ventile (ie, 20 groups of hospitalizations of equal frequency) ordered by predicted suicide risk. Suicide risk of each hospitalization was calculated using coefficients to project risk as of 12 months after hospital discharge regardless of observed hospitalization data and censoring and standardized by time of hospitalization to adjust for temporal variation in suicide risk. Given that the number of hospitalizations per ventile was much larger than the number of suicides, we focused on the CR in the highest-risk ventile in selecting the best penalized model.

Once a best penalized model was selected, a conventional discrete-time survival model with a logistic link function was estimated using the same predictors as the best penalized model to examine how much the penalty reduced model fit. Because the variance inflation factor of coefficients in this model revealed estimates to be unstable, we also used forward stepwise analysis with a .05-level entry criterion to select a stable subset of predictors for a reduced version of the logistic model. Coefficients in this reduced logistic model were then exponentiated to create odds ratios (ORs) for ease of interpretation. Ventiles from the best penalized model were then collapsed into risk strata using the logic of stratum-specific likelihood ratios.43 The CR, AUC, and the standardized (for amount of uncensored time observed after each hospitalization) suicide rates per 100 000 person-years were calculated for these risk strata. Finally, parallel rates of risk were calculated for unintentional injury deaths, attempted suicides, and subsequent hospitalizations in the same ventiles to evaluate other adverse outcomes associated with posthospitalization suicide risk.

Patterns of Posthospitalization Suicide

Sixty-eight hospitalized soldiers died by suicide within 12 months of hospital discharge (263.9 suicides per 100 000 person-years vs 18.5 suicides per 100 000 in the total US Army),23 representing 12.0% of all US Army suicides. An additional 157 hospitalized soldiers died in other ways, and 22 010 others terminated active duty for other reasons (eg, administrative separation and retirement) within 12 months of hospital discharge.

Bivariate Associations of Predictors With Suicide

No interactions emerged in more than 10% of regression trees. However, 131 of the 421 bivariate associations (31.1%) between individual predictors and suicides were significant at the .05 level (eTables 5-9 and eTables 11-15 at http://www.armystarrs.org/publications). All these variables were used in the penalized multivariate models.

Selecting a Best Penalized Survival Model

A 10-fold cross-validation revealed that AUC was maximized across the 20 penalized survival models for an MPP of 1.0 (lasso) with 73 predictors and an MPP of 0.1 to 0.7 with 72 to 122 predictors (Figure 1). Because the lasso model yielded the best cross-validated CR in the highest-risk ventile (52.9%) (Table 1), we estimated a conventional discrete-time survival model with a logistic link function using the same 73 predictors. This model had a much higher AUC (AUC, 0.89) and CR (CR, 61.8%) in the highest-risk ventile than the lasso model with the same predictors, but this was because of overfitting (variance inflation factor >5 for 6 coefficients). Forward stepwise analysis selected a more stable set of predictors in a reduced logistic model, and this model, which contained 20 predictors, had a slightly lower AUC (AUC, 0.84) and CR (CR, 50.0%) in the highest-risk ventile than the lasso model.

Place holder to copy figure label and caption
Figure 1.
Receiver Operating Characteristic (ROC) Curves for Discrete-Time (Person-Month) Elastic Net Penalized Survival Models With Different Mixing Parameter Penalties (MPPs) and for a Conventional Discrete-Time Survival Model Predicting Posthospitalization Suicide

Elastic net penalized survival models were estimated with different MPPs, allowing up to 421 predictors. The best cross-validated model was an MPP of 1.0 with 73 predictors. A conventional discrete-time survival model that contained the same 73 predictors was unstable (variance inflation factor >5.0 for 6 predictors). As a result, we used forward stepwise analysis with a .05-level entry criterion to select a more stable subset of the 73 predictors. Twenty predictors entered that model. The ROC curve shown here for the conventional model is based on those 20 predictors. AUC indicates area under the receiver operating characteristic curve.

Graphic Jump Location
Table Graphic Jump LocationTable 1.  CR, AUC, and Np Values by Mixing Parameter Penaltya

Caution is needed in interpreting predictors in the reduced logistic model because the variable selection algorithm maximized overall prediction accuracy rather than individual coefficient accuracy. It is nonetheless noteworthy that the model included variables in all predictor classes (Table 2): 3 sociodemographic characteristics (male sex, enlistment at ≥27 years of age, and US Armed Forces Qualification Test score >50th percentile; ORs, 1.9 [95% CI, 1.0-3.5] to 7.9 [95% CI, 1.9-32.6]), access to firearms (number of registered pistols; OR, 1.3; 95% CI, 1.0-1.6), crime perpetration (weapons possession or verbal assault; ORs, 2.2 [95% CI, 1.2-4.0] to 5.6 [95% CI, 1.7-18.3]), prior suicidality (ORs, 1.6 [95% CI, 1.1-2.5] to 2.9 [95% CI, 1.7-4.9]), prior psychiatric treatment (ORs, 0.3 [95% CI, 0.2-0.6] to 5.6 [95% CI, 1.8-17.7]), and characteristics of the focal hospitalization (ORs, 0.4 [95% CI, 0.2-0.7] to 6.0 [95% CI, 2.1-17.4]). The 2 ORs less than 1.0 were for (1) being above the 50th percentile on the ratio of number of psychiatric hospitalizations to time in service and (2) posttraumatic stress disorder during current hospitalization.

Table Graphic Jump LocationTable 2.  ORs (95% CIs) and VIFs for the Discrete-Time Logistic Survival Modela
CR and Conditional Risk Distributions

Inspection of the CR across predicted risk ventiles led to creation of 4 risk strata. Most suicides occurred in the highest-risk stratum (which was made up of the 5% of hospitalizations in the highest-risk ventile; CR, 52.9%) (Figure 2). The CR was lower (CR, 8.8%) in the second stratum (made up of the 5% of hospitalizations in the second-highest ventile), lower still (CR, 4.2%) in a third stratum (made up of the 35% of hospitalizations in the next 7 ventiles), and lowest (CR, 0.8%) in the fourth stratum (made up of the 55% of suicides in the lowest 11 ventiles).

Place holder to copy figure label and caption
Figure 2.
Concentration of Risk of Posthospitalization Suicides by Ventile of Predicted Risk Based on the Discrete-Time Penalized Survival Model With a Mixing Parameter Penalty of 1.0

Ventiles are 20 groups of hospitalizations of equal frequency (2688 or 2689 hospitalizations), dividing the total sample of 53 769 hospitalizations into groups defined by level of predicted suicide risk.

Graphic Jump Location

Suicide risk ranged from 1338.8 per 100 000 hospitalizations in the highest-risk stratum to 20.3 per 100 000 hospitalizations in the lowest-risk stratum (Table 3). However, because mean time in service after hospital discharge was considerably less than 12 months, suicide risk per 100 000 person-years was considerably higher than per 100 000 hospitalizations: 3824.1 per 100 000 person-years in the highest-risk stratum to 40.9 per 100 000 in the lowest-risk stratum.

Table Graphic Jump LocationTable 3.  CR and Conditional Risk of Posthospitalization Suicides by Risk Strata Across All Hospitalizations
Stability of Estimates

The CR in the highest-risk stratum did not differ significantly, depending on whether (1) hospitalization was in a facility with a mental health inpatient unit vs a general medical facility without such a unit (48.2% vs 66.7; χ21 = 1.7; P = .19); (2) the suicide occurred before vs after September 1, 2008 (median date of suicides during the study period; 38.7% vs 70.3%; χ21 = 2.4; P = .12); or (3) the suicide did vs did not occur within 3 months of hospital discharge (median time to postdischarge suicide; 52.6% vs 56.7%; χ21 = 0.0; P = .99).

Associations of Suicide Risk With Other Adverse Outcomes

Soldiers in the highest-risk stratum also had elevated risks of other adverse outcomes in the year after hospital discharge, including unintentional injury deaths (CR, 10.1%; χ21 = 7.1; P = .008), suicide attempts (CR, 9.1%; χ21 = 332.7; P < .001), and subsequent hospitalizations (7.5%; χ21 = 893.4; P < .001). Soldiers in the highest predicted suicide risk stratum had 7 unintentional injury deaths, 830 suicide attempts, and 3765 subsequent hospitalizations within 12 months of hospital discharge (492,666.2 per 100 000 person-years). At least one of these outcomes occurred after 46.3% of the highest-risk hospitalizations.

Although risk factors for suicide are widely known, synthesizing this information to optimize suicide prediction has been an elusive goal up to now. This study addressed this problem by using machine learning to generate an actuarial suicide risk algorithm from US Army and DoD administrative data, finding that 52.9% of suicides occurred after the 5% of hospitalizations with highest predicted risk. Although interventions in this high-risk stratum would not solve the entire US Army suicide problem given that posthospitalization suicides account for only 12% of all US Army suicides, the algorithm would presumably help target preventive interventions. Before clinical implementation, though, several key issues must be addressed.

The first question is whether the risk algorithm is sufficiently stable to predict future suicides given that it is based on only 68 prior suicides. It is noteworthy that the machine learning methods used to create the algorithm were designed explicitly to maximize stability of predictions. Within-sample stability analyses found that the CR did not vary significantly by type of inpatient facility, year of hospitalization, or number of months since hospital discharge; however, this does not guarantee future stability. Algorithm stability will consequently be tested again in the 2010-2013 US Army suicide data in a future study to address this question.

The second question is whether the risk algorithm improves on clinical judgment. The study was unable to examine this issue empirically because the US Army electronic medical record does not include a structured field where health care professionals must record suicide risk assessments. In addition, documentation of suicide risk assessment in clinical notes was not consistent during the study period. However, with improved documentation after the VA and DoD CPG, comparison of actuarial to clinical prediction may be possible in the future. As noted in the Introduction, though, previous research has indicated that actuarial suicide prediction is much more accurate than prediction based on clinical judgment.1114 This evidence is consistent with a large body of literature reporting that actuarial methods are superior to expert judgments in many areas of prediction.45,46 At the same time, the comprehensive suicide risk assessments required by the new VA and DoD CPG10 will generate information not included in administrative records. As a result, our algorithm should be seen as a component of this comprehensive clinical assessment rather than a substitute for this assessment.

The third question is whether suicide is sufficiently common in the highest-risk stratum and available interventions sufficiently powerful to make targeted posthospitalization interventions efficient compared with alternative ways of deploying the same clinical resources. Our results shed no light on this question. The potential for harm also has to be taken into consideration because intensive posthospitalization interventions might lead to undue scrutiny by nonmedical leaders that adversely affect soldier careers. This concern is all the more important given that most soldiers identified as being high risk do not commit suicide. Although a formal analysis of comparative risks and benefits is beyond the scope of this report, it is noteworthy that the highest-risk stratum had significantly elevated risks of other adverse outcomes and that prevalence of at least one such outcome was present after 46.3% of highest-risk hospitalizations. Ameliorative effects of expanded high-risk interventions on these outcomes (ie, unintentional injury deaths, suicide attempts, and subsequent hospitalizations) are plausible because numerous risk factors for suicide (eg, depression and substance abuse) are also risk factors for these other outcomes2,47,48 and most suicide prevention interventions recommended for high-risk patients are likely to affect these outcomes as well.7,10 These presumed benefits would have to be considered in a broad-based evaluation of risks and benefits of any future targeted high-risk posthospitalization preventive interventions.

The major limitations of our analysis involve errors in the administrative data used as predictors (missing and inconsistent values and errors in ICD-9-CM diagnoses). In addition, the algorithm could almost certainly be improved if more nuanced risk factor data were available. Because the new VA and DoD CPG contains a checklist of risk factors health care professionals are urged to assess in evaluating suicide risk, creation of a system to record these assessments in the electronic medical record along with the health care professional’s clinical global impression of patient suicide risk might increase the completeness of these assessments and provide a rich source of information for future risk algorithm refinement.

The high concentration of risk of suicides and other adverse outcomes might justify targeting expanded posthospitalization interventions to soldiers classified as having highest posthospitalization suicide risk, although final determination requires careful consideration of intervention costs, comparative effectiveness, and possible adverse effects.

Submitted for Publication: January 29, 2014; final revision received June 12, 2014; accepted July 21, 2014.

Corresponding Author: Ronald C. Kessler, PhD, Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Ste 215, Boston, MA 02115-5899 (kessler@hcp.med.harvard.edu).

Published Online: November 12, 2014. doi:10.1001/jamapsychiatry.2014.1754.

Author Contributions: Dr Kessler had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Kessler, Cai, Colpe, Fullerton, Gilman, Heeringa, Naifeh, Rosellini, Schoenbaum, Wessely, Ursano.

Acquisition, analysis, or interpretation of data: Kessler, Warner, Ivany, Petukhova, Rose, Bromet, Brown, Cox, Gruber, Heeringa, Lewandowski-Romps, Li, Millikan-Bell, Nock, Rosellini, Sampson, Schoenbaum, Stein, Zaslavsky, Ursano.

Drafting of the manuscript: Kessler, Rosellini, Schoenbaum, Ursano.

Critical revision of the manuscript for important intellectual content: Kessler, Warner, Ivany, Petukhova, Rose, Bromet, Brown, Cai, Colpe, Cox, Fullerton, Gilman, Gruber, Heeringa, Lewandowski-Romps, Li, Millikan-Bell, Naifeh, Nock, Rosellini, Sampson, Stein, Wessely, Zaslavsky, Ursano.

Statistical analysis: Kessler, Ivany, Petukhova, Rose, Cai, Gruber, Heeringa, Li, Rosellini, Sampson, Schoenbaum, Zaslavsky.

Obtained funding: Kessler, Ursano.

Administrative, technical, or material support: Kessler, Brown, Colpe, Fullerton, Heeringa, Lewandowski-Romps, Millikan-Bell, Naifeh, Nock, Rosellini, Ursano.

Study supervision: Kessler, Bromet, Sampson, Schoenbaum, Stein, Wessely, Ursano.

Conflict of Interest Disclosures: Dr Kessler reported being a consultant for AstraZeneca, Analysis Group, Bristol-Myers Squibb, Cerner-Galt Associates, Eli Lilly & Company, GlaxoSmithKline Inc, HealthCore Inc, Health Dialog, Hoffman-LaRoche Inc, Integrated Benefits Institute, J & J Wellness & Prevention Inc, John Snow Inc, Kaiser Permanente, Lake Nona Institute, Matria Inc, Mensante, Merck & Co Inc, Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc, Primary Care Network, Research Triangle Institute, Sanofi-Aventis Groupe SA, Shire US Inc, SRA International Inc, Takeda Global Research & Development, Transcept Pharmaceuticals Inc, and Wyeth-Ayerst. Dr Kessler reported serving on advisory boards for Appliance Computing II, Eli Lilly & Company, Mindsite, Ortho-McNeil Janssen Scientific Affairs, Johnson & Johnson, Plus One Health Management, and Wyeth-Ayerst and receiving research support for his epidemiologic studies from Analysis Group Inc, Bristol-Myers Squibb, Eli Lilly & Company, EPI-Q, GlaxoSmithKline, Johnson & Johnson Pharmaceuticals, Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc, Sanofi-Aventis Groupe SA, Shire US Inc, and Walgreens Co. Dr Kessler reported owing a 25% share in DataStat Inc. Dr Stein reported being a consultant for Healthcare Management Technologies and receiving research support for pharmacologic imaging studies from Janssen. No other disclosures were reported.

Funding/Support: The Army STARRS was sponsored by the US Department of the Army and funded under cooperative agreement U01MH087981 with the National Institute of Mental Health, National Institutes of Health, US Department of Health and Human Services.

Role of the Funder/Sponsor: As a cooperative agreement, scientists employed by the National Institute of Mental Health (Drs Colpe and Schoenbaum) and US Army liaisons and consultants (Dr Cox and Steven Cersovsky, MD, MPH) collaborated to develop the study protocol and data collection instruments, supervise data collection, interpret results, and prepare reports. Although a draft of the manuscript was submitted to the US Army and the National Institute of Mental Health for review and comment before submission, this was with the understanding that comments would be only advisory.

Group Information: The Army STARRS coprincipal investigators were Robert J. Ursano, MD (Uniformed Services University of the Health Sciences), and Murray B. Stein, MD, MPH (University of California, San Diego, and Veterans Affairs San Diego Healthcare System). The site principal investigators were Steven G. Heeringa, PhD (University of Michigan), and Ronald C. Kessler, PhD (Harvard Medical School). The National Institute of Mental Health collaborating scientists were Lisa J. Colpe, PhD, MPH, and Michael Schoenbaum, PhD. The US Army liaisons and consultants were Steven Cersovsky, MD, MPH, and Kenneth L. Cox, MD, MPH.

Other team members were Pablo A. Aliaga, MA (Uniformed Services University of the Health Sciences), David M. Benedek, MD (Uniformed Services University of the Health Sciences), Susan Borja, PhD (National Institute of Mental Health), Gregory G. Brown, PhD (University of California, San Diego), Laura Campbell-Sills, PhD (University of California, San Diego), Catherine L. Dempsey, PhD, MPH (Uniformed Services University of the Health Sciences), Richard Frank, PhD (Harvard Medical School), Carol S. Fullerton, PhD (Uniformed Services University of the Health Sciences), Nancy Gebler, MA (University of Michigan), Robert K. Gifford, PhD (Uniformed Services University of the Health Sciences), Stephen E. Gilman, ScD (Harvard School of Public Health), Marjan G. Holloway, PhD (Uniformed Services University of the Health Sciences), Paul E. Hurwitz, MPH (Uniformed Services University of the Health Sciences), Sonia Jain, PhD (University of California, San Diego), Tzu-Cheg Kao, PhD (Uniformed Services University of the Health Sciences), Karestan C. Koenen, PhD (Columbia University), Lisa Lewandowski-Romps, PhD (University of Michigan), Holly Herberman Mash, PhD (Uniformed Services University of the Health Sciences), James E. McCarroll, PhD, MPH (Uniformed Services University of the Health Sciences), Katie A. McLaughlin, PhD (Harvard Medical School), James A. Naifeh, PhD (Uniformed Services University of the Health Sciences), Matthew K. Nock, PhD (Harvard University), Rema Raman, PhD (University of California, San Diego), Sherri Rose, PhD (Harvard Medical School), Anthony Joseph Rosellini, PhD (Harvard Medical School), Nancy A. Sampson, BA (Harvard Medical School), LCDR Patcho Santiago, MD, MPH (Uniformed Services University of the Health Sciences), Michaelle Scanlon, MBA (National Institute of Mental Health), Jordan Smoller, MD, ScD (Harvard Medical School), Michael L. Thomas, PhD (University of California, San Diego), Patti L. Vegella, MS, MA (Uniformed Services University of the Health Sciences), Christina Wassel, PhD (University of Pittsburgh), and Alan M. Zaslavsky, PhD (Harvard Medical School).

Disclaimer: The contents are solely the responsibility of the authors and do not necessarily represent the views of the US Department of Health and Human Services, the National Institute of Mental Health, the US Department of the Army, or the US Department of Defense.

Additional Contributions: John Mann, MD, Maria Oquendo, MD, Barbara Stanley, PhD, Kelly Posner, PhD, and John Keilp, PhD, Department of Psychiatry, Columbia University, College of Physicians and Surgeons, and New York State Psychiatric Institute, New York, contributed to the early stages of the US Army STARRS development.

Armed Forces Health Surveillance Center.  Deaths by suicide while on active duty, active and reserve components, US Armed Forces, 1998-2011. Med Surveill Monthly Rep.2012;19(6):7-10.
Nock  MK, Deming  CA, Fullerton  CS,  et al.  Suicide among soldiers: a review of psychosocial risk and protective factors. Psychiatry. 2013;76(2):97-125.
PubMed
Zamorski  MA.  Suicide prevention in military organizations. Int Rev Psychiatry. 2011;23(2):173-180.
PubMed   |  Link to Article
Valenstein  M, Kim  HM, Ganoczy  D,  et al.  Higher-risk periods for suicide among VA patients receiving depression treatment: prioritizing suicide prevention efforts. J Affect Disord. 2009;112(1-3):50-58.
PubMed   |  Link to Article
Luxton  DD, Trofimovich  L, Clark  LL.  Suicide risk among US Service members after psychiatric hospitalization, 2001-2011. Psychiatr Serv. 2013;64(7):626-629.
PubMed   |  Link to Article
Olfson  M, Marcus  SC, Bridge  JA.  Focusing suicide prevention on periods of high risk. JAMA. 2014;311(11):1107-1108.
PubMed   |  Link to Article
While  D, Bickley  H, Roscoe  A,  et al.  Implementation of mental health service recommendations in England and Wales and suicide rates, 1997-2006: a cross-sectional and before-and-after observational study. Lancet. 2012;379(9820):1005-1012.
PubMed   |  Link to Article
Paton  MB, Large  MM, Ryan  CJ.  Debate: clinical risk categorisation is valuable in the prevention of suicide and severe violence–no. Australas Psychiatry. 2014;22(1):10-12.
PubMed   |  Link to Article
Berrouiguet  S, Gravey  M, Le Galudec  M, Alavi  Z, Walter  M.  Post-acute crisis text messaging outreach for suicide prevention: a pilot study. Psychiatry Res. 2014;217(3):154-157.
PubMed   |  Link to Article
US Department of Veterans Affairs and US Department of Defense. Assessment and Management of Patients at Risk for Suicide. Washington, DC: US Dept of Veterans Affairs and US Dept of Defense; 2013.
Erdman  HP, Greist  JH, Gustafson  DH, Taves  JE, Klein  MH.  Suicide risk prediction by computer interview: a prospective study. J Clin Psychiatry. 1987;48(12):464-467.
PubMed
Gustafson  DH, Greist  JH, Stauss  FF, Erdman  H, Laughren  T.  A probabilistic system for identifying suicide attemptors. Comput Biomed Res. 1977;10(2):83-89.
PubMed   |  Link to Article
Gustafson  DH, Tianen  B, Greist  JH.  A computer-based system for identifying suicide attemptors. Comput Biomed Res. 1981;14(2):144-157.
PubMed   |  Link to Article
Nock  MK, Park  JM, Finn  CT, Deliberto  TL, Dour  HJ, Banaji  MR.  Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychol Sci. 2010;21(4):511-517.
PubMed   |  Link to Article
Bright  TJ, Wong  A, Dhurjati  R,  et al.  Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29-43.
PubMed   |  Link to Article
Garg  AX, Adhikari  NK, McDonald  H,  et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-1238.
PubMed   |  Link to Article
Large  M, Sharma  S, Cannon  E, Ryan  C, Nielssen  O.  Risk factors for suicide within a year of discharge from psychiatric hospital: a systematic meta-analysis. Aust N Z J Psychiatry. 2011;45(8):619-628.
PubMed   |  Link to Article
Weber  GM, Mandl  KD, Kohane  IS.  Finding the missing link for big biomedical data. JAMA. 2014;311(24):2479-2480.
PubMed
Bachynski  KE, Canham-Chervak  M, Black  SA, Dada  EO, Millikan  AM, Jones  BH.  Mental health risk factors for suicides in the US Army, 2007-8. Inj Prev. 2012;18(6):405-412.
PubMed   |  Link to Article
Bell  NS, Harford  TC, Amoroso  PJ, Hollander  IE, Kay  AB.  Prior health care utilization patterns and suicide among U.S. Army soldiers. Suicide Life Threat Behav. 2010;40(4):407-415.
PubMed   |  Link to Article
Black  SA, Gallaway  MS, Bell  MR, Ritchie  EC.  Prevalence and risk factors associated with suicides of Army soldiers 2001–2009. Mil Psychol. 2011;23(4):433-451.
Link to Article
Hyman  J, Ireland  R, Frost  L, Cottrell  L.  Suicide incidence and risk factors in an active duty US military population. Am J Public Health. 2012;102(suppl 1):S138-S146.
PubMed   |  Link to Article
Schoenbaum  M, Kessler  RC, Gilman  SE,  et al; Army STARRS Collaborators.  Predictors of suicide and accident death in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS): results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry. 2014;71(5):493-503.
PubMed   |  Link to Article
Ursano  RJ, Colpe  LJ, Heeringa  SG, Kessler  RC, Schoenbaum  M, Stein  MB; Army STARRS collaborators.  The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Psychiatry. 2014;77(2):107-119.
PubMed
Pirkola  S, Sohlman  B, Wahlbeck  K.  The characteristics of suicides within a week of discharge after psychiatric hospitalisation: a nationwide register study. BMC Psychiatry. 2005;5:32.
PubMed   |  Link to Article
Kessler  RC, Colpe  LJ, Fullerton  CS,  et al.  Design of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Int J Methods Psychiatr Res. 2013;22(4):267-275.
PubMed   |  Link to Article
Troister  T, Links  PS, Cutcliffe  J.  Review of predictors of suicide within 1 year of discharge from a psychiatric hospital. Curr Psychiatry Rep. 2008;10(1):60-65.
PubMed   |  Link to Article
Bickley  H, Hunt  IM, Windfuhr  K, Shaw  J, Appleby  L, Kapur  N.  Suicide within two weeks of discharge from psychiatric inpatient care: a case-control study. Psychiatr Serv. 2013;64(7):653-659.
PubMed   |  Link to Article
Park  S, Choi  JW, Kyoung Yi  K, Hong  JP.  Suicide mortality and risk factors in the 12 months after discharge from psychiatric inpatient care in Korea: 1989-2006. Psychiatry Res. 2013;208(2):145-150.
PubMed   |  Link to Article
Rubin  DB. Introduction. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons; 2008:1-26.
Efron  B.  Logistic regression, survival analysis, and the Kaplan-Meier curve. J Am Stat Assoc. 1988;83(402):414-425.
Link to Article
Thernau  T, Atkinson  B, Ripley  B. Rpart: Recursive Partitioning. R Package 4.1-0. http://CRAN.R-project.org/package=rpart. Accessed December 15, 2013.
Zhang  H, Singer  BH. Recursive Partitioning and Applications.2nd ed. New York, NY: Springer; 2010.
Draper  NR, Smith  H. Applied Regression Analysis.2nd ed. Hoboken, NJ: John Wiley & Sons; 1981.
Berk  RA. Regression Analysis: A Constructive Critique.Vol 11. New York, NY: Sage; 2004.
Bellazzi  R, Zupan  B.  Towards knowledge-based gene expression data mining. J Biomed Inform. 2007;40(6):787-802.
PubMed   |  Link to Article
van der Laan  MJ, Rose  S. Targeted Learning: Causal Inference for Observational and Experimental Data. New York, NY: Springer; 2011.
Zou  H, Hastie  T.  Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol. 2005;67(suppl):301-320.
Link to Article
Berk  RA. Statistical Learning From a Regression Perspective. New York, NY: Springer; 2008.
Hoerl  AE, Kennard  RW.  Ridge regression: biased estimation for nonorthogonal problems. Technometrics. 1970;12(1):55-67.
Link to Article
Tibshirani  R.  Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol. 1996;58:267-288.
Friedman  J, Hastie  T, Tibshirani  R.  Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):1-22.
PubMed
Schmitz  N, Kruse  J, Tress  W.  Application of stratum-specific likelihood ratios in mental health screening. Soc Psychiatry Psychiatr Epidemiol. 2000;35(8):375-379.
PubMed   |  Link to Article
Stine  RA.  Graphical interpretation of variance inflation factors. Am Stat. 1995;49(1):53-56.
Dawes  RM, Faust  D, Meehl  PE.  Clinical versus actuarial judgment. Science. 1989;243(4899):1668-1674.
PubMed   |  Link to Article
Grove  WM, Zald  DH, Lebow  BS, Snitz  BE, Nelson  C.  Clinical versus mechanical prediction: a meta-analysis. Psychol Assess. 2000;12(1):19-30.
PubMed   |  Link to Article
Victor  SE, Klonsky  ED.  Correlates of suicide attempts among self-injurers: a meta-analysis. Clin Psychol Rev. 2014;34(4):282-297.
PubMed   |  Link to Article
Scott-Parker  B, Watson  B, King  MJ, Hyde  MK.  The influence of sensitivity to reward and punishment, propensity for sensation seeking, depression, and anxiety on the risky behaviour of novice drivers: a path model. Br J Psychol. 2012;103(2):248-267.
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Figure 1.
Receiver Operating Characteristic (ROC) Curves for Discrete-Time (Person-Month) Elastic Net Penalized Survival Models With Different Mixing Parameter Penalties (MPPs) and for a Conventional Discrete-Time Survival Model Predicting Posthospitalization Suicide

Elastic net penalized survival models were estimated with different MPPs, allowing up to 421 predictors. The best cross-validated model was an MPP of 1.0 with 73 predictors. A conventional discrete-time survival model that contained the same 73 predictors was unstable (variance inflation factor >5.0 for 6 predictors). As a result, we used forward stepwise analysis with a .05-level entry criterion to select a more stable subset of the 73 predictors. Twenty predictors entered that model. The ROC curve shown here for the conventional model is based on those 20 predictors. AUC indicates area under the receiver operating characteristic curve.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Concentration of Risk of Posthospitalization Suicides by Ventile of Predicted Risk Based on the Discrete-Time Penalized Survival Model With a Mixing Parameter Penalty of 1.0

Ventiles are 20 groups of hospitalizations of equal frequency (2688 or 2689 hospitalizations), dividing the total sample of 53 769 hospitalizations into groups defined by level of predicted suicide risk.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable 1.  CR, AUC, and Np Values by Mixing Parameter Penaltya
Table Graphic Jump LocationTable 2.  ORs (95% CIs) and VIFs for the Discrete-Time Logistic Survival Modela
Table Graphic Jump LocationTable 3.  CR and Conditional Risk of Posthospitalization Suicides by Risk Strata Across All Hospitalizations

References

Armed Forces Health Surveillance Center.  Deaths by suicide while on active duty, active and reserve components, US Armed Forces, 1998-2011. Med Surveill Monthly Rep.2012;19(6):7-10.
Nock  MK, Deming  CA, Fullerton  CS,  et al.  Suicide among soldiers: a review of psychosocial risk and protective factors. Psychiatry. 2013;76(2):97-125.
PubMed
Zamorski  MA.  Suicide prevention in military organizations. Int Rev Psychiatry. 2011;23(2):173-180.
PubMed   |  Link to Article
Valenstein  M, Kim  HM, Ganoczy  D,  et al.  Higher-risk periods for suicide among VA patients receiving depression treatment: prioritizing suicide prevention efforts. J Affect Disord. 2009;112(1-3):50-58.
PubMed   |  Link to Article
Luxton  DD, Trofimovich  L, Clark  LL.  Suicide risk among US Service members after psychiatric hospitalization, 2001-2011. Psychiatr Serv. 2013;64(7):626-629.
PubMed   |  Link to Article
Olfson  M, Marcus  SC, Bridge  JA.  Focusing suicide prevention on periods of high risk. JAMA. 2014;311(11):1107-1108.
PubMed   |  Link to Article
While  D, Bickley  H, Roscoe  A,  et al.  Implementation of mental health service recommendations in England and Wales and suicide rates, 1997-2006: a cross-sectional and before-and-after observational study. Lancet. 2012;379(9820):1005-1012.
PubMed   |  Link to Article
Paton  MB, Large  MM, Ryan  CJ.  Debate: clinical risk categorisation is valuable in the prevention of suicide and severe violence–no. Australas Psychiatry. 2014;22(1):10-12.
PubMed   |  Link to Article
Berrouiguet  S, Gravey  M, Le Galudec  M, Alavi  Z, Walter  M.  Post-acute crisis text messaging outreach for suicide prevention: a pilot study. Psychiatry Res. 2014;217(3):154-157.
PubMed   |  Link to Article
US Department of Veterans Affairs and US Department of Defense. Assessment and Management of Patients at Risk for Suicide. Washington, DC: US Dept of Veterans Affairs and US Dept of Defense; 2013.
Erdman  HP, Greist  JH, Gustafson  DH, Taves  JE, Klein  MH.  Suicide risk prediction by computer interview: a prospective study. J Clin Psychiatry. 1987;48(12):464-467.
PubMed
Gustafson  DH, Greist  JH, Stauss  FF, Erdman  H, Laughren  T.  A probabilistic system for identifying suicide attemptors. Comput Biomed Res. 1977;10(2):83-89.
PubMed   |  Link to Article
Gustafson  DH, Tianen  B, Greist  JH.  A computer-based system for identifying suicide attemptors. Comput Biomed Res. 1981;14(2):144-157.
PubMed   |  Link to Article
Nock  MK, Park  JM, Finn  CT, Deliberto  TL, Dour  HJ, Banaji  MR.  Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychol Sci. 2010;21(4):511-517.
PubMed   |  Link to Article
Bright  TJ, Wong  A, Dhurjati  R,  et al.  Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29-43.
PubMed   |  Link to Article
Garg  AX, Adhikari  NK, McDonald  H,  et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-1238.
PubMed   |  Link to Article
Large  M, Sharma  S, Cannon  E, Ryan  C, Nielssen  O.  Risk factors for suicide within a year of discharge from psychiatric hospital: a systematic meta-analysis. Aust N Z J Psychiatry. 2011;45(8):619-628.
PubMed   |  Link to Article
Weber  GM, Mandl  KD, Kohane  IS.  Finding the missing link for big biomedical data. JAMA. 2014;311(24):2479-2480.
PubMed
Bachynski  KE, Canham-Chervak  M, Black  SA, Dada  EO, Millikan  AM, Jones  BH.  Mental health risk factors for suicides in the US Army, 2007-8. Inj Prev. 2012;18(6):405-412.
PubMed   |  Link to Article
Bell  NS, Harford  TC, Amoroso  PJ, Hollander  IE, Kay  AB.  Prior health care utilization patterns and suicide among U.S. Army soldiers. Suicide Life Threat Behav. 2010;40(4):407-415.
PubMed   |  Link to Article
Black  SA, Gallaway  MS, Bell  MR, Ritchie  EC.  Prevalence and risk factors associated with suicides of Army soldiers 2001–2009. Mil Psychol. 2011;23(4):433-451.
Link to Article
Hyman  J, Ireland  R, Frost  L, Cottrell  L.  Suicide incidence and risk factors in an active duty US military population. Am J Public Health. 2012;102(suppl 1):S138-S146.
PubMed   |  Link to Article
Schoenbaum  M, Kessler  RC, Gilman  SE,  et al; Army STARRS Collaborators.  Predictors of suicide and accident death in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS): results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry. 2014;71(5):493-503.
PubMed   |  Link to Article
Ursano  RJ, Colpe  LJ, Heeringa  SG, Kessler  RC, Schoenbaum  M, Stein  MB; Army STARRS collaborators.  The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Psychiatry. 2014;77(2):107-119.
PubMed
Pirkola  S, Sohlman  B, Wahlbeck  K.  The characteristics of suicides within a week of discharge after psychiatric hospitalisation: a nationwide register study. BMC Psychiatry. 2005;5:32.
PubMed   |  Link to Article
Kessler  RC, Colpe  LJ, Fullerton  CS,  et al.  Design of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Int J Methods Psychiatr Res. 2013;22(4):267-275.
PubMed   |  Link to Article
Troister  T, Links  PS, Cutcliffe  J.  Review of predictors of suicide within 1 year of discharge from a psychiatric hospital. Curr Psychiatry Rep. 2008;10(1):60-65.
PubMed   |  Link to Article
Bickley  H, Hunt  IM, Windfuhr  K, Shaw  J, Appleby  L, Kapur  N.  Suicide within two weeks of discharge from psychiatric inpatient care: a case-control study. Psychiatr Serv. 2013;64(7):653-659.
PubMed   |  Link to Article
Park  S, Choi  JW, Kyoung Yi  K, Hong  JP.  Suicide mortality and risk factors in the 12 months after discharge from psychiatric inpatient care in Korea: 1989-2006. Psychiatry Res. 2013;208(2):145-150.
PubMed   |  Link to Article
Rubin  DB. Introduction. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons; 2008:1-26.
Efron  B.  Logistic regression, survival analysis, and the Kaplan-Meier curve. J Am Stat Assoc. 1988;83(402):414-425.
Link to Article
Thernau  T, Atkinson  B, Ripley  B. Rpart: Recursive Partitioning. R Package 4.1-0. http://CRAN.R-project.org/package=rpart. Accessed December 15, 2013.
Zhang  H, Singer  BH. Recursive Partitioning and Applications.2nd ed. New York, NY: Springer; 2010.
Draper  NR, Smith  H. Applied Regression Analysis.2nd ed. Hoboken, NJ: John Wiley & Sons; 1981.
Berk  RA. Regression Analysis: A Constructive Critique.Vol 11. New York, NY: Sage; 2004.
Bellazzi  R, Zupan  B.  Towards knowledge-based gene expression data mining. J Biomed Inform. 2007;40(6):787-802.
PubMed   |  Link to Article
van der Laan  MJ, Rose  S. Targeted Learning: Causal Inference for Observational and Experimental Data. New York, NY: Springer; 2011.
Zou  H, Hastie  T.  Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol. 2005;67(suppl):301-320.
Link to Article
Berk  RA. Statistical Learning From a Regression Perspective. New York, NY: Springer; 2008.
Hoerl  AE, Kennard  RW.  Ridge regression: biased estimation for nonorthogonal problems. Technometrics. 1970;12(1):55-67.
Link to Article
Tibshirani  R.  Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol. 1996;58:267-288.
Friedman  J, Hastie  T, Tibshirani  R.  Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):1-22.
PubMed
Schmitz  N, Kruse  J, Tress  W.  Application of stratum-specific likelihood ratios in mental health screening. Soc Psychiatry Psychiatr Epidemiol. 2000;35(8):375-379.
PubMed   |  Link to Article
Stine  RA.  Graphical interpretation of variance inflation factors. Am Stat. 1995;49(1):53-56.
Dawes  RM, Faust  D, Meehl  PE.  Clinical versus actuarial judgment. Science. 1989;243(4899):1668-1674.
PubMed   |  Link to Article
Grove  WM, Zald  DH, Lebow  BS, Snitz  BE, Nelson  C.  Clinical versus mechanical prediction: a meta-analysis. Psychol Assess. 2000;12(1):19-30.
PubMed   |  Link to Article
Victor  SE, Klonsky  ED.  Correlates of suicide attempts among self-injurers: a meta-analysis. Clin Psychol Rev. 2014;34(4):282-297.
PubMed   |  Link to Article
Scott-Parker  B, Watson  B, King  MJ, Hyde  MK.  The influence of sensitivity to reward and punishment, propensity for sensation seeking, depression, and anxiety on the risky behaviour of novice drivers: a path model. Br J Psychol. 2012;103(2):248-267.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

6,301 Views
13 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Multimedia

Author Interview

audio player

Articles Related By Topic
Related Collections
PubMed Articles
Jobs