0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Reward Signals, Attempted Suicide, and Impulsivity in Late-Life Depression FREE

Alexandre Y. Dombrovski, MD1; Katalin Szanto, MD1; Luke Clark, DPhil2; Charles F. Reynolds III, MD1; Greg J. Siegle, PhD1
[+] Author Affiliations
1Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania
2Department of Experimental Psychology, University of Cambridge, Cambridge, England
JAMA Psychiatry. 2013;70(10):1020-1030. doi:10.1001/jamapsychiatry.2013.75.
Text Size: A A A
Published online

Importance  Suicide can be viewed as an escape from unendurable punishment at the cost of any future rewards. Could faulty estimation of these outcomes predispose to suicidal behavior? In behavioral studies, many of those who have attempted suicide misestimate expected rewards on gambling and probabilistic learning tasks.

Objectives  To describe the neural circuit abnormalities that underlie disadvantageous choices in people at risk for suicide and to relate these abnormalities to impulsivity, which is one of the components of vulnerability to suicide.

Design  Case-control functional magnetic resonance imaging study of reward learning using a reinforcement learning model.

Setting  University hospital and outpatient clinic.

Patients  Fifty-three participants 60 years or older, including 15 depressed patients who had attempted suicide, 18 depressed patients who had never attempted suicide (depressed control subjects), and 20 psychiatrically healthy controls.

Main Outcomes and Measures  Components of the cortical blood oxygenation level–dependent response tracking expected and unpredicted rewards.

Results  Depressed elderly participants displayed 2 distinct disruptions of control over reward-guided behavior. First, impulsivity and a history of suicide attempts (particularly poorly planned ones) were associated with a weakened expected reward signal in the paralimbic cortex, which in turn predicted the behavioral insensitivity to contingency change. Second, depression was associated with disrupted corticostriatothalamic encoding of unpredicted rewards, which in turn predicted the behavioral oversensitivity to punishment. These results were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular illness, and incipient dementia.

Conclusions and Relevance  Altered paralimbic reward signals and impulsivity and/or carelessness may facilitate unplanned suicidal acts. This pattern, also seen in gambling and cocaine use, may reflect a primary deficit in the paralimbic cortex or in its mesolimbic input. The overreactivity to punishment in depression may be caused in part by a disruption of appetitive learning in the corticostriatothalamic circuits.

Figures in this Article

What causes a person to commit suicide is one of the central questions in psychiatry. We know that stressors precipitate suicidal behavior, particularly in the pathological states of depression, psychosis, intoxication, and pain. However, because only a minority of people facing these circumstances attempt suicide, the individual diathesis must play an important role. This individual diathesis is still not well understood. However, impulsive-aggressive traits,16 cognitive deficits,713 and persisting hopelessness1416 have emerged as important dimensions. Suicidal behavior is heterogeneous, and even less is known about the many pathways that lead to it. Suicidal acts follow a decision, and we argue that a propensity to make bad decisions may be part of one of the pathways to suicidal behavior. This view is supported by at least 3 converging lines of evidence. The first line is the association of suicidal behavior with problem gambling17 and drug addiction,18 behaviors defined by disadvantageous choice. In the second line, several although not all studies find that significant subgroups of individuals with a history of suicide attempts lose on gambling tasks, fail to learn from the experience of rewards and punishments,11,1922 and make short-sighted choices.23 In the third line, suicidal behavior has been linked to disruptions of signaling pathways in the basal ganglia,2430 disruptions that severely and somewhat selectively impair decision-making processes.3135 Akin to the analysis of suicide as an escape from self by Baumeister36 and the entrapment theory of Williams et al,37 one can view suicidal behavior as instrumental, aimed at achieving the subjectively preferred outcome in a desperate situation. Our interpretation of the evidence reviewed above is that at least some people who engage in suicidal behavior tend to misestimate future outcomes and see suicide as unrealistically attractive relative to other options. This perspective is supported by our behavioral findings of disrupted instrumental learning in older depressed people who attempt suicide (suicide attempters).38,39 To extend this evidence, we examined whether older depressed suicide attempters demonstrate disruptions in brain signals involved in reward prediction. We also considered whether impulsivity, a key dimension of the suicidal diathesis, exerts effects on these signals.

We focused on older adults because the rate of suicide is high in the elderly, and we believe that the age-dependent decline in decision competence40,41 puts this group at special risk. We view learning from rewards and punishments in a changing, ambiguous environment as a model of behavioral adaptation to change and uncertainty that accompanies a suicidal crisis. Consistent with this idea, a previous study38 observed a complete breakdown of reward learning in older depressed suicide attempters, with some suicide attempters overreacting to punishments and others displaying insensitivity to a contingency change. Older depressed suicide attempters also failed to grasp the changing contingency in an environment with no misleading feedback, the Wisconsin Card Sorting Task.39 This failure to grasp a changing contingency resembles the behavior of animals and humans with lesions of the ventromedial and ventrolateral prefrontal cortex (vmPFC and vlPFC).4244 To test whether the activity of these areas was disrupted during reward learning in older depressed suicide attempters, we conducted a functional magnetic resonance imaging (fMRI) study with participants facing uncertain and changing rewards and punishments on a probabilistic reversal learning task. To obtain more accurate and interpretable predictions of neural activity, we used a cognitive modeling approach, reinforcement learning, to estimate 2 key reward signals postulated by formal learning theory. The first signal is expected reward, and the second is the discrepancy between the experienced outcome and the prior expectation, or the prediction error.

We were further interested in how these neural signals are affected by impulsivity, a tendency strongly linked to suicidal behavior. We hypothesized that a history of suicide attempts and impulsivity would be associated with weaker expected reward signals in the vmPFC on the basis of studies mapping value representations to the vmPFC4550 and positron emission tomography findings of vmPFC alterations in suicide attempters51,52 and individuals with maladaptive impulsive behaviors.53

Overreactivity to negative feedback is typical of depression5458 and was very prominent among acutely depressed older suicide attempters in our behavioral study.38 On the probabilistic reversal learning task, depressed patients tend to switch response after a misleading punishment,58 as if forgetting the preceding history of rewards. According to learning theory,59,60 such responses have to result from a disproportionately big effect of negative prediction errors on choice. This effect could be caused by incorrect encoding of negative or preceding positive prediction errors or improper maintenance of expected reward representations. This reasoning led us to focus on the neural representation of prediction errors as a correlate of behavioral overreactivity to punishments in depression. A prior study61 has found altered error-related activity in the vlPFC of depressed individuals who overreacted to negative feedback during probabilistic reversal learning, although the study did not estimate prediction error signals. Studies using other reward-learning paradigms have found blunted prediction errors in the cingulate gyrus, striatum, and midbrain of depressed patients.62,63 However, these abnormalities were not linked to any particular behavioral tendency.

Participants

From March 28, 2008, through November 15, 2011, we recruited 53 participants 60 years or older, including 31 with major depression (15 suicide attempters and 18 with no lifetime history of suicidal behavior or ideation; we hereinafter refer to these 31 participants as depressed) and 20 psychiatrically healthy control subjects. Major depression was diagnosed by the Structured Clinical Interview for DSM-IV Axis I Disorders64,65 (SCID). Unlike participants in the previous behavioral study,38 all of whom had an acute depressive episode, participants in this study had different levels of depressive symptoms, from severe to partial remission (Table). To exclude individuals with clinical dementia and to ensure that participants could engage in the task, all were required to have a score of at least 24 on the Mini-Mental State Examination.66 Our reported sample excludes elderly individuals with sensory disorders that precluded cognitive testing, limited English, mental retardation, delirium, neurologic disorders, bipolar disorder, schizophrenia, schizoaffective disorder, exposure to electroconvulsive therapy in the previous 6 months, and circumstances precluding an fMRI assessment (eg, claustrophobia, metal implants [n = 7]). Participant flow is described in more detail in the material available on the authors’ website (https://db.tt/PHaUDMEs).

Table Graphic Jump LocationTable.  Demographic, Clinical, and Cognitive Characteristics

All participants provided written informed consent. The University of Pittsburgh institutional review board approved the study.

Suicide attempters had engaged in a self-injurious act with an intent to die; 9 of 15 made their first suicide attempt after 50 years of age and 6 of them after 60 years of age. Fourteen of the 15 were recruited as inpatients, all of whom were admitted out of concern about suicide risk. These participants displayed a high level of suicidal intent (mean [SD] Suicide Intent Scale score, 18.1 [4.6]) during their attempts and severe suicidal ideation during the index episode (Scale for Suicide Ideation score, 23.9 [6.9]); 7 of 15 made repeated attempts. A history of suicide attempts was verified by a study psychiatrist (A.Y.D. or K.S.) based on the interview, medical records, and information from the treatment team and from family or friends. We excluded participants with significant discrepancies between these sources. None of the suicide attempters had experienced head injuries directly related to the attempt; however, we assessed potential anoxic-ischemic or toxic brain injury based on the Beck Lethality Scale,67 medical records, and the clinical interview. A study psychiatrist (A.Y.D.) identified any attempts with a score greater than 4 on the Beck Lethality Scale and any history of systemic hypotension of longer than 5 minutes, asphyxia, or neurotoxic ingestion in 3 of 15 participants. For 1 additional participant, we could not rule out brain injury during past attempts. Thus, we excluded these 4 participants in sensitivity analyses.

Nonsuicidal depressed elderly were included in the study to detect an association between decision making and suicidal behavior beyond the cognitive effects of depression. These participants had no current or lifetime history of suicide attempts or suicidal ideation as established by clinical interview, review of medical records, SCID, and the Scale for Suicidal Ideation (lifetime). One of the 18 nondepressed participants was recruited as an inpatient. Participants were excluded from this group if they had a current passive death wish or a history of indirect self-destructive behaviors.

Nondepressed controls were included as the benchmark group. They had no lifetime history of any psychiatric disorder as determined by SCID.

Clinical and cognitive assessments used to characterize the study groups and the 3-T MRI scanning parameters are described in detail on the authors’ website.

Probabilistic Reversal Learning

On each of the 300 trials of this instrumental learning task, participants chose between 2 pictures (eFigure 1 in the Supplement). One picture had a higher probability of reward when chosen (varied, .80 ≤ p ≤ .87), and the other, its additive inverse (.13 ≤ [1 − p] ≤ .20). In other words, even while choosing the right picture, participants received occasional misleading (probabilistic) negative feedback. Every 25 trials, the contingency (the good picture) changed without warning (reversal). On this task, one needs to trade off staying with the previously reinforced stimulus despite occasional misleading (probabilistic) feedback and switching when a true reversal occurs. The tendency to stay too long after reversal while ignoring negative feedback leads to perseverative errors. Conversely, the tendency to switch after a single misleading punishment results in probabilistic switch errors, previously linked to depression.58 We used the number of trials with misleading feedback during which the participant resisted this switch as a measure of resilience to it. Spontaneous or win/switch errors occurred when the subject switched away from a stimulus after being rewarded for choosing it. Because spontaneous errors are infrequent, their count is strongly and positively skewed with many values of 0. We also recorded response times, using their variation as an index of whether subjects were consistently on task or distracted at times. To ensure that even the most impaired participants are engaged in the task during scanning, we performed prior behavioral training with 4 reversals, and participants were no longer naive to the change in contingency. Thus, we were not expecting a complete breakdown of contingency learning in suicide attempters, unlike the earlier behavioral study with an unexpected reversal.38

Reinforcement Learning Model

To estimate abstract reward signals from participants’ reinforcement history and behavior, we adapted the Rescorla-Wagner reinforcement learning model, as in the earlier behavioral study.38 On this task, expected reward or expected value tracks the certainty of reward for the best available action. Here, the model’s estimate of expected value is high when the choice of one stimulus is almost certain to be rewarded. Meanwhile, a low expected value means ambiguity about which stimulus is likely to be rewarded. Prediction error is the discrepancy between the prior expectation of reward and the reward actually received on this trial. Prediction errors drive the updates of expected value after feedback. A reward for choosing a stimulus with a low prior expected value produces a positive prediction error and increases expected value. The omission of reward for a stimulus with a previously high expected value produces a negative prediction error and decreases expected value.

Measures of Impulsivity

We were interested in how neural indices of reward learning are affected by impulsivity for 2 reasons. First, strong evidence implicates impulsivity in the suicidal diathesis.5 Second, impulsivity involves a failure of higher-order control, which could plausibly undermine reward learning in an uncertain, changing environment. Assessments of impulsivity included the Impulsive/Careless Style subscale of the Social Problem-Solving Inventory,68,69 the Nonplanning and Attention/Cognitive subscales of the Barratt Impulsiveness Scale,70 and bets against the odds on the Cambridge Gamble Task (CGT),21 all found to be related to suicidal behavior.

Bets against the odds on the CGT, observed in older suicide attempters, reflect a neglect of information about outcome probability,21 possibly owing to a gambler’s fallacy or belief in a lucky streak. On each trial of the CGT, the participant is presented with an array of 10 boxes at the top of the screen, with each colored red or blue. The ratio of red to blue boxes varies from 1:9 to 9:1, in a pseudorandom order. The participant is instructed that the computer has hidden a token in 1 of the boxes and is told to register a guess as to whether the token is hidden under a red or a blue box by selecting 1 of the 2 corresponding colored panels. Given that the proportions of majority vs minority bets tend to follow a nonlinear, beta distribution, we dichotomized all participants into those who always bet on the majority color (good choices) vs those who sometimes bet on the minority color (bad choices). Of the 33 depressed participants included in the analysis, 2 were missing Social Problem-Solving Inventory data, 6 were missing Barratt Impulsiveness Scale data, and 9 were missing CGT data.

Statistical Analysis
Blood Oxygenation Level–Dependent Signal

Given the age-related variance in the hemodynamic response function (HRF), an empirical HRF was estimated in the control group using the (unbiased) switch/stay contrast in the vlPFC,71 as detailed on the authors’ website and in the Supplement (eFigure 2). This empirical HRF was then convolved with the expected value, positive prediction error, and negative prediction error signals from the reinforcement learning model for each subject. Voxelwise blood oxygenation level–dependent (BOLD) signal was regressed on these estimates in single-variable single-subjects analyses using an AFNI program (3dDeconvolve).72 Group differences were estimated by regressing the beta weights against 2 factors—depression and history of suicide attempts—across participants using another AFNI program (3dRegAna). To understand the variation in the HRF across regions, we also obtained estimates of the impulse response function for expected value signals using the 3dDeconvolve program with no assumed HRF and extracted the time series of resulting beta weights for each of the nondepressed controls.

To control type I error, we thresholded voxelwise tests at P < .005 and cluster thresholded them using Monte Carlo simulations based on the spatial autocorrelation of derived maps using the AFNI programs 3dFWHMx and 3dAlphaSim72,73 (67 voxels yielding P < .05, corrected). To estimate effects within the anatomical vmPFC mask from the Talairach atlas, 8 voxels at P < .005 controlled type I error at P < .05 with small-volume correction.

Correlations With Behavior and Self-Report: Extracting Network Activity Indices

To test the relationship between neural responses to expected value and prediction error on one hand and behavioral, clinical, and self-reported measures of impulsivity on the other, we conducted 2 independent analyses described in detail on the authors’ website. In the first, we mapped the networks responsive to expected value and prediction error in healthy controls (n = 20). In the second, we estimated the responses of these networks in the independent group of depressed participants with and without suicide attempts (n = 33) and correlated them with the measures of interest. In these correlational analyses, we chose to examine summary activations across networks responsive to expected value and prediction error instead of activations in single regions of interest to reduce dimensionality and control type I error. This choice was dictated by our initial analyses showing that responses to expected value and prediction error were distributed across large networks and were strongly intercorrelated across regions within each network (Tables 1 and 2 on the authors’ website). Although some regions were less active in suicide attempters and in depressed participants in general regardless of the history of suicide attempts as discussed in the Results section, we found no regions that were more active in the 2 depressed groups than in the controls. Thus, by applying the masks from the controls to the depressed participants, we did not leave out any ostensibly relevant regions.

In an exploratory analysis (described on the authors’ website along with the sensitivity analyses), we examined whether paralimbic structures encoded expected reward when the choices were being made vs when the reward/punishment was received after the choice. Given the low temporal resolution of fMRI, we tested this by comparing the timing of paralimbic response with that of the ostensibly choice-related response in the cortical task networks sensitive to ambiguity.

Suicide Attempts, Impulsivity, and Value Signals

In within-group analyses, controls demonstrated a positively modulated BOLD signal in the vmPFC (Brodmann areas [BAs] 10, 32, and 24) and other paralimbic structures, including the midcingulate cortex (BA 24), precuneus/posterior cingulate (BAs 7 and 31), and posterior insula for high expected reward (Figure 1A [in blue] and Table 1 on the authors’ website). In the whole-brain group comparison, as hypothesized, the pregenual cingulate subregion of the vmPFC (BA 32, 24, and 25 on the posterior periphery of the reward-modulated vmPFC region of interest) tracked the reinforcement history less reliably in suicide attempters than in the comparison groups, controlling for depression group status (P < .05, corrected; Figure 1B).

Place holder to copy figure label and caption
Figure 1.
Modulation by Expected Reward

A, Within-group analyses. In controls, high expected reward (shown in blue) positively modulated the blood oxygenation level–dependent signal in the ventromedial prefrontal cortex (vmPFC; Brodmann areas [BAs] 10, 32, and 24), the posterior insula (area 1), and other paralimbic structures (the midcingulate cortex [BA 24] and precuneus [BAs 7 and 31; area 2]). A large cortical network responded to high ambiguity/low value (shown in red). This network included the lateral prefrontal areas of the ventrolateral PFC (vlPFC; lateral BA 10 and BAs 13, 44, and 45; operculoinsular cortex [area 3]) and the dorsolateral PFC (dlPFC; BA 9 and lateral BA 8) that we expected to be activated a priori and the lateral parietal cortex (lat parietal; BAs 7 and 40) and dorsomedial PFC (dmPFC; medial BA 8; shown in red). Although the maps were qualitatively similar in nonsuicidal depressed participants, suicide attempters did not display paralimbic responses to high expected reward. Area 4 indicates associative striatum and thalamus. P < .005, uncorrected for illustration. B, A history of suicide attempts was related to a weaker response to expected reward in the pericallosal vmPFC (pregenual cingulate [BA 32, 24, and 25] on the posterior periphery of the reward-modulated vmPFC region of interest). P < .05, corrected, controlling for depression group status. The x and z values indicate Montreal Neurological Institute coordinates.

Graphic Jump Location
Correlations With Task Performance

Depressed participants whose paralimbic structures were less responsive to expected reward were more likely to ignore negative feedback after reversal (r33 = 0.47 [P = .006]; after excluding 1 outlier, r32 = 0.64 [P < .001]). However, their learning was otherwise relatively preserved, with no statistically reliable relationship between weak paralimbic modulation and inability to track the contingency, probabilistic, or spontaneous switches (r33 ≤ 0.24 [P ≥ .28]) (Figure 2A-C).

Place holder to copy figure label and caption
Figure 2.
Impulsivity and Paralimbic Expected Reward Signals in Depressed Participants

A, Paralimbic functional network masks were independently derived in controls. B, Depressed participants (nonsuicidal and suicide attempters) with weaker paralimbic responses to expected reward were more likely to ignore negative feedback after a reversal, making multiple perseverative errors. C, Weak paralimbic responses were related to bets against the odds on the Cambridge Gamble Task (CGT). D, Weak paralimbic responses were related to poor attempt planning in suicide attempters. E, Weak paralimbic responses were related to higher scores on the Impulsive/Careless Style subscale of the Social Problem-Solving Inventory. F, Weak paralimbic responses were related to higher scores on the Nonplanning subscale of the Barratt Impulsiveness Scale.

Graphic Jump Location
Correlations With Impulsivity Measures

Confirming our hypothesis in depressed participants, weak paralimbic response to high expected reward was related to nonplanning impulsivity (r27 = −0.43 [P = .03]; after excluding 1 outlier, r26 = −0.50 [P = .01]) and bad choices on the CGT (ρ27 = 0.50 [P = .01]). The correlation with impulsive/careless problem-solving style was not significant (r31 = −0.20 [P = .29]) but became more apparent after excluding one outlier (r30 = −0.43 [P = .02]). We found no relationship with delay discounting (r23 = −0.10 [P = .64]). Among suicide attempters, weak paralimbic modulation was related to poor attempt planning (ρ15 = −0.58 [P = .03]) and not significantly to low attempt lethality (ρ15 = −0.48 [P = .07]) (Figure 2D-F and Supplement [eFigure 3]).

Timing of Reward Signals in the Paralimbic vs Other Cortical Networks

Did paralimbic structures represent the expected reward signal when the actual choice was being made? Our analysis of BOLD waveforms seems to suggest otherwise. Although the opercularinsular, dorsomedial PFC (dmPFC), and lateral-parietal responses to ambiguity peaked at a time point consistent with choice (Figure 3, in red), responses to high expected reward in the vmPFC and precuneus peaked earlier (Figure 3, in blue; repeated-measures analysis of variance, F99,228 = 9.3 [P < .001]). This peak fell approximately between feedback and the next choice. Thus, paralimbic structures appeared to broadly integrate reinforcement history rather than compare the value of presented options or select between actions.

Place holder to copy figure label and caption
Figure 3.
Time Course of Reward Signals in Healthy Controls

We compared the paralimbic (blue) vs neocortical task (red) networks in healthy controls (n = 20). The onset of the time window corresponds to reward delivery on the preceding trial. Paralimbic responses to high expected value peaked earlier than responses to the low expected value in the lateralized frontoparietal network, operculoinsular cortex, ventrolateral prefrontal cortex (vlPFC), and dorsolateral prefrontal cortex (dlPFC), ostensibly aligned with the next stimulus presentation. This alignment suggests that paralimbic structures responded to the reward from the preceding trial. Indeed, the timing of paralimbic response is unlikely to coincide with actual choice because the peak to response would have to be less than 4 seconds. ANOVA indicates analysis of variance; vmPFC, ventromedial prefrontal cortex.

Graphic Jump Location
Depression, Overreactivity to Punishments, and Prediction Error Signals

Unexpected rewards (positive prediction error) modulated the same networks as ambiguity, plus the midcingulate cortex, striatum, and thalamus (Figure 4A and Table 2 on the authors’ website). The behavioral overreactivity of depressed individuals to negative feedback was related to blunted representation of positive prediction errors in the cingulo-opercular and frontoparietal networks (r33 = −0.40 [P = .02]). As a group, depressed participants displayed weaker modulation by positive prediction error in the right thalamus, bilateral superior temporal gyrus (BA 22 and 39), bilateral operculoinsular cortex (BA 13, 45, and 46), bilateral postcentral gyrus (BA 40), and the bilateral supplementary motor area (P < .05, corrected; Figure 4B), and these effects were not related to a history of suicide attempts.

Place holder to copy figure label and caption
Figure 4.
Modulation by Positive Prediction Error

A, Within-group analysis in controls (n = 20). Unexpected rewards (positive prediction error) modulated the same networks as ambiguity, plus the midcingulate cortex, striatum, and thalamus. Area 1 indicates the putamen; area 2, inferior parietal lobule; area 3, operculoinsular cortex; area 4, thalamus; and dmPFC, dorsomedial prefrontal cortex. P < .05, corrected. B, As a group, depressed participants (nonsuicidal and suicide attempters) displayed weaker modulation by positive prediction error in the right thalamus, the bilateral superior temporal gyrus (Brodmann area [BA] 22 and 39), the bilateral operculoinsular cortex (BA 13, 45, and 46), the bilateral postcentral gyrus (BA 40), and the bilateral supplementary motor area/cingulate. P < .05, corrected, controlling for a history of suicide attempts.

Graphic Jump Location
Sensitivity Analyses

Group differences in expected reward and prediction error representations were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular pathological features, incipient dementia, sex, and model fits (see material on the authors’ website). Only depressive severity was a significant covariate, explaining additional variance in expected reward signal in the pericallosal cingulate cortex.

Our study of reward learning in late-life depression uncovered 2 neural patterns associated with disrupted control over reward-guided behavior. The first pattern, manifesting in a blunted modulation of paralimbic structures by expected reward, is associated with impulsivity and attempted suicide. The second pattern, associated with depression, is expressed in a weak thalamocorticostriatal response to unpredicted rewards and in maladaptive overreactivity to punishments. We discuss them in that order.

Reward Value Representations in the Paralimbic Structures, Impulsivity, and Attempted Suicide

In a dynamic environment, paralimbic structures guide approach behavior by tracking the value of available options.45,46 This function is sometimes related to the hedonic aspect of pursuing rewards (liking).45,74,75 In the impulsive depressed elderly in our study and in depressed suicide attempters, the paralimbic structures failed to track the reinforcement history accurately. This failure was seen mostly in patients who had made unplanned suicide attempts, again linking this anomaly to impulsive behavior. Participants displaying such a failure had the tendency to perseverate in approach behavior even when it no longer paid off, suggesting that they did not update the value of the option that was no longer rewarded. This interpretation is consistent with disrupted tracking of expected value by the paralimbic structures, which was associated with betting against the odds on a gambling task, a behavior previously observed in older suicide attempters.21 In both cases, impulsive individuals appear to be ignoring key information—odds (CGT) or reinforcement history (reversal learning)—and experiencing losses as a result. Our findings of the relationship among paralimbic structures’ expected reward signals, trait impulsivity, and bad choices on the CGT converge with those of Cox and colleagues,76 who found that vmPFC-insula resting-state functional connectivity was related to disadvantageous risk seeking. Our observations bear a curious resemblance to the behavioral and neural patterns seen in long-term cocaine users77,78 and in impulsive pathological gamblers79 on the reversal learning task. A similar pattern of perseveration has been produced by long-term cocaine administration in monkeys80 and by its withdrawal in rats.81 Whether these populations experience a primary disruption in the paralimbic cortex itself or in its mesolimbic input remains unclear.82,83 We found little support for the primary cortical deficit account from the literature concerning lesions in monkeys. Although lesions of the medial occipital-frontal cortex induce perseverative responding on a pavlovian reversal learning task,84 their effects on instrumental behavior are more complex and not limited to perseveration.85 Meanwhile, the fact that such perseveration can be induced and ameliorated86 by monoaminergic manipulation points toward a source in the ascending reward pathways.

Depression, Positive Prediction Errors, and Overreactivity to Punishment

Depressed individuals overreact to negative feedback, making error after error.5458 Abnormal response to errors in prefrontal areas, such as the vlPFC and the dmPFC, has been suggested to underlie this tendency.61 Extending that finding, we show that this behavior is also associated with blunted response of the cingulo-opercular and frontoparietal networks to positive prediction errors. Notably, this blunting was seen on a different set of trials: those with surprising positive feedback rather than misleading negative feedback. This effect may be best explained in terms of competition between 2 controllers of behavior. A coherent neurobiological account of such competition is offered by the rodent model of controllable vs uncontrollable stress used by Amat et al.87 In the controllable condition, once the animal encodes the action/reinforcement contingency, resulting cortical safety signals inhibit the primitive brainstem and limbic stress response. Conversely, weaker encoding of unpredicted rewards in depressed patients may fail to suppress a primitive response to noncontingent punishment, resembling the state of learned helplessness. Depressed individuals may thus experience punishments as uncontrollable, shifting toward a primitive, subcortical lose-switch policy instead of strategic choice based on reinforcement history. A plausible computational account of depressive overreactivity to punishments is that a primitive, automatic pavlovian controller responding to an aversive stimulus may override the more sophisticated controllers of instrumental learning. Pavlovian influences on instrumental behavior have been invoked to explain a range of disadvantageous approach-avoidance responses88 and appear to depend on limbic structures, such as the basolateral amygdala89 and the ventral striatum.90

Limitations

Besides the cross-sectional, case-control design and a relatively small sample, several other design weaknesses limit our findings. Imaging assessments are too demanding for our sickest patients. Therefore, being unable to include these patients, we probably lack the opportunity to observe all the neural correlates of suicidal behavior. Because some of our patients were in partial remission, our findings may not apply to the most acutely depressed individuals. With respect to external validity, one can be more confident in the mechanistic findings linking impulsivity and overreactivity to punishment to specific neural aberrations than in group differences as such. Furthermore, expected reward is confounded with (lower) ambiguity and risk in reversal learning, limiting the interpretation of our findings. In addition, because depression status was confounded with antidepressant exposure, we cannot conclusively separate the two, although sensitivity analyses suggest that our results are robust. Also, our correlational study cannot disentangle the exact structure of relationships among the altered paralimbic reward signals, behaviors on reversal learning and gambling tasks, and self-reported impulsivity. Finally, the delta-rule model of associative learning explains only the most basic aspects of human brain activity during reversal learning (additional discussion can be found on the authors’ website). More sophisticated models49,91,92 might provide additional insight into anomalous reward learning in impulsive depressed elderly.

In summary, we found that in older adults, unplanned suicide attempts and impulsivity were associated with disrupted paralimbic tracking of expected reward. Depression was associated with disrupted corticostriatothalamic encoding of unexpected rewards.

Corresponding Author: Alexandre Y. Dombrovski, MD, Department of Psychiatry, University of Pittsburgh, 3811 O’Hara St, Pittsburgh, PA 15213 (dombrovskia@upmc.edu)

Submitted for Publication: July 25, 2012; final revision received November 7, 2012; accepted December 18, 2012.

Published Online: August 7, 2013. doi:10.1001/jamapsychiatry.2013.75.

Author Contributions: Dr Dombrovski had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Dombrovski, Clark, Reynolds, Siegle.

Acquisition of data: Dombrovski, Szanto, Reynolds, Siegle.

Analysis and interpretation of data: Dombrovski, Clark, Siegle.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: Dombrovski, Clark, Siegle.

Statistical analysis: Dombrovski, Siegle.

Obtained funding: Dombrovski, Clark.

Administrative, technical, and material support: Dombrovski, Reynolds.

Study supervision: Dombrovski, Szanto, Reynolds, Siegle.

Funding/Support: The study was supported by grants K23MH086620, K23MH070471, and R01MH085651 from the National Institute of Mental Health and by the American Foundation for Suicide Prevention.

Role of the Sponsors: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.

Previous Presentation: Some of the results reported herein were previously presented at the 50th Annual Meeting of the American College of Neuropsychopharmacology; December 7, 2011; Waikoloa, Hawaii.

Additional Contributions: Michael Hallquist, PhD, and Henry Chase, PhD, provided critical comments on the manuscript. Mandy Collier, BS, contributed to data collection and processing. Swathi Gujral, BS, Natalie Truty, BS, and Cori Shollenberger, BS, assisted with recruitment and assessments. Jan Kalkus, BS, provided help with graphics. Howard Aizenstein, MD, PhD, contributed to the structural imaging protocol.

Correction: This article was corrected on July 3, 2014, to update the URL to the authors’ website.

Keilp  JG, Gorlyn  M, Oquendo  MA,  et al.  Aggressiveness, not impulsiveness or hostility, distinguishes suicide attempters with major depression. Psychol Med. 2006;36(12):1779-1788.
PubMed   |  Link to Article
McGirr  A, Renaud  J, Bureau  A, Seguin  M, Lesage  A, Turecki  G.  Impulsive-aggressive behaviours and completed suicide across the life cycle: a predisposition for younger age of suicide. Psychol Med. 2008;38(3):407-417.
PubMed   |  Link to Article
New  AS, Gelernter  J, Goodman  M,  et al.  Suicide, impulsive aggression, and HTR1B genotype. Biol Psychiatry. 2001;50(1):62-65.
PubMed   |  Link to Article
Swann  AC, Dougherty  DM, Pazzaglia  PJ, Pham  M, Steinberg  JL, Moeller  FG.  Increased impulsivity associated with severity of suicide attempt history in patients with bipolar disorder. Am J Psychiatry. 2005;162(9):1680-1687.
PubMed   |  Link to Article
Turecki  G.  Dissecting the suicide phenotype: the role of impulsive-aggressive behaviours. J Psychiatry Neurosci. 2005;30(6):398-408.
PubMed
Placidi  GP, Oquendo  MA, Malone  KM, Huang  YY, Ellis  SP, Mann  JJ.  Aggressivity, suicide attempts, and depression: relationship to cerebrospinal fluid monoamine metabolite levels. Biol Psychiatry. 2001;50(10):783-791.
PubMed   |  Link to Article
Gunnell  D, Magnusson  PK, Rasmussen  F.  Low intelligence test scores in 18 year old men and risk of suicide: cohort study. BMJ. 2005;330(7484):167. doi:10.1136/bmj.38310.473565.8F.
PubMed   |  Link to Article
Keilp  JG, Gorlyn  M, Oquendo  MA, Burke  AK, Mann  JJ.  Attention deficit in depressed suicide attempters. Psychiatry Res. 2008;159(1-2):7-17.
PubMed   |  Link to Article
Keilp  JG, Sackeim  HA, Brodsky  BS, Oquendo  MA, Malone  KM, Mann  JJ.  Neuropsychological dysfunction in depressed suicide attempters. Am J Psychiatry. 2001;158(5):735-741.
PubMed   |  Link to Article
Bartfai  A, Winborg  IM, Nordström  P, Asberg  M.  Suicidal behavior and cognitive flexibility: design and verbal fluency after attempted suicide. Suicide Life Threat Behav. 1990;20(3):254-266.
PubMed
Jollant  F, Bellivier  F, Leboyer  M,  et al.  Impaired decision making in suicide attempters. Am J Psychiatry. 2005;162(2):304-310.
PubMed   |  Link to Article
Westheide  J, Quednow  BB, Kuhn  KU,  et al.  Executive performance of depressed suicide attempters: the role of suicidal ideation. Eur Arch Psychiatry Clin Neurosci. 2008;258(7):414-421.
PubMed   |  Link to Article
Cha  CB, Najmi  S, Park  JM, Finn  CT, Nock  MK.  Attentional bias toward suicide-related stimuli predicts suicidal behavior. J Abnorm Psychol. 2010;119(3):616-622.
PubMed   |  Link to Article
Beck  AT, Kovacs  M, Weissman  A.  Hopelessness and suicidal behavior: an overview. JAMA. 1975;234(11):1146-1149.
PubMed   |  Link to Article
Young  MA, Fogg  LF, Scheftner  W, Fawcett  J, Akiskal  H, Maser  J.  Stable trait components of hopelessness: baseline and sensitivity to depression. J Abnorm Psychol. 1996;105(2):155-165.
PubMed   |  Link to Article
Szanto  K, Reynolds  CF  III, Conwell  Y, Begley  AE, Houck  P.  High levels of hopelessness persist in geriatric patients with remitted depression and a history of attempted suicide. J Am Geriatr Soc. 1998;46(11):1401-1406.
PubMed
Wong  PW, Cheung  DY, Conner  KR, Conwell  Y, Yip  PS.  Gambling and completed suicide in Hong Kong: a review of coroner court files. Prim Care Companion J Clin Psychiatry. 2010;12(6):PCC.09m00932. doi:10.4088/PCC.09m00932blu.
PubMed
Vijayakumar  L, Kumar  MS, Vijayakumar  V.  Substance use and suicide. Curr Opin Psychiatry. 2011;24(3):197-202.
PubMed   |  Link to Article
Jollant  F, Lawrence  NS, Olie  E,  et al.  Decreased activation of lateral orbitofrontal cortex during risky choices under uncertainty is associated with disadvantageous decision-making and suicidal behavior. Neuroimage. 2010;51(3):1275-1281.
PubMed   |  Link to Article
Malloy-Diniz  LF, Neves  FS, Abrantes  SS, Fuentes  D, Corrêa  H.  Suicide behavior and neuropsychological assessment of type I bipolar patients. J Affect Disord. 2009;112(1-3):231-236.
PubMed   |  Link to Article
Clark  L, Dombrovski  AY, Siegle  GJ,  et al.  Impairment in risk-sensitive decision-making in older suicide attempters with depression. Psychol Aging. 2011;26(2):321-330.
PubMed   |  Link to Article
Martino  DJ, Strejilevich  SA, Torralva  T, Manes  F.  Decision making in euthymic bipolar I and bipolar II disorders. Psychol Med. 2011;41(6):1319-1327.
PubMed   |  Link to Article
Dombrovski  AY, Szanto  K, Siegle  GJ,  et al.  Lethal forethought: delayed reward discounting differentiates high- and low-lethality suicide attempts in old age. Biol Psychiatry. 2011;70(2):138-144.
PubMed   |  Link to Article
Ahearn  EP, Jamison  KR, Steffens  DC,  et al.  MRI correlates of suicide attempt history in unipolar depression. Biol Psychiatry. 2001;50(4):266-270.
PubMed   |  Link to Article
Vang  FJ, Ryding  E, Träskman-Bendz  L, van Westen  D, Lindström  MB.  Size of basal ganglia in suicide attempters, and its association with temperament and serotonin transporter density. Psychiatry Res. 2010;183(2):177-179.
PubMed   |  Link to Article
Di Maio  L, Squitieri  F, Napolitano  G, Campanella  G, Trofatter  JA, Conneally  PM.  Suicide risk in Huntington’s disease. J Med Genet. 1993;30(4):293-295.
PubMed   |  Link to Article
Foncke  EM, Schuurman  PR, Speelman  JD.  Suicide after deep brain stimulation of the internal globus pallidus for dystonia. Neurology. 2006;66(1):142-143.
PubMed   |  Link to Article
Voon  V, Krack  P, Lang  AE,  et al.  A multicentre study on suicide outcomes following subthalamic stimulation for Parkinson’s disease. Brain. 2008;131(pt 10):2720-2728.
PubMed   |  Link to Article
Dombrovski  AY, Siegle  GJ, Szanto  K, Clark  L, Reynolds  CF, Aizenstein  H.  The temptation of suicide: striatal gray matter, discounting of delayed rewards, and suicide attempts in late-life depression [published online October 17, 2011]. Psychol Med. 2012;42(6):1203-1215. doi:10.1017/S0033291711002133.
PubMed   |  Link to Article
Soulas  T, Gurruchaga  JM, Palfi  S, Cesaro  P, Nguyen  JP, Fenelon  G.  Attempted and completed suicides after subthalamic nucleus stimulation for Parkinson's disease. J Neurol Neurosurg Psychiatry. 2008;79(8):952-954.
Link to Article
Housden  CR, O’Sullivan  SS, Joyce  EM, Lees  AJ, Roiser  JP.  Intact reward learning but elevated delay discounting in Parkinson’s disease patients with impulsive-compulsive spectrum behaviors. Neuropsychopharmacology. 2010;35(11):2155-2164.
PubMed   |  Link to Article
Voon  V, Reynolds  B, Brezing  C,  et al.  Impulsive choice and response in dopamine agonist-related impulse control behaviors. Psychopharmacology (Berl). 2010;207(4):645-659.
PubMed   |  Link to Article
Weintraub  D, Siderowf  AD, Potenza  MN,  et al.  Association of dopamine agonist use with impulse control disorders in Parkinson disease. Arch Neurol. 2006;63(7):969-973.
PubMed   |  Link to Article
Hälbig  TD, Tse  W, Frisina  PG,  et al.  Subthalamic deep brain stimulation and impulse control in Parkinson’s disease. Eur J Neurol. 2009;16(4):493-497.
PubMed   |  Link to Article
Frank  MJ, Samanta  J, Moustafa  AA, Sherman  SJ.  Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science. 2007;318(5854):1309-1312.
PubMed   |  Link to Article
Baumeister  RF.  Suicide as escape from self. Psychol Rev. 1990;97(1):90-113.
PubMed   |  Link to Article
Williams  JMG, Crane  C, Barnhofer  T, Duggan  D. Psychology and suicidal behaviour: elaborating the entrapment model. In: Hawton  K, ed. Prevention and Treatment of Suicidal Behaviour: From Science to Practice. New York, NY: Oxford University Press; 2005:71-89.
Dombrovski  AY, Clark  L, Siegle  GJ,  et al.  Reward/punishment reversal learning in older suicide attempters. Am J Psychiatry. 2010;167(6):699-707.
PubMed   |  Link to Article
McGirr  A, Dombrovski  AY, Butters  MA, Clark  L, Szanto  K.  Deterministic learning and attempted suicide among older depressed individuals: cognitive assessment using the Wisconsin Card Sorting Task. J Psychiatr Res. 2012;46(2):226-232.
PubMed   |  Link to Article
Samanez-Larkin  GR, Kuhnen  CM, Yoo  DJ, Knutson  B.  Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J Neurosci. 2010;30(4):1426-1434.
PubMed   |  Link to Article
Denburg  NL, Cole  CA, Hernandez  M,  et al.  The orbitofrontal cortex, real-world decision making, and normal aging. Ann N Y Acad Sci. 2007;1121:480-498.
PubMed   |  Link to Article
Fellows  LK, Farah  MJ.  Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain. 2003;126(pt 8):1830-1837.
PubMed   |  Link to Article
Tsuchida  A, Doll  BB, Fellows  LK.  Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J Neurosci. 2010;30(50):16868-16875.
PubMed   |  Link to Article
Hornak  J, O’Doherty  J, Bramham  J,  et al.  Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J Cogn Neurosci. 2004;16(3):463-478.
PubMed   |  Link to Article
Grabenhorst  F, Rolls  ET.  Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci. 2011;15(2):56-67.
PubMed   |  Link to Article
Rushworth  MF, Noonan  MP, Boorman  ED, Walton  ME, Behrens  TE.  Frontal cortex and reward-guided learning and decision-making. Neuron. 2011;70(6):1054-1069.
PubMed   |  Link to Article
Gläscher  J, Hampton  AN, O’Doherty  JP.  Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2009;19(2):483-495.
PubMed   |  Link to Article
Plassmann  H, O’Doherty  J, Rangel  A.  Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci. 2007;27(37):9984-9988.
PubMed   |  Link to Article
Hampton  AN, Bossaerts  P, O’Doherty  JP.  The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci. 2006;26(32):8360-8367.
PubMed   |  Link to Article
Jocham  G, Neumann  J, Klein  TA, Danielmeier  C, Ullsperger  M.  Adaptive coding of action values in the human rostral cingulate zone. J Neurosci. 2009;29(23):7489-7496.
PubMed   |  Link to Article
Leyton  M, Paquette  V, Gravel  P,  et al.  Alpha-[11C]methyl-L-tryptophan trapping in the orbital and ventral medial prefrontal cortex of suicide attempters. Eur Neuropsychopharmacol. 2006;16(3):220-223.
PubMed   |  Link to Article
Oquendo  MA, Placidi  GP, Malone  KM,  et al.  Positron emission tomography of regional brain metabolic responses to a serotonergic challenge and lethality of suicide attempts in major depression. Arch Gen Psychiatry. 2003;60(1):14-22.
PubMed   |  Link to Article
Fineberg  NA, Potenza  MN, Chamberlain  SR,  et al.  Probing compulsive and impulsive behaviors, from animal models to endophenotypes: a narrative review. Neuropsychopharmacology. 2010;35(3):591-604.
PubMed   |  Link to Article
Nelson  RE, Craighead  WE.  Selective recall of positive and negative feedback, self-control behaviors, and depression. J Abnorm Psychol. 1977;86(4):379-388.
PubMed   |  Link to Article
Steffens  DC, Wagner  HR, Levy  RM, Horn  KA, Krishnan  KR.  Performance feedback deficit in geriatric depression. Biol Psychiatry. 2001;50(5):358-363.
PubMed   |  Link to Article
Elliott  R, Sahakian  BJ, Herrod  JJ, Robbins  TW, Paykel  ES.  Abnormal response to negative feedback in unipolar depression: evidence for a diagnosis specific impairment. J Neurol Neurosurg Psychiatry. 1997;63(1):74-82.
Link to Article
Elliott  R, Sahakian  BJ, McKay  AP, Herrod  JJ, Robbins  TW, Paykel  ES.  Neuropsychological impairments in unipolar depression: the influence of perceived failure on subsequent performance. Psychol Med. 1996;26(5):975-989.
PubMed   |  Link to Article
Murphy  FC, Michael  A, Robbins  TW, Sahakian  BJ.  Neuropsychological impairment in patients with major depressive disorder: the effects of feedback on task performance. Psychol Med. 2003;33(3):455-467.
PubMed   |  Link to Article
Bush  RR, Mosteller  F. Stochastic Models for Learning. Oxford, England: John Wiley & Sons Inc; 1955.
Rescorla  RA, Wagner  AR. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black  AH, Prokasy  WF, eds. Classical Conditioning II. New York, NY: Appleton-Century-Crofts; 1972:64-99.
Taylor Tavares  JV, Clark  L, Furey  ML, Williams  GB, Sahakian  BJ, Drevets  WC.  Neural basis of abnormal response to negative feedback in unmedicated mood disorders. Neuroimage. 2008;42(3):1118-1126.
PubMed   |  Link to Article
Kumar  P, Waiter  G, Ahearn  T, Milders  M, Reid  I, Steele  JD.  Abnormal temporal difference reward-learning signals in major depression. Brain. 2008;131(pt 8):2084-2093.
PubMed   |  Link to Article
Gradin  VB, Kumar  P, Waiter  G,  et al.  Expected value and prediction error abnormalities in depression and schizophrenia. Brain. 2011;134(pt 6):1751-1764.
PubMed   |  Link to Article
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders.ed 4. Washington, DC: American Psychiatric Association; 1994.
First  M Sr, Gibbon  M, Williams  JBW. Structured Clinical Interview for DSM-IV Axis I Disorders–Patient Edition (SCID-I/P), Version 2. New York: Biometrics Research Dept, New York State Psychiatric Institute; 1995.
Folstein  MF, Folstein  SE, McHugh  PR.  “Mini-Mental State”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-198.
PubMed   |  Link to Article
Beck  AT, Beck  R, Kovacs  M.  Classification of suicidal behaviors, I: quantifying intent and medical lethality. Am J Psychiatry. 1975;132(3):285-287.
PubMed
Gibbs  LM, Dombrovski  AY, Morse  J, Siegle  GJ, Houck  PR, Szanto  K.  When the solution is part of the problem: problem solving in elderly suicide attempters. Int J Geriatr Psychiatry. 2009;24(12):1396-1404.
PubMed   |  Link to Article
D’Zurilla  TJ, Nezu  AM.  Development and preliminary evaluation of the Social Problem-Solving Inventory: psychological assessment. J Consult Clin Psychol. 1990;2(2):156-163.
Patton  JH, Stanford  MS, Barratt  ES.  Factor structure of the Barratt Impulsiveness Scale. J Clin Psychol. 1995;51(6):768-774.
Link to Article
Cools  R, Clark  L, Owen  AM, Robbins  TW.  Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 2002;22(11):4563-4567.
PubMed
Cox  RW.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res. 1996;29(3):162-173.
PubMed   |  Link to Article
Forman  SD, Cohen  JD, Fitzgerald  M, Eddy  WF, Mintun  MA, Noll  DC.  Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med. 1995;33(5):636-647.
PubMed   |  Link to Article
Kringelbach  ML.  The human orbitofrontal cortex: linking reward to hedonic experience. Nat Rev Neurosci. 2005;6(9):691-702.
PubMed   |  Link to Article
Peters  J, Büchel  C.  Neural representations of subjective reward value. Behav Brain Res. 2010;213(2):135-141.
PubMed   |  Link to Article
Cox  CL, Gotimer  K, Roy  AK, Castellanos  FX, Milham  MP, Kelly  C.  Your resting brain CAREs about your risky behavior. PLoS One. 2010;5(8):e12296. doi:10.1371/journal.pone.0012296.
PubMed   |  Link to Article
Fernández-Serrano  MJ, Perales  JC, Moreno-López  L, Pérez-García  M, Verdejo-García  A.  Neuropsychological profiling of impulsivity and compulsivity in cocaine dependent individuals. Psychopharmacology (Berl). 2012;219(2):673-683.
PubMed   |  Link to Article
Ersche  KD, Roiser  JP, Robbins  TW, Sahakian  BJ.  Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology (Berl). 2008;197(3):421-431.
PubMed   |  Link to Article
de Ruiter  MB, Veltman  DJ, Goudriaan  AE, Oosterlaan  J, Sjoerds  Z, van den Brink  W.  Response perseveration and ventral prefrontal sensitivity to reward and punishment in male problem gamblers and smokers. Neuropsychopharmacology. 2009;34(4):1027-1038.
PubMed   |  Link to Article
Jentsch  JD, Olausson  P, De La Garza  R  II, Taylor  JR.  Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology. 2002;26(2):183-190.
PubMed   |  Link to Article
Calu  DJ, Stalnaker  TA, Franz  TM, Singh  T, Shaham  Y, Schoenbaum  G.  Withdrawal from cocaine self-administration produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats. Learn Mem. 2007;14(5):325-328.
PubMed   |  Link to Article
Chase  HW, Clark  L.  Gambling severity predicts midbrain response to near-miss outcomes. J Neurosci. 2010;30(18):6180-6187.
PubMed   |  Link to Article
Stalnaker  TA, Roesch  MR, Calu  DJ, Burke  KA, Singh  T, Schoenbaum  G.  Neural correlates of inflexible behavior in the orbitofrontal-amygdalar circuit after cocaine exposure. Ann N Y Acad Sci. December 2007;1121:598-609.
PubMed   |  Link to Article
Chudasama  Y, Robbins  TW.  Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci. 2003;23(25):8771-8780.
PubMed
Noonan  MP, Walton  ME, Behrens  TE, Sallet  J, Buckley  MJ, Rushworth  MF.  Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc Natl Acad Sci U S A. 2010;107(47):20547-20552.
PubMed   |  Link to Article
Ersche  KD, Roiser  JP, Abbott  S,  et al.  Response perseveration in stimulant dependence is associated with striatal dysfunction and can be ameliorated by a D(2/3) receptor agonist. Biol Psychiatry. 2011;70(8):754-762.
PubMed   |  Link to Article
Amat  J, Baratta  MV, Paul  E, Bland  ST, Watkins  LR, Maier  SF.  Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat Neurosci. 2005;8(3):365-371.
PubMed   |  Link to Article
De Martino  B. The effect of context on choice and value. In: Sharot  T, Dolan  RJ, eds. Neuroscience of Preference and Choice. London, England: Academic Press; 2012:97-106.
Corbit  LH, Balleine  BW.  Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J Neurosci. 2005;25(4):962-970.
PubMed   |  Link to Article
Talmi  D, Seymour  B, Dayan  P, Dolan  RJ.  Human pavlovian-instrumental transfer. J Neurosci. 2008;28(2):360-368.
PubMed   |  Link to Article
Behrens  TEJ, Woolrich  MW, Walton  ME, Rushworth  MFS.  Learning the value of information in an uncertain world. Nat Neurosci. 2007;10(9):1214-1221.
PubMed   |  Link to Article
O’Reilly  RC, Frank  MJ, Hazy  TE, Watz  B.  PVLV: the primary value and learned value Pavlovian learning algorithm. Behav Neurosci. 2007;121(1):31-49.
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Figure 1.
Modulation by Expected Reward

A, Within-group analyses. In controls, high expected reward (shown in blue) positively modulated the blood oxygenation level–dependent signal in the ventromedial prefrontal cortex (vmPFC; Brodmann areas [BAs] 10, 32, and 24), the posterior insula (area 1), and other paralimbic structures (the midcingulate cortex [BA 24] and precuneus [BAs 7 and 31; area 2]). A large cortical network responded to high ambiguity/low value (shown in red). This network included the lateral prefrontal areas of the ventrolateral PFC (vlPFC; lateral BA 10 and BAs 13, 44, and 45; operculoinsular cortex [area 3]) and the dorsolateral PFC (dlPFC; BA 9 and lateral BA 8) that we expected to be activated a priori and the lateral parietal cortex (lat parietal; BAs 7 and 40) and dorsomedial PFC (dmPFC; medial BA 8; shown in red). Although the maps were qualitatively similar in nonsuicidal depressed participants, suicide attempters did not display paralimbic responses to high expected reward. Area 4 indicates associative striatum and thalamus. P < .005, uncorrected for illustration. B, A history of suicide attempts was related to a weaker response to expected reward in the pericallosal vmPFC (pregenual cingulate [BA 32, 24, and 25] on the posterior periphery of the reward-modulated vmPFC region of interest). P < .05, corrected, controlling for depression group status. The x and z values indicate Montreal Neurological Institute coordinates.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Impulsivity and Paralimbic Expected Reward Signals in Depressed Participants

A, Paralimbic functional network masks were independently derived in controls. B, Depressed participants (nonsuicidal and suicide attempters) with weaker paralimbic responses to expected reward were more likely to ignore negative feedback after a reversal, making multiple perseverative errors. C, Weak paralimbic responses were related to bets against the odds on the Cambridge Gamble Task (CGT). D, Weak paralimbic responses were related to poor attempt planning in suicide attempters. E, Weak paralimbic responses were related to higher scores on the Impulsive/Careless Style subscale of the Social Problem-Solving Inventory. F, Weak paralimbic responses were related to higher scores on the Nonplanning subscale of the Barratt Impulsiveness Scale.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 3.
Time Course of Reward Signals in Healthy Controls

We compared the paralimbic (blue) vs neocortical task (red) networks in healthy controls (n = 20). The onset of the time window corresponds to reward delivery on the preceding trial. Paralimbic responses to high expected value peaked earlier than responses to the low expected value in the lateralized frontoparietal network, operculoinsular cortex, ventrolateral prefrontal cortex (vlPFC), and dorsolateral prefrontal cortex (dlPFC), ostensibly aligned with the next stimulus presentation. This alignment suggests that paralimbic structures responded to the reward from the preceding trial. Indeed, the timing of paralimbic response is unlikely to coincide with actual choice because the peak to response would have to be less than 4 seconds. ANOVA indicates analysis of variance; vmPFC, ventromedial prefrontal cortex.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 4.
Modulation by Positive Prediction Error

A, Within-group analysis in controls (n = 20). Unexpected rewards (positive prediction error) modulated the same networks as ambiguity, plus the midcingulate cortex, striatum, and thalamus. Area 1 indicates the putamen; area 2, inferior parietal lobule; area 3, operculoinsular cortex; area 4, thalamus; and dmPFC, dorsomedial prefrontal cortex. P < .05, corrected. B, As a group, depressed participants (nonsuicidal and suicide attempters) displayed weaker modulation by positive prediction error in the right thalamus, the bilateral superior temporal gyrus (Brodmann area [BA] 22 and 39), the bilateral operculoinsular cortex (BA 13, 45, and 46), the bilateral postcentral gyrus (BA 40), and the bilateral supplementary motor area/cingulate. P < .05, corrected, controlling for a history of suicide attempts.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable.  Demographic, Clinical, and Cognitive Characteristics

References

Keilp  JG, Gorlyn  M, Oquendo  MA,  et al.  Aggressiveness, not impulsiveness or hostility, distinguishes suicide attempters with major depression. Psychol Med. 2006;36(12):1779-1788.
PubMed   |  Link to Article
McGirr  A, Renaud  J, Bureau  A, Seguin  M, Lesage  A, Turecki  G.  Impulsive-aggressive behaviours and completed suicide across the life cycle: a predisposition for younger age of suicide. Psychol Med. 2008;38(3):407-417.
PubMed   |  Link to Article
New  AS, Gelernter  J, Goodman  M,  et al.  Suicide, impulsive aggression, and HTR1B genotype. Biol Psychiatry. 2001;50(1):62-65.
PubMed   |  Link to Article
Swann  AC, Dougherty  DM, Pazzaglia  PJ, Pham  M, Steinberg  JL, Moeller  FG.  Increased impulsivity associated with severity of suicide attempt history in patients with bipolar disorder. Am J Psychiatry. 2005;162(9):1680-1687.
PubMed   |  Link to Article
Turecki  G.  Dissecting the suicide phenotype: the role of impulsive-aggressive behaviours. J Psychiatry Neurosci. 2005;30(6):398-408.
PubMed
Placidi  GP, Oquendo  MA, Malone  KM, Huang  YY, Ellis  SP, Mann  JJ.  Aggressivity, suicide attempts, and depression: relationship to cerebrospinal fluid monoamine metabolite levels. Biol Psychiatry. 2001;50(10):783-791.
PubMed   |  Link to Article
Gunnell  D, Magnusson  PK, Rasmussen  F.  Low intelligence test scores in 18 year old men and risk of suicide: cohort study. BMJ. 2005;330(7484):167. doi:10.1136/bmj.38310.473565.8F.
PubMed   |  Link to Article
Keilp  JG, Gorlyn  M, Oquendo  MA, Burke  AK, Mann  JJ.  Attention deficit in depressed suicide attempters. Psychiatry Res. 2008;159(1-2):7-17.
PubMed   |  Link to Article
Keilp  JG, Sackeim  HA, Brodsky  BS, Oquendo  MA, Malone  KM, Mann  JJ.  Neuropsychological dysfunction in depressed suicide attempters. Am J Psychiatry. 2001;158(5):735-741.
PubMed   |  Link to Article
Bartfai  A, Winborg  IM, Nordström  P, Asberg  M.  Suicidal behavior and cognitive flexibility: design and verbal fluency after attempted suicide. Suicide Life Threat Behav. 1990;20(3):254-266.
PubMed
Jollant  F, Bellivier  F, Leboyer  M,  et al.  Impaired decision making in suicide attempters. Am J Psychiatry. 2005;162(2):304-310.
PubMed   |  Link to Article
Westheide  J, Quednow  BB, Kuhn  KU,  et al.  Executive performance of depressed suicide attempters: the role of suicidal ideation. Eur Arch Psychiatry Clin Neurosci. 2008;258(7):414-421.
PubMed   |  Link to Article
Cha  CB, Najmi  S, Park  JM, Finn  CT, Nock  MK.  Attentional bias toward suicide-related stimuli predicts suicidal behavior. J Abnorm Psychol. 2010;119(3):616-622.
PubMed   |  Link to Article
Beck  AT, Kovacs  M, Weissman  A.  Hopelessness and suicidal behavior: an overview. JAMA. 1975;234(11):1146-1149.
PubMed   |  Link to Article
Young  MA, Fogg  LF, Scheftner  W, Fawcett  J, Akiskal  H, Maser  J.  Stable trait components of hopelessness: baseline and sensitivity to depression. J Abnorm Psychol. 1996;105(2):155-165.
PubMed   |  Link to Article
Szanto  K, Reynolds  CF  III, Conwell  Y, Begley  AE, Houck  P.  High levels of hopelessness persist in geriatric patients with remitted depression and a history of attempted suicide. J Am Geriatr Soc. 1998;46(11):1401-1406.
PubMed
Wong  PW, Cheung  DY, Conner  KR, Conwell  Y, Yip  PS.  Gambling and completed suicide in Hong Kong: a review of coroner court files. Prim Care Companion J Clin Psychiatry. 2010;12(6):PCC.09m00932. doi:10.4088/PCC.09m00932blu.
PubMed
Vijayakumar  L, Kumar  MS, Vijayakumar  V.  Substance use and suicide. Curr Opin Psychiatry. 2011;24(3):197-202.
PubMed   |  Link to Article
Jollant  F, Lawrence  NS, Olie  E,  et al.  Decreased activation of lateral orbitofrontal cortex during risky choices under uncertainty is associated with disadvantageous decision-making and suicidal behavior. Neuroimage. 2010;51(3):1275-1281.
PubMed   |  Link to Article
Malloy-Diniz  LF, Neves  FS, Abrantes  SS, Fuentes  D, Corrêa  H.  Suicide behavior and neuropsychological assessment of type I bipolar patients. J Affect Disord. 2009;112(1-3):231-236.
PubMed   |  Link to Article
Clark  L, Dombrovski  AY, Siegle  GJ,  et al.  Impairment in risk-sensitive decision-making in older suicide attempters with depression. Psychol Aging. 2011;26(2):321-330.
PubMed   |  Link to Article
Martino  DJ, Strejilevich  SA, Torralva  T, Manes  F.  Decision making in euthymic bipolar I and bipolar II disorders. Psychol Med. 2011;41(6):1319-1327.
PubMed   |  Link to Article
Dombrovski  AY, Szanto  K, Siegle  GJ,  et al.  Lethal forethought: delayed reward discounting differentiates high- and low-lethality suicide attempts in old age. Biol Psychiatry. 2011;70(2):138-144.
PubMed   |  Link to Article
Ahearn  EP, Jamison  KR, Steffens  DC,  et al.  MRI correlates of suicide attempt history in unipolar depression. Biol Psychiatry. 2001;50(4):266-270.
PubMed   |  Link to Article
Vang  FJ, Ryding  E, Träskman-Bendz  L, van Westen  D, Lindström  MB.  Size of basal ganglia in suicide attempters, and its association with temperament and serotonin transporter density. Psychiatry Res. 2010;183(2):177-179.
PubMed   |  Link to Article
Di Maio  L, Squitieri  F, Napolitano  G, Campanella  G, Trofatter  JA, Conneally  PM.  Suicide risk in Huntington’s disease. J Med Genet. 1993;30(4):293-295.
PubMed   |  Link to Article
Foncke  EM, Schuurman  PR, Speelman  JD.  Suicide after deep brain stimulation of the internal globus pallidus for dystonia. Neurology. 2006;66(1):142-143.
PubMed   |  Link to Article
Voon  V, Krack  P, Lang  AE,  et al.  A multicentre study on suicide outcomes following subthalamic stimulation for Parkinson’s disease. Brain. 2008;131(pt 10):2720-2728.
PubMed   |  Link to Article
Dombrovski  AY, Siegle  GJ, Szanto  K, Clark  L, Reynolds  CF, Aizenstein  H.  The temptation of suicide: striatal gray matter, discounting of delayed rewards, and suicide attempts in late-life depression [published online October 17, 2011]. Psychol Med. 2012;42(6):1203-1215. doi:10.1017/S0033291711002133.
PubMed   |  Link to Article
Soulas  T, Gurruchaga  JM, Palfi  S, Cesaro  P, Nguyen  JP, Fenelon  G.  Attempted and completed suicides after subthalamic nucleus stimulation for Parkinson's disease. J Neurol Neurosurg Psychiatry. 2008;79(8):952-954.
Link to Article
Housden  CR, O’Sullivan  SS, Joyce  EM, Lees  AJ, Roiser  JP.  Intact reward learning but elevated delay discounting in Parkinson’s disease patients with impulsive-compulsive spectrum behaviors. Neuropsychopharmacology. 2010;35(11):2155-2164.
PubMed   |  Link to Article
Voon  V, Reynolds  B, Brezing  C,  et al.  Impulsive choice and response in dopamine agonist-related impulse control behaviors. Psychopharmacology (Berl). 2010;207(4):645-659.
PubMed   |  Link to Article
Weintraub  D, Siderowf  AD, Potenza  MN,  et al.  Association of dopamine agonist use with impulse control disorders in Parkinson disease. Arch Neurol. 2006;63(7):969-973.
PubMed   |  Link to Article
Hälbig  TD, Tse  W, Frisina  PG,  et al.  Subthalamic deep brain stimulation and impulse control in Parkinson’s disease. Eur J Neurol. 2009;16(4):493-497.
PubMed   |  Link to Article
Frank  MJ, Samanta  J, Moustafa  AA, Sherman  SJ.  Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science. 2007;318(5854):1309-1312.
PubMed   |  Link to Article
Baumeister  RF.  Suicide as escape from self. Psychol Rev. 1990;97(1):90-113.
PubMed   |  Link to Article
Williams  JMG, Crane  C, Barnhofer  T, Duggan  D. Psychology and suicidal behaviour: elaborating the entrapment model. In: Hawton  K, ed. Prevention and Treatment of Suicidal Behaviour: From Science to Practice. New York, NY: Oxford University Press; 2005:71-89.
Dombrovski  AY, Clark  L, Siegle  GJ,  et al.  Reward/punishment reversal learning in older suicide attempters. Am J Psychiatry. 2010;167(6):699-707.
PubMed   |  Link to Article
McGirr  A, Dombrovski  AY, Butters  MA, Clark  L, Szanto  K.  Deterministic learning and attempted suicide among older depressed individuals: cognitive assessment using the Wisconsin Card Sorting Task. J Psychiatr Res. 2012;46(2):226-232.
PubMed   |  Link to Article
Samanez-Larkin  GR, Kuhnen  CM, Yoo  DJ, Knutson  B.  Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J Neurosci. 2010;30(4):1426-1434.
PubMed   |  Link to Article
Denburg  NL, Cole  CA, Hernandez  M,  et al.  The orbitofrontal cortex, real-world decision making, and normal aging. Ann N Y Acad Sci. 2007;1121:480-498.
PubMed   |  Link to Article
Fellows  LK, Farah  MJ.  Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain. 2003;126(pt 8):1830-1837.
PubMed   |  Link to Article
Tsuchida  A, Doll  BB, Fellows  LK.  Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J Neurosci. 2010;30(50):16868-16875.
PubMed   |  Link to Article
Hornak  J, O’Doherty  J, Bramham  J,  et al.  Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J Cogn Neurosci. 2004;16(3):463-478.
PubMed   |  Link to Article
Grabenhorst  F, Rolls  ET.  Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci. 2011;15(2):56-67.
PubMed   |  Link to Article
Rushworth  MF, Noonan  MP, Boorman  ED, Walton  ME, Behrens  TE.  Frontal cortex and reward-guided learning and decision-making. Neuron. 2011;70(6):1054-1069.
PubMed   |  Link to Article
Gläscher  J, Hampton  AN, O’Doherty  JP.  Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2009;19(2):483-495.
PubMed   |  Link to Article
Plassmann  H, O’Doherty  J, Rangel  A.  Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci. 2007;27(37):9984-9988.
PubMed   |  Link to Article
Hampton  AN, Bossaerts  P, O’Doherty  JP.  The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci. 2006;26(32):8360-8367.
PubMed   |  Link to Article
Jocham  G, Neumann  J, Klein  TA, Danielmeier  C, Ullsperger  M.  Adaptive coding of action values in the human rostral cingulate zone. J Neurosci. 2009;29(23):7489-7496.
PubMed   |  Link to Article
Leyton  M, Paquette  V, Gravel  P,  et al.  Alpha-[11C]methyl-L-tryptophan trapping in the orbital and ventral medial prefrontal cortex of suicide attempters. Eur Neuropsychopharmacol. 2006;16(3):220-223.
PubMed   |  Link to Article
Oquendo  MA, Placidi  GP, Malone  KM,  et al.  Positron emission tomography of regional brain metabolic responses to a serotonergic challenge and lethality of suicide attempts in major depression. Arch Gen Psychiatry. 2003;60(1):14-22.
PubMed   |  Link to Article
Fineberg  NA, Potenza  MN, Chamberlain  SR,  et al.  Probing compulsive and impulsive behaviors, from animal models to endophenotypes: a narrative review. Neuropsychopharmacology. 2010;35(3):591-604.
PubMed   |  Link to Article
Nelson  RE, Craighead  WE.  Selective recall of positive and negative feedback, self-control behaviors, and depression. J Abnorm Psychol. 1977;86(4):379-388.
PubMed   |  Link to Article
Steffens  DC, Wagner  HR, Levy  RM, Horn  KA, Krishnan  KR.  Performance feedback deficit in geriatric depression. Biol Psychiatry. 2001;50(5):358-363.
PubMed   |  Link to Article
Elliott  R, Sahakian  BJ, Herrod  JJ, Robbins  TW, Paykel  ES.  Abnormal response to negative feedback in unipolar depression: evidence for a diagnosis specific impairment. J Neurol Neurosurg Psychiatry. 1997;63(1):74-82.
Link to Article
Elliott  R, Sahakian  BJ, McKay  AP, Herrod  JJ, Robbins  TW, Paykel  ES.  Neuropsychological impairments in unipolar depression: the influence of perceived failure on subsequent performance. Psychol Med. 1996;26(5):975-989.
PubMed   |  Link to Article
Murphy  FC, Michael  A, Robbins  TW, Sahakian  BJ.  Neuropsychological impairment in patients with major depressive disorder: the effects of feedback on task performance. Psychol Med. 2003;33(3):455-467.
PubMed   |  Link to Article
Bush  RR, Mosteller  F. Stochastic Models for Learning. Oxford, England: John Wiley & Sons Inc; 1955.
Rescorla  RA, Wagner  AR. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black  AH, Prokasy  WF, eds. Classical Conditioning II. New York, NY: Appleton-Century-Crofts; 1972:64-99.
Taylor Tavares  JV, Clark  L, Furey  ML, Williams  GB, Sahakian  BJ, Drevets  WC.  Neural basis of abnormal response to negative feedback in unmedicated mood disorders. Neuroimage. 2008;42(3):1118-1126.
PubMed   |  Link to Article
Kumar  P, Waiter  G, Ahearn  T, Milders  M, Reid  I, Steele  JD.  Abnormal temporal difference reward-learning signals in major depression. Brain. 2008;131(pt 8):2084-2093.
PubMed   |  Link to Article
Gradin  VB, Kumar  P, Waiter  G,  et al.  Expected value and prediction error abnormalities in depression and schizophrenia. Brain. 2011;134(pt 6):1751-1764.
PubMed   |  Link to Article
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders.ed 4. Washington, DC: American Psychiatric Association; 1994.
First  M Sr, Gibbon  M, Williams  JBW. Structured Clinical Interview for DSM-IV Axis I Disorders–Patient Edition (SCID-I/P), Version 2. New York: Biometrics Research Dept, New York State Psychiatric Institute; 1995.
Folstein  MF, Folstein  SE, McHugh  PR.  “Mini-Mental State”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-198.
PubMed   |  Link to Article
Beck  AT, Beck  R, Kovacs  M.  Classification of suicidal behaviors, I: quantifying intent and medical lethality. Am J Psychiatry. 1975;132(3):285-287.
PubMed
Gibbs  LM, Dombrovski  AY, Morse  J, Siegle  GJ, Houck  PR, Szanto  K.  When the solution is part of the problem: problem solving in elderly suicide attempters. Int J Geriatr Psychiatry. 2009;24(12):1396-1404.
PubMed   |  Link to Article
D’Zurilla  TJ, Nezu  AM.  Development and preliminary evaluation of the Social Problem-Solving Inventory: psychological assessment. J Consult Clin Psychol. 1990;2(2):156-163.
Patton  JH, Stanford  MS, Barratt  ES.  Factor structure of the Barratt Impulsiveness Scale. J Clin Psychol. 1995;51(6):768-774.
Link to Article
Cools  R, Clark  L, Owen  AM, Robbins  TW.  Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 2002;22(11):4563-4567.
PubMed
Cox  RW.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res. 1996;29(3):162-173.
PubMed   |  Link to Article
Forman  SD, Cohen  JD, Fitzgerald  M, Eddy  WF, Mintun  MA, Noll  DC.  Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med. 1995;33(5):636-647.
PubMed   |  Link to Article
Kringelbach  ML.  The human orbitofrontal cortex: linking reward to hedonic experience. Nat Rev Neurosci. 2005;6(9):691-702.
PubMed   |  Link to Article
Peters  J, Büchel  C.  Neural representations of subjective reward value. Behav Brain Res. 2010;213(2):135-141.
PubMed   |  Link to Article
Cox  CL, Gotimer  K, Roy  AK, Castellanos  FX, Milham  MP, Kelly  C.  Your resting brain CAREs about your risky behavior. PLoS One. 2010;5(8):e12296. doi:10.1371/journal.pone.0012296.
PubMed   |  Link to Article
Fernández-Serrano  MJ, Perales  JC, Moreno-López  L, Pérez-García  M, Verdejo-García  A.  Neuropsychological profiling of impulsivity and compulsivity in cocaine dependent individuals. Psychopharmacology (Berl). 2012;219(2):673-683.
PubMed   |  Link to Article
Ersche  KD, Roiser  JP, Robbins  TW, Sahakian  BJ.  Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology (Berl). 2008;197(3):421-431.
PubMed   |  Link to Article
de Ruiter  MB, Veltman  DJ, Goudriaan  AE, Oosterlaan  J, Sjoerds  Z, van den Brink  W.  Response perseveration and ventral prefrontal sensitivity to reward and punishment in male problem gamblers and smokers. Neuropsychopharmacology. 2009;34(4):1027-1038.
PubMed   |  Link to Article
Jentsch  JD, Olausson  P, De La Garza  R  II, Taylor  JR.  Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology. 2002;26(2):183-190.
PubMed   |  Link to Article
Calu  DJ, Stalnaker  TA, Franz  TM, Singh  T, Shaham  Y, Schoenbaum  G.  Withdrawal from cocaine self-administration produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats. Learn Mem. 2007;14(5):325-328.
PubMed   |  Link to Article
Chase  HW, Clark  L.  Gambling severity predicts midbrain response to near-miss outcomes. J Neurosci. 2010;30(18):6180-6187.
PubMed   |  Link to Article
Stalnaker  TA, Roesch  MR, Calu  DJ, Burke  KA, Singh  T, Schoenbaum  G.  Neural correlates of inflexible behavior in the orbitofrontal-amygdalar circuit after cocaine exposure. Ann N Y Acad Sci. December 2007;1121:598-609.
PubMed   |  Link to Article
Chudasama  Y, Robbins  TW.  Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci. 2003;23(25):8771-8780.
PubMed
Noonan  MP, Walton  ME, Behrens  TE, Sallet  J, Buckley  MJ, Rushworth  MF.  Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc Natl Acad Sci U S A. 2010;107(47):20547-20552.
PubMed   |  Link to Article
Ersche  KD, Roiser  JP, Abbott  S,  et al.  Response perseveration in stimulant dependence is associated with striatal dysfunction and can be ameliorated by a D(2/3) receptor agonist. Biol Psychiatry. 2011;70(8):754-762.
PubMed   |  Link to Article
Amat  J, Baratta  MV, Paul  E, Bland  ST, Watkins  LR, Maier  SF.  Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat Neurosci. 2005;8(3):365-371.
PubMed   |  Link to Article
De Martino  B. The effect of context on choice and value. In: Sharot  T, Dolan  RJ, eds. Neuroscience of Preference and Choice. London, England: Academic Press; 2012:97-106.
Corbit  LH, Balleine  BW.  Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J Neurosci. 2005;25(4):962-970.
PubMed   |  Link to Article
Talmi  D, Seymour  B, Dayan  P, Dolan  RJ.  Human pavlovian-instrumental transfer. J Neurosci. 2008;28(2):360-368.
PubMed   |  Link to Article
Behrens  TEJ, Woolrich  MW, Walton  ME, Rushworth  MFS.  Learning the value of information in an uncertain world. Nat Neurosci. 2007;10(9):1214-1221.
PubMed   |  Link to Article
O’Reilly  RC, Frank  MJ, Hazy  TE, Watz  B.  PVLV: the primary value and learned value Pavlovian learning algorithm. Behav Neurosci. 2007;121(1):31-49.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Supplement.

eFigure 1. Probabilistic reversal learning task

eFigure 2. Estimated hemodynamic response function (HRF) for the switch vs stay contrast in healthy controls (n = 20)

eFigure 3. Behavioral performance, perseverative errors, and impulsivity by group

eReferences.

Supplemental Content

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 6

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles