0
Original Article |

Genetic and Environmental Influences on Pro-Inflammatory Monocytes in Bipolar Disorder:  A Twin Study FREE

Roos C. Padmos, MD; G. Caroline M. Van Baal, PhD; Ronald Vonk, MD; Annemarie J. M. Wijkhuijs, BSc; René S. Kahn, MD, PhD; Willem A. Nolen, MD, PhD; Hemmo A. Drexhage, MD, PhD
[+] Author Affiliations

Author Affiliations: Department of Immunology, Erasmus Medical Center, Rotterdam (Drs Padmos and Drexhage and Ms Wijkhuijs); Department of Psychiatry, Rudolf Magnus Institute of Neuroscience, University Medical Center Utrecht, Utrecht (Drs Van Baal and Kahn); Reinier van Arkel Group, ’s-Hertogenbosch (Dr Vonk); and Department of Psychiatry, University Medical Center Groningen, University of Groningen, Groningen (Dr Nolen), the Netherlands.


Arch Gen Psychiatry. 2009;66(9):957-965. doi:10.1001/archgenpsychiatry.2009.116.
Text Size: A A A
Published online

Context  A monocyte pro-inflammatory state has previously been reported in bipolar disorder (BD).

Objective  To determine the contribution of genetic and environmental influences on the association between monocyte pro-inflammatory state and BD.

Design  A quantitative polymerase chain reaction case-control study of monocytes in bipolar twins. Determination of the influence of additive genetic, common, and unique environmental factors by structural equation modeling (ACE).

Setting  Dutch academic research center.

Participants  Eighteen monozygotic BD twin pairs, 23 dizygotic BD twin pairs, and 18 monozygotic and 16 dizygotic healthy twin pairs.

Main Outcome Measures  Expression levels of monocytes in the previously reported coherent set of 19 genes (signature) reflecting the pro-inflammatory state.

Results  The familial occurrence of the association between the monocyte pro-inflammatory gene-expression signature and BD found in the within-trait/cross-twin correlations (twin correlations) was due to shared environmental factors (ie, both monozygotic and dizygotic ratios in twin correlations approximated 1; ACE modeling data: 94% [95% confidence interval, 53%-99%] explained by common [shared] environmental factors). Although most individual signature genes followed this pattern, there was a small subcluster of genes in which genetic influences could dominate.

Conclusion  The association of the monocyte pro-inflammatory state with BD is primarily the result of a common shared environmental factor.

Figures in this Article

Bipolar disorder is a complex illness, and multiple genes and environmental factors determine its pathogenesis. The reported heritability ranges from 60% to 85%,1,2 which indicates that genes play a major role in pathogenesis of the disorder. However, environment is also important in bipolar disorder, which is shown by a concordance rate of only 40% to 70% in monozygotic twins.3

We recently reported the presence of a pro-inflammatory state of circulating monocytes in a considerable proportion of patients with bipolar disorder, reflected by the coherent expression of a set of 19 inflammation-related genes (the so-called PDE4B-associated pro-inflammatory gene expression signature) in their circulating monocytes. The inflammatory signature could also be found in the monocytes of a large proportion of offspring of patients with bipolar disorder and especially in children developing a mood disorder, showing that the pro-inflammatory state of monocytes precedes the actual mood symptoms.4 Our findings thus lend support to the theory that an activated inflammatory response system is a causal factor for mood symptoms in bipolar disorder. A role for an activated inflammatory response system in the etiology of mood disorders has been implicated previously and postulated as the “macrophage–T cell theory of depression” or the “inflammatory cytokine theory of depression.”57

Our offspring study mentioned above also showed that the inflammatory state of monocytes (as evidenced by the positive pro-inflammatory signature) is familial, but the study could not differentiate between genetic and shared environmental factors that determine the inflammatory signature expression. Twin studies are needed for this particular distinction. We conducted a twin study to determine (1) the contributing part of genes and environmental factors to individual differences in signature expression and (2) to what extent the observed association of bipolar disorder with the inflammatory signature is due to the same genetic effects and environmental factors.

We performed quantitative polymerase chain reactions (Q-PCRs) of the 19 reported genes in the circulating monocytes of 18 bipolar monozygotic, 23 bipolar dizygotic, 18 healthy monozygotic, and 16 healthy dizygotic twin pairs. We adopted a formal genetic model-fitting approach, the structural equation modeling. This method is capable of quantifying genetic and environmental influences on the correlation between bipolar disorder and signature expression. Structural equation modeling has extensively been described and used in schizophrenia and bipolar disorder.811

PARTICIPANTS

Outpatients with DSM-IV bipolar I or II disorder were recruited from an ongoing Dutch twin study on bipolar disorder described in detail elsewhere.12 In short, participants were twin pairs in which at least 1 twin had bipolar disorder. A DSM-IV diagnosis of bipolar disorder was made using the Structured Clinical Interview for DSM-IV Axis I disorders. Present mood states were evaluated via the Young Mania Rating Scale and the Inventory for Depressive Symptomatology. The index twins did not have a history of drug or alcohol dependency for at least 6 months, nor did they have a severe medical illness, verified through a medical history assessment.

In this study, 75 twin pairs were included: 41 bipolar twin pairs and 34 control twin pairs. Of the 41 bipolar twin pairs, 6 were monozygotic (MZ) concordant (both the index twin and co-twin had bipolar disorder), 12 pairs were MZ discordant (co-twin did not have bipolar disorder), 4 pairs were dizygotic (DZ) concordant, and 19 pairs were DZ discordant. Nine discordant co-twins (those without bipolar disorder) had other DSM-IV diagnoses: 7 had a major depressive disorder, 2 had schizophrenia, and 1 had a dissociative disorder. At the time of blood drawing, almost all patients were euthymic except for 4 patients who met the criteria for a depressive episode (Inventory for Depressive Symptomatology score >12). Seventeen index twins with bipolar disorder (8 MZ and 9 DZ) had also been included in our previous study.

Of the control twin pairs, 18 were MZ and 16 were DZ. The healthy control twins did not have an Axis I psychiatric disorder or an Axis II personality disorder. Furthermore, they had no first-degree relatives with a severe psychiatric disorder, such as schizophrenia, psychotic disorders, mood disorders, anxiety disorders, or substance use disorders. They were in self-proclaimed good health and free of any obvious medical illness for at least 2 weeks prior to blood drawing. The healthy controls did not use any psychotropic or other medication (apart from contraception hormonal therapy). The demographics of the twin pairs are summarized in Table 1 and Table 2.

Table Graphic Jump LocationTable 1. Age and Sex of All Twin Pairs
Table Graphic Jump LocationTable 2. Characteristics of Bipolar Patients

The medical ethical review committee of the University Medical Center Utrecht approved the studies. Written informed consent was obtained from all participants after a complete description of the study was given.

LABORATORY METHODS
Zygosity Determination and Blood Collection and Preparation

Zygosity was determined by DNA fingerprinting, using 9 to 11 high polymorphic microsatellite markers in the laboratory of the Division of Genetics, University Medical Center Utrecht. Blood was collected in a clotting tube for serum preparation (frozen and stored at −80°C) and in sodium-heparin tubes for immune cell preparation. From the heparinized blood, peripheral blood mononuclear cell suspensions were prepared using low-density gradient centrifugation (as described in detail before13) within 8 hours to avoid ex vivo activation of the monocytes. Peripheral blood mononuclear cells were frozen in RPMI-1640 (Sigma-Aldrich, St Louis, Missouri) with 25mM 4-(2-hydroxyethyl)-1-piperazineethanesulfonic acid and UltraGlutamine containing 10% fetal calf serum and 10% dimethylsulfoxide and stored in liquid nitrogen. This enabled us to store the samples in the hospital where the patients were seen and where the blood was drawn, to ship them deep-frozen to Rotterdam, and to test patient and control immune cells in the same series of experiments.

Isolation of Monocytes

CD14+ monocytes were isolated from frozen peripheral blood mononuclear cells from selected subjects (Table 1) using a magnetic cell-sorting system (Miltenyi Biotec, Bergisch Gladbach, Germany). The viability and purity of monocytes was more than 95% (determined by morphological screening after trypan blue staining and flow cytometry analysis).

Quantitative PCR

The RNA was isolated from monocytes using RNeasy columns (Qiagen, Hilden, Germany). After extraction, the RNA concentration was determined and RNA was stored at −80°C until use.

To obtain complementary DNA for the Q-PCR, we used an extensively described, optimized protocol, as previously described.14 One microgram of RNA was reverse-transcribed using SuperScriptII (Invitrogen, Carlsbad, California) and random hexamers (Amersham Biosciences, Roosendaal, the Netherlands) for 50 minutes at 42°C.

Quantitative PCR was performed with TaqMan Universal PCR master mix (Applied Biosystems, Foster City, California). All Taqman probes and consensus primers were preformulated and designed by Applied Biosystems (eTable 1). Polymerase chain reactions were performed for 2 minutes at 50°C and 10 minutes at 95°C, followed by 40 cycles of 15 seconds at 95°C and 1 minute at 60°C. Polymerase chain reaction amplification of the reference gene ABL was performed for each sample to allow normalization between the samples. ABL was chosen as the reference gene because it was previously shown to be the most consistently expressed endogenous control in hematopoietic cells.15 The quantitative value obtained from Q-PCR is a cycle threshold (CT). The fold change values between different groups were determined from the normalized CT values (CT gene minus CT housekeeping gene) by the ΔΔCT method (2−ΔΔCT, User Bulletin 2, Applied Biosystems) (eTable 2). To correct for interassay variance, we set the mean of the studied genes found in the healthy control index twins in the same assay for each gene to 1 (∑ ΔCT healthy control index twins = 0; 2−0 = 1). The fold change values of the genes in patient monocytes were expressed relative to this set mean of 1 in the healthy controls. These fold change values were used in the statistical methods described below.

STATISTICAL ANALYSIS

For the structural equation modeling approach, the statistical package Mx was used for analysis,16 and bivariate twin models were set up to describe the data. In these models, the variance of a certain trait (such as signature positivity) or the covariance between 2 traits (such as signature positivity and bipolar disorder) can be due to additive genetic (A); common environmental (C), such as socioeconomic status and childhood diet; and/or unique environmental influences (E), eg, accidents (Figure).

Place holder to copy figure label and caption
Figure.

Path diagram of a bivariate twin model (Cholesky decomposition). Bipolar disorder (BD) and monocyte signature expression (SIG) (on a 5-point scale) in the index twin and co-twin are influenced by additive genetic (A), common environmental (C), and unique environmental (E) unobserved latent factors. One set of factors loads on both observed traits; the other set loads on signature expression only. The magnitude of the effects of A, C, and E on BD (reflected in the factor loadings) are fixed at population values so that h2, c2, and e2, are 70%, 15% and 15%, respectively. Factor loadings of the 2 sets of latent factors on signature expression are estimated (aBD, cBD, and eBD for the factors that load on both traits and aSIG, cSIG, and eSIG for the factors that load on monocyte signature expression only). A strong genetic correlation is reflected in a large aBD compared with aSIG and vice versa (same for common environmental and unique environmental influences). The 6 different parameters can be estimated because the latent factors show a distinctive pattern of correlations among each other: additive genetic factors between twins are correlated at 1 for monozygotic (MZ) twins and 0.5 for dizygotic (DZ) twins, while common environmental factors correlate at 1 for both types of twin pairs. Unique environmental factors are by definition uncorrelated.

Graphic Jump Location

Identical (MZ) twins share all of their genes; therefore they correlate at 1 for additive genetic effects, whereas nonidentical twins (DZ) share on average only half of their genes and thus correlate at 0.5 for additive genetic effects. However, with regard to the shared environment, both MZ and DZ twin pairs correlate at 1. For example, parenting, exposure to infections, and diet are in most cases experienced the same for both MZ and DZ twins living together. In contrast, the correlations between unique environmental influences are modeled as 0, because they are by definition not shared between twins. This means that if a higher correlation for a certain trait is found in MZ twins than in DZ twins it must be due to genetic effects (because the correlation for environmental influences is the same in DZ and MZ twins), but if the same correlation (no 0) is found in MZ and DZ twin pairs, then it must be due to shared environment.

Bivariate twin modeling was used to estimate the magnitude of the effects of A, C, and E on signature positivity and its interdependence with bipolar disorder, which is expressed in parameter estimates a, c, and e, respectively. The relative contribution of A to the variance of signature positivity (univariate heritability) and to the covariance between bipolar disorder liability and signature positivity (bivariate heritability) was expressed as the percentage of genetic variance within total (co)variance. The univariate (narrow sense) heritability is calculated thusly: h2univariate = Var A/(Var A + Var C + Var E). The bivariate heritability is calculated with the equation h2bivariate =(Cov A)/(Cov A + Cov C + Cov E). Univariate and bivariate c2 and e2 are calculated in a similar way.

Prior to structural equation modeling, various correlations were determined in MZ and DZ twins: (1) cross-trait/within-twin correlations, which are called trait correlations (in our case, bipolar disorder with signature positivity within the same subject), (2) within-trait/cross-twin correlations, which are called twin correlations (signature positivity in the index twin and his or her co-twin in MZ and DZ twin pairs), and (3) cross-trait/cross-twin correlations, which are called cross-correlations (bipolar disorder in the index twin and signature positivity of his or her co-twin and vice versa in MZ and DZ twin pairs). A significant trait correlation suggests a common etiologic influence on both traits. Significant twin correlations, or cross-correlations, imply that this common influence is familial. Monozygotic and DZ twin correlation, or cross-correlation, ratios of 2:1 suggest that this influence is due to genetic effects (A), while ratios of 1:1 imply the effect of shared environment (C). A nonsignificant cross-correlation combined with a significant trait correlation means that the common etiologic influence on both traits is probably due to a unique environmental influence (E). Using the information from these correlations, the phenotypic correlation (rph) between 2 traits (bipolar disorder and signature positivity) can be determined and decomposed into genetic (rg), common environmental (rc), and unique environmental (re) components. The rg indicates to which degree the same genes influence both bipolar disorder and signature positivity; rc indicates to which degree the same shared environmental factors influence both bipolar disorder and signature positivity; and re provides information about the possible effect of unique environmental factors on both traits. By combining information from the rg, rc, and re correlations with a2, c2, and e2, the influence of genetic, common, and unique environmental factors on the total correlation between bipolar disorder and signature positivity could be established.

Ninety-five percent confidence intervals (CIs) of the parameter estimates were calculated and used to draw conclusions.17 The parameter estimate can be fixed at all the values in the CI without resulting in a significantly worse fit of the model (at α = .05).

For the bivariate twin modeling liability threshold models were used, meaning that both affected and unaffected individuals have a liability to develop bipolar disorder and that if a certain threshold was crossed a person has bipolar disorder and if not the person is considered healthy. However, since twin pairs in our sample were specifically selected for bipolar disorder, we could not estimate h2, c2, or prevalence in this sample. Hence, parameters were fixed to values that agree with the literature (h2 = 70%, c2 = 15%, prevalence = 1%).2,18,19 We allowed for common environmental influences on bipolar disorder to detect a common environmentally mediated association between signature positivity and bipolar disorder. Fixing h2, c2, and prevalence to different values (h2 = 80% or 60%, c2 = 25% or 5%, prevalence = 0.5%) did not alter conclusions, though parameter estimates were slightly different (results available from corresponding author on request).

The Mx program cannot process dichotomous and continuous data together; therefore the monocyte gene-expression levels were categorized into a 5-category ordinal scale (approximately 20% per category). Overall, the data were slightly negatively skewed (except for PTGS2, ATF3, NAB2, and MAPK6). For a few genes, the distribution was significantly leptokurtic: IL6, PTX3, BCL2A1, EMP1, and MAPK6. However, because the data were transformed to an ordinal scale (quintiles), this had no effect on the results.

PREVALENCE OF THE PDE4B-ASSOCIATED PRO-INFLAMMATORY MONOCYTE SIGNATURE

The pro-inflammatory PDE4B-associated signature appeared to be present—depending on the definition—in 20% to 41% of the presently studied bipolar index cases (Table 3). Signature positivity, PDE4B expression, and the fold change values of the separate signature genes of all twin groups are presented in Table 4 and eTable 2 and eTable 4. The correlation of the separate signature genes with each other is shown in the eFigure. In essence, these data confirm our previous data of a high prevalence of pro-inflammatory monocytes in singletons.4

Table Graphic Jump LocationTable 3. Presence of a PDE4B-Associated mRNA Signature in Bipolar Index Twins and Healthy Twin Pairs
Table Graphic Jump LocationTable 4. Concordance of PDE4B Expression in Monozygotic and Dizygotic Twins
ASSOCIATION OF BIPOLAR DISORDER WITH SIGNATURE POSITIVITY

We continued performing the specific twin statistical analysis on our twin data. Table 5 shows that the cross-trait/within-twin correlations (trait correlations) of the monocyte pro-inflammatory signature are significant, again illustrating that bipolar disorder is associated with the presence of the monocyte pro-inflammatory signature in this twin sample.

Table Graphic Jump LocationTable 5. Cross-Trait/Within-Twin, Within-Trait/Cross-Twin, and Cross-Trait/Cross-Twin Correlations on Signature Positivity and mRNA Expression
EFFECT OF GENES AND ENVIRONMENTAL FACTORS ON SIGNATURE POSITIVITY

A familial effect on the presence of pro-inflammatory monocytes, as determined by signature positivity, is indicated by the significance of the within-trait/cross-twin correlations (twin-correlations, Table 5). This familial effect is most likely mainly due to shared environmental factors, which is suggested by the MZ and DZ twin correlation ratios approximating 1 (Table 5). The ACE modeling data presented in Table 6 also demonstrate that the variance of signature positivity is probably not caused by genetic effects (h2 = 0%; 95% CI, 0%-44%) but is most likely due to shared environmental influences (c2 = 95%; 95% CI, 53%-99%).

Table Graphic Jump LocationTable 6. Estimated Influences of Additive Genetic (h2), Common (c2), and Unique (e2) Environmental Factors on Monocyte Gene Expression Signature and the Association Between Monocyte Gene Expression Signature and Bipolar Disorder
EFFECT OF GENES AND ENVIRONMENTAL FACTORS ON THE ASSOCIATION BETWEEN BIPOLAR DISORDER AND SIGNATURE POSITIVITY

The association of pro-inflammatory monocytes with bipolar disorder through a familial effect is indicated by the significance of the cross-trait/cross-twin correlations (cross-correlation, Table 5). This familial effect is probably mainly the result of shared environmental factors, which is suggested by the MZ and DZ cross-correlation ratios approximating 1 (Table 5). The ACE modeling data confirm the large contribution of common environment to the covariance between bipolar disorder and signature positivity; it is shown in Table 6 that c2 must be at least 45% responsible for the association of signature positivity with bipolar disorder. This means that similar shared/common environmental factors influence the cooccurrence of bipolar disorder and signature expression.

ASSOCIATION OF BIPOLAR DISORDER WITH MESSENGER RNA EXPRESSION OF SEPARATE INFLAMMATORY SIGNATURE GENES

Analysis of the separate signature genes confirmed our and others' findings. Table 5 shows that bipolar disorder is associated with aberrant messenger RNA expression levels of almost all signature genes tested (because of the significance of the trait correlations), except for NAB2.

EFFECT OF GENES AND ENVIRONMENTAL FACTORS ON EXPRESSION OF SEPARATE INFLAMMATORY SIGNATURE GENES

In most genes, a familial effect was found on the expression of the separate signature genes (the twin correlations were significant, Table 5). For the entire signature, this effect was primarily due to common environment (MZ and DZ twin correlation ratios mostly approximated 1).

The separate signature gene expressions were fitted into an ACE model. The estimated heritabilities (h2) and environmental variances (c2 and e2) of the gene-expression levels indicate that the variance of gene expression is, for most separate genes, explained by common environmental influences (Table 6). The additive genetic effect does not seem to explain much of the variance of the gene-expression levels.

EFFECT OF GENES AND ENVIRONMENTAL FACTORS ON THE ASSOCIATION BETWEEN BIOPOLAR DISORDER AND MESSENGER RNA EXPRESSION IN SEPARATE SIGNATURE GENES

The familial effect seen on the association of bipolar disorder and most inflammatory signature genes is suggested to be shared environment, because the MZ and DZ cross-correlation ratios approximate 1. However, it must be noted that for some of the motility/chemotaxis genes (CCL2, CCL7, EMP1, and CDC42), the MZ cross-correlations are higher than the DZ cross-correlations (approximating a 2:1 ratio), suggesting that genetic effects influence the association of those particular genes with bipolar disorder.

The influence of h2, c2, and e2 on the covariance of the separate genes with bipolar disorder is presented in Table 6. For the separate signature genes, it is difficult to assign either h2, c2, or e2 as the predominant contributor, since the CIs are wide. However, it can be noted that for most genes, h2 and/or c2 must explain some of the covariance, since the 95% CI of e2 never reaches 100%. More information can be obtained from the analysis of signature positivity described above, in which separate gene-expression levels are combined into a signature. It is shown that common environment is the most important factor influencing the association of bipolar disorder with pro-inflammatory monocytes.

This study points to common environmental factors as the main contributing factors to the pro-inflammatory state of monocytes. By applying structural equation modeling, we demonstrated (by univariate analysis) that of the total variance for PDE4B-associated signature positivity, 95% (95% CI, 53%-99%) could be explained by shared environmental factors.

In addition, this study demonstrates that common environmental factors are the main contributing factors to the association of the pro-inflammatory monocytes with bipolar disorder. By using a bivariate model, we showed that of the total covariance between bipolar disorder and PDE4B-asociated signature positivity, 94% (95% CI, 45%-100%) was due to shared common environment and not to genetic effects (h2 = 1%; 95% CI, 0%-50%) or unique environmental factors (e2 = 5%; 95% CI, 0%-24%).

Our study has limitations that need to be considered. First, because a shared common environment is such an important factor in explaining variance of signature expression, the question arises whether our experimental setup could be the shared environmental factor and if this could bias results. Though most blood drawing, blood processing, and experiments were done at the same time in twin pairs (the putative common shared environmental factor), we also had twin pairs who were handled separately in each step. The same high concordance rates were also observed in both the MZ and DZ twin pairs. Also, when we examined data from only 1 experiment (thus excluding experiment as a confounding factor), we saw the same results. Furthermore, “experiment” was included in the analysis as an independent variable, which excluded experimental setting as a confounding factor. We are therefore confident that the observed data are indeed explained by shared environmental factors other than the experimental setup.

Second, various confounding factors, such as medication use, smoking, and physical exercise, could have affected our data. We corrected for the most important confounding factor, lithium treatment, as lithium is known to have immune-modulating effects and is capable of influencing gene expression.4,20 This analysis did not alter results or conclusions. Because a relatively low number of subjects were without lithium treatment, this analysis is sensitive to outliers. Therefore, we have chosen to report the more conservative non–lithium-corrected analyses (lithium-corrected data in eTable 5 and eTable 6). Unfortunately, we could not correct for the other mentioned possible confounding factors, as that information was not available.

Third, concordance in this study was defined as the co-twin having bipolar disorder. However, some of the co-twins had a broader psychiatric phenotype (7 had a major depressive disorder, 2 had schizophrenia, and 1 had a dissociative disorder). We repeated the analysis, treating the 10 discordant twins with a broader psychiatric phenotype as concordant. This kind of analysis did not change our results or conclusions (eTable 7 and eTable 8).

Fourth, data on protein expression would have strengthened our results, yet it is our experience that the gene-expression signatures in monocytes correlate with the serum cytokine levels, yet the latter are far less sensitive and precise than the signature test.4 Fifth, it should also be noted that the 19 signature genes tested might only form part of a more extensive molecular pro-inflammatory system in the monocytes of bipolar patients. Nevertheless, most of the tested genes clearly indicate an environmental dependence. Lastly, although our bipolar disorder twin sample is, as far as we know, the largest studied to date, it is still not large enough to make definite conclusions.

Theoretically, the correlation between bipolar disorder and pro-inflammatory monocytes can be explained in 4 ways. First, bipolar disorder itself induces the pro-inflammatory state of monocytes (eg, the stressful state of the illness might induce monocyte activation). Second, the pro-inflammatory monocytes cause mood disorders in patients (the macrophage theory of depression). Third, there could be a separate underlying factor that influences bipolar disorder as well as monocytes independently from each other (eg, patients and their family members are present in an infectious/stressful environment that affects both their monocyte systems as well as their brains). Fourth, it is possible that there are 2 unrelated underlying factors shared in the environment, one leading to monocyte activation and the other to bipolar disorder (eg, 2 separate infections due to a common shared environmental factor, eg, a deprived condition, one leading to monocyte activation and the other to bipolar disorder). Because our data demonstrate that the pro-inflammatory state of the monocytes is mainly explained by a shared environmental factor, while it is known from literature that bipolar disorder itself is heritable for 60% to 85%, we can use the patterns of cross-twin/cross-trait correlations (cross-correlations) to falsify some of these theoretical explanations.21 If bipolar disorder itself was responsible for the monocyte activation, the observed MZ and DZ cross-correlation ratios should approximate 0.5, because genes are very important for bipolar disorder itself (and not for signature expression). Therefore, because DZ twins share only half of their genes, their observed correlation should be less than that in MZ twins. However, in our study the MZ and DZ cross-correlation ratios approximate 1. It is therefore reasonable to conclude that bipolar disorder is not a causative factor for monocyte activation. Regarding the other 3 explanations (whether pro-inflammatory monocytes cause bipolar disorder or whether a separate underlying factor influences both bipolar disorder as well as the monocyte or both separately), our data do not provide a solution.

It must be noted that this strong effect of common environment did not apply to all genes in the signature. Some of the genes (CCL2, CCL7, EMP1, and CDC42) were clearly influenced by genetic factors, and interestingly in a previous study,22 we found that CCL2, CCL7, EMP1, and CDC42 belong to a separate subcluster of motility/chemotaxis and adhesion genes, which are—though weaker—correlated with the core inflammatory genes (such as PDE4B, the cytokines, and inflammatory compounds). It is tempting to speculate that the expression of this subcluster of genes in monocytes is more under the control of genetic factors and that common environmental factors play a minor role.

There is a multitude of environmental factors that can act as shared environmental factors associated with both bipolar disorder and pro-inflammatory monocyte activation. Various chronic infections, such as infections with Toxoplasma gondii,23 herpes simplex virus, Cytomegalovirus, Borna virus,2426 influenza,27 and rubella,28 have been described as being associated with psychiatric diseases. Some of these microbes are known as good stimulators of monocytes, eg, T gondii. It is thought that infections with these microbes can be introduced in early youth via prenatal exposure27,28 or via household crowding.29

One of the next possible common environmental factors is stress. Increasing amounts of evidence suggest that an early exposure to stress is a risk factor for psychopathology. There is a large body of literature showing that maternal prenatal stress leads to altered set points in the immunoneuroendocrine system of the offspring and a later development of various depressive behavioral problems.3035 Stress experienced later in life is also able to influence the immune system as well as mood disorder development.3639 Apart from infections and stress, diet, in particular a diet insufficient in specific nutrients (such as ω-3 fatty acids), is another set of possibly shared environmental factors linking bipolar disorder with the immune system.40

In conclusion, our twin study shows that common environmental factors are most likely responsible for the pro-inflammatory monocyte activation seen in bipolar disorder. When acting on a susceptible genetic background, the environmentally induced pro-inflammatory monocyte activation can be seen as a factor precipitating disease.

Correspondence: Hemmo A. Drexhage, MD, PhD, Department of Immunology, PO Box 1738, 3000 DR Rotterdam, the Netherlands (h.drexhage@erasmusmc.nl).

Submitted for Publication: August 26, 2008; final revision received January 6, 2009; accepted February 11, 2009.

Financial Disclosure: None reported.

Additional Contributions: Harm de Wit, BSc, provided technical assistance.

Farmer  AElkin  A McGuffin  P The genetics of bipolar affective disorder. Curr Opin Psychiatry 2007;20 (1) 8- 12
PubMed
Smoller  JWFinn  CT Family, twin, and adoption studies of bipolar disorder. Am J Med Genet C Semin Med Genet 2003;123C (1) 48- 58
PubMed
Taylor  LFaraone  SVTsuang  MT Family, twin, and adoption studies of bipolar disease. Curr Psychiatry Rep 2002;4 (2) 130- 133
PubMed
Padmos  RCHillegers  MHKnijff  EMVonk  RBouvy  AStaal  FJde Ridder  DKupka  RWNolen  WADrexhage  HA A discriminating messenger RNA signature for bipolar disorder formed by an aberrant expression of inflammatory genes in monocytes. Arch Gen Psychiatry 2008;65 (4) 395- 407
PubMed
Smith  RS The macrophage theory of depression. Med Hypotheses 1991;35 (4) 298- 306
PubMed
Leonard  BE The immune system, depression and the action of antidepressants. Prog Neuropsychopharmacol Biol Psychiatry 2001;25 (4) 767- 780
PubMed
Maes  M Evidence for an immune response in major depression: a review and hypothesis. Prog Neuropsychopharmacol Biol Psychiatry 1995;19 (1) 11- 38
PubMed
Brans  RGvan Haren  NEvan Baal  GCSchnack  HGKahn  RSHulshoff Pol  HE Heritability of changes in brain volume over time in twin pairs discordant for schizophrenia. Arch Gen Psychiatry 2008;65 (11) 1259- 1268
PubMed
Hall  MHRijsdijk  FPicchioni  MSchulze  KEttinger  UToulopoulou  TBramon  EMurray  RMSham  P Substantial shared genetic influences on schizophrenia and event-related potentials. Am J Psychiatry 2007;164 (5) 804- 812
PubMed
Toulopoulou  TPicchioni  MRijsdijk  FHua-Hall  MEttinger  USham  PMurray  R Substantial genetic overlap between neurocognition and schizophrenia: genetic modeling in twin samples. Arch Gen Psychiatry 2007;64 (12) 1348- 1355
PubMed
Rijsdijk  FVvan Haren  NEPicchioni  MM McDonald  CToulopoulou  THulshoff Pol  HEKahn  RSMurray  RSham  PC Brain MRI abnormalities in schizophrenia: same genes or same environment? Psychol Med 2005;35 (10) 1399- 1409
PubMed
Vonk  Rvan der Schot  ACKahn  RSNolen  WADrexhage  HA Is autoimmune thyroiditis part of the genetic vulnerability (or an endophenotype) for bipolar disorder? Biol Psychiatry 2007;62 (2) 135- 140
PubMed
Knijff  EMBreunis  MNvan Geest  MCKupka  RWRuwhof  Cde Wit  HJNolen  WADrexhage  HA A relative resistance of T cells to dexamethasone in bipolar disorder. Bipolar Disord 2006;8 (6) 740- 750
PubMed
Gabert  JBeillard  Evan der Velden  VHBi  WGrimwade  DPallisgaard  NBarbany  GCazzaniga  GCayuela  JMCave  HPane  FAerts  JLDe Micheli  DThirion  XPradel  VGonzalez  MViehmann  SMalec  MSaglio  Gvan Dongen  JJ Standardization and quality control studies of ‘real-time’ quantitative reverse transcriptase polymerase chain reaction of fusion gene transcripts for residual disease detection in leukemia: a Europe Against Cancer program. Leukemia 2003;17 (12) 2318- 2357
PubMed
Beillard  EPallisgaard  Nvan der Velden  VHBi  WDee  Rvan der Schoot  EDelabesse  EMacintyre  EGottardi  ESaglio  GWatzinger  FLion  Tvan Dongen  JJHokland  PGabert  J Evaluation of candidate control genes for diagnosis and residual disease detection in leukemic patients using ‘real-time’ quantitative reverse-transcriptase polymerase chain reaction (RQ-PCR): a Europe against cancer program. Leukemia 2003;17 (12) 2474- 2486
PubMed
Neale  MCBSXie  GMaes  HH Mx: statistical modeling. 7th ed. Richmond, VA Dept of Psychiatry, Virginia Commonwealth University2006;
Neale  MCMiller  MB The use of likelihood-based confidence intervals in genetic models. Behav Genet 1997;27 (2) 113- 120
PubMed
Regeer  EJten Have  MRosso  MLHakkaart-van Roijen  LVollebergh  WNolen  WA Prevalence of bipolar disorder in the general population: a reappraisal study of the Netherlands Mental Health Survey and Incidence Study. Acta Psychiatr Scand 2004;110 (5) 374- 382
PubMed
ten Have  MVollebergh  WBijl  RNolen  WA Bipolar disorder in the general population in the Netherlands (prevalence, consequences and care utilisation): results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). J Affect Disord 2002;68 (2-3) 203- 213
PubMed
Rybakowski  JK Antiviral and immunomodulatory effect of lithium. Pharmacopsychiatry 2000;33 (5) 159- 164
PubMed
Gillespie  NAM Direction of causation models. Everitt  BSHowell  DCThe Encyclopedia of Statistics in Behavioral Science. Hoboken, NJ Wiley2005;496- 499
Padmos  RCSchloot  NCBeyan  HRuwhof  CStaal  FJde Ridder  DAanstoot  HJTse  WKde Wit  HHerder  CDrexhage  RCMenart  BLeslie  RDDrexhage  HA Distinct monocyte gene-expression profiles in autoimmune diabetes. Diabetes 2008;57 (10) 2768- 2773
PubMed
Torrey  EFBartko  JJLun  ZRYolken  RH Antibodies to Toxoplasma gondii in patients with schizophrenia: a meta-analysis. Schizophr Bull 2007;33 (3) 729- 736
PubMed
Hinze-Selch  D Infection, treatment and immune response in patients with bipolar disorder versus patients with major depression, schizophrenia or healthy controls. Bipolar Disord 2002;4 ((suppl 1)) 81- 83
PubMed
Taieb  OBaleyte  JMMazet  PFillet  AM Borna disease virus and psychiatry. Eur Psychiatry 2001;16 (1) 3- 10
PubMed
Yolken  RHTorrey  EF Viruses, schizophrenia, and bipolar disorder. Clin Microbiol Rev 1995;8 (1) 131- 145
PubMed
Machón  RAMednick  SAHuttunen  MO Adult major affective disorder after prenatal exposure to an influenza epidemic. Arch Gen Psychiatry 1997;54 (4) 322- 328
PubMed
Brown  AS Prenatal infection as a risk factor for schizophrenia. Schizophr Bull 2006;32 (2) 200- 202
PubMed
Torrey  EFYolken  RH At issue: is household crowding a risk factor for schizophrenia and bipolar disorder? Schizophr Bull 1998;24 (3) 321- 324
PubMed
Abe  HHidaka  NKawagoe  COdagiri  KWatanabe  YIkeda  TIshizuka  YHashiguchi  HTakeda  RNishimori  TIshida  Y Prenatal psychological stress causes higher emotionality, depression-like behavior, and elevated activity in the hypothalamo-pituitary-adrenal axis. Neurosci Res 2007;59 (2) 145- 151
PubMed
Costello  EJWorthman  CErkanli  AAngold  A Prediction from low birth weight to female adolescent depression: a test of competing hypotheses. Arch Gen Psychiatry 2007;64 (3) 338- 344
PubMed
Wals  MReichart  CGHillegers  MHVan Os  JVerhulst  FCNolen  WAOrmel  J Impact of birth weight and genetic liability on psychopathology in children of bipolar parents. J Am Acad Child Adolesc Psychiatry 2003;42 (9) 1116- 1121
PubMed
Van den Bergh  BRMulder  EJMennes  MGlover  V Antenatal maternal anxiety and stress and the neurobehavioural development of the fetus and child: links and possible mechanisms—a review. Neurosci Biobehav Rev 2005;29 (2) 237- 258
PubMed
Maccari  SMorley-Fletcher  S Effects of prenatal restraint stress on the hypothalamus-pituitary-adrenal axis and related behavioural and neurobiological alterations. Psychoneuroendocrinology 2007;32 ((suppl 1)) S10- S15
PubMed
Austin  MPLeader  LRReilly  N Prenatal stress, the hypothalamic-pituitary-adrenal axis, and fetal and infant neurobehaviour. Early Hum Dev 2005;81 (11) 917- 926
PubMed
Leonard  BE HPA and immune axes in stress: involvement of the serotonergic system. Neuroimmunomodulation 2006;13 (5-6) 268- 276
PubMed
Padgett  DAGlaser  R How stress influences the immune response. Trends Immunol 2003;24 (8) 444- 448
PubMed
Willner  P Chronic mild stress (CMS) revisited: consistency and behavioural-neurobiological concordance in the effects of CMS. Neuropsychobiology 2005;52 (2) 90- 110
PubMed
Hillegers  MHBurger  HWals  MReichart  CGVerhulst  FCNolen  WAOrmel  J Impact of stressful life events, familial loading and their interaction on the onset of mood disorders: study in a high-risk cohort of adolescent offspring of parents with bipolar disorder. Br J Psychiatry 2004;18597- 101
PubMed
Parker  GGibson  NABrotchie  HHeruc  GRees  AMHadzi-Pavlovic  D Omega-3 fatty acids and mood disorders. Am J Psychiatry 2006;163 (6) 969- 978
PubMed

Figures

Place holder to copy figure label and caption
Figure.

Path diagram of a bivariate twin model (Cholesky decomposition). Bipolar disorder (BD) and monocyte signature expression (SIG) (on a 5-point scale) in the index twin and co-twin are influenced by additive genetic (A), common environmental (C), and unique environmental (E) unobserved latent factors. One set of factors loads on both observed traits; the other set loads on signature expression only. The magnitude of the effects of A, C, and E on BD (reflected in the factor loadings) are fixed at population values so that h2, c2, and e2, are 70%, 15% and 15%, respectively. Factor loadings of the 2 sets of latent factors on signature expression are estimated (aBD, cBD, and eBD for the factors that load on both traits and aSIG, cSIG, and eSIG for the factors that load on monocyte signature expression only). A strong genetic correlation is reflected in a large aBD compared with aSIG and vice versa (same for common environmental and unique environmental influences). The 6 different parameters can be estimated because the latent factors show a distinctive pattern of correlations among each other: additive genetic factors between twins are correlated at 1 for monozygotic (MZ) twins and 0.5 for dizygotic (DZ) twins, while common environmental factors correlate at 1 for both types of twin pairs. Unique environmental factors are by definition uncorrelated.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable 1. Age and Sex of All Twin Pairs
Table Graphic Jump LocationTable 2. Characteristics of Bipolar Patients
Table Graphic Jump LocationTable 3. Presence of a PDE4B-Associated mRNA Signature in Bipolar Index Twins and Healthy Twin Pairs
Table Graphic Jump LocationTable 4. Concordance of PDE4B Expression in Monozygotic and Dizygotic Twins
Table Graphic Jump LocationTable 5. Cross-Trait/Within-Twin, Within-Trait/Cross-Twin, and Cross-Trait/Cross-Twin Correlations on Signature Positivity and mRNA Expression
Table Graphic Jump LocationTable 6. Estimated Influences of Additive Genetic (h2), Common (c2), and Unique (e2) Environmental Factors on Monocyte Gene Expression Signature and the Association Between Monocyte Gene Expression Signature and Bipolar Disorder

References

Farmer  AElkin  A McGuffin  P The genetics of bipolar affective disorder. Curr Opin Psychiatry 2007;20 (1) 8- 12
PubMed
Smoller  JWFinn  CT Family, twin, and adoption studies of bipolar disorder. Am J Med Genet C Semin Med Genet 2003;123C (1) 48- 58
PubMed
Taylor  LFaraone  SVTsuang  MT Family, twin, and adoption studies of bipolar disease. Curr Psychiatry Rep 2002;4 (2) 130- 133
PubMed
Padmos  RCHillegers  MHKnijff  EMVonk  RBouvy  AStaal  FJde Ridder  DKupka  RWNolen  WADrexhage  HA A discriminating messenger RNA signature for bipolar disorder formed by an aberrant expression of inflammatory genes in monocytes. Arch Gen Psychiatry 2008;65 (4) 395- 407
PubMed
Smith  RS The macrophage theory of depression. Med Hypotheses 1991;35 (4) 298- 306
PubMed
Leonard  BE The immune system, depression and the action of antidepressants. Prog Neuropsychopharmacol Biol Psychiatry 2001;25 (4) 767- 780
PubMed
Maes  M Evidence for an immune response in major depression: a review and hypothesis. Prog Neuropsychopharmacol Biol Psychiatry 1995;19 (1) 11- 38
PubMed
Brans  RGvan Haren  NEvan Baal  GCSchnack  HGKahn  RSHulshoff Pol  HE Heritability of changes in brain volume over time in twin pairs discordant for schizophrenia. Arch Gen Psychiatry 2008;65 (11) 1259- 1268
PubMed
Hall  MHRijsdijk  FPicchioni  MSchulze  KEttinger  UToulopoulou  TBramon  EMurray  RMSham  P Substantial shared genetic influences on schizophrenia and event-related potentials. Am J Psychiatry 2007;164 (5) 804- 812
PubMed
Toulopoulou  TPicchioni  MRijsdijk  FHua-Hall  MEttinger  USham  PMurray  R Substantial genetic overlap between neurocognition and schizophrenia: genetic modeling in twin samples. Arch Gen Psychiatry 2007;64 (12) 1348- 1355
PubMed
Rijsdijk  FVvan Haren  NEPicchioni  MM McDonald  CToulopoulou  THulshoff Pol  HEKahn  RSMurray  RSham  PC Brain MRI abnormalities in schizophrenia: same genes or same environment? Psychol Med 2005;35 (10) 1399- 1409
PubMed
Vonk  Rvan der Schot  ACKahn  RSNolen  WADrexhage  HA Is autoimmune thyroiditis part of the genetic vulnerability (or an endophenotype) for bipolar disorder? Biol Psychiatry 2007;62 (2) 135- 140
PubMed
Knijff  EMBreunis  MNvan Geest  MCKupka  RWRuwhof  Cde Wit  HJNolen  WADrexhage  HA A relative resistance of T cells to dexamethasone in bipolar disorder. Bipolar Disord 2006;8 (6) 740- 750
PubMed
Gabert  JBeillard  Evan der Velden  VHBi  WGrimwade  DPallisgaard  NBarbany  GCazzaniga  GCayuela  JMCave  HPane  FAerts  JLDe Micheli  DThirion  XPradel  VGonzalez  MViehmann  SMalec  MSaglio  Gvan Dongen  JJ Standardization and quality control studies of ‘real-time’ quantitative reverse transcriptase polymerase chain reaction of fusion gene transcripts for residual disease detection in leukemia: a Europe Against Cancer program. Leukemia 2003;17 (12) 2318- 2357
PubMed
Beillard  EPallisgaard  Nvan der Velden  VHBi  WDee  Rvan der Schoot  EDelabesse  EMacintyre  EGottardi  ESaglio  GWatzinger  FLion  Tvan Dongen  JJHokland  PGabert  J Evaluation of candidate control genes for diagnosis and residual disease detection in leukemic patients using ‘real-time’ quantitative reverse-transcriptase polymerase chain reaction (RQ-PCR): a Europe against cancer program. Leukemia 2003;17 (12) 2474- 2486
PubMed
Neale  MCBSXie  GMaes  HH Mx: statistical modeling. 7th ed. Richmond, VA Dept of Psychiatry, Virginia Commonwealth University2006;
Neale  MCMiller  MB The use of likelihood-based confidence intervals in genetic models. Behav Genet 1997;27 (2) 113- 120
PubMed
Regeer  EJten Have  MRosso  MLHakkaart-van Roijen  LVollebergh  WNolen  WA Prevalence of bipolar disorder in the general population: a reappraisal study of the Netherlands Mental Health Survey and Incidence Study. Acta Psychiatr Scand 2004;110 (5) 374- 382
PubMed
ten Have  MVollebergh  WBijl  RNolen  WA Bipolar disorder in the general population in the Netherlands (prevalence, consequences and care utilisation): results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). J Affect Disord 2002;68 (2-3) 203- 213
PubMed
Rybakowski  JK Antiviral and immunomodulatory effect of lithium. Pharmacopsychiatry 2000;33 (5) 159- 164
PubMed
Gillespie  NAM Direction of causation models. Everitt  BSHowell  DCThe Encyclopedia of Statistics in Behavioral Science. Hoboken, NJ Wiley2005;496- 499
Padmos  RCSchloot  NCBeyan  HRuwhof  CStaal  FJde Ridder  DAanstoot  HJTse  WKde Wit  HHerder  CDrexhage  RCMenart  BLeslie  RDDrexhage  HA Distinct monocyte gene-expression profiles in autoimmune diabetes. Diabetes 2008;57 (10) 2768- 2773
PubMed
Torrey  EFBartko  JJLun  ZRYolken  RH Antibodies to Toxoplasma gondii in patients with schizophrenia: a meta-analysis. Schizophr Bull 2007;33 (3) 729- 736
PubMed
Hinze-Selch  D Infection, treatment and immune response in patients with bipolar disorder versus patients with major depression, schizophrenia or healthy controls. Bipolar Disord 2002;4 ((suppl 1)) 81- 83
PubMed
Taieb  OBaleyte  JMMazet  PFillet  AM Borna disease virus and psychiatry. Eur Psychiatry 2001;16 (1) 3- 10
PubMed
Yolken  RHTorrey  EF Viruses, schizophrenia, and bipolar disorder. Clin Microbiol Rev 1995;8 (1) 131- 145
PubMed
Machón  RAMednick  SAHuttunen  MO Adult major affective disorder after prenatal exposure to an influenza epidemic. Arch Gen Psychiatry 1997;54 (4) 322- 328
PubMed
Brown  AS Prenatal infection as a risk factor for schizophrenia. Schizophr Bull 2006;32 (2) 200- 202
PubMed
Torrey  EFYolken  RH At issue: is household crowding a risk factor for schizophrenia and bipolar disorder? Schizophr Bull 1998;24 (3) 321- 324
PubMed
Abe  HHidaka  NKawagoe  COdagiri  KWatanabe  YIkeda  TIshizuka  YHashiguchi  HTakeda  RNishimori  TIshida  Y Prenatal psychological stress causes higher emotionality, depression-like behavior, and elevated activity in the hypothalamo-pituitary-adrenal axis. Neurosci Res 2007;59 (2) 145- 151
PubMed
Costello  EJWorthman  CErkanli  AAngold  A Prediction from low birth weight to female adolescent depression: a test of competing hypotheses. Arch Gen Psychiatry 2007;64 (3) 338- 344
PubMed
Wals  MReichart  CGHillegers  MHVan Os  JVerhulst  FCNolen  WAOrmel  J Impact of birth weight and genetic liability on psychopathology in children of bipolar parents. J Am Acad Child Adolesc Psychiatry 2003;42 (9) 1116- 1121
PubMed
Van den Bergh  BRMulder  EJMennes  MGlover  V Antenatal maternal anxiety and stress and the neurobehavioural development of the fetus and child: links and possible mechanisms—a review. Neurosci Biobehav Rev 2005;29 (2) 237- 258
PubMed
Maccari  SMorley-Fletcher  S Effects of prenatal restraint stress on the hypothalamus-pituitary-adrenal axis and related behavioural and neurobiological alterations. Psychoneuroendocrinology 2007;32 ((suppl 1)) S10- S15
PubMed
Austin  MPLeader  LRReilly  N Prenatal stress, the hypothalamic-pituitary-adrenal axis, and fetal and infant neurobehaviour. Early Hum Dev 2005;81 (11) 917- 926
PubMed
Leonard  BE HPA and immune axes in stress: involvement of the serotonergic system. Neuroimmunomodulation 2006;13 (5-6) 268- 276
PubMed
Padgett  DAGlaser  R How stress influences the immune response. Trends Immunol 2003;24 (8) 444- 448
PubMed
Willner  P Chronic mild stress (CMS) revisited: consistency and behavioural-neurobiological concordance in the effects of CMS. Neuropsychobiology 2005;52 (2) 90- 110
PubMed
Hillegers  MHBurger  HWals  MReichart  CGVerhulst  FCNolen  WAOrmel  J Impact of stressful life events, familial loading and their interaction on the onset of mood disorders: study in a high-risk cohort of adolescent offspring of parents with bipolar disorder. Br J Psychiatry 2004;18597- 101
PubMed
Parker  GGibson  NABrotchie  HHeruc  GRees  AMHadzi-Pavlovic  D Omega-3 fatty acids and mood disorders. Am J Psychiatry 2006;163 (6) 969- 978
PubMed

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics
PubMed Articles