Contextual Predictors of Mental Health Service Use Among Children Opento Child Welfare FREE

Availability of and access to mental health services for children inchild welfare/child protective services (“child welfare”) shouldbe a high priority. Several studies have documented high rates of emotionaland behavioral problems among children removed from their homes^1- 3 orwho remain with their families with active child welfare cases.^1,4- 5 Deliveryof appropriate and timely mental health services may be an important elementin reducing long-term negative consequences for children served by child welfareagencies and in decreasing placement instability among children removed fromtheir homes.

Little is known at a national level, however, about rates and patternsof specialty mental health service use among children served by child welfareagencies. Several regional studies have shown relatively high rates of specialtymental health service use among children in foster care⁶ comparedwith relevant groups of impoverished children.^7- 8 Lessis known about rates of mental health service use among children served bychild welfare agencies who remain in their homes of origin.

Recent analyses from the National Survey of Child and Adolescent Well-being(NSCAW) confirm and extend these findings.¹ Analysesof NSCAW data reveal (1) substantial specialty mental health service use inchildren placed out of the home in the first few months after contact withthe child welfare system (kinship care, 26.1%; foster care, 28.4%; and groupcare, 59.9%); (2) much higher rates of specialty mental health service useby children placed in out-of-home care compared with those who remain in theirhomes, even after controlling for clinical need; and (3) limited use of specialtymental health services despite clear evidence of need in many children. Numerousstudies have demonstrated that clinical and nonclinical factors predict children’sspecialty mental health service use, suggesting that level of emotional andbehavioral problems, maltreatment type, age, and race/ethnicity may all bepowerful predictors of mental health service use.⁶

The literature to date, however, has focused primarily on child-levelfactors associated with children’s specialty mental health service useand has not examined contextual factors that may explain variations in serviceuse. Our study extends previous work by examining the unique contributionof several county-level contextual variables to rates and patterns of specialtymental health service use, beyond important child- and family-level predictors.Specifically, 2 county-level factors—the degree of integration betweenchild welfare and mental health service systems and the available supply ofmental health providers—have been discussed as policy contexts thatmay affect rates and patterns of service use. To date, there have been fewopportunities to examine the interplay of these key contextual variables withknown drivers of service use.

We first ask, What is the relationship between child service use patternsand the strength of interagency linkages between local child welfare and mentalhealth agencies? Strong interagency linkages have been proposed as a criticalmechanism for enhancing the effectiveness, efficiency, and continuity of servicesfor high-risk target populations.⁹ Linkagesare developed through mechanisms such as greater communication between agencies,heightened awareness of concerns that extend across organizational boundaries,simplified and streamlined referrals between agencies, and co-location ofproviders to increase sharing of information on specific cases.^10- 11 Interagencylinkage has a long history in broader system integration efforts, dating backto the 1960s’ War on Poverty^9,11 andextending through children’s system-of-care efforts.^12- 13

Explicit tests of the effects of system integration efforts are recentand focus on improving individual client-level outcomes, with no study todate finding client-level benefit from service integration manipulations.^10,14- 17 Froma theoretical standpoint, equally important and less studied are potentialincreases in the efficiency of service delivery owing to system integrationefforts.¹⁸ We predict that children with aclinically significant need for specialty mental health services in countieswith strong interagency linkages are more likely to receive specialty mentalhealth services than children in counties with weak interagency linkages.Such linkages may result in overall increases in rates of service use, butthey may also result in a stronger association between need and service use.In addition, we hypothesize that stronger interagency linkages will resultin greater emphasis on clinical factors and in weaker associations betweenspecialty mental health service use and nonclinical factors, especially race/ethnicity.

Our second research question asks, What role does the differential supplyof mental health providers across counties, specified as a per capita rate,play in explaining service use? Differences in provider supply may be dueto economic conditions in an area^19- 20;environmental characteristics of a region, such as the presence of educationalinstitutions¹⁹; or other factors. Regardlessof the specific determinants, many argue that a lower relative supply of specialtymental health providers in a county will cause a reduction in the percentageof children receiving specialty mental health services or increases in thestrength of the relationship between need and use if lower relative supplyleads to more restrictive criteria for children accessing services. Federal-and state-level programs exist to try to address mental and physical healthprovider shortages.

We begin by examining specialty mental health service use during 1 yearamong children whose families are referred to child welfare and have a childwelfare case opened, first modeled as a function of known child- and family-levelpredictors of service use. We then address whether the 2 contextual variables,linkages and relative specialty mental health provider supply, relate to patternsof specialty mental health service use among children in child welfare, witha particular focus on whether these variables moderate key relationships betweenneed and service use.

The NSCAW aims to learn about the experiences of children who are subjectsof child abuse and child neglect investigations (or assessments) conductedby child welfare agencies, and it is the first nationally representative studyof its kind. The NSCAW used a stratified 2-stage cluster-sampling strategyto select a nationally representative sample of children. Detailed discussionsof the NSCAW methods have been published elsewhere.²¹

The linked Caring for Children in Child Welfare study collected additionalinformation about characteristics of the counties and agencies in which childrenwere sampled.²² In each NSCAW county, interviewswere conducted with key informants from a variety of state and local agencies,with supplemental information obtained from agency Web sites and internalpublications and from the Area Resource File, maintained by Quality ResourceSystems Inc, Fairfax, Va, for the National Center for Health Workforce Analysis,Rockville, Md. Detailed discussions of the Caring for Children in Child Welfaremethods have been published elsewhere.²²

Children in the NSCAW were sampled from 92 selected primary samplingunits (PSUs) consisting of 97 counties. The PSUs were typically defined asgeographic areas that encompass the population served by a single child welfareagency (usually 1 county; hereafter, PSUs will frequently be referred to ascounties). The PSUs were selected with probability proportional to the sizeof the county child welfare population. From an original sample of 100 PSUs,8 were removed because of regulations that made study recruitment unworkable.This had a small effect on the definition of the sample, but the sample continuesto represent the national target population.²³

In participating counties, children were selected from among the populationof children from birth to age 14 years for whom an investigation of abuseor neglect had been opened by the child welfare system during a 15-month periodbeginning October 1, 1999. The final child welfare sample included 5504 children,resulting in an overall weighted response rate of 64%. Extensive analysesconcluded that nonresponse bias was minimal and unlikely to be consequentialfor most analyses.²¹ This article focuses specificallyon children who were removed from their homes or were living in a family inwhich a case was opened for child welfare services after substantiation ofabuse or neglect (N = 2823). Child welfare services could rangefrom minimal (eg, information and referral) to intense (eg, removal of thechild from the home).

Approval for this study was provided by the US Office of Management;by the budget and institutional review boards of Research Triangle Institute(Research Triangle Park, NC), University of California at Berkeley, and Children’sHospital, San Diego (San Diego, Calif); and by numerous state and county institutionalreview boards representing PSUs involved in the study. Informed consent wasobtained from all participants.

This study uses data from initial interviews with child welfare workersand initial and 12-month follow-up interviews with current caregivers. Forchildren remaining in their homes (n = 1961), the current caregiverwas most often the child’s biological parent. Among children removedfrom their homes (n = 862), current caregiver interviews were completedwith relative or nonrelative foster caregivers or group care staff. Initialcurrent caregiver interviews were completed a mean (SD) of 5.6 (2.7) monthsafter the onset of the child welfare investigation, and follow-up interviewswere completed after a mean (SD) of 13.5 (1.6) months. Initial interviewswith child welfare workers were completed a mean (SD) of 5.2 (2.7) monthsafter the onset of the child welfare investigation.

Trained research assistants collected interview data from agency informantsbetween September 21, 2000, and July 25, 2001. When the primary identifiedagency respondent did not feel qualified to answer some or all questions,he or she was encouraged to identify alternate informants who could providethe relevant information.

Children’s age, sex, and race/ethnicity were collected as partof the initial case identification procedure and were confirmed by caregiverand child welfare worker interviews. The child’s placement at the timeof the initial caregiver interview was classified into 1 of 4 categories:(1) home of origin, (2) kinship foster care, (3) nonrelative foster care,and (4) group care. Excluded from analyses were 32 children in unknown settings.

Child welfare workers identified the types of suspected maltreatmentusing a modified Maltreatment Classification Scale.²⁴ Sixnon–mutually exclusive dichotomous variables that describe maltreatmenttypes were created: (1) physical abuse,(2) sexual abuse, (3) emotional abuse,(4) supervisory neglect, (5) physical neglect, and (6) abandonment.

For each case in the NSCAW, caseworkers reported the presence or absenceof risk factors that resulted in the family having contact with child welfare.Risks fell into the following 7 categories: (1) alcohol or drug abuse by theprimary caregiver or recent arrest history; (2) poor parenting skills; (3)physical, mental, or cognitive limitations of the primary caregiver;(4) lowsocial or economic support; (5) history of domestic violence; (6) historyof substantiated child abuse; and (7) poor cooperation with the child welfareinvestigation. Cases were categorized regarding the number of risks reportedby caseworkers. Based on caregiver reports at the initial interview, children’sinsurance was classified as (1) Medicaid, (2) private or CHAMPUS (CivilianHealth and Medical Program of the Uniformed Services), or (3) no insurance.

The Child Behavior Checklist (CBCL), a widely used and psychometricallyestablished measure,²⁵ was used to estimateemotional and behavioral problems for youth and the need for mental healthtreatment. Two caregiver report forms of the CBCL were used, one for childrenin the study aged 2 to 3 years and another for children aged 4 to 14 years.Children at or above the clinical cutoff point (t = 64)on any of the internalizing, externalizing, or total scale scores were categorizedas having clinically significant levels of need. Our research team used thismethod to identify need¹ rather than just thetotal CBCL score because it identifies a slightly larger group of childrenas having need while still representing a very conservative clinical thresholdwith respect to evaluation of met and unmet needs.

Current caregivers responded to questions about children’s mentalhealth service use in an adapted version of the Child and Adolescent ServicesAssessment.²⁶ The present study included informationon the use of outpatient specialty mental health services from investigationonset through approximately 1 year, including (1) clinic-based specialty mentalhealth services (eg, community mental health clinics), (2) therapeutic nursery,(3) day treatment, and (4) private professionals, such as psychiatrists, psychologists,social workers, and psychiatric nurses.

The strength of ties existing between child welfare and mental healthagencies at the local level (linkages) was assessed through 2 different interviewmodules, one focusing on mental health services available to children in thechild welfare system and one focusing on characteristics of the local mentalhealth agency in the county. Linkages were defined by a count of 26 concreteindicators of linkage between the 2 local agencies (eg, co-location of childwelfare and mental health services, existence of a formal child welfare committeethat reviews mental health service use on a case-by-case basis, shared officespace, joint service provision at the caseworker level, and joint training).The full set of indicators was drawn from the ACCESS (Access to CommunityCare and Effective Services and Supports) program’s study of serviceintegration efforts to improve care for homeless mentally ill adults,¹⁰ and a complete list can be obtained from one of us(M.S.H.).

Regional variation in specialty mental health provider supply was measuredusing Area Resource File variables. Approximations of federal methods fordefining provider supply when identifying mental health shortage areas wereapplied to each NSCAWcounty.²⁷ Using data from1990 (the most recent data available for all necessary human resource variables),the numbers of psychiatrists (adult and child), psychologists, and socialworkers in each county were summed, and a rate per 10 000 populationwas derived. For reference, the average child in the target population livedin a county with a total of 30 such providers per 10 000 individualsin the population (range, 9-69 providers).

Variables that describe the child population size and the level of povertyin the county were included as control variables in multivariate models.

All analyses used sampling weights that account for the sampling plan.Descriptive, bivariate, and multivariate analyses were conducted using statisticalanalysis software (SUDAAN version 8.0; Research Triangle Institute to accountfor the sampling design. Modeling of specialty mental health service use followed3 steps. First, predictor variables were examined for collinearity with oneanother or inadequate cell sizes. Second, child- and family-level variableswere entered into an initial model of service use. Finally, each of the primarycontextual variables of interest was added to the model to evaluate its uniquecontribution.

Multivariate models were also developed using appropriately weightedrandom-effects logistic regression (GLIMMIX macro; SAS Institute, Cary, NC)with a random intercept. Similar results were observed between the 2 approaches;therefore, results based on models in SUDAAN are reported herein.

Across 1 year after the initial child welfare investigation, an estimated28.3% of children in the population studied received outpatient specialtymental health services. Many of the predicted relationships between child-and family-level factors and specialty mental health service use were confirmedby using bivariate analyses (Table 1).Children with higher rates of use were those who were placed outside of theirhomes and those with clinical-level CBCL scores. Other significant predictorsof service use at the bivariate level included age, several different abusetypes, family risk factors, and insurance status.

The 3 multivariate models in Table 2 correspondto the 3 analytic steps: (1) entry of child- and family-level predictors,(2) addition of several county-level control variables, and (3) addition ofthe 2 primary variables of interest (mental health provider supply and strengthof interagency linkages). In Table 2,step 1 summarizes the relationships between specialty mental health serviceuse and each of the child- and family-level predictors. Dominant predictorsin the multivariate model were children’s CBCL scores and placementstatus. Children above the clinical cutoff point on the CBCL were 4.0 timesas likely to receive specialty mental health services as those below the cutoffpoint. Adjusting for other variables, we found that children placed outsidethe home were much more likely to receive services than those remaining intheir homes of origin, with those in nonrelative foster care (odds ratio [OR],3.92) and group care (OR, 5.51) being the most likely to receive services.

Other variables also accounted for service use patterns. Despite controllingfor level of clinical need, younger children were much less likely to receivespecialty mental health services than older children. Race/ethnicity alsoaccounted for differentials in service use; specifically, African Americanchildren were 0.61 times as likely and Hispanic children were 0.51 times aslikely to use services as white children. In the multivariate model, insurancestatus and family risk factors no longer had significant associations withspecialty mental health service use.

After entering child-level predictors of service use, which served thefunction of risk adjustment across counties, we entered the contextual variablesindividually. In Table 2, step 2 summarizesthe second multivariate model with the addition of 2 county-level controlvariables. The addition of county poverty level to the model did not contributesignificantly to the prediction of children’s service use (OR, 1.25),and neither did the size of the county child population (OR, 1.03 for smallcompared with large counties; OR, 1.41 for medium compared with large counties).The 2 independent variables of interest, agency linkages and provider supply,were then sequentially added to the model, along with their interactions withCBCL score and race/ethnicity. Substantial relationships were present on additionof the linkage variable and its interactions (Table 2, step 3, and the Figure).For children with CBCL scores below the clinical cutoff point, increased interagencylinkages were related to a decreased likelihood of service use (OR, 0.93 perlinkage item). This relationship was moderated by a strong and significantinteraction of interagency linkage with CBCL score. In counties with stronginteragency linkages compared with those with weaker linkages, the differentialin service use rates was greater between children above and below the CBCLclinical cutoff point. For example, in counties with linkage levels 1 SD abovethe mean level of interagency linkage, children above the clinical CBCL cutoffpoint were estimated to receive specialty mental health services at 6.40 timesthe rate of children below the cutoff point:

Exp[(β_CBCL) × 1 + (β_{CBCL × Linkage}) × Linkage Score]= Exp [0.38(1) + 0.09 (16.40)] = 6.40.

The same OR was estimated to be 2.64 in counties 1 SD below the mean:

Exp[0.38 (1) + 0.09 (6.56)] = 2.64.

The mean level of interagency linkage applicable to children in thisstudy was 11.48 (weighted), with an SD of 4.92 (unweighted). This mean isbased on attaching a linkage score to each child; a mean based only on 1 observationper county would not account for the number of children served in each county.

Linkages also moderated the relationship between race/ethnicity andservice use (see Table 2 for interactionORs), with the effect primarily focused on service use patterns by AfricanAmerican children (χ²₃ = 8.17; P = .04). In countieswith stronger linkages (vs those with weaker linkages), differentials in serviceuse between African American children and white children diminished. In counties1 SD above the mean on the interagency linkage variable, African Americanchildren were estimated to be 0.89 times as likely to receive services aswhite children:

Exp[(β_{African American}) × 1 + (β_{African American × Linkage}) ×Linkage Score] = Exp[−1.91 (1) + 0.11 (16.40)] = 0.89.

The same OR was estimated to be 0.30 in counties 1 SD below the mean:

Exp[−1.91 (1) + 0.11 (6.56)] = 0.30.

The interactions of interagency linkage with CBCL score and race/ethnicityare also shown in the Figure, with linkagesegmented into low, medium, and high tertiles. As linkage levels increase,the likelihood of specialty mental health service use increases for childrenabove the clinical cutoff point and decreases for those below the cutoff point(Figure, A). As linkage levels increase,differences in rates of service use between white and African American childrendiminish (Figure, B).

Specialty mental health provider supply did not contribute to predictionof service use (OR, 1.01 per additional professional). The same result wasfound when supply was trichotomized to examine for nonlinear patterns. Theinteraction of provider supply and CBCL score did not contribute significantlyto the model (χ²₁ = 1.64; P = .20), and neither did its interaction with race/ethnicity(χ²₃ = 1.03; P = .79).

This article examined the relationship between 2 contextual variablesand specialty mental health service use, controlling for other predictorsof service use, in a nationally representative sample of children referredto the child welfare system who were subsequently removed from their homesor who remained with their families but had a child welfare case opened.

At a broad level, our analyses confirm findings that rates of mentalhealth need are exceptionally high in this population.^3,28 Atthe time of the first interview wave, 42.4% of children had clinical-levelCBCL scores. Within approximately 1 year of the child welfare investigation,28.3% of children had received specialty mental health services. Despite largeincreases in specialty mental health service use during this time, these analysesdemonstrate that many children with strong clinical indications of need forservice continue not to receive such services, consistent with our earlierwork examining service use among children shortly after contact with the childwelfare system.¹

The present study extends that earlier work by focusing on how contextualpredictors moderate service use patterns among children open to child welfareduring a full year after contact with child welfare. Results from this studyrevealed that child- and contextual-level variables serve as powerful predictorsof rates and patterns of specialty mental health services. At the child level,out-of-home placement status and older age predicted increased service use,even after controlling forlevel of need in multivariate models. At a minimum,these findings raise questions about the degree to which counties and childwelfare agencies are missing opportunities to address mental health concernsthat are already substantial among younger children and children remainingin their homes of origin. In addition, race/ethnicity and CBCL scores contributeto patterns of specialty mental health service use.

Child-level variables must be examined jointly with county-level contextualvariables. The 2 most significant results of this analysis concern the interactionsof CBCL score and race/ethnicity with the strength of interagency linkagesbetween the local child welfare and mental health service systems. First,our data suggest that child welfare systems are responsive to the level ofemotional and behavioral problems that children experience but that localinteragency linkages increase the focus of specialty mental health servicedelivery to children with clinical levels of need. Whether targeting specialtymental health services to children with the most substantial levels of clinicalneed represents the most efficient and effective allocation of mental healthresources is unclear. Some might argue that given the overall high levelsof need and the comparatively low rates of specialty mental health serviceuse, it is critical to increase rates of specialty mental health service useby all children. Others might argue that specialty mental health servicesshould focus primarily on children with the most significant levels of need,paired with appropriate prevention approaches for children and families withlower risks. Regardless, the results of this study suggest that interagencycoordination may have important effects on the patterns of children receivingservices.

The strength of interagency ties also seems to affect racial/ethnicdisparities in service use. Without taking county contextual variables intoaccount, we found that African American and Hispanic children are markedlyless likely to receive specialty mental health services than white children,replicating findings from other local studies^29- 31 ofchildren in the child welfare system. Increased interagency linkages, however,seem to decrease disparities for African American children but not for childrenfrom other minority groups. A variety of potential mechanisms have been proposedto explain lower use rates by racial/ethnic groups, such as language barriers,knowledge of services, concerns about stigma, and differences in beliefs aboutthe nature of mental health problems. Although this study does not directlyidentify specific mechanisms that contribute to lower use rates, efforts bychild welfare and mental health agencies to coordinate around the mental healthneeds of children may be able to prevent disparities in mental health careuse among African American children, who are heavily overrepresented in thechild welfare system.

These results are in accord with other studies that show the importanceof specific linkage mechanisms (eg, providers’ knowledge of and connectionwith mental health service resources) to increased identification of children’sneed for services.¹⁸ However, system integrationefforts emphasize a broader array of mechanisms designed to streamline theidentification and referral process. Items in the linkage construct exemplifythese mechanisms. It is important to note that the linkage construct itemsare not highly correlated. This poses no theoretical problem because the linkageindicators are best conceptualized as cause indicators.³² However,if interagency linkage is causally related to patterns of service use, a pragmaticimplication is that communities may need to focus on multiple aspects of coordinationto make significant changes in the way services are targeted to children inthe child welfare system.

The other contextual variable studied herein, total supply of mentalhealth providers in a county, does not seem to have a strong relationshipwith specialty mental health service use among children in the child welfaresystem. Using an approximation of federal methods for defining mental healthcare supply, only a relatively weak and nonsignificant trend was found forincreased use of services in areas with greater relative provider supply.

This study has 2 major strengths: it includes a nationally representativesample of children involved with child welfare systems from 92 PSUs and thusallows for the unique opportunity to study how contextual characteristicsof counties and service systems relate to service delivery rates and patterns.

These strengths suggest some inherent limitations. First, there havebeen few opportunities to formally develop and test measures of some constructsof interest, such as interagency linkage and provider supply. However, themeasure of interagency linkage used is the most concrete operationalizationwe are aware of. Likewise, the provider supply variable is an approximationof a federal approach to assessing mental health resource adequacy, althoughit does not take into account other master’s-level providers who makeup large segments of the mental health care system. Second, although associationsbetween linkages and service use were identified in this study, it was notpossible to evaluate the causality of the observed relationships. Despitethese limitations, this study is the first to explore contextual predictorsof specialty mental health service use among children in the child welfaresystem, and it reveals potentially important associations.

These results confirm previous findings regarding child- and family-levelvariables that affect specialty mental health service use in children servedby child welfare agencies. In addition to these findings, 2 new primary messagesemerged from this study. First, counties and child welfare systems may largelybe missing several important opportunities to improve the well-being of childrenand reduce the likelihood of families’ reinvolvement with child welfare.Specifically, counties and child welfare agencies may want to evaluate opportunitiesfor addressing the high rates of clinically significant emotional and behavioralproblems among young children and children who remain in their homes of originthat currently are not adequately addressed.

Second, the degree of linkage between the local child welfare and mentalhealth service systems may have important effects on the pattern of childrenreceiving specialty mental health services. Increasing coordination betweenthe 2 agencies at the local level may facilitate targeting of scarce serviceresources to those children with the greatest levels of need. Furthermore,it may help mitigate the effects of other forces that frequently lead to disparitiesin service use by African Americans. The specific mechanisms by which theseeffects occur are not known and may be multiple. Future research is warrantedto understand whether increased collaboration between agencies can improvethe targeting of resources and minimize inappropriate disparities.

Correspondence: Michael S. Hurlburt, PhD,Child and Adolescent Services Research Center at Children’s HospitalSan Diego, 3020 Children’s Way, MC 5033, San Diego, CA 92123 (mhurlburt@casrc.org).

Submitted for Publication: March 10, 2004;final revision received May 28, 2004; accepted June 9, 2004.

Funding/Support: This research was supportedthrough the Caring for Children in Child Welfare study, which was funded bygrant MH59672-02 from the National Institute of Mental Health, Bethesda, Md.

Disclaimer: The information and opinions expressedherein reflect solely the position of the authors. Nothing herein should beconstrued to indicate the support or endorsement of its content by the Administrationon Children, Youth, and Families, US Department of Health and Human Services,Washington, DC.

Acknowledgment: The Caring for Children inChild Welfare project is a collaborative effort between the Child and AdolescentServices Research Center at Children’s Hospital San Diego; the Departmentof Psychiatry at the University of Pittsburgh, Pittsburgh, Pa; the ColumbusChildren’s Hospital, Columbus, Ohio; the Services Effectiveness ResearchProgram at Duke University, Durham, NC; and Research Triangle Institute. Acomplete description of the study and a list of key personnel are availableat http://www.casrc.org/projects/CCCW/index.htm. This article alsoincludes data from the NSCAW, which was developed under contract to ResearchTriangle Institute from the Administration on Children, Youth, and Families,US Department of Health and Human Services. The Caring for Children in ChildWelfare project maintains ongoing collaboration with the NSCAW Research Group.