Cost-effectiveness of Intensive Psychiatric Community Care for High Users of Inpatient Services FREE

EXPERIMENTAL studies published during the past 15 years have reported that concentrating treatment resources in community-based assertive community treatment (ACT) teams or intensive case-management programs can result in improved clinical status of severely mentally ill patients at no additional cost.^1- 6 Some studies, however, have found case management to be associated with no clinical change and/or increased service utilization and cost.^7- 11 Three recent reviews^12- 14 concluded that intensive community treatment frequently reduces hospital use but does not always achieve net cost savings, and far less consistently achieves clinical improvement.

Most published controlled trials of ACT were conducted at single sites as specially funded research or demonstration programs implemented by leaders in the field. Although these demonstrations illustrate the potential value of intensive community treatment under optimal operating circumstances, the applicability of their findings to ordinary conditions is uncertain.¹⁵ There is, in addition, a need to identify clinical settings in which such programs are consistently cost-effective.

In this article we evaluate the effects of 10 intensive psychiatric community care (IPCC) programs on the clinical status, community adjustment, and total costs to society (health care costs and non–health care costs) of high users of Department of Veterans Affairs (VA) psychiatric inpatient care. The study is distinctive because it involves multiple sites comprising 2 distinct types of facility, neuropsychiatric hospitals and general medical and surgical hospitals, and was implemented under routine circumstances, as would be the case with dissemination across large public health care systems.

Veterans were eligible for the study if they: (1) were currently hospitalized on a VA psychiatric inpatient unit, (2) had a primary psychiatric diagnosis (as determined by the primary clinician) other than substance abuse or organic brain disorder, and (3) met criteria for high recent psychiatric inpatient utilization (see below). Patients who met these criteria were referred to the research team by their inpatient clinicians. Recruitment occurred from 1987 to 1990 and the last follow-up interview was completed in mid 1992. Virtually all patients referred for screening met the eligibility criteria and 85% agreed to participate in the study. After giving written informed consent to participate, patients were randomized to IPCC or standard VA treatment by the coin-flip of an independent research coordinator. Those assigned to IPCC were introduced to their clinicians shortly after randomization. Control patients continued their prior inpatient treatment until they were determined on clinical grounds to be ready for discharge.

Two contrasting types of VA facilities were involved. Neuropsychiatric sites, typically located in suburban or rural settings, are large facilities that provide long-term mental health care to severely disabled patients. General medical and surgical sites are located in urban centers and provide shorter-term, crisis-oriented inpatient care.

Utilization criteria for program entry were site type–specific because of very different utilization patterns.¹⁶ Entry criteria required 180 days of hospitalization during the previous year or 4 or more prior admissions at long-stay neuropsychiatric facilities; and 40 or more days of psychiatric hospitalization or 2 prior admissions at general medical and surgical sites.

The IPCC program was designed through a comprehensive literature review supplemented by consultation from national experts in the Wisconsin ACT program. The IPCC program was designed to operate under 4 core principles: (1) Intensity. Patients were to be seen as frequently as clinically indicated and case loads were to be low (7-15 patients per clinician). (2) Flexibility and community orientation. Clinicians were urged to provide the majority of contacts in community settings where maximal clinical leverage could be obtained. (3) Rehabilitation focus. A broad range of rehabilitation services was emphasized, such as training in practical problem solving and adaptive skill building. (4) Continuity of care. The IPCC teams were to be a "fixed point of continuing responsibility,"¹⁷ assertively maintaining contact with even the most reluctant patients, although parallel involvement in other VA services was not discouraged.

Most team members were masters-level social workers and nurses, and each team was staffed with a part-time psychiatrist. Teams met to review cases at least weekly (in some instances daily), although team focus was not emphasized in the IPCC program as much as it has been emphasized in recent writing about the Wisconsin ACT model.¹⁸ Staff were available to other VA clinicians and to clients after hours and on weekends.

Standard VA care provided to controls included inpatient psychiatric and psychopharmacologic treatment, outpatient psychiatric treatment, and rehabilitation services, such as work therapy. These services did not differ programmatically between general medical and surgical and neuropsychiatric sites. There was substantial fiscal and administrative pressure for both IPCC and standard-care patients to be discharged from the hospital as soon as possible.

Program implementation was monitored using weekly clinician logs, structured clinical summaries completed on each case every 6 months, and a standardized rating instrument of team operation based on a checklist of 23 ACT program characteristics, previously shown to predict program effectiveness.^18- 20 These data allow evaluation of program implementation at each site with respect to: (1) the frequency, intensity, and location of clinical contacts; (2) the types of services provided; (3) duration of involvement; and (4) overall fidelity to the ACT model.¹⁶

Data on clinical status and community adjustment were obtained through face-to-face patient interviews conducted by independent interviewers every 6 months. Interviewers received 3 days of intensive training in the use of all instruments at the beginning of the project and periodically thereafter.

Psychiatric symptoms were assessed with both an observer-rated instrument, the Brief Psychiatric Rating Scale (BPRS),²¹ and a self-report symptom measure, the Brief Symptom Inventory.²² Substance abuse was evaluated with the alcohol and drug composite indices of the Addiction Severity Index.²³ Functional status was assessed with the Global Assessment Scale (GAS) (although scores on this measure are somewhat contaminated by symptoms)²⁴ and with a self-report measure of community functioning that documented capability to shop, socialize, and perform other basic activities independently. Satisfaction with services was evaluated with a series of 5-point Likert-type questions (ranging from 0=very dissatisfied to 4=very satisfied) that addressed satisfaction with IPCC treatment and other VA and non-VA mental health services.

Community tenure was assessed by subtracting all known days of hospitalization or other days residing in controlled environments (such as nursing homes and jails) from 730 days (2 years).

Health care costs were estimated by multiplying the number of units of service by their estimated unit costs. Veterans Affairs service use was determined from national computerized workload databases and both medication and non-VA service use from interviews. Use of non-VA services may have been undercounted owing to recall difficulties during a 6-month period. Unit costs for VA general psychiatry, substance abuse, medical and surgical, inpatient, and outpatient care (including group treatment and day hospital care) were estimated for each medical center from site-specific cost data available from the local cost distribution report and computerized workload data, using methods described elsewhere.^16,25- 26 Reported non-VA costs were minimal (<3% of all costs) and unit costs were estimated on the basis of published data from private hospitals.

Interview data were also used to estimate non–health care service use and costs including: (1) the administrative costs of transferring payments (eg, disability and welfare)^27- 28; (2) criminal justice system costs (police contacts, arrests, court appearances, nights in jail, and probation visits)^29- 30; and (3) productivity (estimated by employment earnings, included as negative costs).

Cost data were summarized and analyzed from the perspectives of the health care system (VA and non-VA, inpatient and outpatient, psychiatric and medical and surgical, plus drug costs) and society (health care costs plus all non–health care costs, minus productivity).

Analyses were based on intention-to-treat principles including all subjects as randomized. Although 83% of planned interviews were completed at general medical and surgical facilities (88% among IPCC patients vs 77% among controls) and 77% at neuropsychiatric facilities (77% for IPCC vs 76% for controls), primary longitudinal outcomes were analyzed using random effects regression models³¹ to maximize statistical power for testing hypotheses with longitudinal data. These analyses were conducted using the PROC MIXED function from the SAS statistical computer software package, version 6.09 (SAS Institute Inc, Cary, NC). These models accommodate correlations among the repeated observations and missing data at some assessment periods. The significance of differences between treatment conditions was tested using the likelihood ratio χ² statistic. Specifically, we compared a model that included the effects of time, time squared, and treatment condition to a model that adds group × time interactions. The hypothesis of interest is the group × time interaction. A secondary analysis of outcomes was conducted using analysis of covariance (ANCOVA) comparing groups at the final follow-up interview, controlling for baseline status on the follow-up measure. Follow-up rates for this interview at general medical and surgical facilities were 82% for IPCC patients and 73% for control patients and 71% for IPCC patients and 67% for control patients at neuropsychiatric facilities. At general medical and surgical sites, patients who were followed up had higher GAS scores (P<.009) and were more likely to be married (P<.03) than patients who were not. At neuropsychiatric sites, those who were followed up had lower BPRS scores (P<.02) and were also more likely to be married (P<.01).

Cost data were analyzed in their raw form and with natural log transformation using ANCOVA, controlling for utilization and cost during the 6 months before study entry.

Findings were also summarized as incremental cost-effectiveness ratios, the ratio of the incremental cost of the treatment (the average cost of IPCC minus the average cost of standard VA care) to the incremental effectiveness of the treatment (the average effectiveness of IPCC minus the average effectiveness of standard VA care).^32- 33

Table 1 shows that patients at neuropsychiatric sites were older, less likely to have been married or working, and less likely to have substance abuse problems (which were relatively infrequent at both types of site). The cohort had substantial hospitalization histories, averaging 452 days during the 2 years prior to program entry at neuropsychiatric sites and 123 days at general medical and surgical sites.

Comparison of sociodemographic and baseline clinical characteristics of experimental and control patients showed no significant differences on any of 21 measures at general medical and surgical sites, but significantly different scores on 2 measures at neuropsychiatric sites: the BPRS (mean=16.0 for IPCC patients vs 19.6 for controls; t=2.97, df=273, P<.01) and the GAS (mean=42.1 for IPCC patients vs 37.9 for controls; t=3.05, df=340, P<.01).

Across the entire IPCC program, caseloads averaged 13.2 patients per clinician. The IPCC patients had face-to-face clinical contacts with their IPCC clinicians during 60 of the 104 weeks (58%) of the study, and averaged 1.9 contacts per patient per week and 41 minutes per contact during those weeks. More than half (64%) of face-to-face clinical time took place in community settings. Seventy-eight percent of patients were receiving at least 1 type of rehabilitative service after 1 year (the midpoint of the study) and 75% after 2 years. Only 12% of IPCC patients dropped out before the study period was over.

The fidelity of IPCC treatment to the ACT model was compared with 14 model ACT teams implemented under closely supervised conditions and a set of standard mental health clinics. Intensive psychiatric community care programs (mean score=3.7, SD=0.55 [potential range, 0-5]) were substantially more ACT-like than the control programs (mean score=2.4, SD=0.36), but slightly less ACT-like than the model programs (mean score=4.0, SD=0.39).²⁰

Site-by-site implementation data presented elsewhere¹⁶ showed that 2 general medical and surgical sites saw patients only once every 3 weeks during the 2-year period, as compared with about twice every 3 weeks at the other sites. These 2 sites also had the lowest proportion of time spent seeing patients in community settings (43% and 0%, as compared with 72% at other sites), and the lowest scores on the ACT fidelity measure (3.1 and 2.4), both more than 1 SD below the mean of other IPCC sites.

Table 2 presents data on clinical and social adjustment outcomes at baseline and during the 2 years following study entry. Comparison of outcomes at general medical and surgical sites across all time points using random regression analysis showed significant differences on only 1 measure (community living skills), with better functioning for IPCC patients (χ² test=8.65, df=2, P<.01). There were no significant differences between IPCC clients and controls on any measures at neuropsychiatric sites.

Figures in the "Comparative Change" columns of Table 2 show the difference between IPCC and standard VA treatment in the amount of change from baseline. Differences are minimal at most time points, but substantial differences favoring IPCC appear at general medical and surgical sites at the 2-year assessment on several measures. Significant differences favor IPCC treatment on total symptoms (BPRS) (F=8.27; df=1,402; P<.004); community living skills (F=11.62; df=1,366; P<.001); global functioning (GAS) (F=11.12; df=1,409; P<.001); and satisfaction with services (F=5.93; df=1,324; P<.02).

Comparison of IPCC and standard VA care on total days living in the community showed no significant difference at general medical and surgical sites (591 days for IPCC vs 579 for standard VA care; F=1.12; df=1,525; P=.29), but substantially greater community days at neuropsychiatric sites (375 days for IPCC vs 224 for standard VA care; F=23.7; df=1,342; P<.001). All but 2 of these differences remain significant even after reducing the standard significance level to .004 to adjust for 14 comparisons.

Table 3 shows that there were no statistically significant differences at general medical and surgical sites in inpatient or outpatient service utilization during the 2-year follow-up period. Analysis of cost data also showed no significant differences between groups on standard inpatient or outpatient health care costs. However, adding the cost of IPCC treatment ($5793 at general medical and surgical sites) increased total health care costs for the IPCC group by about $5000 over the standard care group (P<.002). Non–health care costs were relatively small in magnitude and not significantly different across groups with 1 exception—the administrative cost of transferring payments was larger in the IPCC group, probably reflecting the case managers' facilitation of access to benefits. In the final intention-to-treat analysis of societal costs at general medical and surgical sites, IPCC was about $5000 more expensive than standard VA care (P<.01).

At neuropsychiatric sites there were significantly fewer hospital days and significantly greater outpatient visits for IPCC (Table 3). The most dramatic difference is noted for non–mental health hospital days reflecting care on long-term "intermediate care" units, a level of VA hospital care between acute hospitalization and nursing home care. Differences in hospital utilization at neuropsychiatric sites were attributable both to a greater proportion of patients having been discharged (82% for IPCC vs 57% for standard VA care; χ² test=24.9, df=1, P<.001) and to reduced hospital utilization among those who were discharged (average days, 216 for IPCC vs 275 for standard VA care; t=2.2, df=241, P<.02).

These differences in inpatient use resulted in more than $35000 lower inpatient costs for IPCC patients, and modestly increased outpatient costs. Total societal costs averaged $34000 less for IPCC patients at neuropsychiatric sites (P<.001).

Analyses were repeated excluding the 2 general medical and surgical sites that did not adequately implement the IPCC program. Although clinical outcome results did not change, hospital days at the remaining sites were lower for IPCC patients (99.8 hospital days for IPCC vs 128.9 for standard VA care; F=6.01, df=1,344, P<.02), as were inpatient costs ($27776 for IPCC vs $35570 for standard VA care; F=5.21, df=1,344, P<.05). Community days were significantly higher for IPCC patients (605.0 for IPCC vs 577.2 for standard VA care; F=3.7, df=1,344, P<.05). Societal costs dropped to $44772 for IPCC and $44810 for standard VA care, a nonsignificant difference of $38.

At general medical and surgical sites cost-effectiveness ratios reveal costs of $1705 per unit of improvement on the BPRS (95% confidence interval [CI], −$1200 to $4609); and $1268 per unit of improvement on the GAS (95% CI, −$880 to $3416). However, with the 2 poor implementation sites excluded, cost-effectiveness ratios at general medical and surgical sites become −$13 per unit of improvement on the BPRS (95% CI, −$2494 to $2520) and −$10 per unit of improvement on the GAS (95% CI, −$880 to $3416). Negative ratios are not unambiguously interpretable, because large negative ratios could reflect either greater cost savings (a desirable result) or less effectiveness (an undesirable result). A negative cost-effectiveness ratio thus means that the treatment has greater cost-effectiveness but does not reflect its relative magnitude. The large CIs about the cost-effectiveness ratios are due to the characteristically large SDs of the cost data. Cost-effectiveness ratios were not calculated for neuropsychiatric sites, at which IPCC yielded no significant gain in effectiveness.

Although IPCC is one of the most thoroughly studied psychosocial interventions for people with severe mental illness,^{1- 14,16- 19} the present study adds to our understanding of this approach in 5 ways.

First and most broadly, this study confirms findings of other experimental studies by demonstrating that IPCC is a cost-effective treatment associated with long-term improvement in psychiatric status, social functioning, and consumer satisfaction at acute care sites and no deterioration at neuropsychiatric sites. After excluding sites that did not fully implement the program, both health care and total societal costs were equivalent at general medical and surgical sites and significantly lower at neuropsychiatric sites. Thus, IPCC is a cost-effective treatment that merits implementation among high users of hosptial services of the type sampled here.

Second, because the programs evaluated in this study were implemented under more typical administrative arrangements than those in previous studies, our results suggest that the benefits of IPCC can be obtained under ordinary conditions, even when programs are not implemented by their original innovators. The IPCC program was implemented without either a detailed manual or extensive and continuous training. It thus appears to be effective as well as efficacious and illustrates the specific contribution of effectiveness evaluation to psychiatric research.³⁴

Third, this study illustrates the importance of tracking program implementation and, albeit unintentionally, demonstrates the relationship of faithful implementation of IPCC to program effectiveness. The 2 general medical and surgical sites that failed to implement an IPCC program also failed to demonstrate the desired effect on hospital utilization. One of them, in fact, had the opposite of the intended effect, generating considerably greater hospital use among IPCC patients than among controls. Increased hospitalization has recently been reported in another study of case management, with the interpretation that when programs are poorly implemented, case managers tend to facilitate hospital admission for their patients because they are aware of the depth of patients' pain but have only hospital care to offer.³⁵ These data suggest that costly interventions for vulnerable populations should not be implemented without appropriate training and continuous monitoring of structure, process, and outcomes of service delivery.

Fourth, we observe that even when the clinical benefits of IPCC are not apparent for as long as 18 months of treatment, they may yet appear after further treatment. This finding underlines the importance of long-term follow-up and demonstrates the potential for delay in improvement with patients with severe and persistent mental illness.

Fifth and finally, the study shows that the effects of IPCC programs may be quite different in settings that treat different types of patients. The IPCC-related improvement in clinical status and social adjustment was only observed at acute care facilities treating younger, higher-functioning patients with greater subjective distress—patients who were more likely to have the capacity and motivation for change.

Total cost savings, in contrast, were significant only among the patients treated at neuropsychiatric facilities—older patients who had longer hospital stays. A study of deinstitutionalization at Northampton State Hospital, Northampton, Mass, also found older longer-stay patients to be least vulnerable for rehospitalization after community placement.³⁶

Perhaps the most important lesson of this study, in this respect, is that intensive, high-cost interventions must be carefully targeted at appropriate patient subgroups. In a previous article,³⁷ we showed that savings potentials are substantial only among psychiatric patients with the highest health care costs. It can not be overemphasized that all patients treated in this study were high service utilizers, representing the upper 5% to 10% of VA resource consumers. While many patients might derive clinical benefit from IPCC treatment, only those with high levels of inpatient service use are likely to show cost savings.

Before concluding, several methodological limitations must be addressed. First, although raters were not part of the clinical team, their ratings were not conducted under double-blind conditions and it is possible that rater bias influenced the results. We think this is unlikely because 2 of the measures that showed significant response to IPCC treatment (community functioning and satisfaction) were based on self-report data and were therefore less subject to such bias. Second, it is possible that patients who were lost to follow-up were doing more poorly than others, artificially inflating ratings at the final interview. Follow-up rates at 24 months, however, were quite reasonable (78% at general medical and surgical facilities 69% at neuropsychiatric facilities) and were in fact somewhat higher than at some earlier time points.

A final limitation is that the study was conducted entirely at VA facilities in the northeastern United States, in a sample of older men with limited substance abuse comorbidity, and thus may not be generalizable to the other settings. Further studies of this type are clearly needed to test the generality of our findings in other health systems and for other subgroups.

Accepted for publication November 18, 1997.

We would like to thank the many Veterans Affairs staff members (program directors, clinical, clerical, and evaluation staff) and the hundreds of veterans at Intensive Psychiatric Community Care demonstration sites who made this project possible. In particular, we are grateful for the support of the following people: M. Hahn, ACSW; N. Pazienza, PhD; and M. Coppola, ACSW, Albany, NY; N. Denton, MSW; J Winarski, LICSW; D. Bitman, PhD; D. Chew, MSW; and W. Penk, PhD; Boston, Mass; M. Tsuang, MD, PhD; J. Simpson, PhD; E. Brown, PhD; G. Lockwood, MSS; N. Noel, PhD; J. O'Connor, OTR; M. Hastings-Procaccino, MSW; and J. Spoltore, PhD, Brockton/West Roxbury, Mass; D. Heimowitz, MSW; D. Harris-Cobbinah, CNS; S. Rochester, CNS; S. Gentile, MA; and E. Mayer, BSW, Bronx, NY; S. Krishnaswamy, MD; J. King; J. Jordan, MSW; C. Shams, RN; L. Smith, MSW; C. Glair, RN; and R. Sullivan, MSW, Buffalo, NY; B. Lewis, RN; R. Linares, MD; D. Fogarty; R. Davis, MS; and T. Roach, BA, Canandaigua, NY; A. Russo, MSW; E. Garrett, RN; and J. Ammazzalorso, Montrose, NY; L. Lantinga, PhD; G. Kline, MA; and M. Baker, Syracuse, NY; L. Harkness, PhD; J. Brady, LPN; D. Deegan, MSW; B. Johnson, MSN; C. Sweeney, RN; H. Spencer, MA; and M. Cinquanta, BA, West Haven, Conn. We also acknowledge R. Baldino; D. Thompson, BS; B. Zigler, MS; A. Ackles, BS; and D. Showalter, BS, of the Northeast Program Evaluation Center, West Haven, Conn, for critical assistance with data preparation and analysis. Philip Leaf, PhD, and Robert Milstein, MD, PhD, assisted with the design of the study. Alan Fontana, PhD; Donald Hedeker, PhD; Marilyn Stolar; and Weichun Xu, PhD, assisted with statistical analysis and Linda Frisman, PhD, and Henry Dove, PhD, with cost estimation methods. Finally, we are grateful to Paul Errera, MD, VA Central Office; Donald Burnette, former Director, VA Eastern Region; Al Washko, former VA Region 1 Director; and Ted Stefos, PhD, for their continuous encouragement and support. Boris Astrachan, MD, provided valuable advice at many junctures. Elizabeth Brown, PhD; David Gittleson, DSW; Terrence Keane, PhD; Stephen Price, MD; and Ming Tsuang, MD, PhD, served on the advisory Northeast Regional Mental Health Task Force during the implementation of this project.

Reprints: Robert Rosenheck, MD, Northeast Program Evaluation Center, 950 Campbell Ave, West Haven, CT 06516 (e-mail: Robert.Rosenheck@Yale.edu).