Cost-effectiveness of Supported Housing for Homeless Persons With Mental Illness FREE

DELIVERY OF effective services to homeless people with serious psychiatric or addictive disorders or both has been difficult, in part, because of their need for assistance from diverse agencies and the difficulty of integrating services at the interorganizational level.¹ A recent 18-site demonstration project² found that extensive and well-funded efforts to promote integration of service delivery across dozens of agencies by implementing global, systemwide integration strategies did not result in improved access to services or better client outcomes. In contrast, a more focused, agency-specific approach, in which pairs of agencies map out specific ways of coordinating their efforts, increased access of homeless veterans to social security benefits³ and improved their quality of life.⁴

Clinical services for this population have included (1) community outreach,⁵ (2) case management,^6- 8 and(3) housing assistance involving either time-limited halfway house treatment^9- 10 or longer-term mainstream community housing with support.^11- 12 Recently, experimental studies^6- 14 have demonstrated superior outcomes for diverse interventions, typically described as supported housing programs in which case management and housing resources are combined, with benefits more often demonstrated for housing outcomes than for clinical status.¹⁵ Although no paradigmatic standards for this approach have emerged, it received a strong endorsement from the congressionally appointed Bipartisan Millennial Housing Commission.^16(p49)

An important unanswered question is whether setting aside housing resources is either necessary or sufficient for facilitating exit from homelessness in this population. On the one hand, provision of intensive clinical services may result in receipt of sufficient access to health care, income support, or rehabilitation services to facilitate exit from homelessness without formal linkage to housing subsidies. On the other hand, even when given priority access to housing subsidies, people with serious behavioral disorders may not be able to take advantage of them.

Only one experimental study^12,17 has attempted to disentangle the effect of housing subsidies and intensive case management for this population. That study used a 2 × 2 study design, crossing rent subsidies with intensive case management, and reported that clients who received rent subsidies were more likely to be independently housed after 18 months but that intensive case management was not associated with greater improvement than standard case management in any outcome domain. However, these findings are ambiguous because (1) receipt of housing subsidies did not reduce nights of homelessness and (2) the intensive case management intervention as actually delivered was not dissimilar from the standard care intervention. Access to housing subsidies did not reduce homelessness.

Supported housing services can be costly.¹⁸ However, a recent study¹⁹ that assessed the costs for clients placed in the New York–New York (NY/NY) supported housing initiative and a matched control group found substantially greater reductions in hospital use among NY/NY clients than controls, offsetting almost the entire $19 000 annual program cost. In the absence of random assignment, however, it is possible that these savings reflected unmeasured (and unmatched) client characteristics rather than placement in NY/NY housing.

To evaluate the cost-effectiveness of an agency-specific approach to the integration of clinical and housing services, we conducted an experimental evaluation of a joint program of the US Department of Housing and Urban Development (HUD) and the US Department of Veterans Affairs (VA)—the HUD-VA Supported Housing (HUD-VASH) program—in which HUD Section 8 housing vouchers were paired with intensive case management services provided by VA clinicians.

In a 3-year prospective experimental study, we compared outcomes and societal costs among clients randomly assigned to (1) HUD-VASH, (2) intensive case management without special access to Section 8 vouchers, or (3) standard VA homeless services.¹⁷ We hypothesized that case management combined with housing subsidies in HUD-VASH would result in better housing, mental health, and social adjustment outcomes than either control condition and that intensive case management, in turn, would result in better outcomes than standard care. We further hypothesized that HUD-VASH would generate sufficient savings in hospital, halfway house, criminal justice, and emergency shelter costs to offset the additional costs of intensive case management services but that case management alone would be almost as expensive as HUD-VASH but less effective.

Through an interagency agreement, HUD allocated funds for approximately 1000 housing vouchers for a program providing housing and case management assistance for literally homeless veterans with psychiatric or substance abuse problems or both.²⁰ Participants were offered priority access to Section 8 housing vouchers administered by local housing authorities. These vouchers authorize payment of a standardized local fair-market rent (established by HUD using surveys of local rents) less 30% of the individual beneficiary's income.

Professional staff of the VA's Health Care for Homeless Veterans (HCHV) program,²¹ to which each experimental HUD-VASH program was linked, identified potential candidates for the program. To participate in the study, each veteran had to agree to a treatment plan involving further participation in case management and other specified services if randomized to either HUD-VASH or case management only. However, once assigned, retention of the voucher was not contingent on participation in treatment.

The case managers linked clients with the local housing authority and facilitated administrative access and use of the voucher. Case managers also eased the transition to independent living by helping clients (1) locate an apartment, (2) negotiate the lease (through face-to-face landlord meetings if the client wished), and (3) furnish and move into their new apartment. The case management model used in HUD-VASH was modified from the Assertive Community Treatment (ACT) model²² and encouraged at least weekly face-to-face contact, community-based service delivery, and more intensive involvement in crisis situations. Case managers, most of whom were experienced social workers and nurses, were also encouraged to provide substance abuse and employment counseling and to facilitate linkage with other VA services. Teams in this study consisted of 3 case managers and an allocation of 50 vouchers, allowing maximal caseloads of 25:1, including 25 case management– only clients. Training and monitoring were conducted with written materials and through conference calls, case reviews, and evaluation forms. There were no on-site monitoring visits.

Eligibility was determined from administrative intake forms documenting housing and clinical status at the time of the initial outreach assessment by the HCHV program.²¹ After providing written informed consent and completing baseline assessments, 460 veterans were randomly assigned through a telephone call to the central evaluation staff, who identified the next assignment from a deck of cards specific to each site. Veterans were assigned to (1) HUD-VASH (case management plus vouchers) (n = 182), (2) case management only (n = 90), or (3) standard care (n = 188), which consisted of short-term broker case management as provided by HCHV program outreach workers. The randomization was weighted so that half as many veterans were assigned to case management alone as to the other 2 groups. In the case management– only condition, case managers were to provide the same intensity of services as in the HUD-VASH condition and were encouraged to use whatever housing resources could be obtained for their clients. Neither veterans nor staff could be masked to group assignment.

Baseline and follow-up assessment interviews every 3 months were conducted by trained evaluation assistants. The VASH clinicians used (1) structured forms to document their efforts to assist their clients in obtaining their vouchers and apartments and (2) quarterly structured summaries of case management services.

The study took place at VA medical centers in San Francisco, Calif (n= 107); San Diego, Calif (n = 91); New Orleans, La (n = 165); and Cleveland, Ohio (n = 97). Veterans were eligible if they were literally homeless at the time of outreach assessment (ie, living in a homeless shelter or on the streets), had been homeless for 1 month or longer, and had received a diagnosis of a major psychiatric disorder (schizophrenia, bipolar disorder, major affective disorder, or posttraumatic stress disorder) or an alcohol or drug abuse disorder or both.

All veterans provided written informed consent to participate in the study, and the protocol was approved by the human investigations committee at each medical center. Veterans received $20 after each interview.

Recruitment for the study took place between June 1, 1992, and December 31, 1995. During this time, 3489 veterans contacted through outreach at the 4 sites met minimal eligibility criteria, and 460 (13.2%) gave written informed consent to participate in the study. The major reason most veterans did not participate was limited program capacity.

Comprehensive intake data from all HCHV program participants allow detailed comparison of participants and nonparticipants. Compared with other eligible veterans, those who joined the study were slightly younger (42.0 vs 42.8 years, t = 2.2₆₃₇; P<.03), were more likely to be female (4.2% vs 1.8%, χ²₁ =11.3; P<.001), were more likely to be African American (64% vs 57%, χ²₁ = 7.1; P<.008), and had slightly fewer nights of literal homelessness in the 60 days before intake (26.1 vs 27.1 nights, t =2.8; P<.01), but they had a greater likelihood of past hospitalization for drug abuse (49.8% vs 39.5%, χ²₁ = 17.3; P<.001). They were more likely to have been admitted to the residential treatment component of the HCHV program(29.1% vs 10.9%, χ²₁ = 119.7; P<.001). Participants thus showed evidence of more severe illness on some measures and greater involvement in treatment.

Data were obtained on current sociodemographic characteristics, duration of the current episode of homelessness, and housing status during the 90 days before each interview. We recorded the number of days in the previous 90 that the client spent in each of 11 different types of housing. The primary outcome measures were the number of days housed in the previous 90 (ie, sleeping in an apartment, room, or house of one's own or of a family member or friend) and the number of days homeless in the previous 90 (ie, sleeping in an emergency shelter; a substandard, single-room occupancy hotel; or outdoors, on the sidewalk, or in a park, abandoned building, automobile, truck, or boat).

Psychiatric, alcohol, and drug problems were assessed using specific items and composite scores from the Addiction Severity Index.²³ Psychological distress was measured using the Brief Symptom Index.²⁴ Diagnoses were based on the working clinical diagnoses of the case management teams. Quality of life was evaluated using selected subscales from the Lehman Quality of Life Interview.²⁵

Among those who were housed, the quality of their residence was further assessed using one scale that addressed positive characteristics of the residence (eg, safety, proximity to shopping, affordability, adequate size, and privacy) and another that measured housing problems (eg, pests, broken windows, neighborhood crime, and plumbing problems).²⁶

Social support was measured by the number of people in 9 different categories to whom the veteran reported feeling close, an index of the total frequency of contacts with these people, and the average number of types of people who would help with a loan, with transportation, or in an emotional crisis.^27- 28

Five kinds of indicators were used to compare services provided to each treatment group during the study. First, computerized VA workload data were used to measure contacts with the HUD-VASH or HCHV programs. Second, the nature of the therapeutic alliance was measured by a 5-item rating scale completed separately by the clinician (Cronbach α = .85) and the veteran (Cronbach α= .86) using a modified version of the Working Alliance Inventory.^29- 30 Veterans' assessments of their alliance with their case managers were self-administered in private and mailed in a separate envelope. Third, data were obtained on the use of Section 8 housing vouchers in all 3 groups and on the initial housing search for those assigned to the HUD-VASH group using a structured activities questionnaire. Fourth, data from the quarterly case manager summaries were used to compare case management services provided to each group. Finally, we measured use of regular VA mental health outpatient services (ie, beyond those from HUD-VASH or HCHV program staff).

The VA health care costs were estimated by multiplying the number of units of service consumed by each patient by the estimated unit cost of each type of service using VA cost data from fiscal year 1996 and methods developed previously.³¹ Unit costs of the HUD-VASH case management were estimated separately using more detailed data on program expenditures and services delivered during a sample year (1996) when the program was fully operational.³²

The VA health service utilization data were derived from the VA's comprehensive national workload data systems: the Patient Treatment File for inpatient care, the Outpatient Care File for outpatient care, and program monitoring data on the delivery of residential treatment through VA contracts with community agencies.

Unit costs for VA inpatient, residential care, and outpatient treatment were estimated on the basis of data from the VA's Cost Distribution Report, which is a facility-by-facility accounting record that identifies total expenditures and unit costs associated with VA inpatient and outpatient health care services nationwide.³¹

Utilization of non-VA services was documented through quarterly patient interviews, during which was recorded the use of non-VA medical and mental health inpatient, residential, and nursing home care; non-VA medical-surgical outpatient care; and non-VA mental health outpatient care. Non-VA unit costs were estimated from several sources, including analysis of costs in the 1998 MarketScan (The Medstat Group, Ann Arbor, Mich) data set³³ and published studies^34- 35 that identify unit costs in large non-VA health care systems.

Non–health care costs were also evaluated and used to estimate costs from the perspective of governmental agencies or taxpayers and of society as a whole (total resource consumption). Interview data documented the number of days spent in shelter beds or in jail or prison, cash transfer payments (eg, VA benefits, Supplemental Security Income, and Social Security disability), earnings, and the cost of the Section 8 vouchers. Although cash transfer payments (including housing subsidies) were included in the evaluation of costs from the perspective of governmental agencies, only the administrative cost of these payments was included in societal cost estimates.³⁶ Productivity (employment earnings) was also included in the societal cost estimate, as a negative cost.

Per diem estimates of the cost of shelter and jail days were based on published literature.¹⁹ The administrative cost of the Section 8 vouchers were estimated by multiplying the number of months each veteran received a Section 8 subsidy during 3-year follow-up by HUD's estimated monthly administrative cost for the Section 8 program at each locality.³⁷ The value of housing subsidies received by program participants was calculated by subtracting 30% of monthly income from reported monthly rent among clients who identified themselves as Section 8 beneficiaries. This figure was only included in the cost estimate from the perspective of the government.

First, we evaluated the effectiveness of the randomization by comparing baseline characteristics of clients randomly assigned to each of the 3 groups. Next, we compared housing procurement processes and delivery of case management services across the 3 groups to determine whether the intended differences in access to housing subsidies and case management were achieved.

Third, housing and clinical outcomes across the groups were compared. The follow-up periods selected for analysis were baseline and 6, 12, 18, 24, and 36 months, and all interviews conducted during each interval were included. Because we planned to compare the groups during 5 intervals after the baseline assessment, we used a repeated-measures with mixed-effects analytic strategy. This method was chosen to allow use of all available data from each participant during each follow-up interval. Using hierarchical linear modeling, we modeled random effects for each participant to adjust standard errors for the nonindependence of observations within participants.³⁸ The repeated-measures mixed-effects model approach was chosen because it allows(1) comparison of each experimental group (HUD-VASH and case management only) with the standard care group at each specific point and (2) comparison of groups averaged across all points (ie, area under the curve). These analyses were conducted using PROC MIXED of SAS version 8.0 (SAS Institute Inc, Cary, NC), with α<.05.

Comparison across groups showed significant differences in follow-up rates within each assessment period from the 6-month assessment (χ²₂ = 10.3; P<.006) through the 3-year assessment (χ²₂ = 55.1; P<.001), with higher follow-up rates in the voucher plus case management group (146 [80%], 153 [84%], 142 [78%], 140 [77%], and 127 participants [70%] at 6 months and 1, 1.5, 2, and 3 years, respectively) than in the case management– only group (59 [66%], 62 [69%], 66 [73%], 55 [61%], and 43 participants [48%]); follow-up rates were even lower in the standard care group (126 [67%], 113[60%], 111 [59%], 92 [49%], and 75 participants [40%]). Two strategies were used to address the potential bias from data loss. First, patients actually followed up at each point were compared on baseline characteristics, and measures for which significant differences were identified were included as covariates in subsequent analyses.

Second, marginal structural modeling was used to inversely weight observations from patients on the basis of their likelihood of being followed up.^39- 40 In this approach, survival analysis is first used to model the likelihood that each observation for each participant will be available for analysis. The predicted probabilities are then used to inversely weight each observation so that available observations from clients with characteristics similar to those who were not followed up are given a greater weight. Since the results did not differ substantially across these analytic strategies, we present data from the analysis that adjusted for baseline measures that were significantly different among interviewed groups at any point and dichotomous dummy codes representing 3 of the 4 sites.

Incremental cost-effectiveness ratios were used to compare the cost-effectiveness of each of the experimental conditions with standard care⁴¹ from each of 4 cost perspectives: the VA, the total health care system (VA and non-VA), the government (or taxpayers), and society as a whole. The incremental cost-effectiveness ratio is the ratio of the difference between groups in costs to the difference in effectiveness. The 95% confidence interval (CI) of the combined incremental cost-effectiveness ratio was computed using the methods described by O'Brien et al.⁴² In addition, cost-effectiveness acceptability curves were constructed using recently published methods⁴³ for analyzing net benefits⁴⁴ in which the probability that benefits equal costs is plotted across a range of possible shadow prices for a day of independent housing.

These analyses require complete data for all participants during 3-year follow-up. Although complete data were available for VA health costs, some data were missing on housing outcomes and non-VA resource use. We used multiple regression models to impute missing data. First, a series of models were estimated in which housing outcomes or service use were the dependent variables and measures of baseline clinical status and dichotomous dummy-coded variables representing treatment group and time interval were the independent variables. These models were then used to generate estimated values of these dependent variables for each client at each point, which were used to impute missing data.

To examine longitudinal time trends in costs, we conducted a series of analyses of variance comparing VA health care costs among groups at 6-month intervals from the year before randomization to 3 years after. We used VA costs for longitudinal analysis because no data are missing and they account for 77% of all health costs (range, 73%-82% across groups).

There were no significant differences among groups on any sociodemographic, clinical, or community-adjustment measures at baseline (data available from the authors on request). The sample was diagnostically heterogeneous: 9.7% had serious psychiatric diagnoses, 50.4% had alcohol or drug disorders, and 35.2% had dual diagnoses, and 4.7% has other psychiatric disorders.

Comparison of workload data showed greater participation rates and greater numbers of visits among participants in the HUD-VASH group than in the case management– only group and in the case management– only group than in the standard care group across all years (Table 1).

Clinician- and veteran-rated therapeutic alliance scores in the HUD-VASH group were significantly higher than those in the other 2 groups during the first year of treatment (Table 1).

More than three fourths of veterans assigned to the voucher condition received vouchers (78.6%) compared with only 5.6% of the case management– only group and 1.6% of the standard care group (Table 1). As planned, HUD-VASH case managers were actively involved in the housing search, meeting with prospective landlords for 72% of clients and helping furnish apartments for 54%. During the first year of case management, HUD-VASH clients received more specialized services than the other groups, especially in the housing domain. These data provide evidence that the treatment conditions differed in the expected ways, although the case management– only group received somewhat less intensive services than the HUD-VASH group.

All 3 groups showed substantial use of VA mental health services, declining progressively within each group from year 1 to year 3 (Table 1). Service use was substantially higher in the HUD-VASH group than in the other 2 groups in the second year.

Averaging across all 3 years, repeated-measures mixed-effects analysis shows that veterans assigned to the HUD-VASH group had 25% more days in an apartment, room, or house than the standard care group (59.4 vs 47.6 days)(t = 4.88; P<.001) and 16.9% more days housed than the case management– only group (59.4 vs 50.8 days) (t = 2.90; P<.004)(Table 2). Differences were significant across time for the first 2 years but attenuated in year 3 (Figure 1). The case management– only group had only 7% more days housed than the standard care group (t = 1.06; P = .29) (Table 2).

Veterans assigned to the HUD-VASH group had 36.2% fewer days homeless than the standard treatment group (13.1 vs 20.5 days) (t = 3.56; P<.001) and 35.8% fewer days homeless than the case management– only group (13.1 vs 20.3 days) (t = 2.87; P = .004). There was no significant difference between the case management– only group and the standard care group (t = 0.05; P = .96) (Table 2). Differences in days homeless were significant across time for the first 2 years and then attenuated in years 2 and 3 (Figure 2).

Consistent with these objective findings, veterans in the HUD-VASH group reported greater subjective satisfaction with housing than either of the other groups and, among those who were housed, experienced fewer housing problems(Table 2). Those who were housed also experienced higher housing quality (ie, more desirable features) than the standard care group but not than the case management– only group.

Veterans assigned to the HUD-VASH group had larger social networks overall (numbers of people they felt close to) and were more satisfied with their family relationships than either of the other groups (Table 2). There were no significant differences on other clinical or community-adjustment measures in either the cumulative analysis (Table 2) or at any specific point.

Outcome analyses were repeated among subgroups of veterans (1) with severe mental illness, (2) dually diagnosed as having psychiatric and substance abuse disorders, (3) with substance abuse disorders, (4) who had been homeless for longer than 1 year at the time of program entry, (5) who were members of racial or ethnocultural minority groups, and (6) with high and low levels of social support at program entry. Results were not different from the analyses of the entire sample or in a set of analyses that excluded "crossovers," that is, veterans in the voucher plus case management group who did not receive a voucher, veterans in the other groups who did receive a voucher, and veterans in the case management– only group who did not participate in case management for at least 6 months. The HUD-VASH was consistently associated with improved housing outcomes but no other outcomes.

Total 3-year VA health costs for HUD-VASH clients were $8009 (28%) greater than those for the standard care group (Table 3), and costs for the case management– only group were$6580 (23%) greater (results of cost analyses are discussed as differences and percentage differences between the means presented in the tables).

The difference between the HUD-VASH group and the standard care group was almost entirely attributable to the $4905 (774%) greater homeless program costs (ie, HUD-VASH case management) and $2454 (32%) greater outpatient mental health costs.

Longitudinal analysis showed substantial declines in VA costs for all 3 groups after randomization (Figure 3 and Figure 4). There were no significant differences among groups in either total VA health costs (data not shown) or VA inpatient and residential treatment costs (Figure 3) at any specific points. The HUD-VASH clients, however, had significantly higher VA outpatient costs (including HUD-VASH costs) than both of the other groups during the first year (Figure 4). Between 18 months and 3 years, the HUD-VASH and case management–only groups had greater outpatient costs than the standard care group (Figure 4).

Three-year non-VA health costs were estimated to be $1047 (10%) lower for HUD-VASH clients than for standard care clients and $3468 (32%) less for case management– only clients (Table 4).

Combining VA and non-VA health cost data, we estimate that from the perspective of the health care system as a whole, costs for HUD-VASH clients were $6962 (18%) greater than costs for standard care clients (Table 4).

Although not included in societal cost estimates,³⁶ the value of Section 8 vouchers averaged $6775 for veterans in HUD-VASH who received them. These costs, along with shelter, incarceration, and other transfer payments, were included in the governmental cost estimates, which showed HUD-VASH to be $10 295 (17.8%) more expensive than standard care from this perspective(Table 4).

Total non–health care costs in the HUD-VASH group were only $762(47%) less than those in the standard care group (see component details in Table 4). Combining health care and non–health care resource consumption to estimate costs from the perspective of society as a whole, HUD-VASH clients consumed $6200 (15%) more resources than standard care clients.

Incremental cost-effectiveness ratios show that each additional day housed among HUD-VASH clients cost $58 (95% CI, $4-$111) from the perspective of the VA, $50 (95% CI, −$17-$117) from the perspective of the total heath system, $74 (95% CI, $5-$143) from the perspective of governmental agencies, and $45 (95% CI, −$19-$108) from the perspective of society as a whole.

Cost-effectiveness acceptability curves show that from the societal perspective, benefits are likely to outweigh costs with a probability of 56% if a day of housing is valued at $50; 80% if valued at $75; 92% at $100; and 97% above $125 (Figure 5). Probabilities of achieving cost-effectiveness were modestly greater from the societal perspective than from the perspective of the health care system or VA and modestly smaller from the governmental perspective. If valued at $125 per day housed, there was a 90% chance of benefits exceeding costs from all perspectives.

The main hypotheses of this study were only partially supported. Assignment to HUD-VASH was associated with improved housing outcomes and greater social contacts but no other benefits, and costs increased. Case management, by itself, yielded no advantage over standard care.

The housing benefits reported for HUD-VASH are more robust than those of the only previous effort to differentiate the impact of housing subsidies and case management. Unlike HUD-VASH, the San Diego Supported Housing Study^12,17 found that assignment to the voucher condition resulted in more days of independent housing but no reduction in homelessness. However, similar to the San Diego study, we found no advantage for case management in the absence of vouchers and no clinical benefits.

Other studies,^{6- 7,13- 15,45} in contrast, have reported gains in housing and some clinical measures with case management interventions based on the ACT model. One possible explanation for these differences is that virtually all the evaluations of ACT in homeless populations have included special access to housing resources and thus may have been more like the HUD-VASH intervention than like the case management–only intervention evaluated herein. In addition, the intended caseloads in the HUD-VASH program and the San Diego program (20-25 cases per case manager) were larger than is typical in ACT programs (10 cases per case manager), and the delivery of services was less intensive. It is also possible that the diagnostic heterogeneity of our sample or receipt of extensive nonprogram services attenuated differences between the 2 case management groups and the standard care group.

Our cost findings are also different from those reported from a nonexperimental study of homeless mentally ill clients placed in the NY/NY supported housing program.¹⁹ Although the NY/NY evaluation compared service use and costs in matched samples, the HUD-VASH evaluation compared costs prospectively between randomly assigned treatment groups. Since the matching procedure used in the NY/NY evaluation was based entirely on administrative data, it is possible that the matching procedure was imperfect and that clients were selected for NY/NY housing on clinical grounds, for example, because they had made progress toward recovery from acute psychiatric or addictive problems and thus were entering a period in their lives when they would have experienced lower hospital use even without participating in the NY/NY program. Such selection bias is highly unlikely in the HUD-VASH evaluation because treatment groups were identified on the basis of random assignment.

Although net costs in the HUD-VASH program were greater than in the NY/NY program, the increase in annual costs (from $2067 from the societal perspective to $3431 from the governmental perspective) was modest in magnitude and comparable to those of other widely used and demonstrably effective psychiatric treatments such as atypical antipsychotic medications⁴⁶ or methadone maintenenace.⁴⁷ Further cost-effectiveness analyses suggest that although we do not know the monetary value of a day of housing, if we examine a range of possible values, we find that if a day of housing is valued at $50 the probability of benefits equaling costs is only 25% to 58% across the various cost perspectives, but the probability rises to 90% to 97% at a shadow price of $125 per day housed.

The principal limitation of this study is the substantial and differential follow-up attrition across treatment groups after the first year, with participants in the 2 experimental conditions more likely to be reinterviewed than those in standard care. It is somewhat reassuring that there were few differences at baseline between those who were successfully followed up and those who were not and that results did not change when analyzed using marginal structural models. However, since one would expect that veterans who were doing less well clinically—especially those who had relapsed to substance abuse—would be more likely to be lost to follow-up, the measured outcomes in the standard care group may be biased in the favorable direction. Since such bias would tend to obscure between-group differences, the negative findings for clinical outcomes should be interpreted with some caution.

This program evaluation study was different from a manual-guided psychotherapy trial in that care provided at local sites was neither monitored through site visits or tape-recorded sessions nor guided by quantitative program fidelity standards—which have yet to be developed for this kind of program. However, the implementation approach used in this effectiveness study is of the kind that would occur in a nonresearch dissemination effort, thus increasing its relevance to "real-world" practice. In addition, because the same case managers implemented both experimental conditions, they may have believed that they were offering inferior services to case management– only clients and they (perhaps reinforced by their clients) may have experienced some demoralization. The alternative approach—of assigning different case managers to each of the treatment conditions—was not feasible.

In addition to these limitations of measurement and program implementation, an intrinsic limitation of any cost-effectiveness analysis is that in the absence of a monetary evaluation of the outcomes it is not possible to decide whether the added costs of a program are justified by the benefits. Cost-effectiveness acceptability curves suggest that if a day of housing for a homeless person with mental illness is valued at $125 or more, HUD-VASH is likely to be an efficient investment from all 4 cost perspectives. But it is unclear whether$125 is an appropriate shadow price for a day of housing for this population.

There are a variety of methods for estimating willingness to pay for various states of health or welfare,^48- 49 but they are often difficult to administer to people with mental illness, and it has been argued that the preferences of the general (tax-paying) public are more germane to the valuation process than those of the direct beneficiaries.⁴¹ However, the utility associated with basic housing for a person who has been homeless is likely to be far greater than for the public at large, even allowing for the uncertainties associated with interpersonal comparisons of well-being, adding another level of complexity to monetizing the outcomes of this study.^50- 51

Furthermore, in any large-scale dissemination of HUD-VASH, the increased costs associated with the program would be likely to require increased taxation. Taxation incurs deadweight losses that have been estimated at 16% to 30% of revenue.^52- 53 Under the assumption of a 20% deadweight loss due to taxation, HUD-VASH client costs would increase to $8265 (15.7%) more than standard care, and the incremental cost-effectiveness ratio would be $59 per day housed (95% CI, −$30-$149). The HUD-VASH program is not an unambiguously cost-effective intervention.

Cost considerations aside, some authorities argue that housing is so fundamental to realizing the worth of liberty and the pursuit of happiness that it must be regarded as a right guaranteed to all citizens.^54- 56 Society, in this view, is categorically or constitutionally obligated to ensure access to housing, and costs are irrelevant. This line of inquiry moves us from considerations of efficiency to the just distribution of social resources, and from the domain of health economics to law and philosophy, domains that are beyond the scope of this study but perhaps deserving of greater attention.

Finally, because this study was conducted on a diagnostically heterogeneous population within the VA health care system, the generalizability of the findings to diagnostically homogeneous populations, to women, or to health care systems other than the VA cannot be assumed.

The absence of differences between case management only and standard care in this study and the San Diego study raises the question of whether housing vouchers could be provided to homeless clients without being linked to intensive case management services. No study or program has offered vouchers to people with serious mental illness without some special program supports,⁵⁷ so no answer to this question is available. Some studies suggest that case management services might be effectively delivered on a time-limited basis,⁵⁸ as in the critical time intervention,¹³ reducing total health care costs while ensuring access to necessary services and supports. However, our data show that as the intensity of case management weakened, in the third year, group differences in outcomes also attenuated, suggesting that service intensity may need to be maintained. Further research is needed on this issue.

To avoid any misunderstanding, it should be emphasized that our findings should not be taken to suggest that case management, in general, does not result in improved health status or community adjustment for homeless people with mental illness. This study compared 2 case management interventions with standard care in a full-service health care system in which homeless veterans had an outreach clinician to link them with a full range of health care services. Housing outcomes, in fact, were impressive even for the standard care group. To evaluate case management in an absolute sense, one would have to compare outcomes for recipients of those services with outcomes for clients who were kept from using any such services at all, which is not a feasible alternative.

Although systemwide efforts to improve client outcomes by fostering services integration across dozens of agencies have been ineffective,² the agency-specific approach demonstrated here was successful at integrating clinical and housing services and in improving housing outcomes. This study demonstrates the potential benefit of housing vouchers for this population, although the associated clinical costs are not inconsiderable.

Corresponding author and reprints: Robert Rosenheck, MD, VA Northeast Program Evaluation Center, 950 Campbell Ave, West Haven, CT 06516 (e-mail: Robert.Rosenheck@Yale.Edu).

Submitted for publication June 27, 2002; final revision received October 24, 2002; accepted January 9, 2003.

We thank Haiqun Lin, PhD, for statistical consultation and Jennifer Cahill, MS, and David Dausey, MPhil, for programming support.