Improved Outcomes

Research has shown that longer lengths of stay in treatment or treatment completion improve client outcomes including healthcare utilization, criminal justice activity, and employment. At the same time, NIATx has introduced a process improvement model that has been successfully adopted by substance abuse treatment providers across the country to improve access and retention. Despite the lessons learned and an ever-expanding knowledge base, the examination and assessment of the relationship between treatment process measures and client treatment outcomes remains a key challenge in the assessment of quality of service in behavioral health care. While research by the Washington Circle Group found a relationship between quality indicators related to retention and criminal justice activity (Garnick et al, 2007), it is important to explore the juxtaposition of approaches to process improvement targeting provider process measures and the long term effect of these changes on both proximal and distal client outcomes.

The information provided here helps set the stage for future research by:

• Identifying the different types of client outcomes
• Discussing the importance of measuring these outcomes
• Providing an overview of the literature related to a few key client outcomes

In recent years, significant changes have been introduced in behavioral health. The Institute of Medicine report for behavioral health care has provided a quality improvement framework (IOM, 2006). Subsequently, the Substance Abuse and Mental Health Services Administration’s National Outcomes Monitoring System outlines ten key state level outcome domains for mental health, substance abuse and prevention in order to evaluate the long term impact of treatment. The evolution of the Substance Abuse Prevention and Treatment Block Grant into the Performance Partnership Grant increased federal and state pressure on publicly funded alcohol and drug abuse treatment providers to demonstrate improved client outcomes on measures of health care utilization, mental health, criminal justice involvement, and employment as well as reductions in alcohol and drug use. As a result, state authorities are requiring community treatment programs to monitor client outcomes and assess treatment performance. Ongoing assessments of performance can improve provider processes by using data to drive change (Commons et al., 1997; McLellan et al., 2008).

Proximal and Distal Outcomes

When tracking outcomes, it is important to distinguish between proximal versus distal outcomes from the client perspective. In general, a proximal client outcome is one that can be measured on more of a short term basis and these measures often rely on client self-reports to evaluate changes over time. For example, an agency might look at changes in self-reported drug use from admission to discharge or use the BASIS-24, measured at distinct points in the client treatment episode, as a measure of improvement. An underlying challenge with client self-report measures often is tracking down the client for follow-up, which is an argument for conducting short yet meaningful surveys at times other than discharge during the clients treatment episode.

Distal outcomes are often tracked longitudinally over time and these may involve either client self-reports or linkages to state administrative data sets. In the former, the clients are often tracked over a designated time period post discharge (12 months, 3 years etc) for a specific measure. The ones that are most prevalent in the literature are abstinence, re-admission, or changes in employment. Again, it is very important to remember that these are client self-reported changes that involve tracking down the client for follow-up—not always an easy task. Most of these studies involve small samples, certainly no larger than a couple of hundred clients. When linking to state administrative data sets, researchers are able to track the impact of treatment on a large population set for specific measures which, again relying on the literature, are more likely to be changes in health care utilization and criminal justice activity. The level of sophistication of the analysis or the availability of these distal client outcome measures depends on the quality of the state administrative data systems and the ability to link client records together.

Measuring Treatment Outcomes

There is increasing demand for evidence of the effectiveness of drug treatment services after care. Policymakers, consumers, and other stakeholders require publicly funded substance abuse treatment systems to document that services are appropriate, effective, and cost-effective (McCarty et al., 1998). The Institute of Medicine (1990) reported that state information systems are often underutilized, poorly maintained, and underdeveloped. Following this criticism, many state administrations improved their data systems to better track delivery of drug abuse treatment services. The data systems remain idiosyncratic and have limitations (McCarty et al., 1998). Researchers, in partnership with state administrators, have begun overcoming these limitations by leveraging technology to link across state databases (Hser & Evans 2008; Campbell, 2009; Campbell et al., 2008). In order to develop appropriate research protocols, researchers working with state administrative datasets need to identify and select the appropriate data sources, determine how to obtain the required data, and understand the advantages and challenges of the selected data sources (Garnick et al., 2002).

Treatment Outcomes

Drug Abuse and Criminal Activity

Criminal justice involvement is an important outcome measure due to the prevalence of criminal behavior by those with substance abuse problems and the association of drug use with crime (Rajkumar & French, 1997). The associated economic and societal cost of drug and alcohol related crime has been estimated at approximately $72 billion, in 1999 dollars, for medical and mental health care and other tangible expenses such as public property damage and services (Miller et al., 2006). Two-thirds of all inmates report symptoms meeting criteria for substance abuse, are more likely to have a prior criminal record (70 percent versus 46 percent), and are twice as likely to have three or more prior incarceration sentences (U.S. Dept. of Justice, 2005). The cost to society per incarceration is approximately $23,000 per annum (U.S. Dept. of Justice, 2004). Drug treatment’s most substantial and consistent economic benefits are avoided costs of criminal activity (McCollister & French, 2003). Drug treatment can reduce those costs; analysis of California Treatment Outcome Protocol data found decreases in costs associated with victimization from criminal activities and additional costs associated with crimes and incarceration (Ettner et al., 2006).

Drug treatment also reduces criminal activity. Longitudinal studies involving one-year and five-year follow-ups found that individuals who remained in treatment at least one week were less likely to commit a crime (Zarkin et al., 2002) and illegal activity or re-offenses were reduced for clients treated for six months or longer (French et al., 1993; Holloway, 2006; Hubbard et al., 2003). The Drug Abuse Reporting Program (DARP) and the Treatment Outcome Prospective Study (TOPS) also document large decreases in criminal involvement as measured by arrests, convictions, and incarcerations post treatment (Farabee et al., 2004; Grella et al., 2006; Hubbard et al., 1989; Hser & Evans, 2008; Sells & Simpson, 1980; Simpson, 1981; Simpson & Sells, 1982; 1990). Research has also shown that length of stay and treatment engagement impacts post-treatment criminal activity. Longer lengths of stay in treatment have a negative and statistically significant impact on subsequent criminal activity (French et al., 1993). A recent study in Oklahoma found that clients who initiate a new outpatient treatment episode and remain engaged in treatment are significantly less likely to be arrested or incarcerated within the next year (Garnick et al., 2007).

Drug Abuse and Employment

Chronic unemployment is common among drug users and is related to criminal behavior, continued drug use, and poor treatment results (Platt, 1995). In 2005, 29 percent of TEDS admissions aged 16 and over were employed (Office of Applied Studies, 2006). Employment contributes to drug abuse treatment success and is an important treatment outcome (French et al., 1993; Institute of Medicine, 1990; Leshner, 1997). Employment is often examined in terms of prior employability and an individual’s ability to reconnect with the job market upon treatment completion. Individuals who successfully complete treatment are more likely to be employed post discharge, have fewer work related problems, and earn higher incomes (Kamara & Van Der Hyde, 1998; Slaymaker et al., 2006; Wickizer et al., 2000; Zarkin et al., 2002; Finigan et al., 1996; Luchansky et al., 2000; Metsch et al., 2003). For example, Brown et al. (1997) found that clients who complete treatment earned an average of $145 more per month than clients who were assessed but not admitted. Studies also show a relationship between length of stay in treatment and employment outcomes. Results from the TOPPS II Interstate Cooperative Study Group (Arria, 2003), which examined the impact of treatment completion on employment in three states, found that earned income improved from 22 percent to 49 percent for completers and that the length of time in treatment, regardless of completion status, was significantly associated with the likelihood of employment one year post treatment. Successful post-treatment employment is also associated with decreased relapse among drug users following treatment (Wolkstein & Spiller, 1998) and post-treatment community functioning and community reintegration (Comerford, 1999; Platt, 1995; Room, 1998).

Drug Abuse and Medicaid Utilization

Drug abuse can have a substantial impact on Medicaid utilization and cost given the high rates of physical and mental health co-morbidity among drug-using adults and the resulting need for care. Medicaid beneficiaries diagnosed with a drug use disorder use more medical care and have higher medical expenditures than beneficiaries with other behavioral health conditions but no drug use disorder (Clark et al., 2009). A significant portion of medical encounters with drug-using adults occur in high cost, acute care settings. Drug-using adults are more than twice as likely to visit an emergency department and more than six times more likely to be hospitalized than the general population (Stein, 1993). Outcome research suggests that clients who successfully receive treatment use fewer health services and reduce other medical costs (Parthasarathy et al., 2001; McCollister & French, 2003; Ettner et al., 2006; Wickizer et al., 2006), including Medicaid expenditures (McConnell et al., 2008). In their study of the costs of substance abuse to the Medicaid hospital care program, Fox et al. (1995) discovered more 60 medical conditions involving 1100 diagnoses that were at least partially attributable to the patients’ substance abuse. Given these findings, an estimated 4 million days (1 of 5 Medicaid hospital days) were devoted to care that was substance abuse related, accounting for almost $8 billion in Medicaid expenditures.

Long-term costs of Medicaid-insured patients as compared to demographically matched commercially insured patients decline after enrollment in drug treatment (Walter et al., 2005). Studies linking substance abuse data found that treatment completion reduces inpatient health care utilization, including mental health services; these gains are offset by a subsequent increase in outpatient utilization (Hser & Evans, 2008; Maynard et al., 2000; Shephard et al., 2002; Wickizer et al., 2006). Similar studies examining the impact of policy changes on health care utilization (Deck et al., 2000, 2004, 2005, 2006; Fuller et al., 2006) illustrate how public policies affecting access to addiction treatment can influence Medicaid utilization.

References

• Arria, A.M. TOPPS-II Interstate Cooperative Study Group. (2003). Drug treatment completion and post-discharge employment in the TOPPS-II Interstate Cooperative Study. Journal of Substance Abuse Treatment, 25(1), 9-18.
• Brown, M., Longhi, D. & Luchansky, B. (1997). Employment Outcomes of Chemical Dependency Treatment and Additional Vocational Services Publicly Funded by Washington State. A Four-and-a-Half Year Follow-up Study of Indigent Persons served by Washington State’s Alcoholism and Drug Addiction Treatment and Support Act (ADATSA). Key Findings. Briefing Paper. Olympia, WA: Washington State Department of Social and Health Services, Research and Data Analysis. Briefing Paper 4.29bp.
• Campbell, K.M. (2009). Impact of record-linkage methodology on performance indicators and multivariate relationships. Journal of Substance Abuse Treatment, 36(1), 110-7.
• Campbell, K.M., Deck, D. & Krupski, A. (2008). Record linkage software in the public domain: a comparison of Link Plus, The Link King, and a ‘basic’ deterministic algorithm. Health Informatics Journal, 14(1), 5-15.
• Clark, R.E., Samnaliev, M. & McGovern, M.P. (2009). Impact of substance disorders on medical expenditures for Medicaid beneficiaries with behavioral health disorders. Psychiatric Services, 60(1), 35-42.
• Comerford, A.W. (1999). Work dysfunction and addiction: Common roots. Journal of Substance Abuse Treatment, 16(3), 247-53.
• Commons, M., McGuire, T.G. & Riordan, M.H. (1997). Performance contracting for substance abuse treatment. Health Services Research, 32(5), 631-50.
• Deck, D. & Carlson, M.J. (2004). Access to publicly funded methadone maintenance treatment in two western states. Journal of Behavioral Health Services and Research, 31(2), 164-77.
• Deck, D. & Carlson, M.J. (2005). Retention in publicly funded methadone maintenance treatment in two Western States. Journal of Behavioral Health Services and Research, 32(1), 43-60.
• Deck, D.D., McFarland, B.H., Titus, J.M., Laws, K.E. & Gabriel, R.M. (2000). Access to substance abuse treatment services under the Oregon Health Plan. JAMA, 284(16), 2093-9.
• Deck, D.D., Wiitala, W.L. & Laws, K.E. (2006). Medicaid coverage and access to publicly funded opiate treatment. Journal of Behavioral Health Services and Research, 33(3), 324-34.
• Ettner, S.L., Huang, D., Evans, E., Ash, D.R., Hardy, M., Jourabchi, M. & Hser, Y.I. (2006). Benefit-cost in the California treatment outcome project: does substance abuse treatment “pay for itself”? Health Services Research, 41(1), 192-213.
• Farabee, D., Hser, Y., Anglin, M.D. & Huang, D. (2004). Recidivism among an early cohort of California’s Proposition 36 offenders. Criminology & Public Policy 3(4), 563-584.
• Finigan, M. (1996). Societal outcomes and cost savings of drug and alcohol treatment in the state of Oregon. Salem, OR: Oregon Office of Drug Abuse Programs.
• Fox, K., Merrill, J.C., Chang, H.H. & Califano, J.A. (1995). Estimating the costs of substance abuse to the Medicaid hospital care program. American Journal of Public Health, 85(1), 48-54.
• French, M.T., Zarkin, G.A., Hubbard, R.L. & Rachal, J.V. (1993). The effects of time in drug abuse treatment and employment on post treatment drug use and criminal activity. American Journal of Drug and Alcohol Abuse, 19(1), 19-33.
• Fuller, B.E., Rieckmann, T.R., McCarty, D.J., Ringor-Carty, R. & Kennard, S. (2006). Elimination of methadone benefits in the Oregon Health Plan and its effects on patients. Psychiatric Services, 57(5), 686-91.
• Garnick, D.W., Horgan, C.M., Lee, M.T., Panas, L., Ritter, G.A., Davis, S. et al. (2007). Are Washington Circle performance measures associated with decreased criminal activity following treatment? Journal of Substance Abuse Treatment, 33(4), 341-52.
• Garnick, D.W., Lee, M.T., Chalk, M., Gastfriend, D., Horgan, C.M., McCorry, F. et al. (2002). Establishing the feasibility of performance measures for alcohol and other drugs. Journal of Substance Abuse Treatment, 23(4), 375-85.
• Grella, C.E., Hser, Y.I. & Hsieh, S.C. (2003). Predictors of drug treatment re-entry following relapse to cocaine use in DATOS. Journal of Substance Abuse Treatment, 25(3), 145-54.
• Holloway, K.R., Bennett, T.H. & Farrington, D.P. (2006). The effectiveness of drug treatment programs in reducing criminal behavior: a meta-analysis. Psicothema, 18(3), 620-9.
• Hser, Y.I. & Evans, E. (2008). Cross-system data linkage for treatment outcome evaluation: lessons learned from the California Treatment Outcome Project. Evaluation and Program Planning, 31(2), 125-35.
• Hubbard, R.L., Craddock, S.G. & Anderson, J. (2003). Overview of 5-year follow-up outcomes in the drug abuse treatment outcome studies (DATOS). Journal of Substance Abuse Treatment, 25(3), 125-34.
• Hubbard, R.L., Marsden, M.E., Rachal, J.V., Harwood, H.J., Cavanaugh, E. & Ginzburg, H.M. (1989). Drug abuse treatment: A natural study of effectiveness. Chapel Hill, North Carolina: The University of North Carolina Press.
• Institute of Medicine. (1990) Treating Drug Problems. Washington, D.C.: National Academy Press.
• Institute of Medicine. (1997). Managing managed care: Quality improvement in behavioral health. Washington, DC: National Academy Press.
• Institute of Medicine. (2006). Improving the Quality of Health Care for Mental and Substance-Use Disorders: Quality Chasm Series. Washington, DC: National Academy Press.
• Kamara, S.G. & Van der Hyde, V.A. (1998). Employment outcomes of regular versus extended outpatient alcohol and drug treatment. Medicine and Law, 17(4), 625-32.
• Leshner, A.I. (1997). Addiction is a brain disease, and it matters. Science, 278(5335), 45-7.
• Luchansky, B., He, L., Krupski, A. & Stark, K.D. (2000). Predicting readmission to substance abuse treatment using state information systems. The impact of client and treatment characteristics. Journal of Substance Abuse, 12(3), 255-70.
• Maynard, C., Cox, G.B., Krupski, A. & Stark, K. (2000). Utilization of services by persons discharged from involuntary chemical dependency treatment. Journal of Addictive Diseases, 19(2), 83-93.
• McCarty, D., McGuire, T.G., Harwood, H.J. & Field, T. (1998). Using state information systems for drug abuse services research. American Behavioral Scientist, 41(8), 1090-1106.
• McCollister, K.E. & French, M.T. (2003). The relative contribution of outcome domains in the total economic benefit of addiction interventions: a review of first findings. Addiction, 98(12), 1647-59.
• Metsch, L.R., Pereyra, M., Miles, C.C. & McCoy, C.B. (2003). Welfare and work outcomes after substance abuse treatment. Social Service Review, 77(2), 237-254.
• Miller, T.R., Levy, D.T., Cohen, M.A. & Cox, K.L. (2006). Costs of alcohol and drug-involved crime. Prevention Science, 7(4), 333-42.
• Office of Applied Studies. Treatment Episode Data Set (TEDS) Highlights – 2005. (2006). DHHS Publication No. (SMA) 07-4229. Rockville, MD: Substance Abuse and Mental Health Services Administration.
• Parthasarathy, S., Weisner, C., Hu, T.W. & Moore, C. (2001). Association of outpatient alcohol and drug treatment with health care utilization and cost: revisiting the offset hypothesis. Journal of Studies on Alcohol, 62(1), 89-97.
• Platt, J.J. (1995). Vocational rehabilitation of drug abusers. Psychological Bulletin, 117(3), 416-33.
• Rajkumar, A.S. & French, M.T. (1997). Drug abuse, crime cost and the economic benefits of treatment. Journal of Quantitative Criminology, 13, 291-323.
• Room, R. (1998). The co-occurrence of mental disorders and addictions: Evidence on epidemiology, utilization, and treatment outcomes. (No. 141). Toronto, ON: Centre for Addiction and Mental Health.
• Sells, S.B. & Simpson, D.D. (1980). The case for drug abuse treatment effectiveness, based on the DARP research program. British Journal of Addiction, 75(2), 117-31.
  
• Shepard, D.S., Daley, M., Ritter, G.A., Hodgkin, D. & Beinecke, R.H. (2002). Managed care and the quality of substance abuse treatment. Journal of Mental Health Policy and Economics, 5(4), 163-74.
• Simpson, D.D. (1981). Treatment for drug abuse. Follow-up outcomes and length of time spent. Archives of General Psychiatry, 38(8), 875-80.
• Simpson, D.D. & Sells, S.B. (1982). Effectiveness of treatment for drug abuse: An overview of the DARP research program. Advances in Alcohol and Substance Abuse, 2(1), 7-29.
• Simpson, D.D. & Sells, S.B. (1990). Opioid Addiction and Treatment: A 12-Year Follow-up. Malabar, Florida: Krieger Publishing Company.
• Slaymaker, V.J. & Owen, P.L. (2006). Employed men and women substance abusers: job troubles and treatment outcomes. Journal of Substance Abuse Treatment, 31(4), 347-54.
• U.S. Department of Justice, Office of Justice Programs. (2004). State Prison Expenditures 2001. Bureau of Justice Statistics Special Report.
• U.S. Department of Justice, Office of Justice Programs. (2005). Substance Dependence, Abuse, and Treatment of Jail Inmates, 2002. Bureau of Justice Statistics Special Report.
• Wickizer, T.M., Campbell, K., Krupski, A. & Stark, K. (2000). Employment outcomes among AFDC recipients treated for substance abuse in Washington State. Milbank Quarterly, 78(4), 585-608.
• Wolkstein, E. & Spiller, H. (1998). Providing vocational services to clients in substance abuse rehabilitation. Directions in Rehabilitation Counseling, 9, 65-78.
• Zarkin, G.A., Dunlap, L.J., Bray, J.W. & Wechsberg, W.M. (2002). The effect of treatment completion and length of stay on employment and crime in outpatient drug-free treatment. Journal of Substance Abuse Treatment, 23(4), 261-71.