The Evaluation of the Illinois Multi-Site Police-Initiated Deflection Initiative

Introduction

The Illinois Multi-Site Deflection Initiative (Initiative), funded by the Illinois Department of Human Services (IDHS) Division of Substance Use Prevention and Recovery, is a multifaceted effort to improve behavioral health outcomes. Deflection is a relatively new field at the intersection of public safety and public health that connects individuals who have contact with police to behavioral health and other services and resources.^[1] Deflection builds upon pre-arrest diversion, established several decades ago, which aims to reduce the justice system involvement following arrest.^[2] To increase preventative public health measures, many jurisdictions have turned to deflection, which addresses behavioral health needs and reduces criminal justice system involvement prior to arrest.^[3] Deflection leverages law enforcement contacts with individuals who could benefit from behavioral health, housing, or other services, and utilizes a warm handoff to deflection specialists, who bridge the gap between law enforcement and service providers (Figure 1).

Figure 1

Deflection Evaluation Challenges

Evaluating deflection programs presents unique challenges. Deflection is a voluntary referral program in which participants can start or discontinue participation at any time. These programs typically rely upon police officers to identify persons at their discretion, making it difficult obtaining a control group for outcome comparisons. Additionally, deflection has expanded beyond law enforcement to include referrals from fire departments, EMS, public health, and social service agencies. Furthermore, participating individuals may receive different interventions and levels of service.^[4] Some participants may request only food or clothing, while others engage in long-term substance use disorder treatment or find permanent housing.

Another challenge is that deflection programs generally do not follow a standardized curriculum or set of guidelines. Due to the relative newness of the field, deflection currently has a limited evidence base, making it difficult to establish best practices.^[5] Programs may engage participants through various models or “pathways”; there are currently six recognized deflection pathways.^[6] While many of these pathways rely upon police referrals, there is also the self-referral pathway, where individuals voluntarily enter a police department to seek help, and the post-overdose pathway, where EMS teams or deflection staff reach out to individuals who have experienced a non-fatal overdose.

Programs also operate under a variety of names, such as Angel,^[7] Quick Response Teams (QRT), ^[8] Law Enforcement Assisted Diversion (LEAD),^[9] and Stop, Triage, Engage, Educate and Rehabilitate (STEER).^[10] While most deflection programs support individuals with substance use disorders or mental health issues, some serve additional populations. For example, among the Illinois initiative’s deflection programs, most serve individuals with substance use disorders, but one program supports crime victim, and another focuses on people who are unhoused.

Deflection also faces data-related challenges. Deflection participants will have contact with multiple agencies, from law enforcement, to deflection staff, to services providers in their communities. Data may not be collected consistently across agencies, or may not be shared due to privacy concerns. Some behavioral health providers express concerns about compliance with the Health Insurance Portability and Accountability Act (HIPAA) and 42 CFR part 2.^[11] The latter is a federal regulation that protects the confidentiality of patient records for persons undergoing substance use disorder treatment.^[12] However, some exceptions allow the use of those records for research purposes.^[13] Obtaining outcome data, such as justice system contacts, treatment records, overdose data, and death certificates, can be particularly difficult.

Background on the Illinois Deflection Initiative

The Initiative is a state-level partnership between IDHS, the Illinois State Police (ISP), and the Illinois Criminal Justice Information Authority (ICJIA). Deflection sites across the state partner with ISP multijurisdictional drug task forces.^[14] These task forces are composed of ISP officers and local law enforcement officers from one or more counties and aim to reduce drug distribution and trafficking, or to address other large-scale issues such as violent crime. The deflection sites rely on ISP officers to provide referrals to the deflection team and forge relationships with local law enforcement agencies. The Initiative began planning its first program in June 2021 and launched in January 2022. As of July 2024, the initiative has established five operational programs, with six more in the planning stage, and will span 45 counties (45% of the state) (Figure 2).

Figure 2

Map of Illinois Multi-Site Deflection Initiative Program Sites

Note. Deflection program sites as of July 2024.

Most deflection programs developed through the Initiative are multijurisdictional, covering between three and nine counties, and serve a mix of rural and urban populations (Table 1). These programs tend to focus on providing services to those with substance use, mental health, or co-occurring disorders. Some programs serve slightly different populations. One in East St. Louis connects individuals who are victims of violent crime to appropriate services; another in Springfield focuses on providing services to individuals experiencing homelessness; and one in development in the Chicagoland area will serve youth who have encounters with ISP troopers.

Table 1

Illinois Multi-Site Deflection Evaluation Programs by Characteristics

Note. As of July 2024. SUD = Substance use disorder.

In addition to sponsoring the Initiative, IDHS dedicated funding to support an evaluation. Our evaluation team conducted a formative and process evaluation of each program as it developed. We began with a research design plan, but, as with such a multifaceted and complex initiative, we pivoted to an adaptive approach as the programs evolved. We employed a mixed-methods approach, combining quantitative and qualitative analyses. We strived to provide regular feedback to the funder and stakeholders through meetings, presentations, briefs, and reports.

The Evaluation Team

ICJIA’s Center for Justice Research and Evaluation leads the evaluation.^[15] As the programs grew in number, so did the evaluation team. The team now consists of six staff members—four full-time and two part-time—working on the evaluation project. All team members have master’s degrees in social science-related fields and have research and evaluation experience. At the end of year two, we hired a program manager. This position significantly enhanced the evaluation process’s efficiency, effectiveness, and quality. This program manager provides coordination, quality control, resource allocation, and timeline management.

The Evaluation Design

We established a logic model^[16] specifically for the evaluation, shown below (Figure 3). It depicts our resources, activities, and evaluation goals. Through our work, we hope to understand, inform, and document the programs’ implementation and operations in the short term. In the long term, we aim to increase the understanding of these programs and deflection in general. The program staff and stakeholders can use our findings for program improvement and can leverage the findings to support new or continued funding and program sustainability. Since deflection is a relatively new field, we hope that other researchers and programs can use our work, and that we can offer guidance to the field as a whole.

Figure 3

Logic Model of the Illinois Multi-Site, Deflection Initiative Evaluation

Evaluation Challenges and Lessons Learned

Evaluating a large, complex initiative comes with multiple challenges. The approaches taken by the evaluation team to mitigate some of these common challenges in large-scale evaluations may interest others conducting or funding similar work.

Coordination and Collaboration

The team working on such a large evaluation project must dedicate time to data collection, analysis, writing, collaboration, and communication. We hold weekly internal, sub-team, and ad hoc group meetings as needed. We use daily email and messaging and engage in informal conversations among team members. Due to the many facets of the evaluation, we utilize a work management platform, Smartsheet, to help coordinate the evaluation.

Over the past two years, we have attended and presented at local and national conferences to learn from and collaborate with others in the deflection field. These conferences included those hosted by the National Criminal Justice Association, the Illinois Governor’s Office Deflection Learning Lab, the International Co-Responder Alliance, the Police, Treatment, and Community Collaborative (PTACC) Summit, and the Police Assisted Addiction and Recovery Initiative (PAARI) Conference.

Program Relationships

To increase collaboration with program stakeholders, our team has increased the number of meetings with funders, program development training and technical assistance providers, and program staff. Through a contract with IDHS, the TASC Center for Health and Justice (TASC CHJ) develops and implements the deflection programs by bringing together a leadership team of dedicated law enforcement officers, service providers, and community members at each deflection site. Upon launching of a deflection program, TASC Deflection including regional managers, site managers, and deflection specialists. We began attending post-event debrief sessions held by TASC CHJ to observe how program partners collaborate and to build relationships between our organizations. In response to requests for quicker turnaround on event evaluation and survey findings, we have begun providing research briefs to program funders and stakeholders following deflection program kickoffs, launches, and other similar events. These briefs allow our partners to review and incorporate findings and feedback more quickly, using an iterative process to enhance development and operations for each deflection program.

Our research team is based in Chicago, which makes it challenging to fully engage in Community-Based Participatory Research (CBPR) and conduct regular site visits.^[17] CBPR is an approach that engages community members in all stages of the research process, from developing research questions and methodologies, to collecting, analyzing, and interpreting the data. The shared decision-making promotes mutual respect between researchers and community, and increases the relevance of the research to the community. Engaging in CBPR typically involves meeting with community members in-person and in their preferred community settings. Some Illinois deflection programs are located over 300 miles away, limiting our ability to engage with those teams.

Data Collection

Administrative Data

Each month, TASC Ops. shares program data with ICJIA. This data, while non-identifiable, includes demographic information and services and/or treatment data for all deflection participants. While analyzing the program data, we encountered inconsistencies related to administrative data collection, requiring modifications to capture more variables and more accurate data. We also learned that providing different stakeholders with a data summary during quarterly presentations or upon request was insufficient. Therefore, a dashboard was created using Tableau data visualization software. This was intended to replace the need for regular or ad hoc data requests from the programs, stakeholders, and funders. The dashboard includes the number of participant referrals, the number of active deflection participants, and the rate of active participants. It also includes referral and participant demographics, referral sources, and the types of services requested by participants. There is also an interactive map of the state of Illinois by county, so users can see which deflection site and ISP task forces are operational in each county. The dashboard is updated monthly and allows users to sort program data by time period and deflection program. By providing law enforcement, deflection staff, and other deflection stakeholders with this dashboard, we minimize the need for monthly data reports and ad hoc data requests, as the updated data is readily available to all partners.

From the first site’s launch in January 2022 through June 2024, the five active deflection sites received 781 referrals: 61% (476) from state police, 28% (222) from local police, 9% (68) were self-referrals, and the remaining 2% (15) were community-based referrals. The sites had a combined 225 individuals who agreed to participate in the program following referrals, for a participation rate of 28.8%.

Planning Process Surveys

Each program is developed through action planning, guided by TASC CHJ. At the end of each action planning session, usually completed in six sessions, we surveyed program stakeholders, including law enforcement officers and local social service providers. Survey questions included satisfaction with the sessions, understanding of the program’s purpose, readiness to contribute to the program, and perceived sustainability of the initiative.

Action planning surveys indicated that satisfaction with the process and understanding of deflection increased throughout the action planning proceedings. As TASC CHJ further developed the sessions, feedback from later deflection sites became increasingly positive.

Collaboration Inventory

To measure how program leadership teams view collaboration in their program, we administered the Wilder Collaboration Inventory^[18] as a paper survey at the end of the program action planning. We then administered the Inventory via Qualtrics to program stakeholders and staff approximately one year after the programs became operational. This allows us to compare collaboration items between time points and across different deflection programs in the initiative. Collaboration surveys consistently showed that action planning participants were concerned about insufficient resources to support their deflection programs and that few sites felt they had the appropriate cross-section of community members at the table.^[19]

The collaboration inventory identified several common strengths across sites, including that teams felt collaboration was in their best interest, that team members held each other in high regard, and that they believed that the deflection programs would not be successful without collaboration between their agencies.

Stigma Measure

We administered a survey at program kickoff and launch events to gather feedback on the event and to measure stigma toward people living with substance use. This will eventually allow us to compare whether program stakeholders experience a change in stigma between when they first join the deflection program and after approximately a year of developing a deflection program.

We have collected 65 responses from individuals who attended a site kickoff event and 93 responses from those who attended a site launch event. We are particularly interested in measuring how stigma may evolve among law enforcement members engaged in deflection programs and have 25 responses (16%) from police officers.

Pre- and Post-Testing

We conducted pre- and post-tests for deflection participants based on the unique characteristics of each site. We used the Brief Cope survey for the East St. Louis deflection program that serves victims of violent crimes; the Social Needs Screening Tool for the Springfield deflection program that serves people who are unhoused;^[20] and the Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES) survey for all other deflection programs.^[21] We provided training to deflection staff to help them administer the surveys to participants. We incentivized participants who completed the surveys and deflected staff who administered them. Following continued low survey completion rates, we are reevaluating how we wish to gather participant outcome data.

Only one of the five operational sites provided a sufficient number of matched pre- and post-tests for us to be able to analyze the data. We are currently analyzing the pre-test data for all sites and the matched pre- and post-test data for the SI CERT site.

Police Survey

We developed a survey for law enforcement officers to measure their awareness of and attitudes toward the deflection program in their area. This was intended to measure improvements in officer views toward deflection after program implementation and participation.

The online questionnaire was sent to 26 ISP and East St. Louis police officers assigned to the department’s investigative unit. The response rate was 80% (21), and the vast majority of them worked for ISP. Initial analysis showed that the majority of officers (15 of 21) had heard of the ESL CERT deflection program and that all those who had heard of it were aware that making referrals to the program was at least somewhat encouraged. Of the 15 officers who had heard of the program, nine had made referrals, four referred someone directly to TASC Ops., and the rest forwarded the referral information to the ISP point of contact for the program.

Interviews

After programs have been operational for at least six months, we recruit stakeholders and program staff for interviews. We recruited for interviews by advertising them at local leadership team meetings and by sending individualized emails to members of a deflection site’s staff and stakeholder team. Interviews were conducted virtually, were semi-structured, and typically lasted an hour. Interview topics focused on the staff and stakeholders’ experiences with the program’s development and operations, and their perceptions of the program’s processes, strengths, and challenges. We recorded and transcribed the interviews, developed a codebook, checked for intercoder consistency, then coded each interview and reported on common themes.

We completed 11 interviews with staff and stakeholders from the ESL CERT site and the same number at the SI CERT deflection site. ESL CERT law enforcement officers and service providers identified the TASC deflection staff as one of the major strengths of their program. However, service providers requested that the deflection specialists increase their outreach to service organizations in the area and forge stronger relationships with providers. Overall, the ESL CERT interviews uncovered a need for additional communication and stronger relationships between all program partners.

SI CERT service providers noted that they are operating at full capacity and experiencing a shortage of mental health professionals, substance use treatment facilities, and housing resources, which limits their ability to serve the deflection program participants. Service providers also noted that lack of law enforcement buy-in has limited the program’s effectiveness. Some speculated that law enforcement officers in the area hold a stigma towards those with substance use disorders, including deflection specialists who may have lived experience of justice system involvement or be in recovery.

We also began recruiting program participants for interviews and provided a gift card for compensation. Due to initial low interest, we are trying to engage participants in new ways, such as through the service providers and recruiting in person at the program sites.

Field Observations

We traveled to deflection sites across the state and conducted field observations, including at deflection action planning sessions, community kickoff and launch events, and community engagement and celebratory events. In addition, we conducted additional site visits to meet with deflection staff and police officers on-site. Field observations included handwritten or typed notes of proceedings, observation of community member engagement and interpersonal dynamics, and observation of interactions between local community members and external program partners.

Our field observations showed the importance of strengthening partnerships across communities and professions. This finding aligns with what was reflected in the collaboration inventory, in which program stakeholders noted the need for additional community members at the table and stronger partnerships across the program. Implementing a Community Standards document helped site partners navigate these complex relationships. We observed that holding an in-person meeting helped reinvigorate sites following months of virtual meetings. Many deflection teams enjoyed connecting and expanding their knowledge of local organizations and resources.

Deliverables

For each program that has been operational to date, we produced an action planning report,^[22] research briefs summarizing kickoff and launch events, a staff and stakeholder feedback report, a local data presentation, and regular summaries of program administrative data. These deliverables typically include a recommendations section based on the content of the report; each recommendation we provide is supported with peer reviewed references. The recommendations provided are not only for the betterment of future deflection programs in this Initiative, but to aid in the development of any other similar deflection sites. We have received feedback from our partner organizations that our deliverables are read thoroughly and taken into account when improving current sites and planning for future deflection sites. We continue to develop our data dashboard in order to disseminate program data to program partners and staff regularly. To summarize the entire deflection initiative, we have produced an end-of-year publication summary and this evaluation design summary.

Future Research

To further address the challenges that we have encountered in survey administration and data completeness, we plan to seek feedback directly from program staff and participants regarding the content and format of the surveys we use. We will also explore the potential for new and creative methods of survey administration, including implementing them into service touchpoints or via a mobile app.

To build upon the findings from our evaluation activities thus far, our team will focus on collecting both quantitative and qualitative data from deflection program participants. This will include expanded use of the pre- and post-tests, additional participant interviews, and a review of which outcome metrics are most appropriate for each deflection site. While each site has unique components, we plan to incorporate standardized measures across sites to enhance generalizability and comparability between sites. Use of standardized measures may open up opportunities to collaborate with external research partners and compare deflection processes and outcomes across program models nationwide.

Conclusion

We have completed the second year of an evaluation of the Illinois Multi-Site Deflection Initiative. We evaluated a training for deflection specialists,^[23] attended action planning sessions at 11 deflection programs, analyzed program data for the current five operational programs, and conducted interviews with program staff, community stakeholders, and participants in three programs. These evaluation activities and others have been disseminated in reports, research briefs, and presentations. This summary provided a point-in-time review of our evaluation efforts and our vision for future avenues for evaluation activities.