Today’s Critical Issues Report is about a significant new research study that evaluates the impact of PERF’s ICAT training program as it was implemented by the Louisville, KY Metro Police Department in 2019. 

The study, cited in a Washington Post article, was conducted by Dr. Robin S. Engel and her colleagues at the University of Cincinnati, under the auspices of the IACP/University of Cincinnati Center for Police Research and Policy.  PERF was not involved in the research.

The key findings are found on page xii in the Executive Summary:

  • ICAT training was associated with a statistically significant 28% reduction in use of force by Louisville Metro officers.
  • ICAT also was associated with a 26% decline in citizen injuries.
  • And ICAT was associated with a 36% reduction in officer injuries.

“Notably, these results were beyond chance,” Professor Engel said. “These significant reductions in force and injuries occurred above and beyond observed changes in arrest patterns. This is the first known independent evaluation of a police de-escalation training that has demonstrated significant changes in police behavior.”

PERF Executive Director Chuck Wexler spoke with Professor Engel about the study’s history, methodology, and findings.

In addition to her academic work, Professor Engel previously served as UC’s Vice President for Safety and Reform, where her duties included oversight of the operations and reform efforts of the University of Cincinnati Police Division.

Chuck Wexler: Tell us about how this study came about.

Professsor Engel:  When I took over the University of Cincinnati Police Division (UCPD) after an officer-involved shooting, I wanted to change use-of-force policy and training. That’s a heavy burden for a law enforcement executive to take on, because the lives of officers and citizens hang in the balance of the policies and training we provide. So I looked to the evidence to see what would best guide me to make those decisions.  But there was no evidence.

So we picked the training, ICAT, that we thought was the best available, and then we tested it, because I did not want another law enforcement executive to have to wonder whether he or she was making a good decision for their agency. The only way we can improve training is to be willing, as police executives, to test our training programs and put that information out to the rest of the field. That’s why I did this study.

When we brought ICAT to the UCPD, first we looked at changes in officers’ attitudes, knowledge, and confidence in using de-escalation tactics. We did pre-training, post-training, and follow-up surveys with our officers. We found that the training was positively received, and that changes in officers’ attitudes toward interactions with the public and use of force were trending in a positive direction.

At UCPD, we didn’t have enough uses of force to determine whether ICAT had an impact on use of force. So I had to find a larger agency implementing de-escalation training that was willing to be studied.

That was about the same time the Louisville Metro Police Department, which has about 1,200 officers, was getting ready to implement ICAT training, and they were willing to have it systematically evaluated.

Wexler:  As a researcher, is it difficult for you to get access to a police department to measure something like this?

Professsor Engel:  Our team is fortunate, because we have established a good reputation and track record of working with law enforcement agencies, so they’re more willing to work with our team. But this is a challenging issue for law enforcement executives. There’s a need to build evidence in the field, but some law enforcement executives are reluctant to do that. For one thing, it’s time consuming to engage in research. It takes up additional resources, and sometimes the research designs are hard to implement. You also have to be able to address your public if the findings are negative, and explain what that means for your agency.

Wexler:   What was the research design for this study?

Professsor Engel:   A randomized control trial design is considered one of the highest standards in research. A randomized control trial design means that people are randomly selected to be “treated,” which in this case means that officers received training. Then you have a group that is not trained, and you compare them to one another.

In the realities of our profession, you really can’t randomly train some and not others within your agency. There’s the expectation that all will receive training, particularly if it turns out it’s beneficial. But you can adjust the timing of the training.

We also know that if an officer is trained and is assigned to work with an officer who is not trained, that contaminates the experiment.

So what we used is a “stepped wedge design,” which in the case of Louisville Metro Police means that we randomized which divisions were trained first, and we compared those divisions to the other divisions that hadn’t been trained. You continue to do those comparisons until the entire police force is trained.

So it’s a form of a randomized control trial, which is one of the strongest research designs we have.

Wexler:   How long did it take for the Louisville Metro Police Department to be trained?

Professsor Engel:   Louisville Metro began their training in February 2019, and they completed it at the end of November 2019. Our study looked at data beginning in January 2018, which was a year before the training, and continued through April 2020, which was beyond the training period.

Wexler:   What were the key findings of the study?

Professsor Engel:   The first thing we found was that this training was generally well-received by the officers, which we saw in the satisfaction measures.

We found there was some fall-off in the reported satisfaction during the follow-up period, but it was still positively received overall. So we found that the officers liked the training and, more importantly, they learned and felt confident and comfortable in their de-escalation skills after the training.

We also found that they changed their attitudes significantly in terms of their views towards citizens, and also in terms of their views towards use of force more generally. So all the attitudinal measures we looked at were trending in a positive direction.

We found that sergeants and lieutenants who were interviewed were positive about the training but did not take active steps to reinforce the training in the field. We think this is an area of improvement for any agency implementing de-escalation training department-wide. There’s real opportunity for a more holistic approach across the agency.

The most important finding was that there was a significant reduction in officers’ uses of force, a reduction in citizen injuries, and a reduction in officer injuries. This is the first independent study that has examined the impact of de-escalation training on police behavior and found an effect.

Wexler:   Did these findings surprise you?

Professsor Engel:   When I first looked at the findings, I thought it must be because there were fewer opportunities to use force, meaning that arrest situations were going down. But when we controlled for that, the findings still held. So it wasn’t about fewer opportunities to have to use force.

That said to me that something is really different about this training. Officers are responding to it in a positive way, and they’re implementing it in the field. That’s a really positive and important finding for policing generally, not just for the Louisville Metro Police Department.

Wexler:   The findings about the Critical Decision-Making Model weren’t as positive, correct?

Professsor Engel:   Yes, that particular component of ICAT training needs to be fleshed out a little bit better with trainers, to emphasize how it fits together with the other tactics they’re learning.

And we found in the follow-up survey that this is one of the components that has more training decay. So there needs to be better training up front, but also make sure it’s reinforced in the field and there’s follow-up training.

Wexler:   Louisville Metro has been in the news this summer for use of force in the death of Breonna Taylor. What would you say to those who might question the efficacy of LMPD’s use-of-force training based on that incident?

Professsor Engel:   In this study, we were focused on the patrol-officer level. The officers who executed the search warrant in the Breonna Taylor case were in a specialized unit. That specialized unit was trained in de-escalation like all other officers. I think every police executive knows that one situation doesn’t define an agency.

Wexler:  And it’s important to emphasize that ICAT focuses on non-firearm situations.

Professsor Engel:   Absolutely, I think that’s where it applies. Although you can think about using de-escalation training and tactics in a whole host of situations, even, for instance, in de-escalating an interaction with your own peers.

I think the important thing to keep in mind with the Louisville Metro Police Department is that they did experience significant reductions in overall uses of force, the severity of the force that was used, officer injuries, which were reduced by 36%, and citizen injuries, which were reduced by 26%, after the introduction of this training.

This is not going to prevent every bad situation from happening. But it is going to have an impact overall, at the aggregate level.

Wexler:   This study only looked at one agency. How generalizable are these findings to other departments?

Professsor Engel:   That is an important caveat: this is one study in one agency. Clearly we need to build the evidence base. But this is the evidence we have at this moment.

For this study, Louisville Metro officers were surveyed and asked how they felt about their agency. Only about a quarter of the officers agreed or strongly agreed that the Louisville Metro Police Department was a good agency to work for. This suggested that there were some issues with officer morale, and this might be a challenging agency to provide new training in. But they overcame those hurdles, and those officers liked this training, responded to it, and used it in the field.

I actually found that very encouraging in terms of how generalizable these finding may be. When you implement a training in challenging circumstances and still have a positive impact, that bodes well for what might happen in other agencies with the training.

Wexler:  Is training by itself enough to change officer behavior?

Professsor Engel:   I think it’s so important for law enforcement executives across the country to realize that they can’t just pick a training program, put it in place, and feel like that’s enough. We’ve learned from this study that it really does need to be a holistic approach. You need to make sure that the de-escalation training is emphasized in policy, and that your field supervisors are encouraging officers to use those tactics and skills in the field. You need to make sure that there’s managerial support all the way through the agency.

You need to make sure that the training dosage is appropriate. This is not a “one and done.” This is something that you have to build upon, and these tactics have to be reinforced throughout the agency. Take those core concepts and embed them in other trainings. Capture the information in the data you collect. Support officers with de-escalation awards. That’s the only way we’re going to be able to change culture.

I hope this study does not lead police executives to say, “We’ll do de-escalation training and that’s enough.” It’s not enough.

Wexler:   Instructors can have a substantial influence on how a training is received. Did you evaluate the trainers who taught ICAT in Louisville Metro?

Professsor Engel:   We observed the training ourselves to make sure there was fidelity to the model and they were training the core concepts of ICAT in an effective way. When I sat in on the training, I was particularly impressed with the trainers’ command of the room, the interest level of the officers, and the way they were engaging in the activities that are part of the training. For me, that was a real positive.

Wexler:   What related research questions are you hoping to study in the future?

Professsor Engel:   One thing I want to look at is officer characteristics, and whether that impacts their level of positivity about the training and their use of it in the field. For example, what are the differences between younger and older officers? Male vs. female officers? What about different assignments or ranks? All those things are important for our trainers to better understand what officers they’re more likely to reach with this training, and where they might need some additional assistance.

We also want to better understand the circumstances in which de-escalation tactics are being used.

We have a very robust dataset, and we’re going to be able to continue to look at some of the nuances to better inform the training moving forward. 

 

The PERF Critical Issues Report is part of the Critical Issues in Policing project, supported by the Motorola Solutions Foundation.

 

PERF also is grateful to the Howard G. Buffett Foundation for supporting this work.