Evidence Based Oversight on Police Use of Force

Note: This is one of a series of posts I created for the American Society of Evidence Based Policing, called The Criminal Justician. With the migration to the new site, these posts were not migrated, so putting them back here on the Crime De-Coder site for exposure. I encourage everyone to join ASEBP!

I have been asked a few times to give comments on different racial bias metrics, such as reports the ACLU puts out, or general Police Scorecards. Not that I don’t think a piece going over them is appropriate, but they are actually very hard to write about in an evidence based manner. While they are metrics devised to potentially identify problematic behavior, they often don’t have much evidence that the metrics are effective in reducing said problematic behavior. Proper data measurement may be a necessary condition to be evidence based, but it is definitely not sufficient all by itself.

One needs to do something with the information to result in subsequent behavior change. That action tied to metrics is often some type of formal oversight within the police department when individuals do not meet particular standards. And there is one example of oversight that reaches what I would consider a decent evidence base to suggest it is worthwhile for most police departments to pursue: formal reporting procedures for more serious levels of use of force, such as officer involved shootings and even lower levels of force (such as pointing a weapon at an individual or using a conducted energy device).

The Evidence

The story starts with the work James Fyfe conducted with the NYPD for his dissertation (Fyfe, 1979; 1982). So NYPD did two things in early 1970s, administratively implement more restrictive circumstances in which officer involved shootings (OIS) were acceptable (e.g. no fleeing felons1, no warning shots, no shooting at moving vehicles), as well as a board to review cases when a firearm was discharged2. These policy changes resulted in the total number of OIS decreasing (by around 30-50%) in the 1970s, despite crime rates going up in this period.3

Fyfe was motivated to examine changes after discovering someone within the NYPD was actually tabulating the incidents of OIS (Fyfe, 1982). So while simply collecting the data did not itself cause the reduction, data reporting/collection in conjunction with administrative oversight I believe is reasonable to say caused the decrease in OIS at this time for the NYPD.

Claiming something is “evidence based” has no bright line rules. But generally it should be replicated in different settings and by different scientists. Similar findings on reductions in OIS in response to changes in administrative policies for upping standards from fleeing felon to defense of life resulted in reduced OIS in several different cities in the 70s and 80s (Geller & Scott, 1992; Meyer, 1980; Sherman, 1980, 2018; Walker, 1993).4 In Philadelphia in the 70s going the opposite direction, White (2001) found that the removal of administrative policy increased OIS, and reinstatement of administrative policy in 1980 caused OIS to go back down.

And we have additional work that extends administrative oversight from OIS to lower levels of use of force. Stricter administrative policies have evidence of reducing pepper spray and conducted energy device usage across several jurisdictions (Bishopp et al., 2015; Ferdik et al., 2014; Morabito & Doerner, 1997; Thomas et al., 2010). Those examples are typically focused on specifying on when some use of force is appropriate, but sometimes the administrative policy can simply be the actual formal reporting procedure itself. Recent examples of asking officers to document when they point their firearm at an individual have evidence for reducing OIS. Two examples: Jennings & Rubado (2017) using national level data show officer involved deaths are lower in jurisdictions with formal documentation procedures, and Shjarback et al., (2021) show in Dallas implementing lower level use of force reporting resulted in slightly fewer officer shootings in Dallas. (Much of the work Chief Brown did in Dallas, in terms of both policies and formal reporting procedures, appears to me to be a good model for other police departments to follow.)

Lessons for Other Areas?

So it is a kaleidoscope of studies and outcomes I have given here, and it is a mix of administrative oversight policies as well as simply documentation. Taking a broader view, the majority of good examples of reducing officer use of force often have the ingredients:

So there is ultimately a trade-off – it is easy for me to blog from my chair ‘make police officers fill out for paperwork’. I think part of the key that makes the more severe use of force oversight work, not just in terms of reducing those incidents but not causing extraneous levels of bureaucratic paperwork, is that OIS is a fairly rare event.

So things like post-hoc root cause reviews don’t make sense for lower levels of use of force (at least not for every time they occur during an arrest). But what does make sense is to create standard forms to report such events, and then it is as simple as a drop-down in a form to say “I used X level of force on individual Y during the incident”. The more rare events are difficult to identify individual long term problematic behavior in officers, but the more common lower levels of use of force can be used to identify problematic officer behavior over time (Ridgeway & MacDonald, 2009 is an example with stops, but could be extended to use of force).

That at least ties the racial bias metrics I started the post off with. As I said at the beginning, the metrics are often only loosely tied to actual change in police departments. I think metrics in conjunction with specific administrative oversight is the most appropriate way to reduce problematic behavior. But in terms of promoting the racial outcomes police departments (or outside community groups) want, there is more work that needs to be done in that bullet point list to be evidence based.6

References

  1. Note this is before Tennessee v. Garner in 1985 that limited these shootings for all police departments in the US.↩︎

  2. I feel obligated to say that Frank Serpico has no good things to say about Jim Fyfe and the way he conducted internal reviews of officer involved shootings at NYPD.↩︎

  3. Simply reducing the number of shootings is not unambiguously a good thing. Officer involved shootings are sometimes a necessity, and so if oversight is making police more hesitant it could potentially result in more officer or civilian injuries overall. In terms of total officer involved injuries, Fyfe found no differences in the rates of officer injuries conditional on an officer involved shooting. This has generally been the case in subsequent analyses I will note as well (Jetelina et al., 2018; Shjarback et al. 2021).↩︎

  4. John Shjarback provided extensive feedback and additional examples of police oversight from the initial draft of this post. Thank you John!↩︎

  5. This should be done in conjunction with training on said formal policies.↩︎

  6. There is some evidence that forced police oversight via Department of Justice consent decrees in certain cities reduced problematic behavior (MacDonald & Braga, 2019; Morgan et al., 2017). But examining outcomes more broadly for multiple cities is not as conclusive (Goh, 2020).↩︎