Cost-benefit analysis of Gun Shot Detection Tech

Note: This is one of a series of posts I created for the American Society of Evidence Based Policing, called The Criminal Justician. With the migration to the new site, these posts were not migrated, so putting them back here on the Crime De-Coder site for exposure. Encourage everyone to join ASEBP!

If you asked me a year or two ago, I would have said that gun-shot-detection technology (GSD) was clearly in the camp of not worth it. Recent evidence has slightly moved my opinion of that (such as Dennis Mares group, CCSVP, 2023), but it is still in my opinion too expensive. Recently saw an article about Baltimore paying for a $730,000 contract, and I will walk through some of my cost-benefit logic.

I think Baltimore is pretty much an upper bound of cost-effectiveness – it is a city with large number of shootings. So if GSD is not a clear net benefit in Baltimore, it is not going to be for pretty much every other city in the US.

I would break down the potential value that GSD has into two categories:

The lives saved in Baltimore is a discussion, although not clear cut to me whether GSD is worth it. Simultaneously the majority of US cities don’t have the level of gun violence Baltimore does, so most cities GSD will not be worth it. GSD and subsequent violent crime reductions is much fuzzier evidence of efficacy, and many people I have seen use what to me are questionable cost of crime estimates even when they do estimate crime reductions.

Rapid Response

This to me is the main benefit of GSD – more rapid police response will lead to more lives saved. Really it is just a question of how many lives will a GSD system save. This is a function of how much time does GSD shave off of response times, and how many shootings are covered in a particular area.

Several recent studies have estimated on average less than 2 minutes in reduced response time (CCSVP, 2023; Piza et al., 2023).

Based on work I have done with Philly shootings (Circo & Wheeler, 2021), the marginal effect estimate of an two added minutes of drive time to a shooting is approximately 1 percentage point increased chance in dying (and from research in other cities like Chicago, e.g. Crandell et al., 2013, it appears very similar). What this means, say you had a 10% chance of dying based on where you were shot, if you got to the hospital 2 minutes faster, I would expect your probability to go down to 9%.

There are all sorts of simplyfing factors here, and you could do a more rigorous estimate given a cities actual data if you wanted, but this 1 percentage point marginal effect estimate is likely to be on the high side although is ballpark reasonable. This suggests if GSD reduces response to the scene by 2 minutes on average, it will reduce 1 homicide per 100 shooting victims (so a shooting where someone is hit with a bullet).

Most cities don’t have GSD coverage over 100% of the city – the Baltimore article says the contract limits coverage to slightly over 5 square miles. I am guessing you can maybe squeeze in around 300 shootings/homicides into a 5 square mile area in Baltimore (it is silly for cities to redact this info, there is no public safety concern). So I am guessing GSD in Baltimore will save around 3 people per year, but could calculate more accurate estimates of this if they published the coverage areas. (All the assumptions I am making in this lives saved section are on the high side, so erring my estimates on more lives saved than fewer lives saved.)

I’d note two things, 1) prior research of GSD has not found clear reductions in fatalities. Put in this light though – needing to treat 100 shootings to save 1 life, shows these prior studies are likely underpowered to detect an overall reduction in fatalities. 2) How you valuate those reductions in homicides can make or break the cost-benefit estimate for GSD in this scenario. So if you think, yeah I would spend $300,000 to save a life, then GSD I believe is worth it in an absolute sense in Baltimore.

I’d note GSD potentially has other opportunity costs though. So simply having more EMS units stationed at hot spots may have a higher return on investment than GSD here (Hosler et al., 2019). Ditto with having officers stationed at a hotspot, although with difficulty hiring it may be easier to do a GSD contract than hire 4 more officers. So even if in an absolute sense you think GSD is worth it to save 3 lives per year in Baltimore, there may be better initiatives to invest that money in (including just hiring more police officers).

I honestly don’t know how to value a life here from the cities perspective. But at least this puts it pretty plain – I think GSD will save 1 life per 100 shooting victims in its coverage area. Majority of cities in the US this is going to be much less than 1 life saved per year, and so the (typically mid 6 figure to start) GSD contracts I see are probably not justified based on a lives saved argument.

In Baltimore though it is a reasonable question from a public policy perspective.

How to Value Crime Reductions

Before talking about the evidence for whether GSD reduces crime, it is worth unpacking “how do you value” different public safety interventions. To me we can break these costs down into these categories:

  1. preventing crime has labor cost savings (less work for police, courts, prosecutors, public defenders, etc.)
  2. personal costs to the victim (e.g. lost future productivity, personal medical expenses, suffering and hardship)
  3. more generalized societal costs (e.g. Medicaid, reductions in home prices and businesses)

My prior work has just focused purely on A, labor cost savings (Wheeler & Reuter, 2021). They are relatively uncontroversial, and because they are realized costs for the actual public sector agencies I work with, they are much easier to make clear-cut policy decisions.

C is of course relevant to public policy discussions. Medicaid suggests it may make sense for states to invest in gun violence prevention to a certain extent (like GIVE in New York). Businesses and home price impacts make sense for cities to weigh directly as well, but are hard to calculate in real life. C however are not costs bourne directly by the public sector agencies themselves, so don’t necessarily make sense for police to bear that cost.

B is the one I am not so sure about, but academic estimates of this tend to drawf the other two. These often involve future productivity losses in the estimate, as well as willingness-to-pay estimates. For example, the value of a statistical life is often estimated in the millions.

So the Chicago crime lab estimates that reducing shooting victimizations saves several million dollars, and Dennis Mares says preventing an aggravated assault saves over $100,000 (CCSVP, 2023) – but these to me are gross over-estimates. I don’t doubt that individuals bear these costs over their life-course. Being murdered from a personal perspective is essentially infinite cost. But I don’t think it makes sense for cities to bear all of that burden when making public policy decisions.

The reason for this is that those costs are not realized by cities in any direct manner. In terms of productivity losses (e.g. one fewer worker) or tax base reduction, if you stop working you will be replaced. From the cities perspective, the B category only matters to the extent it flows to C (the greater societal negative externalities).

Additionally I am very skeptical of willingness-to-pay estimates, talk is cheap. Many I have seen are to be frank absurd – if you pay someone $1 to do a survey and they answer they are willing to pay $200 in increased taxes, I find that to be pretty dubious. Even if those estimates are accurate for those individuals, they are not the ones paying property taxes (or voting for local politicians), so their WTP is probably irrelevant (their ability to pay is probably not aligned with their stated preferences).

So you can see how “saving a life” in my cost-benefit framework is difficult to measure. The saying “an ounce of prevention is worth a pound of cure” I believe is correct. In the “prevent crime” scenario, there are clear and tangible benefits to A/C, so the counterfactual is Cost(Crime) - 0 = Benefit – preventing crime is all upside. Under the save a life scenario, the algebra is Cost(Death) - Cost(Still shot but alive) = Benefit – the costs for A/C in my list are pretty much the same in the shot but still alive scenario – the general harm to society has already been dealt. In that scenario we are just talking B. And I don’t know whether police departments should count B at all when divying up their budget.

Everytown for gun safety has a much more justifiable estimate in my opinion – for non-fatal shootings it is $25,000 currently (and for fatal shootings it is $274,000). Even those the bulk are quality of life (section B, personal costs) in the prior section. But those appear to me more on their face justifiable.

So in that framework, the cost savings from Baltimore saving 3 people is about $250,000*3 – just break even with the GSD contract.

Does GSD reduce crime?

So now we can discuss the second potential benefit of GSD – whether it reduces subsequent gun violence. In short, while you can find a few examples of mixed evidence of crime reductions in specific cities (CCSVP, 2023; Lawrence et al., 2019), I think it is safe to say the majority of prior studies have not found crime reductions post GSD implementation (Mares, 2022; Mares & Blackburn, 2021; Ratcliffe et al., 2019).

To take a step back, we should think of the mechanism via which GSD can potentially reduce future crimes. The main mechanism is via GSD being able to help investigators collect evidence, and subsequently result in more cases solved. GSD can limit the spatial location of forensic collection, think maybe 1,000 square feet vs 20,000+ when using CAD calls (Wheeler et al., 2020). More cases solved means those same individuals are incarcerated, and so cannot commit future gun violence.

I have not seen consistent evidence that GSD results in more case clearances though (Choi et al., 2014; Doucette et al., 2021), so expecting downstream effects of reduced violence is unrealistic. In cases with a victim, which will have a larger probability of being solved (over purely property damage), the spatial precision probably matters less given other available evidence.

I’d note that GSD currently has significant issues with false positives. So while Dennis Mares thinks that it is unrealistic that the majority of GSD calls are false positives (Mares, 2022), the evidence I have seen personally (and others as well) a 50% false positive rate is not that unrealistic (Drange, 2016; Ratcliffe et al., 2019).

Where this matters is that all of the false positives not only I think offset any potential gains in crimes reduced/cases cleared, but result in net negative ROI from a police departments perspective (Blackburn & Mares, 2019; Ratcliffe et al., 2019). Human labor is significant, so even when GSD companies give cities free trials, it costs the police departments 6 figures in labor to implement.

So is GSD worth it?

To wrap up, specifically for Baltimore, this almost entirely hinges on how you value saving ~3 lives. If we go with Everytown’s estimates (net benefit of around $250k for a shooting victimization vs a gun homicide), Baltimore is close to break even with their contract. That even to me is quite optimistic, as I think preventing crime has much higher upsides than harm reduction in this scenario.

While some prior studies have found crime decreases (CCSVP, 2023; Lawrence et al., 2019), many have not. Even in the cases that did find crime decreases, they use to me irrelevant WTP estimates that dramatically inflate cost of crime estimates. If you go with my A/C categories above, even in those specific cities with local estimated crime reductions, it is probably not clear cut that the GSD contract is worth it.

GSD to me needs to both reduce the false positives, as well as become cheaper, to make it a much clearer net benefit proposition for police departments. Currently I don’t think there are scenarios for police departments where it makes more sense to pay for GSD than to hire more officers. If a city has a high enough concentration of crime that the lives saved are worth it, I think just doing hotspots is a better value proposition.

I don’t know the technical challenges for whether GSD tech can make those improvements. For folks not aware, GSD has a human in the loop making yes/no determinations, hence it is hard to really reduce costs all that much.

References