Subway Gun Scanners: The Wrong Tech in the Wrong Place

One of the signature moves of the NYPD under Eric Adams’ administration is to turn to new technologies to solve complex problems — or sometimes no problem at all. Technology has its promise and benefits, of course, but also its risks and problems. Seeing only one side of this equation is another signature of the Adams administration, which is to spend a lot of money on a new technology, only to quietly push it aside when it fails. 

We are seeing that pattern happen again, this time as the NYPD pilots a new AI-powered weapons-detection technology in the New York City subway system, made by a company called Evolv. The Evolv trial — scheduled to run for 30 days with a scanner tested at different, undisclosed stations — is likely to prove an expensive and intrusive waste of resources that could be better spent on real solutions for keeping New Yorkers safe when they ride the transit system.

Evolv may well go the route of another Adams failure, Knightscope K5, a mobile robot that supposedly would help police patrol a busy station like Times Square. The robot raised serious civil liberties concerns. And, in reality, the robot spent most of its time tethered to its charging station and being escorted by real-life NYPD officers. Just a few months after its debut, it was retired. Look for Evolv to suffer the same fate, only at greater expense to the people of New York. 

There is a fairly simple rubric for evaluating whether to deploy new policing technologies. It balances benefits against costs, and considers community needs. The NYPD — and indeed jurisdictions around the country — would do well to apply this principle, being thoughtful before they invest time and money into technological tools that are likely to meet with public opposition, for good reason.

The technology rubric requires asking a few basic questions. First, has testing shown there will be real benefits from this technology? If there is no evidence of clear benefits, the discussion can just end there. Second, if there are benefits, are they outweighed by risks to the public, whether from violation of rights, racial disparities, or simple but significant inconvenience? Finally, are there any regulations or approaches that mitigate the harms, such that the benefits can be achieved in a sensible way? 

It’s impossible to know if the Evolv pilot will extend beyond the 30-day pilot, but if Evolv were to become the norm at NYC subway stations, it would fail all three tests.

The claimed benefits

The Adams administration claims that deploying Evolv will keep weapons off the transit system by scanning passengers as they enter. (It appears that the scanners are to be used, at least at present, only for firearms.) This trial is part of an effort to restore a sense of safety on the subways and rebuild ridership, which has not fully rebounded since the pandemic. But there is little evidence Evolv will make any headway toward addressing these real challenges. 

It’s true that annual subway ridership, through April 2024, was about 25% below the same period in 2019, pre-pandemic. But an analysis from the City Comptroller’s Office notes that weekend subway ridership has returned close to 2019 levels while weekday ridership remains decreased. This suggests that a structural move to hybrid and remote work is the major force affecting ridership, not concerns about public safety. (Survey data also indicate that “people behaving erratically” is a primary driver of lower ridership.) 

But even if safety worries are driving decreased ridership, it doesn’t appear that Evolv can provide the solution. Weapons detectors don’t detect or deter erratic behavior. Overall, crime on the New York City subways is decreasing after a pandemic-era spike. In reporting Q1 data for 2024, the NYPD noted that overall crime was down only slightly year-over-year, but that major crimes — categories like robbery, grand larceny and felony assault — were all down by double-digit percentages. With 1,000 additional cops in the system starting in January, arrests were up across the board. 

It’s true that gun arrests on the subway saw a seemingly whopping increase of 83% year over year — but that is only because the actual number of gun arrests, in a system that handles almost 4 million riders every day, was low to start with. The increase was from 12 to 22. 

So Evolv, were it deployed widely, would remove a couple dozen guns from the subway system. In other words, we may have a solution here in search of a problem. 

Nor is it clear Evolv would be up to the task of catching weapons — of whatever kind — even if that were needed. The mayor said that “several thousand tests” have proved the system’s efficacy, but City Hall hasn’t provided any additional information about this testing. 

Meanwhile, Evolv’s own CEO has admitted that subways are “not a good use case” for its technology for multiple reasons, including electromagnetic interference concerns from being below ground. Institutions already using the system have reported that it keeps alerting to people who have no weapons, while missing a stream of real ones. In Utica, N.Y., Evolv’s scanners missed a knife in the knapsack of a student who ended up stabbing a classmate but identified a 7-years-old’s lunchbox as a bomb. A Bronx hospital that used the system for seven months in 2022 found that 50,000 of 194,000 scans tripped the alarm, or about one in four. Of the positive scans, 85% were false positives, meaning there were no weapons. 

What we are seeing with Evolv is a basic (and hard to solve) problem with weapons scanning in heavily trafficked places: How do you set the sensors to catch what they’re meant to without disrupting life for everyone else? Make the settings too sensitive, and riders’ umbrellas will set off scanners; not sensitive enough, and hidden weapons get through. 

The proven costs

While the benefits are dubious, the costs here likely will prove immense. Each Evolv unit will cost about $125,000 over the course of a four-year contract. Although the pilot will involve only using the scanners where the NYPD already employs bag checks, in the long run that would accomplish pretty much nothing. To truly be effective, Evolv would have to be placed in all stations and operate at all times. Multiply that $125,000 cost by all the entrances to major subway stations like Times Square and Union Square, Grand Central and Penn Station, Queensboro Plaza and Atlantic Ave-Barclays Center, and you’re talking about millions of dollars in expenses — before factoring in the other 450+ stations in the system. And if the scanners are not everywhere, riders bent on carrying weapons will simply go to another stop. 

Then there are all the NYPD officers needed to run the scanners, control the crowds that will surely develop, ensure compliance and stop offenders. Already, the administration’s decision to increase police presence on the subways has resulted in a massive increase in overtime pay from $4 million in 2022 to $155 million in 2023. And that significant figure came from just adding officers to some stations. For the Evolv plan to have a shot at working, it would require officers at every station entrance, 24 hours a day, seven days a week. This would represent a scaling up of police staffing that is almost hard to fathom. Surely those officers — and taxpayer dollars — could be put to better use. Every officer standing idly by waiting for an Evolv alert to go off is one more officer who is not available to respond to an actual emergency or public safety crisis.

And that’s before we even take into consideration the implications for the subway system itself, and its millions of riders. Want to hurt subway ridership? Imagine, given Evolv’s demonstrated false positive rate, how many subway riders are going to be stopped erroneously by the scanners. Besides, some of those riders are workers who carry metal tools as part of their livelihood. Evolv is likely to drive away more riders than it attracts. These are very real costs that cannot be ignored.

Finally, being stopped and searched by the police because of a false alert is no mere inconvenience. It is an unwelcome police encounter that — if past is prologue — inevitably is going to result in greater impacts on Black and Brown riders.

The mayor says that people who do not want to be scanned will simply have to exit the station. The City also claims that their system for selecting who will be scanned is constitutional. A 2006 federal court decision about bag checks to prevent terrorist incidents, MacWade v. Kelly, provides some support for these claims. But it’s not clear that this 2006 decision makes any sense in the present context.

First, the decision rested on the notion that the stops were acceptable so long as they were truly random — say every 10th person going through a turnstile. The MacWade v. Kelly court stressed that the officers had virtually no discretion about who was stopped. But forgive some skepticism that at a busy station during rush hour, anyone can say with confidence who the tenth person is. Perhaps it is no surprise that following this decision from the federal court, the City subsequently was sued by Jangir Sultan, a man of South Asian descent who was stopped over 20 times. There simply is no way that many stops could have been random, and it is telling  that the City settled the case out of court. 

If stops are not truly random — and we are indeed skeptical that they feasibly can be — then they rest on the discretion of officers, and that discretion historically has been used in racially discriminatory ways. Beyond that, though, it’s one thing to take extreme measures to stop a terrorist incident — a “paramount” government interest is the standard the federal court set. It’s an entirely different thing to subject people to searches without any suspicion they did anything wrong, especially when — as we argued above — the City’s argument for the searches is not an entirely rational response to the real problems on the subway.

Instead of indulging the mayor’s penchant for the latest tech toy, the City should stick to solutions to subway crime that are much more likely to work — indeed, some of them already are working. Build better civic infrastructure, such as making sure lights in stations and streets around them are working. Adopt non-police approaches to people in mental health crisis, getting them off the platforms and into care they need. Put police officers where they can actually protect people — on the trains and the platforms — not gathered in stations scrolling on their phones, or, worse, standing at entrances checking on faulty technology. 

All of us want easy solutions to our problems, especially when it comes to public safety. Sometimes technology can help provide them. But instead of blindly embracing technology as a public safety panacea, only to dump the expensive technology in the trash when it proves to be more trouble than it’s worth, a better approach is to think through the problem before diving in. This is a lesson the Adams administration has yet to learn.