The injured teenage survivor of a January 2025 shooting at a Nashville, Tennessee high school recently sued the manufacturer of an “AI gun detection” system that failed to detect the handgun that left two dead, including the shooter.
According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”
Omnilert cofounder Ara Bagdasarian declined Ars’ invitation to answer questions about the lawsuit. System Integrations, the other defendant in the case, which resold the Omnilert system, also did not respond to Ars’ request for comment.
I am so tired of AI being shoved into everything and then people surprised when it doesn’t work. There’s no AI I think that could have detected a small firearm easily concealed. Hell as it is with legal concealed carry you can’t tell who is legally carrying as it is even with some of the most observant eyes watching.
People (and by this I mean the company) keep think that AI can give actual answers. It can’t. It’s a non-detrrminustic system, but they want it to behave deterministically. I’m sure the engineers gave the probability stats up to the business and marketing, who then immediately lowered their pants and shit on them, and then rolled it out as the perfect amazing product
The people who profit from this company don’t think that. They think that dumb school administrators think that, and will spend money on it.
The idea with these kinds of systems are meant to allow early warning when possible.
No system is going to be 100%.
AI in this context is useless though, you could paint marker “not a gun” on the side of a gun and guess what would happen.
It has some uses, but 95% of what is being used for and 100% of the data centers aren’t it.
It would flag it as a gun. How do I know? I worked on and developed a similar system at one point. It worked extremely well. We weren’t an American company and ultimately covid killed us (it was US American orgs that were the most interested in our stuff).
Do you think LLMs are being used for this sort of thing? Putting aside the sheer technical mountain of a hurdle that slapping an LLM vision model on top of dozens and dozens of real-time camera streams, the hardware requirements would put the company out of business before they made their first sale.
Computer vision models, which are NOT LLMs, have been around for quite a while now and are very good at doing one thing and one thing only. And they’ll do it well for a miniscule fraction of what it takes to run an LLM.
No, datacentres are not being used for real-time gun detection. The company might have other kinds of infrastructure located in a DC, but not the main video processing hardware.
Yes. It took all of five seconds to find out too.
You’ve already been wrong once, care to try for two?
Using a LLM for detecting a specific object on an image is possible but stupid: if your object is always the same (like in this case) it’s several orders of magnitude cheaper to train once on that specific object then use the computer vision model running directly on the local server that’s recording the video.
Otherwise:
Use cases for LLM-based image recognition is if the object changes at every request or it’s ultra specific with brands and colors