Cybersecurity has an unusual relationship with AI: the industry has been shipping machine learning for two decades, so it is past the excitement and well into the part where you know exactly which promises don’t survive contact with production. That makes it one of the more honest use cases to study.
The problem is the base rate, not the model
Start here, because everything else follows from it.
Attacks are rare. A large enterprise network carries billions of ordinary events a day and perhaps a handful of genuinely malicious ones. Run a detector that is 99% accurate against that traffic and the math is brutal: the 1% false-positive rate applied to millions of benign events buries the true positives completely. The alerts are mostly wrong, not because the model is bad but because the base rate is extreme.
This is why “our AI detects threats with 99% accuracy” is a marketing claim rather than a useful one, and why the practical bottleneck in a security operations center was never detection — it was the analyst drowning in alerts.
Where AI genuinely helps
Alert triage and correlation. The highest-value use today, and it’s unglamorous: taking the flood a SOC already receives, clustering related alerts into a single incident, pulling in context (what is this host, who is this user, what changed), and drafting a summary an analyst can judge in seconds instead of minutes. It doesn’t find new attacks; it makes the existing pile survivable. That is worth more than it sounds.
Anomaly detection on behavior. Models that learn what normal looks like for a user or a host — login times, geography, data volumes, process trees — and flag drift. Genuinely useful for insider threat and compromised credentials, where nothing “malicious” is happening in a signature sense; a legitimate account is simply behaving unlike itself.
The caveat is written into the name: it finds unusual, not malicious. Someone working late from a new laptop before a holiday is anomalous and fine. That gap is where the false positives live, and it never fully closes.
Phishing and business email compromise. Models reading tone, sender relationships, and intent catch what link-blocklists miss. A well-matched problem: high volume, fuzzy patterns, and a human safety net already in the loop.
Malware classification. Grouping new samples by behavioral similarity to known families. Mature, effective, and mostly invisible — it’s been in endpoint products for years.
Natural-language querying. Letting an analyst ask “which hosts talked to this IP last week” without writing the query language. Small, real, quietly popular.
Where it disappoints
Autonomous response. Letting a model isolate hosts or disable accounts on its own is where enthusiasm meets the base rate again. A false positive with autonomous response isn’t a wasted ten minutes: it’s a self-inflicted outage. Almost everyone doing this well keeps a human approving anything destructive, and the ones who didn’t have a story about it.
Novel attack detection. The pitch is that AI catches the zero-day that signatures miss. Sometimes. But a model trained on what normal looks like flags deviation, and a competent attacker’s goal is to look normal — living off the land, using your admin tools, moving at your pace. The attacks worth worrying about are engineered to be unremarkable.
Explainability. “This is anomalous, confidence 0.87” is not something an analyst can act on or a regulator can accept. Detection without an explanation an analyst can verify just relocates the work.
The other side: attackers have the same tools
The uncomfortable symmetry. LLMs have removed the broken-English tell that trained a generation of employees to spot phishing — messages are now fluent, contextual, and cheap to tailor per target from public information. Voice cloning has made the “urgent call from the CFO” attack real. Reconnaissance and lure-writing, the labor-intensive parts of a targeted campaign, got much cheaper.
None of this is a new category of attack. It’s a cost collapse for existing ones, which means volume and quality both go up. The defensive implication is unromantic: the controls that don’t depend on humans spotting a fake — phishing-resistant MFA, out-of-band verification for payments, least privilege — matter more than the training slide telling staff to watch for typos.
There’s also a new attack surface to defend: your own AI. If you’re deploying agents with tool access, prompt injection is now a security problem you own — see LLM security and red teaming for how those systems get broken.
How to think about it
Put AI in its place inside a framework you already use — the NIST Cybersecurity Framework is the common anchor. AI is a control that strengthens functions (mostly Detect and Respond), not a replacement for the program. If your identity hygiene, asset inventory, and patching are weak, a detection model doesn’t compensate; it just tells you about the consequences faster.
The realistic summary: AI in security is a force multiplier for the analysts you have, not a replacement for them. It’s the difference between a team that triages 40% of its alerts and one that triages all of them. That’s a genuinely good outcome — it’s just not the one on the booth banner.
For more on how enterprises are deploying AI, browse our enterprise AI coverage.