Insider Threat Detection: When the Attacker Already Has the Keys

The Problem

Insider threats are a different category of security problem. Firewalls, intrusion detection systems, signature-based tools: all of them assume the attacker is trying to break in. But insiders already have legitimate access. They have credentials, permissions, and knowledge of the systems they're targeting. Traditional machine learning approaches need labeled examples of insider attacks to train on. The problem is that most organizations have never experienced a confirmed insider incident. The labels don't exist. So the question becomes: can you detect anomalous behavior without knowing what an attack looks like?

The Approach: Unsupervised Anomaly Detection

I tested the hypothesis that unsupervised methods (algorithms that learn "normal" and flag deviations) could identify insider threats without labeled training data. Specifically, I wanted to know whether temporal sequence modeling (analyzing behavior patterns over time) would catch attacks that static methods miss. Four models, one dataset, controlled conditions:

Model	Type	What It Does
Isolation Forest	Static	Isolates anomalies by randomly partitioning feature space
PCA Reconstruction	Static	Flags observations that reconstruct poorly from principal components
Dense Autoencoder	Static	Neural network that learns to compress and reconstruct normal behavior
LSTM Autoencoder	Temporal	Learns sequential patterns in behavior over time windows

The dataset was CMU-CERT r4.2, the standard benchmark for insider threat research. Around 1,000 simulated employees over 18 months, with 70 known malicious users across three attack scenario types. Activity logs covering authentication, USB device connections, file operations, email metadata, and web browsing. I extracted 24 behavioral features per user per day and tested 7-day temporal windows for the sequence models.

The Result That Matters

Here's the simplest way to think about it. A security team of five people monitors 1,000 employees. They can realistically investigate 50 flagged users per day: a 5% false positive rate. Everything beyond that top 50 gets ignored. So the question isn't "which model ranks suspicion best across all 1,000 employees?" The question is: "of the 50 people you actually investigate, how many are real threats?" These are not the same question. The gap between them is where real security happens.

Model	AUC-ROC	Recall@5%FPR	Recall@10%FPR
Isolation Forest	0.799	0.044	0.220
PCA Reconstruction	0.612	0.049	0.129
Dense Autoencoder	0.659	0.048	0.118
LSTM Autoencoder	0.770	0.149	0.254

AUC-ROC (the metric every paper reports, every benchmark uses, every vendor cites) says Isolation Forest wins. It ranks anomalies better overall. Most studies would stop here. But Recall@5%FPR asks the operational question: of the people you can afford to investigate, how many are real threats? The LSTM Autoencoder catches 3.4x more attacks at that threshold. Not because it's a fancier algorithm. Because it watches behavior over time. An employee who accesses unusual files on Monday, plugs in a USB on Wednesday, and downloads a large archive on Friday looks normal on any individual day. The LSTM sees the week-long sequence. Isolation Forest sees three unremarkable snapshots.

The standard evaluation metric for insider threat detection (the one in every benchmark, every paper, every vendor whitepaper) systematically undervalues the models that perform best under real operational constraints. If you're building a security product and optimizing for AUC-ROC, you're optimizing for a scenario your customers will never use. The gap between "best on paper" and "most useful in practice" isn't a minor footnote. It's a 3.4x difference in whether attacks get caught.

What Didn't Work

The honest part: "boiling frog" attacks evade detection almost entirely. These are insiders who escalate so gradually that their behavior never deviates sharply from baseline. They shift slowly enough that the model's concept of "normal" drifts with them. 85% of these subtle attacks were missed across all four models. This isn't a failure of the specific models. It's a fundamental challenge in unsupervised anomaly detection. If the attacker is patient enough, the boundary between normal and anomalous blurs beyond what any reconstruction-based method can reliably detect.

Additional Findings

USB device activity was the strongest single indicator of insider attacks (correlation: 0.075 with threat labels)
7-day temporal windows hit the sweet spot. Shorter windows missed multi-day patterns; longer windows added noise without improving detection.
The ensemble of all four models didn't meaningfully outperform the LSTM alone, suggesting the temporal signal was doing the heavy lifting.

What I Took Away

This project shaped how I think about AI systems generally, not just security. The gap between "best metric on paper" and "most useful in practice" is the gap between academic evaluation and operational deployment. It's the same pattern I later saw at EkLine: the simple system that works reliably beats the sophisticated system that works impressively. It also led me somewhere I didn't expect. The behavioral profiling architecture I built for insider threats (learn normal, flag deviations, investigate anomalies) is structurally identical to what AI agent monitoring systems need to do. I followed that thread into a new project: repurposing these models for AI agent governance. The models transferred. The blind spots transferred too.

What Comes Next

The paper is being prepared for journal submission targeting a security venue in mid-2026. But the more interesting direction is what the thesis opened up. The behavioral profiling architecture I built here (learn what normal looks like, flag deviations, investigate anomalies) turns out to be the exact same architecture that AI agent monitoring systems need. I tested that hypothesis, and it held: an Isolation Forest trained entirely on employee data retains 97% of its detection power on AI agent traces. That work became its own project: From Rogue Employees to Rogue Agents. The code, full reproducibility guide, and paper are on GitHub. Everything needed to reproduce the 5-seed experiments on consumer hardware.