Network Intrusion Detection with AI

From signature matching to AI-powered anomaly detection — building a modern blue team detection pipeline.

Today's Agenda

1. IDS/IPS Fundamentals — Signature vs. anomaly-based detection
2. Snort & Suricata Rule Writing — with AI assistance
3. Network Baseline & Deviation Detection
4. AI-Powered Traffic Classification — normal vs. malicious
5. Detecting Common Attacks — port scans, brute force, DDoS, C2, DNS tunneling
6. Zeek Log Analysis with AI
7. Detection Playbooks — building with AI
8. False Positive Management & Alert Tuning
9. CTF Challenge — write rules to catch attack patterns

What Is an Intrusion Detection System?

An IDS monitors network traffic (or host activity) and generates alerts when suspicious behavior is detected. An IPS adds the ability to block traffic inline.

Signature-Based Detection

Matches packets against a database of known attack patterns. Fast and precise for known threats, but blind to novel attacks.

• How it works: Each rule describes specific byte patterns, protocol fields, or behavioral sequences
• Strengths: Low false-positive rate, easy to understand, fast at wire speed
• Weaknesses: Cannot detect zero-day attacks, requires constant rule updates, evasion via encoding/fragmentation

# Example: Snort detecting a known exploit signature
alert tcp $EXTERNAL_NET any -> $HOME_NET 445 (
  msg:"EXPLOIT MS17-010 EternalBlue Attempt";
  content:"|ff|SMB|72 00 00 00 00|";
  offset:4; depth:9;
  sid:2024001; rev:1;
)

Anomaly-Based Detection

Builds a model of "normal" behavior and alerts when traffic deviates from it. Can detect novel threats but generates more false positives.

• Statistical models: Track mean/stddev of packet rates, connection durations, payload sizes
• Machine learning models: Isolation Forest, autoencoders, clustering on netflow features
• Behavioral baselines: Time-of-day patterns, protocol distributions, DNS query frequencies

Signature vs. Anomaly: Side by Side

Best practice: Use both together — signatures for known threats, anomaly detection for the unknown.

The IDS/IPS Tooling Landscape

Anatomy of a Snort Rule

Every Snort rule has a header (action, protocol, IPs, ports, direction) and options (detection logic, metadata).

# Rule structure:
# action proto src_ip src_port -> dst_ip dst_port (options;)

alert tcp $EXTERNAL_NET any -> $HOME_NET 80 (
  msg:"WEB-ATTACKS SQL Injection attempt";
  flow:to_server,established;
  content:"UNION"; nocase;
  content:"SELECT"; nocase; distance:0;
  pcre:"/UNION\s+(ALL\s+)?SELECT/i";
  classtype:web-application-attack;
  sid:2024010; rev:1;
)

• alert — action (alert, log, pass, drop, reject)
• flow:to_server,established — only matches established TCP sessions
• content — byte/string matching; nocase for case-insensitive
• pcre — Perl-compatible regex for complex patterns

Key Snort Rule Options

AI-Assisted Snort Rule Writing

Use an LLM to generate, validate, and optimize detection rules from natural language descriptions or raw packet data.

import anthropic

client = anthropic.Anthropic()

prompt = """Write a Snort 3 rule that detects:
- Outbound DNS queries to TXT records larger than 200 bytes
- Only on port 53 UDP traffic
- This pattern is consistent with DNS tunneling/exfiltration
- Include a threshold of 10 queries in 60 seconds from same source"""

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[{"role": "user", "content": prompt}]
)
print(response.content[0].text)

Suricata Rule Examples

Suricata uses the same rule syntax as Snort with additional keywords for protocol-aware detection.

# Detect SSH brute force (>10 failed attempts in 60s)
alert ssh $EXTERNAL_NET any -> $HOME_NET 22 (
  msg:"ET SCAN SSH Brute Force Attempt";
  flow:to_server,established;
  app-layer-event:ssh.banner_missing;
  threshold:type both, track by_src, count 10, seconds 60;
  classtype:attempted-admin;
  sid:2024020; rev:1;
)

# Detect TLS connection to known-bad JA3 hash (Cobalt Strike)
alert tls $HOME_NET any -> $EXTERNAL_NET any (
  msg:"ET MALWARE Cobalt Strike C2 JA3 Hash";
  ja3.hash; content:"72a589da586844d7f0818ce684948eea";
  classtype:trojan-activity;
  sid:2024021; rev:1;
)

Establishing a Network Baseline

Before detecting anomalies, you need to know what "normal" looks like. Capture baseline metrics over 1-4 weeks of typical operations.

Key Baseline Metrics

• Traffic volume: Bytes/packets per second by protocol (TCP, UDP, ICMP)
• Connection patterns: Average concurrent connections, session durations
• Protocol distribution: HTTP 45%, HTTPS 30%, DNS 10%, SSH 5%, Other 10%
• Time-of-day patterns: Peak hours, off-hours expected traffic levels
• DNS behavior: Query rate, top queried domains, average response sizes
• Endpoint behavior: Typical destinations per host, ports used

# Example: Netflow baseline statistics
baseline = {
    "avg_packets_per_sec": 12450,
    "stddev_packets_per_sec": 2100,
    "avg_dns_queries_per_min": 340,
    "avg_unique_dst_ips_per_host": 45,
    "avg_session_duration_sec": 28.5,
    "protocol_distribution": {
        "TCP": 0.72, "UDP": 0.22, "ICMP": 0.03, "Other": 0.03
    }
}

Deviation Detection with AI

Once a baseline exists, use statistical and ML methods to flag deviations.

import numpy as np
from sklearn.ensemble import IsolationForest

# Netflow features: [packets/s, bytes/s, unique_dst, avg_pkt_size, dns_queries/min]
training_data = np.array([
    [12400, 8500000, 42, 684, 335],
    [12800, 8700000, 47, 680, 350],
    [11900, 8200000, 40, 690, 320],
    # ... weeks of normal traffic ...
])

model = IsolationForest(contamination=0.01, random_state=42)
model.fit(training_data)

# New observation during potential attack
current = np.array([[45000, 32000000, 1200, 72, 890]])
prediction = model.predict(current)
# -1 = anomaly, 1 = normal
print(f"Verdict: {'ANOMALY' if prediction[0] == -1 else 'Normal'}")
# Output: Verdict: ANOMALY

AI-Powered Traffic Classification

Train classifiers to categorize network flows into normal, suspicious, and malicious categories using labeled netflow data.

Feature Engineering for Network ML

import pandas as pd

def extract_flow_features(netflow_records):
    """Extract ML features from raw netflow data."""
    features = []
    for flow in netflow_records:
        features.append({
            "duration":       flow["end_time"] - flow["start_time"],
            "total_packets":  flow["packets_fwd"] + flow["packets_bwd"],
            "total_bytes":    flow["bytes_fwd"] + flow["bytes_bwd"],
            "avg_pkt_size":   flow["total_bytes"] / max(flow["total_packets"], 1),
            "fwd_bwd_ratio":  flow["packets_fwd"] / max(flow["packets_bwd"], 1),
            "unique_flags":   len(set(flow["tcp_flags"])),
            "dst_port":       flow["dst_port"],
            "entropy_payload": calculate_entropy(flow["payload_sample"]),
            "is_well_known_port": 1 if flow["dst_port"] < 1024 else 0,
            "inter_arrival_std":  np.std(flow["inter_arrival_times"]),
        })
    return pd.DataFrame(features)

Payload entropy is especially useful: encrypted C2 traffic and DNS tunneling payloads have high entropy (6.5-8.0), while normal text is lower (3.5-5.0).

Detecting Port Scans

Port scans probe many ports on a target to discover open services. They produce distinctive traffic patterns.

# Snort: Detect SYN scan (>20 ports in 5 seconds)
alert tcp $EXTERNAL_NET any -> $HOME_NET any (
  msg:"SCAN nmap SYN scan detected";
  flags:S,12;
  threshold:type both, track by_src, count 20, seconds 5;
  classtype:attempted-recon;
  sid:2024030; rev:1;
)

# Snort: Detect XMAS scan (FIN+PSH+URG flags)
alert tcp $EXTERNAL_NET any -> $HOME_NET any (
  msg:"SCAN nmap XMAS scan detected";
  flags:FPU,12;
  classtype:attempted-recon;
  sid:2024031; rev:1;
)

Detecting Brute Force Attacks

Brute force attacks send many authentication attempts in rapid succession. Detect by rate-limiting per source IP.

# SSH brute force: >5 connection attempts in 30 seconds
alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (
  msg:"BRUTE-FORCE SSH login attempt";
  flow:to_server,established;
  content:"SSH-";
  threshold:type both, track by_src, count 5, seconds 30;
  sid:2024040; rev:1;
)

# HTTP login brute force (POST to /login)
alert http $EXTERNAL_NET any -> $HOME_NET any (
  msg:"BRUTE-FORCE HTTP login attempt";
  flow:to_server,established;
  http.method; content:"POST";
  http.uri; content:"/login";
  threshold:type both, track by_src, count 20, seconds 60;
  sid:2024041; rev:1;
)

# FTP brute force
alert tcp $EXTERNAL_NET any -> $HOME_NET 21 (
  msg:"BRUTE-FORCE FTP login attempt";
  flow:to_server,established;
  content:"USER ";
  threshold:type both, track by_src, count 10, seconds 60;
  sid:2024042; rev:1;
)

Detecting DDoS Attacks

Distributed Denial of Service attacks flood targets with traffic from many sources. Detection relies on traffic volume anomalies and pattern analysis.

# SYN flood detection
alert tcp any any -> $HOME_NET any (
  msg:"DDOS SYN Flood detected";
  flags:S,12;
  threshold:type both, track by_dst, count 500, seconds 10;
  classtype:attempted-dos;
  sid:2024050; rev:1;
)

# DNS amplification (large DNS responses from external)
alert udp $EXTERNAL_NET 53 -> $HOME_NET any (
  msg:"DDOS DNS Amplification Attack";
  dsize:>512;
  threshold:type both, track by_dst, count 100, seconds 10;
  classtype:attempted-dos;
  sid:2024051; rev:1;
)

AI Enhancement: Multi-Source Correlation

# Detect distributed attacks by correlating source diversity
def detect_ddos(flows, window_sec=60):
    unique_sources = flows.groupby("dst_ip")["src_ip"].nunique()
    total_packets = flows.groupby("dst_ip")["packets"].sum()
    suspects = unique_sources[
        (unique_sources > 100) & (total_packets > 50000)
    ]
    return suspects  # IPs under likely DDoS

Detecting Command & Control (C2) Traffic

C2 beacons phone home at regular intervals. The hallmark is periodic connections with consistent timing and small payloads.

# Detect potential C2 beacon: regular HTTPS to single host
alert tls $HOME_NET any -> $EXTERNAL_NET 443 (
  msg:"MALWARE Potential C2 Beacon - Regular Interval";
  flow:to_server,established;
  threshold:type both, track by_src, count 30, seconds 3600;
  classtype:trojan-activity;
  sid:2024060; rev:1;
)

AI-Based Beacon Detection

import numpy as np

def detect_beacon(connection_times, jitter_threshold=0.15):
    """Detect C2 beacons by analyzing connection periodicity."""
    intervals = np.diff(connection_times)
    if len(intervals) < 5:
        return False, {}

    mean_interval = np.mean(intervals)
    std_interval  = np.std(intervals)
    jitter_ratio  = std_interval / mean_interval if mean_interval > 0 else 1

    is_beacon = (
        jitter_ratio < jitter_threshold  # Low jitter = regular timing
        and mean_interval < 3600         # Beacons within 1 hour
        and len(intervals) > 10          # Enough samples
    )
    return is_beacon, {
        "mean_interval": mean_interval,
        "jitter_ratio": round(jitter_ratio, 4),
        "sample_count": len(intervals)
    }

# Example: connections every ~60s with slight jitter
times = [0, 61, 119, 182, 240, 301, 359, 421, 480, 541, 600, 662]
result, stats = detect_beacon(times)
# result: True, stats: {mean_interval: 60.18, jitter_ratio: 0.044, ...}

Detecting DNS Tunneling

DNS tunneling encodes data in DNS queries/responses (often TXT or CNAME records) to exfiltrate data or establish covert channels.

Indicators of DNS Tunneling

• Long subdomain labels: aGVsbG8gd29ybGQ.evil.com (base64-encoded data)
• High query rate to a single domain
• Unusual record types: TXT, NULL, CNAME with large responses
• High entropy in subdomain strings

# Snort: Detect long DNS subdomain (>50 chars before first dot)
alert udp $HOME_NET any -> any 53 (
  msg:"EXFILTRATION DNS Tunneling - Long Subdomain";
  content:"|01 00|";  # Standard query
  byte_test:1,>,50,12;  # First label length > 50
  threshold:type both, track by_src, count 5, seconds 60;
  classtype:policy-violation;
  sid:2024070; rev:1;
)

# Detect high-frequency DNS to single domain
alert udp $HOME_NET any -> any 53 (
  msg:"EXFILTRATION DNS Tunneling - High Query Rate";
  threshold:type both, track by_src, count 100, seconds 60;
  sid:2024071; rev:1;
)

DNS Tunneling: AI-Powered Detection

import math
from collections import Counter

def calculate_entropy(domain_label):
    """Shannon entropy of a string. High entropy suggests encoded data."""
    if not domain_label:
        return 0
    freq = Counter(domain_label)
    length = len(domain_label)
    return -sum(
        (count/length) * math.log2(count/length)
        for count in freq.values()
    )

def analyze_dns_query(query_name):
    """Score a DNS query for tunneling indicators."""
    labels = query_name.split(".")
    subdomain = labels[0] if len(labels) > 2 else ""

    features = {
        "subdomain_length": len(subdomain),
        "subdomain_entropy": round(calculate_entropy(subdomain), 3),
        "label_count": len(labels),
        "total_length": len(query_name),
        "has_digits_ratio": sum(c.isdigit() for c in subdomain) / max(len(subdomain), 1),
    }

    # Scoring heuristic
    score = 0
    if features["subdomain_length"] > 30: score += 3
    if features["subdomain_entropy"] > 4.0: score += 3
    if features["has_digits_ratio"] > 0.3: score += 2
    if features["label_count"] > 5: score += 1

    features["risk_score"] = score
    features["verdict"] = "SUSPICIOUS" if score >= 5 else "NORMAL"
    return features

# Normal query
print(analyze_dns_query("www.google.com"))
# {'subdomain_length': 3, 'subdomain_entropy': 1.585, 'risk_score': 0, 'verdict': 'NORMAL'}

# Tunneling query
print(analyze_dns_query("aGVsbG8gd29ybGQhIFRoaXMgaXMgZXhmaWx0cmF0ZWQ.data.evil.com"))
# {'subdomain_length': 45, 'subdomain_entropy': 4.312, 'risk_score': 8, 'verdict': 'SUSPICIOUS'}

Packet Header Analysis

Understanding packet headers is essential for writing detection rules and analyzing captures.

TCP Packet Header (typical SYN scan packet):
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Source Port: 54321        |    Dest Port: 80              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Sequence Number: 0x1a2b3c4d                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 Acknowledgment Number: 0x00000000             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Offset|Rsvd |C|E|U|A|P|R|S|F|       Window: 1024            |
|   5   | 000 |0|0|0|0|0|0|1|0|                                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         Checksum: 0xa1b2      |    Urgent Ptr: 0             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Flags: S = SYN only (no ACK) --> This is a scan probe
Window: 1024 --> Unusually small, common in nmap scans

Netflow Data for Detection

Netflow records summarize connection metadata without capturing full payloads — ideal for large-scale traffic analysis.

# Sample netflow records (CSV format)
timestamp,src_ip,dst_ip,src_port,dst_port,proto,packets,bytes,duration,flags
2024-01-15 02:14:01,10.0.1.50,203.0.113.10,443,52341,TCP,145,98400,120.5,APSF
2024-01-15 02:14:03,10.0.1.50,203.0.113.10,443,52342,TCP,142,97200,119.8,APSF
2024-01-15 02:16:02,10.0.1.50,203.0.113.10,443,52350,TCP,148,99100,121.2,APSF
2024-01-15 02:18:01,10.0.1.50,203.0.113.10,443,52358,TCP,140,96800,118.9,APSF
                                                                 ^^^^^^^^^
                                               Regular ~120s intervals = C2 beacon!

# Compare with normal HTTPS browsing:
2024-01-15 02:14:01,10.0.1.22,142.250.80.46,443,61234,TCP,45,32000,5.2,APSF
2024-01-15 02:14:08,10.0.1.22,151.101.1.69,443,61240,TCP,120,185000,12.1,APSF
2024-01-15 02:17:45,10.0.1.22,104.16.132.229,443,61255,TCP,30,22000,3.8,APSF
                             Variable destinations, variable timing = normal browsing

Zeek (Bro) Log Analysis

Zeek generates rich, structured logs from network traffic. Each log type captures different aspects of network activity.

Analyzing Zeek conn.log

# Zeek conn.log fields (tab-separated)
#fields ts      uid             id.orig_h   id.orig_p  id.resp_h     id.resp_p proto  service duration  orig_bytes resp_bytes conn_state
1705276441.123  CYz1234abc      10.0.1.50   52341      203.0.113.10  443       tcp    ssl    120.50    4200       94200      SF
1705276561.456  CYz1234def      10.0.1.50   52342      203.0.113.10  443       tcp    ssl    119.80    4100       93100      SF
1705276681.789  CYz1234ghi      10.0.1.50   52350      203.0.113.10  443       tcp    ssl    121.20    4300       94800      SF

# conn_state values:
# SF  = Normal established and finished
# S0  = SYN sent, no reply (scan)
# REJ = Connection rejected
# S1  = Connection established, not terminated
# OTH = No SYN (mid-stream pickup)

AI-Powered Zeek Log Analysis

import anthropic

client = anthropic.Anthropic()

zeek_logs = """
# conn.log excerpt (suspicious host 10.0.1.50)
1705276441.123 10.0.1.50  52341 203.0.113.10 443 tcp ssl 120.50 4200  94200  SF
1705276561.456 10.0.1.50  52342 203.0.113.10 443 tcp ssl 119.80 4100  93100  SF
1705276681.789 10.0.1.50  52350 203.0.113.10 443 tcp ssl 121.20 4300  94800  SF
1705276801.012 10.0.1.50  52358 203.0.113.10 443 tcp ssl 118.90 4050  92900  SF
# dns.log excerpt
1705276440.100 10.0.1.50  52000 10.0.0.1 53 udp dns 0.002 update.legit-software.com A
1705276560.200 10.0.1.50  52001 10.0.0.1 53 udp dns 0.001 update.legit-software.com A
"""

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=2048,
    messages=[{"role": "user", "content": f"""Analyze these Zeek logs for signs of
compromise. Look for C2 beacons, data exfiltration, lateral movement, or other
suspicious patterns. Provide specific evidence.

{zeek_logs}"""}]
)
print(response.content[0].text)
# AI identifies: regular 120s beacon interval, consistent dst, DNS preceding each conn

Zeek dns.log: Tunneling Detection

# Normal DNS queries in dns.log
1705276440.100 10.0.1.22  10.0.0.1 53 www.google.com       A      NOERROR 142.250.80.46
1705276440.200 10.0.1.22  10.0.0.1 53 cdn.jsdelivr.net     AAAA   NOERROR 2606:4700::6810:...
1705276441.500 10.0.1.23  10.0.0.1 53 api.github.com       A      NOERROR 140.82.121.5

# DNS tunneling in dns.log (note: long encoded subdomains, TXT records)
1705276440.100 10.0.1.50  10.0.0.1 53 aGVsbG8gd29ybGQ.t.evil.com               TXT  NOERROR
1705276440.150 10.0.1.50  10.0.0.1 53 dGhpcyBpcyBleGZpbA.t.evil.com            TXT  NOERROR
1705276440.200 10.0.1.50  10.0.0.1 53 dHJhdGVkIGRhdGEgZnJvbQ.t.evil.com        TXT  NOERROR
1705276440.250 10.0.1.50  10.0.0.1 53 dGhlIGNvbXByb21pc2VkIGhvc3Q.t.evil.com   TXT  NOERROR

Red flags: All queries go to subdomains of t.evil.com; all use TXT records; subdomain labels are base64-encoded; query rate is 20/second from single host.

Building Detection Playbooks with AI

A detection playbook documents what to detect, how to detect it, and what to do when an alert fires.

playbook:
  name: "DNS Tunneling Detection"
  id: PB-NET-005
  mitre_attack: T1071.004  # Application Layer Protocol: DNS
  severity: HIGH

  detection_sources:
    - zeek_dns_log:
        query: |
          dns.log | where query_length > 50
                  | where record_type in ["TXT", "NULL", "CNAME"]
                  | stats count by src_ip, parent_domain
                  | where count > 50
    - suricata_rule: "sid:2024070"
    - ml_model: "dns_tunneling_classifier_v2"

  triage_steps:
    - "Verify domain reputation (VirusTotal, PassiveTotal)"
    - "Check if domain is used by legitimate software (e.g., AV updates)"
    - "Examine subdomain entropy distribution"
    - "Correlate with endpoint logs for process originating DNS queries"
    - "Check for matching IOCs in threat intelligence feeds"

  response_actions:
    - "Block domain at DNS resolver"
    - "Isolate affected host"
    - "Capture full PCAP for forensic analysis"
    - "Escalate to incident response team"

Generating Playbooks with AI

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=4096,
    messages=[{"role": "user", "content": """Create a detection playbook for
lateral movement via SMB/PsExec in a Windows network.

Include:
1. MITRE ATT&CK mapping
2. Snort/Suricata rules
3. Zeek log queries
4. Windows Event Log correlation
5. Triage steps for the SOC analyst
6. Automated response actions
7. False positive scenarios and how to distinguish them

Format as structured YAML."""}]
)

# AI generates a complete, deployable playbook
# covering detection -> triage -> response -> tuning

False Positive Management

High false positive rates cause alert fatigue — the number one enemy of effective detection. Tuning is an ongoing process.

Common Sources of False Positives

Alert Tuning with AI

import anthropic, json

client = anthropic.Anthropic()

alert_data = {
    "rule": "sid:2024070 - DNS Tunneling Long Subdomain",
    "total_alerts_7d": 2847,
    "unique_sources": 12,
    "top_triggering_domains": [
        {"domain": "*.cdn.cloudflare.com", "count": 1200, "known_legitimate": True},
        {"domain": "*.dkim.amazonses.com", "count": 890, "known_legitimate": True},
        {"domain": "*.t.evil-c2.xyz", "count": 450, "known_legitimate": False},
        {"domain": "*.update.corp-tool.internal", "count": 307, "known_legitimate": True}
    ],
    "true_positive_rate": "15.8%"
}

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=2048,
    messages=[{"role": "user", "content": f"""Analyze this IDS alert data and
recommend tuning actions to reduce false positives while maintaining detection
of real threats.

{json.dumps(alert_data, indent=2)}

Provide specific suppress/threshold rules in Suricata format."""}]
)
# AI recommends: suppress rules for cloudflare/amazonses/internal domains
# Keeps detection active for unknown domains like evil-c2.xyz

Suricata Suppress & Threshold Tuning

# /etc/suricata/threshold.config

# Suppress DNS tunneling alerts for known CDN domains
suppress gen_id 1, sig_id 2024070, track by_src, ip 10.0.1.100
suppress gen_id 1, sig_id 2024070, track by_src, ip 10.0.1.101

# Rate-limit noisy scan detection to 1 alert per source per hour
threshold gen_id 1, sig_id 2024030, type limit, \
  track by_src, count 1, seconds 3600

# Raise threshold for brute force on bastion hosts (expected auth volume)
threshold gen_id 1, sig_id 2024040, type both, \
  track by_src, count 50, seconds 60

# Event filter: only alert on SSH brute force if >20 unique users attempted
# (requires Suricata 7+ with Lua scripting for this complexity)

Complete Detection Pipeline

Network Traffic
      |
      v
  [TAP / SPAN Port]
      |
      +---> [Suricata IDS] ---> Signature Alerts (EVE JSON)
      |                              |
      +---> [Zeek]         ---> Structured Logs (conn, dns, http, ssl)
      |                              |
      v                              v
  [Packet Broker]          [Log Aggregation (ELK/Splunk)]
                                     |
                                     v
                           [AI/ML Detection Engine]
                           - Anomaly detection (Isolation Forest)
                           - Beacon detection (periodicity analysis)
                           - DNS tunneling classifier
                           - Traffic classification model
                                     |
                                     v
                           [Alert Correlation & Enrichment]
                           - Deduplicate alerts
                           - Enrich with threat intel
                           - Score by confidence
                                     |
                                     v
                           [SOAR / Analyst Dashboard]
                           - Automated triage
                           - Playbook execution
                           - Incident creation

MITRE ATT&CK Coverage Map

Map your detection rules to ATT&CK techniques to identify coverage gaps.

CTF Challenge: Network Traffic Detection

Scenario: You are a SOC analyst. A PCAP capture from the last 24 hours contains multiple attack patterns hidden among normal traffic. Write detection rules to catch them all.

Attack Patterns to Find

1. SYN scan from an external host targeting the DMZ
2. SSH brute force followed by successful login
3. DNS tunneling exfiltrating data to an external domain
4. C2 beacon over HTTPS with regular 300-second intervals
5. DDoS amplification using DNS reflection

CTF Challenge: Expected Deliverables

# Your submission: detection_rules.rules
# Must contain valid Snort/Suricata rules for each attack

# Rule 1: SYN Scan Detection
alert tcp $EXTERNAL_NET any -> $DMZ_NET any (
  msg:"CTF-02 SYN Scan Detected";
  flags:S,12;
  # YOUR LOGIC HERE: threshold, tracking
  sid:9000001; rev:1;
)

# Rule 2: SSH Brute Force
alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (
  msg:"CTF-02 SSH Brute Force";
  # YOUR LOGIC HERE
  sid:9000002; rev:1;
)

# Rule 3: DNS Tunneling
# Rule 4: C2 Beacon
# Rule 5: DDoS Amplification

Scoring

Using AI to Solve Detection CTFs

import anthropic

client = anthropic.Anthropic()

# Step 1: Feed Zeek logs to AI for pattern identification
zeek_excerpt = open("challenge_conn.log").read()[:8000]

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=4096,
    messages=[{"role": "user", "content": f"""You are a SOC analyst in a CTF.
Analyze these Zeek conn.log entries and identify all suspicious patterns.
For each pattern, specify:
1. Type of attack
2. Source and destination IPs involved
3. Evidence (specific log entries)
4. A Snort/Suricata rule to detect it

Zeek conn.log:
{zeek_excerpt}"""}]
)

# Step 2: Validate AI-generated rules against the PCAP
# Step 3: Tune thresholds to eliminate false positives
# Step 4: Submit final rule set

Key Takeaways

• Layer your defenses: Signature detection (Snort/Suricata) catches known threats; anomaly detection (ML models) catches the unknown
• Baselines are essential: You cannot detect deviations without knowing what normal looks like
• Zeek logs are gold: Structured network metadata enables powerful AI-driven analysis
• AI accelerates rule writing: Generate candidate rules from natural language, then validate against real traffic
• Tuning never stops: False positives degrade detection; continuously refine thresholds and suppressions
• Map to ATT&CK: Know your coverage gaps and prioritize detection development
• Playbooks save time: Document detection logic, triage steps, and response actions for every rule category

Resources & Further Reading

• Snort 3 Documentation: snort.org/documents
• Suricata Rule Writing: docs.suricata.io/en/latest/rules/
• Zeek Documentation: docs.zeek.org
• Emerging Threats Rules: rules.emergingthreats.net
• MITRE ATT&CK: attack.mitre.org
• Security Onion: securityonionsolutions.com
• CIC-IDS Dataset: Labeled network intrusion detection dataset for ML training
• PCAP repositories: malware-traffic-analysis.net, netresec.com/pcap