ARC Labs Blog

Chain of Custody: Teaching an AI Detection Engineer to Prove Its Work

Name: Dodgy Code
Price range: $

Matthew McCaslin

Detection engineering has a credibility problem: a rule that compiles and references the right-looking fields can still fail silently in production, and nothing about that failure announces itself. A query that never matches anything looks identical to a quiet, well-tuned detection, right up until an incident responder goes looking for an alert that should have fired.

I wanted to find out whether an AI agent could be held to a stricter standard than that. The agent, built against Splunk, has one task: for a given MITRE ATT&CK technique and tenant, either produce a detection it can prove works against that environment's data, or report back why it can't yet. To test it, I built a local two-customer lab: one environment representative of a typical mid-maturity Splunk deployment with realistic, imperfect telemetry, and one representative of an environment with... less usable telemetry. This piece walks through one full run of the agent against both, including the parts that weren't as pretty.

Key Takeaways

An existing, non-functional detection, caught before the postmortem.
A working detection tuned with evidence and three uncertainties appropriately escalated to a human.
A poorly deployed environment honestly reported as not covered.

How the Agent is Organized

Rather than one prompt trying to hold an entire tenant's security posture in its head at once, the agent runs as three narrow modules that hand off in sequence, each scoped to only the tools its job requires.

Inventory audits what's indexed, what's field-extracted, and what detections are already deployed, characterized against live data rather than assumed from documentation. It also checks the tooling landscape: is there an alerting mechanism in place that would actually surface a new detection to an analyst?

Gap Analysis takes Inventory's findings, fetches the requested technique's current MITRE ATT&CK detection strategy, and searches live for existing open-source detection content. It diffs what MITRE and the open-source community say a detection needs against what Inventory found actually deployed and populated, and hands Authoring a verdict: covered, broken, missing, or unverifiable with the data on hand.

Authoring takes Gap Analysis's verdict and either verifies an existing detection against production history, tunes one that's structurally sound but underperforming, or writes a new one, then runs it against historical data and keeps tuning until what fires is something an analyst could actually act on.

Each module only sees what its job needs. Inventory never queries MITRE's site. Authoring doesn't do its own open-source research. That narrowness is deliberate: a model with the whole picture in front of it at every step is a model that can quietly paper over a gap in one stage with a guess borrowed from another.

Think of it less like one detective working a case alone, and more like how evidence actually moves through a real investigation. One person logs exactly what's at the scene before anyone touches it. A separate person checks that against what the case actually requires, not what would be convenient to find. Only then does a specific person act, and they act on what was actually logged and actually required, not on a hunch. That's a chain of custody, and it's what Inventory, Gap Analysis, and Authoring do for each other: nobody downstream re-does the previous handler's work, and nobody skips a handoff to save time.

One deliberate limitation: the agent does not fire synthetic attack samples at a tenant's index to confirm a detection catches them. That's reasonable in an isolated lab, but writing fabricated attack telemetry into a live, monitored SIEM to test a detection is itself a risky write action against production infrastructure. Recall (whether a detection would catch a real attack it hasn't already seen) is left an open, explicitly unmeasured question rather than something the agent claims to validate.

A detection that could never fire

The first lab environment had three saved searches already on the books for credential-dumping detection (MITRE T1003.001, LSASS Memory) before this run touched anything. Inventory's job was to check what each one actually references, not what its name claims. One, aimed at LSASS dumping via rundll32.exe and comsvcs.dll's MiniDump export, filters on two fields, Image and CommandLine, that don't exist anywhere in this index. The sourcetype it queries is indexed and populated, the same sourcetype every other detection in this environment runs against; it simply never extracts those two fields.

This isn't a tuning problem, and it isn't a query that's too broad or too narrow. It's a detection that has been reporting a quiet, well-tuned appearance while never once evaluating real activity, because the data it needs isn't configured to arrive in the form it expects. No red-team exercise is required to find this: the proof is in the field list.

MITRE's current detection strategy for this technique describes exactly this pattern as its second stage: a process opens a full-access handle to lsass.exe, then writes a dump file. The recommendation here isn't to leave the rule alone. A detection that can structurally never fire is worse than having no detection at all: it sits in the search head looking active while reporting coverage that doesn't exist. Authoring's recommendation was to disable it and document exactly why, rather than quietly rewrite it into something that won't mean anything without first fixing the underlying field extraction.

That's a smaller action than it might look like from the outside. A disabled detection with a documented reason is a tracked gap, not a forgotten one: whoever owns this environment now has one specific field extraction to fix, and a specific, already-written detection to turn back on and tune for real the moment they fix it. Nothing here got quietly buried. It got handed off, with the next step already written down.

Separating Signal From Noise, and Flagging What It Couldn't Resolve

The second existing detection, built to catch processes opening a memory-readable handle on lsass.exe, already worked in the sense that it fired on genuine memory-read attempts. It also fired on routine, legitimate activity: of 162 access events in the available window, seven came back as results, and four of those were WMI enumerating processes, Windows Backup taking a shadow copy snapshot, Microsoft Defender running a scan, and a one-off handle wininit.exe picks up when it creates lsass.exe in the first place.

None of those four were excluded on a hunch. Each one was confirmed against a specific access mask and module in its call stack that only makes sense for the process claiming it. Each got its own line in a new allowlist rather than a blanket exclusion rule. The Defender entry is pinned to the exact platform version currently installed, on purpose: when that version changes, the entry stops matching, and Defender starts showing up in results again until the allowlist is updated.

The remaining three results are the more interesting outcome. None of them received a verdict, because the available evidence didn't support one either way:

A dump utility run from an administrative tools directory, requesting full access to lsass.exe with the memory-dump API present in its call stack. That could be a sysadmin pulling a routine crash dump, or it could be credential theft. No process-creation record exists for that process ID in the available window, so there is no command line, parent process, or user to check. Authoring's conclusion: this needs a person to confirm.
Windows Task Manager, also requesting full access with a dump API present, consistent with a user selecting "Create dump file" on lsass.exe. Here Authoring went further on its own: it pivoted into the raw process-creation record for that host and identified who launched it, an administrator account, interactively, from the desktop. That narrows the question considerably, but doesn't answer it. An administrator manually dumping a sensitive process is routine on some days and exactly what a SOC should want to know about on others.
A single PowerShell access with no dump API present at all, consistent with ordinary process enumeration or with a script doing something its name doesn't reveal. No script-block logging exists in this environment to check further, so this result stays exactly as unresolved as it started.

All three results remain in Splunk, unresolved, specifically because Authoring didn't have enough evidence to resolve them responsibly. The recommendation to retire the detections this one replaces, and to disable the one described above that can never fire, were both written into the report as actions for a human with write access to take, not actions Authoring took on its own.

An Honest "Not Covered"

The second lab environment presented the opposite case: a working credential and a live connection into an index with nothing in it. Not "nothing relevant to this technique," nothing at all: zero sourcetypes, zero fields, zero events, confirmed three separate ways (by sourcetype, by field, and by deployed detection) before Gap Analysis even started.

Gap Analysis still ran its full live research pass, the same MITRE ATT&CK fetch and open-source search the first environment received, because the technique's requirements don't change by customer, only the available data does. Four real open-source references came back: two SigmaHQ rules for the access pattern, a SigmaHQ rule for the dump filenames themselves, and a Splunk ESCU detection with a tuning macro already built in. Every one of them depends on fields this index simply doesn't have.

The verdict: a real coverage gap, but a data gap, not a detection gap. Writing a detection anyway was the available shortcut, and it would have been the wrong one. A query built against fields that don't exist returns zero matches today, and zero matches on an index with zero events is indistinguishable from a quiet, well-tuned detection. The honest label here is "not covered." Gap Analysis's report named exactly what's missing to change that: Sysmon process-access, file-creation, and registry telemetry, plus the Splunk add-on needed to actually extract the relevant fields, with the ESCU search identified as the recommended starting point once that data exists.

Why This Matters

A detection that compiles isn't a detection that works. Every claim the agent made, that a rule fires, that it doesn't, that a field exists, that a technique requires a specific log source, was checked against live data or live research before it was reported, not assumed.
"I don't know" is a legitimate output. Three results that evidence couldn't resolve were reported as open questions for a human, not guessed at in either direction. An empty environment was reported as empty, not quietly given a detection that would never prove anything.
The agent never took a destructive or unverifiable action on its own. Disabling a broken detection, scheduling a new one, confirming an ambiguous result's intent: all of it went into the report as a recommendation for a human with the right access and context, not an action the agent took unilaterally.