Your LLM Triage Tool Is Guessing. Here Is the Workaround.

The structured workflow that produced 93 percent accuracy in the Oslo study works like this. It is a multi-agent loop with three roles. Agent one receives the alert and a list of allowed data sources (SQL tables, grep targets, API endpoints). It outputs a plan: which queries to run and in what order. Each query is a constrained action. The model cannot run arbitrary SQL. It selects from a predefined set of parameterized queries. Agent two receives the raw query results and produces a structured summary: time range, source IPs, destination IPs, protocols, and notable events. Agent three gets the summary and issues a verdict: malicious, benign, or uncertain. If uncertain, the loop recycles to agent one with a request for more specific evidence. Here is a simplified YAML representation of the agent loop configuration: yaml agents: - role: planner tools: - name: sql_suricata params: [src_ip, dst_port, time_window] - name: grep_logs params: [pattern, file_path] output: query_plan - role: summarizer input: query_results output: structured_summary - role: adjudicator input: structured_summary output: verdict loop_condition: verdict == "uncertain" Every query result is escaped and length-limited. The summarizer cannot see the original alert. The adjudicator cannot see raw logs. This compartmentalization is what prevents the model from overfitting to noise.

The sequence matters. Do not deploy the model and add guardrails later. Guardrails must be in place before the model sees any data. Step 1: Security architect defines the query set. Enumerate exactly what the LLM is allowed to request from each data source. No open-ended natural language queries. Parameterize everything. Step 2: SOC engineer builds a loop mechanism. If the adjudicator returns uncertain, the planner must be called again with a flag to narrow the search. Most teams skip this loop. Without it, uncertain verdicts get forwarded to humans and the automation fails. Step 3: Platform team implements output validation. Every verdict must pass a structural check: does it have one of three allowed verdicts? Is the summary within character limits? Reject malformed outputs silently. Step 4: Incident response lead runs a benchmark before sign-off. Use a labeled dataset of 100 malicious and 100 benign alerts. Accept only if malicious detection > 85 percent and false positive rate < 5 percent.

When an LLM-based triage decision leads to a missed attack or a false escalation, the recovery depends on who owns each piece. The CISO owns the root cause analysis: was the failure in the model, the guardrails, or the algorithm? The IR team owns the re-review of the specific incident and all similar alerts in the window. The SOC manager triggers a calibration check: rerun the missed alert through the pipeline with logging enabled. The DevOps team verifies that the agent loop logs are intact. The legal team should only be involved if the missed alert led to regulatory notification. The handoff that stalls most post-incidents is between IR and SOC manager: the IR team finds the evidence of failure but the SOC manager does not know how to update the query set. A written procedure for updating query parameters must exist before any incident.

Three lessons generalize beyond LLM triage. First, any security tool that produces decisions without a structured reasoning path is untestable. If you cannot replay the exact inputs and get the exact same output, you cannot improve it. Second, uncertainty is a feature not a bug. A model that never says "I don't know" is either lying or finished learning. Third, the most dangerous assumption in security automation is that accuracy benchmarks on public datasets translate to operational accuracy. They do not. Private data shifts the distribution. If the industry does not start requiring organization-specific benchmark baselines for AI security tools, the next wave of automation will produce confident wrong answers at machine speed.