How to Audit AI Agents: From Mutable Logs to Tamper-Evident History
When an AI agent fails, hallucinates, or makes a destructive API call, the first question is always: why did it do that? The second question, increasingly asked by security and compliance teams, is: can you prove it?
As AI agents move from experimental sandboxes to production, the distinction between observability and auditability starts to matter. For most teams it is an engineering discipline question. For high-risk systems under the EU AI Act, it becomes a traceability requirement with a real deadline. This guide covers why standard logging falls short for autonomous systems, what you actually need to capture, how a cryptographically verifiable audit trail works, and why the same record lets you replay any failed run locally without paying for new LLM calls.
Why logs are not enough: claims vs. evidence
If you are using standard observability tools or conventional LLM observability platforms, you don't have an audit trail. You have a list of claims.
Traditional logs are mutable. They show what the agent reportedly did, but anyone with database access, or a rogue script, can alter that history after the fact. A trace that can be silently rewritten is not evidence of anything. It is a statement, and statements need proof. And even when nobody touches them, conventional traces are tied to mutable in-memory state: they rarely capture the exact configuration and context a failed run started with, so you cannot reconstruct that run, let alone replay it.
Auditing an agent means bridging the gap between "this was logged" and "this record has provably not been altered since it was written."
What to log for AI agents
To audit AI agent decisions, you must capture the entire context of the execution, not just the final output or the token count. We call this the decision trail.
Your payload schema should include:
- Inputs: the exact prompt, task, or message payload the agent received, plus the model configuration it ran with (model, temperature, system prompt version).
- Reasoning: the intermediate steps, including the paths the agent considered and rejected (e.g.
gf.reasoning.thought,gf.reasoning.considered,gf.reasoning.rejected). Discarded paths matter as much as the chosen one. - Tool calls: the exact arguments passed to external tools (APIs, shell executions, database queries) and the results returned. This is where side effects happen.
- Outputs and approvals: the raw completion, structured decisions, and any human-in-the-loop approval metadata.
Capturing these into a single session-based unit of analysis saves you from reconstructing disconnected LLM calls after an incident.
The evidence layer: tamper-evidence and the hash chain
How do you upgrade a log into an audit trail? You cryptographically chain the events together.
In a tamper-evident system, every action the agent takes is appended to a ledger as a discrete entry. Each entry carries a SHA-256 hash computed from its own canonical payload, its timestamp, and the hash of the previous entry. This creates a continuous hash chain: if a single byte of a past record is altered, deleted, or reordered, subsequent hashes stop matching and verification fails immediately.
Two precise limits are worth stating, because they define what this guarantee means:
- A hash chain proves the integrity of what was captured. It does not prove completeness. If a run was never instrumented, no chain will conjure it. Instrument at the boundary, automatically, not as an optional step.
- A seal on a ledger you host yourself proves internal consistency. To make the record defensible against a party who claims you rewrote the whole chain, you anchor the ledger externally (trusted timestamping, RFC 3161). That is the difference between tamper-evident and tamper-proof.
Auditing must not endanger the run
A recording layer that can take down the system it records is worse than no recording layer at all. The audit path has to sit out of band, off the critical execution path.
That implies two architectural rules. First, the agent never blocks on the record layer: spans are exported asynchronously, batched, and buffered on failure, so a slow or unreachable backend costs you telemetry latency, never the run. Second, faults stay isolated on the receiving side too: one misbehaving run must not corrupt the recording of another. In practice this means the orchestrator and the evidence layer are separate systems with a one-way data flow between them. The agent finishes its work whether or not the ledger is reachable, and the ledger stays consistent whether or not an agent crashes mid-run.
This separation is also what makes the record comparable across runs. Because the evidence layer only receives what happened (inputs, configuration, reasoning, tool calls) and never participates in producing it, the record survives process crashes and can be diffed between runs with different prompts, models, or code.
Compliance anchors
An evidence layer maps directly onto existing regulatory and audit frameworks:
- EU AI Act, Article 12: requires traceability and automated logging for high-risk AI systems, with Annex III obligations applying from 2 December 2027. An append-only, verifiable ledger is a credible technical implementation of these traceability requirements.
- SOC 2 Type II: requires organizations to demonstrate that system access and modifications are authorized and auditable. A tamper-evident trail supports these audit-trail controls with verifiable, not merely asserted, records.
- NIST AI RMF (GOVERN): calls for mapped accountability and transparent tracking of AI system behavior.
Span Chain as a reference implementation
Span Chain is an open-source evidence layer for AI agents, built in Elixir/OTP. It does not run or orchestrate your agents. Execution stays entirely on your side; the backend passively ingests OpenTelemetry (OTLP/HTTP) spans, isolates every run in its own process, and seals each entry into a SHA-256 hash chain you can verify at any time with verify_ledger. Ingest is out of band: the Python SDK batches spans and buffers them on failure, so a slow or unreachable backend never blocks the agent.
It sits beneath your observability stack rather than replacing it. Your observability shows what happened; the ledger keeps the proof. A comparison with LangSmith and Langfuse is on the project page.
Because the ledger captures a deterministic record of each run, you also get VCR-style cassette replay: a recorded run can be replayed locally with zero live LLM calls, and a structural diff against a baseline shows the exact deviation point. New comparison runs against modified code are real executions and cost what any run costs; replaying what was already recorded costs nothing.
External TSA anchoring (RFC 3161) is on the roadmap for regulated deployments, where tamper-evident needs to become tamper-proof.
Self-host with Docker Compose: github.com/ghostfactory-art/spanchain. MIT licensed.
FAQ: Auditing AI Agents
What is an AI audit trail?
An AI audit trail is a chronological record of every input, reasoning step, tool call, and output generated by an AI agent. Unlike standard logs, a true audit trail is cryptographically chained (tamper-evident), so any alteration of the historical record is immediately detectable.
How do you audit AI agent decisions?
You audit AI agent decisions by inspecting their tamper-evident decision trail: the exact context the agent had at execution time, including the system prompt version, the retrieved context, the reasoning paths it rejected, and the arguments sent to external tools.
What should you log for AI agents?
Log the complete execution context: the initial trigger and input, the exact model configuration (model, temperature, prompt version), the step-by-step reasoning, arguments and responses for all external tool calls, delegation metadata in multi-agent setups, and the final structured output.