How to prove an AI agent did what it claims
An agent tells you it ran the tools and got the result. Can you prove it? Here is why ordinary logs cannot, and how tamper-evident, cryptographically verifiable receipts can.
To prove an AI agent did what it claims, you cannot rely on the agent to tell you, or on ordinary logs to record it; you need a tamper-evident receipt for each action, signed by something the agent does not control and chained so any change is detectable. An agent can report that it ran a tool and got a result while, in reality, it skipped the step, called the wrong tool, or invented the output, and a normal log will happily record whatever the agent says because logs are usually mutable and self-reported. That gap is now a compliance problem as well as a trust one: the EU AI Act, whose Article 12 record-keeping duties are being enforced from August 2026, requires high-risk AI systems to keep logs that are traceable and tamper-evident, not just present (Help Net Security1). I build agent attestation and verifiable-record tooling at RankShield, and the question underneath almost every incident I see is the same: "what did the agent actually do?", with no trustworthy answer. What most guidance leaves out is the concrete mechanism, how you turn an agent action into a receipt a reviewer can verify instead of a log entry they have to trust. That is what this covers. One honest note: no receipt proves an agent was honest or correct, and true third-party verifiability requires anchoring the record to an external, witnessed log; what a good receipt does prove is exactly what was done, by which agent, when, and on whose authority, in a form where tampering is detectable.
Why can’t you trust an AI agent’s own logs?
You cannot trust an agent’s own logs because they are self-reported and mutable: the agent writes what it believes or claims happened, and nothing stops that record from being wrong or altered afterward. An agent can hallucinate a tool call it never made, report success on a step that failed, or call a different tool than the one it names, and an ordinary log captures the agent’s version of events, not the ground truth. The log is a story the agent tells, and a story is exactly what you cannot rely on when money, data, or compliance is on the line.
Mutability makes it worse. Standard application logs can be edited, deleted, or simply never written, so even an accurate log is only as trustworthy as everyone with write access to it. When an incident happens and you go looking for what the agent did, you are reading a record that could have been changed, by an attacker, by a bug, or by the same compromised process you are investigating. "We think it was fine" is not an answer a regulator, an auditor, or a customer will accept, and it is the honest state of most agent logging today.
The core problem is that trust is pointed the wrong way. Ordinary logging asks you to trust the agent and the log; verification flips that, so you do not have to. Instead of a record you hope is accurate, you want a record whose integrity you can check independently, where an alteration is not just against the rules but mathematically detectable. That is the difference between a log and a receipt, and it is the shift the rest of this guide is about.
What does it mean to prove an agent did what it claims?
Proving an agent did what it claims means having execution provenance: a verifiable record, at the level of each tool call, of what was done, by which agent, when, and on whose authority, that a reviewer can check rather than take on faith. It is the difference between an agent asserting "I processed the refund" and a receipt that shows the refund tool was invoked with these arguments, at this time, under this authorization, in a record where any later change would be visible. Provenance turns a claim into evidence.
The unit that matters is the individual action, not the session. Session-level summaries are still self-reported narratives; provenance operates one level down, capturing each consequential tool invocation as its own verifiable event. This is what recent work on verifiability-first agents describes: moving beyond mutable execution logs toward cryptographically anchored, tamper-evident provenance, where every valid tool interaction passes through attestation and an agent cannot perform an undetected operation (Verifiability-First Agents, arXiv3). The goal is that the record of what happened is bound to the happening, not narrated after it.
Crucially, provenance proves conduct, not correctness or honesty. A receipt can prove an agent called a tool with certain arguments; it cannot prove the agent should have, or that the result was right. That is the honest boundary of what this buys you, and it is still enormous: most agent disputes and incidents turn on "what did it actually do," and a verifiable answer to that question is what lets you assign responsibility, satisfy an auditor, and respond to an incident with evidence instead of guesswork. If you want that record produced automatically per action, that is what RankShield’s verifiable AI security is built to do.
How do cryptographically verifiable agent receipts work?
A verifiable agent receipt works by signing each action with a key the agent does not hold, chaining each receipt to the one before it, and timestamping it, so that altering any entry breaks the chain in a way anyone with the record can detect. The signature binds the receipt to a signer the agent cannot impersonate; the hash chain makes the receipts an ordered, tamper-evident sequence where changing one entry invalidates every entry after it; and the timestamp fixes when it happened. Microsoft’s guidance on securing AI agents describes exactly this pattern: each action gets a tamper-evident, signed receipt capturing what was done, by which agent, when, and on whose authority (Microsoft2).
The building blocks are standard, well-understood cryptography. Receipts are typically signed with an algorithm like Ed25519, hash-chained with SHA-256 so each links to the last, and anchored to trusted timestamps such as RFC 3161 or eIDAS to prove ordering in time, with post-quantum signature schemes like ML-DSA available for records that must stay verifiable for years. None of this is exotic; it is the same tamper-evidence that secures transparency logs and software supply chains, applied to agent actions. The point is that verification requires only the record and the public keys, not access to the agent or trust in whoever ran it.
One honest qualification matters here, because it is easy to overstate. A signed, chained receipt lets you detect tampering and check the record against the signer’s public key, which is a real, independent form of verifiability. But full third-party trustlessness, where any party can verify with zero trust in the operator, additionally requires anchoring the receipts to an external, witnessed log that the operator does not control, so they cannot quietly rewrite history before anyone reads it. Without that external anchor, receipts still make alteration detectable and the record independently checkable; with it, verification becomes genuinely operator-independent. Being precise about that line is the difference between honest security and marketing.
What does the EU AI Act require for agent logs?
The EU AI Act’s Article 12 requires high-risk AI systems to automatically keep logs that enable traceability throughout the system’s lifecycle, and the practical reading in 2026 is that those logs must be tamper-evident, not merely present. Enforcement of these record-keeping duties runs from August 2026, and the guidance emerging around it is explicit that mutable, self-reported logs do not satisfy the intent: the record has to support after-the-fact traceability and risk identification in a way an auditor can rely on (Help Net Security1). For agentic systems, that points directly at verifiable, per-action provenance.
This turns execution provenance from a good engineering idea into a compliance requirement for anyone operating high-risk AI in scope. If your logs can be edited and are only the agent’s own account of events, you have a record that may not withstand scrutiny when a regulator or an auditor asks you to demonstrate what the system did. Tamper-evident receipts are a direct, defensible answer: they show the actions, bound to when and by whom, in a form where alteration is detectable, which is exactly the traceability the Article is reaching for.
It also connects to liability. As we covered in our guide to who is liable when an AI agent makes a mistake, the deployer generally carries the exposure, and what limits it is demonstrable oversight. A verifiable record of what your agents did and who authorized their consequential actions is precisely the evidence that turns "we think it was fine" into "here is the proof," which serves compliance and liability at once. This is general information rather than legal advice; confirm how Article 12 applies to your systems with qualified counsel.
How do you make your agents’ actions provable?
You make agent actions provable by generating a signed, chained receipt at the moment each consequential action happens, capturing what was done, by which agent, when, and under what authorization, and storing those receipts so their integrity can be checked later. The key discipline is to capture at the point of action, not to reconstruct after the fact, because a record built after an incident is exactly the mutable, self-reported artifact you are trying to move beyond. Wrap your agents’ tool calls so that invoking a tool also emits its receipt, making the record a byproduct of acting rather than a separate step someone has to remember.
Prioritize the actions that matter. You do not need a cryptographic receipt for every trivial read; you need one for the consequential and hard-to-reverse actions, moving money, changing data, sending communications, granting access, where "what did it do" is a question you will actually have to answer. Bind each receipt to the agent’s identity and to the human or policy that authorized the action, so the record answers not just what happened but on whose authority, which is what turns it into evidence for both compliance and liability.
Then make verification real by anchoring. Storing receipts in your own system gives you tamper-evidence you can check; anchoring their chain to an external, witnessed log is what makes the record verifiable by someone who does not trust you, which is the standard true third-party verification requires. Be honest about which level you are at, because the two are different guarantees. Producing that receipt layer, signed, chained, timestamped, and anchored, is exactly what RankShield’s attestation layer is built to provide, so proving what your agents did becomes a property of the system rather than a scramble after something goes wrong.
How do you move from trusting agents to verifying them?
Stop asking your agents to be trustworthy narrators of their own actions and start making those actions provable. Ordinary logs fail because they are self-reported and mutable, so they record the agent’s story rather than the ground truth. The fix is execution provenance: a tamper-evident receipt for each consequential action, signed with a key the agent does not hold, hash-chained so any change is visible, timestamped, and bound to the agent’s identity and the authority behind it. Capture it at the moment of action, and prioritize the consequential, hard-to-reverse operations where "what did it do" is a question you will actually face.
Be honest about the guarantee: a receipt proves conduct, not correctness or honesty, and full operator-independent verification also requires anchoring the record to an external, witnessed log. Within those bounds, verifiable receipts turn "we think it was fine" into "here is the proof," which is what satisfies EU AI Act Article 12, limits liability, and lets you respond to incidents with evidence. As agents take on more consequential work, the ability to prove what they did stops being optional. If you want that receipt layer, signed, chained, timestamped, and anchored, produced automatically per action, see how RankShield makes AI agent actions verifiable.
Questions, answered.
Why can’t I just use my AI agent’s logs to see what it did?
Because ordinary agent logs are self-reported and mutable: the agent records what it believes or claims happened, and nothing stops that record from being wrong or altered afterward. An agent can hallucinate a tool call it never made, report success on a step that failed, or call a different tool than the one it names, and the log captures the agent’s version rather than the ground truth. On top of that, standard logs can be edited, deleted, or never written, so even an accurate log is only as trustworthy as everyone with write access to it. When an incident happens and you need to know what the agent did, you are reading a record that could have been changed, which is why "we think it was fine" is the honest state of most agent logging and why verifiable receipts, whose integrity you can check independently, are the fix.
What is execution provenance for an AI agent?
Execution provenance is a verifiable record, at the level of each tool call, of what an agent actually did, by which agent, when, and on whose authority, that a reviewer can check rather than take on faith. It is the difference between an agent asserting "I processed the refund" and a receipt showing the refund tool was invoked with these arguments, at this time, under this authorization, in a record where any later change would be detectable. The unit that matters is the individual consequential action, not a session-level summary, because summaries are still self-reported narratives. Provenance proves conduct, not correctness or honesty: it can show an agent called a tool with certain arguments, but not that it should have or that the result was right. That bounded proof is still what most agent incidents and audits actually turn on.
How does a cryptographic agent receipt work?
A verifiable agent receipt is created by signing each action with a key the agent does not hold, chaining each receipt to the one before it, and timestamping it, so altering any entry breaks the chain in a way anyone with the record can detect. The signature binds the receipt to a signer the agent cannot impersonate; the hash chain, typically using SHA-256, makes the receipts an ordered, tamper-evident sequence where changing one entry invalidates every later one; and a trusted timestamp such as RFC 3161 or eIDAS fixes when it happened, with post-quantum signatures like ML-DSA available for long-lived records. These are standard, well-understood cryptographic building blocks, the same tamper-evidence used for transparency logs and software supply chains. Verification then requires only the record and the public keys, not access to the agent or trust in whoever operated it.
Does the EU AI Act require verifiable agent logs?
The EU AI Act Article 12 requires high-risk AI systems to automatically keep logs that enable traceability throughout the system lifecycle, and the practical reading in 2026 is that those logs must be tamper-evident, not merely present. Enforcement of these record-keeping duties runs from August 2026, and the guidance around it is explicit that mutable, self-reported logs do not satisfy the intent, because the record has to support after-the-fact traceability and risk identification an auditor can rely on. For agentic systems that points directly at verifiable, per-action provenance. It also connects to liability: the deployer generally carries the exposure, and a verifiable record of what agents did and who authorized their actions is the evidence that demonstrates oversight. This is general information rather than legal advice; confirm how Article 12 applies to your systems with qualified counsel.
Can you prove an AI agent is honest or correct with receipts?
No, and it is important to be precise about this. A verifiable receipt proves conduct, not correctness or honesty: it can prove that an agent called a specific tool with specific arguments at a specific time under a specific authorization, in a record where tampering is detectable. It cannot prove the agent should have taken that action, that the action was the right one, or that the result was accurate. Those are separate questions that provenance does not answer. What receipts do give you is a trustworthy answer to "what did it actually do," which is the question most agent disputes, incidents, and audits actually turn on. Claiming receipts prove an agent is trustworthy or its outputs are correct would be an overstatement; claiming they make its actions tamper-evident and checkable is accurate.
What does it take to make an agent record verifiable by a third party?
Two things. First, tamper-evidence: sign each receipt with a key the agent does not hold and hash-chain the receipts so any alteration is detectable, which lets someone check the record against the signer’s public key without trusting the agent. Second, and this is the part often glossed over, operator independence: to let a third party verify with no trust in the operator, you have to anchor the receipt chain to an external, witnessed log that the operator does not control, so they cannot quietly rewrite history before anyone reads it. Without that external anchor, receipts still make tampering detectable and the record independently checkable, which is valuable, but full third-party trustlessness requires the anchor. Being honest about which of these two levels you have is the difference between real verifiability and marketing language.
References
- Help Net Security. What the EU AI Act requires for AI agent logging (Article 12 tamper-evident, traceable logs; record-keeping enforced from August 2026).
- Microsoft. Securing AI Agents with Cryptographic Receipts (each action gets a tamper-evident, signed receipt: what, which agent, when, on whose authority).
- arXiv. Verifiability-First Agents: provable observability and lightweight audit agents (moving from mutable execution logs to cryptographically anchored, tamper-evident provenance).
Jamie Kloncz
Founder & CEO, RankShield
Jamie Kloncz is the founder and CEO of RankShield, the verifiable AI and quantum security platform. He started the company after two attacks landed in a single week: his phone was cloned, and his business was hit by a click-fraud campaign. One targeted him as a person, the other his livelihood, and no single tool defended both. That experience, together with surviving an AI voice-clone scam, shaped RankShield’s core belief: the threats of the AI age are personal first, and trust should be something you can check, not just extend.
Make every AI action provable.
RankShield is the verifiable, quantum-safe AI security platform — protection you can check, not just trust.