Using AI in audit without breaking the chain of custody

A language model summarises a 200-page report in a minute, classifies three thousand findings by category and drafts an acceptable remediation plan. The temptation to put it inside the audit process is reasonable and the saving is real.

The problem shows up when somebody asks, eighteen months later, why a specific finding was closed as a false positive. If the answer involves a model whose version no longer exists, with a prompt nobody kept and an output that was copied and pasted into a document, the evidence has broken. Not because the model got it wrong, but because you can no longer demonstrate that it did not.

What exactly breaks

The chain of custody of a piece of evidence is the continuous trail that lets you assert, to a third party, that what you are showing came from where you say and has not changed on the way. Every step must have an author, a date and a describable transformation.

When a model steps in, three new gaps appear:

• Non-determinism. The same input can produce different outputs. You cannot reproduce the step by running it again.
• Blurred authorship. Who made the decision, the model or the person who accepted its suggestion without reading it?
• Opaque transformation. “Summarised with AI” does not describe a transformation. It describes a box.

All three have solutions, and none of them is to stop using the model.

The line: the model proposes, the source system proves

The rule that works best in practice is simple. Evidence is always generated in the source system — the scanner, the log, the console, the repository — and never in the model. The model may read it, order it, correlate it and suggest what to do about it, but it neither creates it nor replaces it.

Put differently: if you deleted every model output, the file should still be complete and auditable. If deleting them leaves holes, you have used the model to produce evidence, and that is the thing you cannot do.

With that line drawn, the tasks sort themselves.

Where the model helps and compromises nothing: summarising, clustering similar findings, translating, drafting, extracting fields from long documents, suggesting the mapping between a control and a regulatory obligation, spotting inconsistencies between documents, preparing interview questions.

Where it must not act without explicit human review: declaring a false positive, closing a finding, rating residual risk, accepting an exception, issuing a conformity conclusion, deciding whether an incident crosses a reporting threshold.

The difference between the two lists is not technical difficulty. It is whether the action carries legal consequence or whether somebody will have to defend it personally.

What has to be recorded at each intervention

For a model-assisted step to remain auditable, six things need to be stored, and all six are cheap if they are automated from the start:

1. The exact input the model received, not a summary of it. If the input was a document, its hash.
2. The instruction or prompt template, with its version. Prompts change; if you do not version them, you cannot explain why the same process gave different results in March and September.
3. Model identification and version, plus the relevant parameters — temperature, whether context retrieval was used and from which corpus.
4. The literal output, in full, before any human edit.
5. Who reviewed it, when, and what they changed. This is the decisive field: it turns a suggestion into a decision with an author.
6. The final decision, linked to the corresponding finding or control.

With those six fields, a model-assisted step is as traceable as a manual one. Without the fifth, it is not traceable at all.

Automation bias is the real risk

The failure I have seen most is not the model hallucinating a finding that does not exist. It is the reviewer accepting three hundred suggestions in a row because the first twenty-five were correct. Past a certain volume, human review becomes a click.

Two countermeasures work reasonably well:

• Mandatory verification sampling. A fixed percentage of assisted decisions is re-tested manually against the source system, and the result is recorded. If the error rate rises, the scope of automation narrows. As a side effect this produces exactly the effectiveness evidence that Article 21(2)(f) of NIS2 asks for.
• Friction proportional to consequence. Accepting a category reclassification can be one click. Closing a critical finding as a false positive should require typing the reason. Friction is not bureaucracy when it protects the decision somebody will have to defend.

Confidentiality and where the data lives

There is a second front, about control rather than evidence. Audit files contain internal paths, test credentials, architecture, unfixed vulnerabilities and sometimes personal data. Before sending them to a model there are three questions to answer in writing, because an auditor will ask them:

• Where is it processed and where is it stored? Location matters for the GDPR and for the ENS, and your own risk assessment should cover it.
• Does the model provider retain inputs or use them for training? The answer has to be in the contract, not on the marketing page.
• What is anonymised before sending, and who verifies that?

And one obligation that tends to be forgotten: if the model is supplied by a third party and supports a relevant function, that third party enters your supplier register with everything that entails.

What you need to be able to say

The acid test is being able to say this sentence without discomfort: “this finding was classified as a false positive on 14 May; the suggestion came from model X version Y with prompt template v3, the literal output is archived, Ana Ruiz reviewed it the same day, checked the original trace in the scanner and confirmed the classification”.

If you can say that, use the model for everything that saves you time. If you cannot, the saving is borrowed: you are taking it from the future, and it is repaid with interest on audit day. That is why, in Violet Security, every assistant suggestion is archived next to the finding with its prompt, its version and its reviewer, instead of disappearing into the final document.