An AI incident record should let a team reconstruct what happened without turning the incident database into a second copy of every prompt, document and personal detail.
Source review: October 29, 2026. Framework reference: NIST AI RMF 1.0 and its current Playbook; Generative AI Profile NIST AI 600-1, published July 26, 2024 and updated on the NIST publication page April 8, 2026.
Start with the user-visible event
Open the record with the observed event: what the system returned or did, where it appeared, when it was reported and who or what detected it. Keep the expected behavior beside the observed behavior. A label such as “hallucination” is too broad to support a fix.
The record should distinguish the product action from the model output. A model can generate an incorrect answer that is caught before a user sees it. A correct model response can also be displayed to the wrong account or used by an unsafe workflow. Those incidents have different owners and controls.
NIST AI RMF Measure guidance recommends histories, audit logs and other records that help reviewers find sources of error, bias or vulnerability. It also calls for documenting human oversight, overrides, reported errors, response times and adjudication activity. That supports a record built around evidence and decisions rather than a free-form narrative.
Pin the system version
An incident cannot be reproduced from a model name alone. Record the application release, model or endpoint identifier, prompt-template version, retrieval or tool configuration, policy version and evaluation-set version that were active.
If the product uses retrieval, save the identifiers and versions of the source documents selected for the response. The article on data provenance in RAG answers explains why a citation without a document version cannot prove which evidence the system actually used.
Keep configuration references immutable where possible. A link to a live prompt or policy page may show today’s state instead of the state at incident time. A content hash, release identifier or snapshot reference gives investigators a stable anchor.
Capture the path through the system
A compact event timeline should show the main steps:
- request accepted;
- preprocessing and policy checks;
- retrieval, tool calls or other external actions;
- model response;
- post-processing and human review;
- product action or user-visible result.
Each step needs an outcome and trace reference. The incident record does not need every debug line. It needs enough structure to show where the behavior changed and which evidence is held elsewhere.
For tool-using systems, record the tool name, permission context, sanitized arguments, result status and whether the action had an external effect. Do not assume that a good final answer means every intermediate action was safe.
Separate impact from severity
Impact describes what happened. Severity is the team’s decision about how urgently and widely to respond. Record affected users or workflows, exposure duration, reversibility, safety or financial consequences, and whether the issue is still active.
Include uncertainty. Early estimates can be marked provisional and updated with a timestamp. Keeping the first estimate helps reviewers understand why the team escalated or waited.
The NIST AI RMF Manage Playbook suggests maintaining reported errors, near misses, incidents and negative impacts with the date reported, report count, impact and severity assessment, and responses. It separately recommends a record of system changes, including why each change was made and how it was tested and deployed.
Record decisions and ownership
Every incident needs a current owner, an accountable decision maker and a next review time. Decision entries should say who chose to contain, continue, roll back or close the incident, what evidence they used and what condition would change the decision.
Human intervention belongs in the timeline. Record whether an operator approved, edited, rejected or overrode the output. Avoid interpreting an override count by itself. A high rate might signal poor model behavior, a cautious workflow, or both.
Link the incident to the evaluation case that prevents recurrence. AI evaluation set versioning shows how to freeze the case, references, grader and run settings so the repaired system is tested against the same failure.
Protect sensitive evidence
A complete record is not the same as an unrestricted record. Prompts, retrieved documents, user messages and tool arguments can contain personal data, credentials or confidential material. Store restricted evidence in an access-controlled system and put references, classifications and retention rules in the incident record.
Redact copies used for broad review. Preserve the original only when the investigation and retention policy require it. Record each redaction so investigators know that the visible artifact is not the raw event.
The general boundary in what an audit log can prove also applies here: logs support reconstruction only when their identity, ordering, access and retention controls are trustworthy.
Close with proof of repair
Closure should include the root cause as currently understood, containment, permanent change, tests run, evaluation results, deployment identifier and monitoring window. If the cause remains uncertain, say so and list the leading hypotheses that were ruled out or remain open.
A near miss deserves the same structure when it reveals a control failure. Marking it as a near miss preserves the distinction without discarding the evidence.
