The four-stage framework: not reinvented, just adapted
According to the US standards body NIST (framework SP 800-61, most recent revision from April 2025), every security incident runs through four phases: preparation, detection and analysis, containment/eradication/recovery, and post-incident activity. This framework is well established and doesn't need reinventing for AI incidents – it only needs to be supplemented with AI-specific signals a classic IT security team would otherwise miss.
Why AI-layer incidents produce different signals
A classic IT incident typically shows up through compromised credentials or unusual network traffic. An AI-layer incident – prompt injection in a production application, an agent's escalating tool calls, an attempt to extract a model through targeted queries, prompt-based data exfiltration – often looks completely unremarkable on classic monitoring dashboards, because it disguises itself as normal application use. Anyone only looking for classic signals reliably misses this incident class.
The four phases, applied to AI incidents
For AI systems, preparation means: logging what an agent actually does (not just what it answers), before an incident is even suspected. Detection and analysis means: specifically looking for AI signals – unusual tool-call patterns, repeated failed instruction attempts, unexpected data volumes in outputs. Containment for an agent often means: stopping the agent and immediately revoking its tool privileges, not just blocking a network account. Post-incident activity means: adjusting the underlying rights structure or input checking so the same incident type doesn't recur identically.
Practice section: the roadmap before the first incident, not after
The most important practical advice: this roadmap belongs on paper before the first incident happens, not while it's underway. Concretely, that means: decide in advance who gets informed in an emergency, how an agent can be technically stopped in an emergency (see the architectural defense from an earlier module in this course), and which logs need to exist beforehand at all to reconstruct what happened afterwards. A company that only answers these questions during an ongoing incident loses valuable time at exactly the moment it's most expensive.