Add AI to the plan you already use
An AI incident response plan should extend the organisation’s existing cyber, privacy and operational response arrangements. A separate document that uses different contacts and is never exercised can slow the response.
Start by listing deployed AI systems, their owners and the business processes they affect. Include ordinary assistants, API integrations, browser or computer-use agents, customer-facing chatbots and AI features embedded in other software. Record the data each system can access and the external actions it can take.
Define what counts as an AI incident
Useful examples include an agent accessing information outside its approved scope, sending or publishing material without valid approval, exposing personal or confidential data, repeatedly attempting a blocked action, producing harmful discriminatory output, or making a high-impact recommendation without the required review.
Not every wrong answer is a security incident. The plan should distinguish routine quality problems from events involving unauthorised access, data exposure, consequential external action, systemic harm or loss of control. Both should be recorded, but they may need different escalation paths.
Assign roles before anything happens
Name an incident lead and an alternate. Identify who can disable the AI integration, revoke credentials, preserve logs, contact the vendor, assess privacy impact and communicate with customers or authorities. Small businesses may use external IT, legal or cyber specialists; record after-hours details rather than assuming the usual contact will be available.
The system owner should not be the only person able to investigate. Keep an accessible register of products, accounts, model versions, service contacts, API keys or credential owners, and connected systems. Do not place secrets directly in the plan.
Detect and preserve evidence
Monitor for unusual tool calls, repeated access failures, unexpected data volumes, changes to permissions, abnormal costs and external actions that do not match an approved workflow. Staff need a simple way to report surprising AI behaviour without first proving a breach occurred.
Preserve relevant prompts, outputs, tool calls, timestamps, user and service identities, approval events, model and application versions, network or application logs and affected records. Avoid repeatedly running the same action to “see if it happens again” when the outcome could be consequential.
Contain safely
Containment may involve pausing an automation, removing write access, revoking a token, isolating a connected account or switching to a manual process. Preserve evidence before deleting records or resetting systems where practical. Do not rely on changing the prompt if the agent retains the same powerful credentials and access.
Check whether the event is continuing and whether another customer, system or supplier is affected. If an external action had an uncertain outcome, inspect the destination before retrying. Duplicate messages, payments or record changes can compound the incident.
Assess notification and assistance
Contact the AI supplier through its security or incident channel and request a reference number, preservation of relevant logs and a clear response timetable. Review contracts for notice requirements to customers, insurers and business partners.
For a suspected cyber security incident, the Australian Signals Directorate’s Australian Cyber Security Centre encourages early reporting and provides technical advice. Its Australian Cyber Security Hotline is 1300 CYBER1 (1300 292 371), available 24/7. See cyber.gov.au for reporting options.
Existing legal duties may also apply. Under the Notifiable Data Breaches scheme, covered entities that suspect an eligible data breach must conduct a reasonable, expeditious assessment and take all reasonable steps to complete it within 30 calendar days. If they have reasonable grounds to believe an eligible breach has occurred, they must notify affected individuals and the OAIC as soon as practicable, unless an exception applies. Not every AI error or data breach meets the threshold. See the OAIC guidance linked below and obtain advice for the circumstances; sector-specific requirements may also apply.
Recover and learn
Before restoring the system, confirm the cause is understood well enough to prevent immediate recurrence. Reduce privileges, correct the workflow, add missing approval steps, test the change and communicate any temporary limitations to staff.
Hold a short review after recovery. Record what happened, which controls worked, where information was missing and who owns each improvement. Update the AI register and risk assessment. Share a useful internal lesson without exposing sensitive details or blaming the person who reported the problem.
A tabletop exercise to run this month
Use a scenario in which a connected AI agent reads a restricted file and then attempts to send a summary to an external address. Ask the team:
- Who notices first and how?
- Who can stop the integration immediately?
- Which logs prove what the agent read and whether the message was sent?
- Who contacts the vendor, affected person, customer or authority?
- What manual process keeps the business operating?
- What evidence must be preserved before credentials are changed?
Run the exercise in under an hour and turn gaps into named actions with dates. A modest rehearsed plan is more useful than a comprehensive document no one can execute.