AI is already in the control room. The safety case has not caught up.

Dr Rizwan Ahmad, CEO and Founder, Cianaa

Based on the keynote I delivered at the 2026 NZ ICS/OT Cyber Summit, hosted by the NZ ICS Cyber Technical Network at the Lower Hutt Events Centre, Wellington, on 4 November 2026, and on my paper “Artificial Intelligence in SCADA and Operational Technology Environments: Governance Through ISO/IEC 42001:2023”.

Ask an operator whether they run AI in their control environment and the answer is usually no. Ask whether they run anomaly detection on their historian, predictive maintenance on rotating plant, or an intrusion detection model watching Modbus traffic, and the answer changes.

The technology arrived before the governance did. That is the gap I set out to measure.

I reviewed 78 sources from an initial 847, across power generation, water treatment, oil and gas, and manufacturing, and mapped what is actually deployed against what the safety and security standards actually require. The conclusion is not that AI has no place in operational technology. It is that most organisations cannot currently demonstrate the one thing a regulator or an auditor will ask for: evidence that the system is under control.

The tension nobody designed for

Operational technology is built on determinism. A protection relay must respond inside a bounded time, every time, and fail safe when it cannot. That expectation is written into IEC 61508 and IEC 61511 as Safety Integrity Levels, and the whole discipline of functional safety rests on it.

Machine learning is probabilistic by construction. It produces a confidence, not a guarantee. Its execution path is unbounded, its behaviour drifts as the plant changes, and it can be induced to fail by inputs designed to make it fail.

These two worlds are now sharing a control room. Most of the risk sits in that sentence.

The number that settles the argument

The most useful finding is a comparison that is rarely made explicitly.

Data-driven models in common classification tasks carry a likelihood of misclassification between 10⁻¹ and 10⁻³. A SIL 4 safety function requires a probability of failure on demand between 10⁻⁵ and 10⁻⁴.

That is a gap of two to four orders of magnitude, and the paper’s position is that it cannot be closed by governance alone. It is a property of the technology, not of the paperwork around it.

The practical consequence is blunt. AI should not serve as a primary safety function at SIL 3 or SIL 4 without additional architectural safeguards and a formal safety case demonstrating that the compound failure probability actually meets the SIL requirement. Absent that, AI advises, monitors, prioritises and detects. The safety instrumented system under IEC 61511 remains primary, and the fallback to it must be deterministic and documented.

Any vendor who tells you otherwise is selling something they cannot evidence.

Three risks that behave differently in OT

In an IT environment, a wrong answer is an inconvenience. In a plant, it moves something physical.

Drift. Models degrade quietly as equipment is replaced, processes change or seasons turn. Three distinct kinds matter, and each needs its own detection method: covariate shift in the input distribution, concept drift in the relationship between inputs and outcomes, and prior probability shift when the underlying rate of anomalies changes. A useful trigger is a Population Stability Index above 0.2, which in industrial control benchmarks corresponds to roughly a five percent accuracy decline over thirty days.

False alarms. A spurious anomaly alert in a SCADA system can trigger an emergency shutdown, costing hundreds of thousands in lost production and creating hazardous transient conditions on restart. The governance threshold that matters is not accuracy, it is false alarm rate: no more than one per shift at control level, and for safety-critical alarms, one per six months. Beyond that, operators stop believing the system, which is the worst outcome of all.

Adversarial manipulation at the physical layer. This is where OT differs most. Attacks are not limited to the model. They reach the sensors and the controllers feeding it. Parasitic implants on PLC circuit boards can hijack signals between controllers and actuators, penetrate air-gapped networks through the supply chain, and leave no software integrity violation to detect. In published work, adversarial samples reduced the precision of one detection model by 66 percent. Triton in 2017 showed that attackers specifically target safety systems.

Latency decides where AI can live

One of the more practical findings is that the OT hierarchy imposes hard limits, and they determine what is even possible.

If a proposal puts a deep learning model at protection level, the latency budget has already answered the question.

A way to decide, not just a list of concerns

The part of the paper that practitioners have responded to most is a five-gate decision framework. Before an AI system is deployed in an OT environment, it passes through five gates in order, and the gate it fails determines the automation level it is allowed.

1. Regulatory. Is there a compliance pathway for a high-risk classification? If not, recommendation only.
2. Safety. Does it directly influence a safety-critical function? If it does, it passes only with a completed SIL assessment, human-in-the-loop authorisation and a documented fail-safe fallback to the SIS. Otherwise, recommendation only.
3. Determinism. Is the output bounded and the worst-case latency inside the control loop tolerance? If not, supervised with fallback.
4. Explainability. Can an operator understand why it said what it said, within the latency budget? If not, supervised with explainability tooling.
5. Maturity. Has it run for at least six months without significant drift, passed adversarial robustness testing, and been through an ISO 42001 internal audit? If not, supervised rather than autonomous.

Only a system that passes all five earns full automation, inside a defined envelope and with continuous monitoring.

Applying this to six scenarios on the SWaT water treatment testbed produced a result worth sitting with: strong performance metrics were necessary but never sufficient. Every scenario with safety-critical implications routed to recommendation only regardless of its performance metrics. One was an edge model on a field PLC, blocked purely because it sat adjacent to the safety instrumented system, where a false positive could trigger a spurious shutdown. Performance is not a governance argument.

Why one standard is not enough

ISO/IEC 42001 provides the AI management system, but it was not written for plants. IEC 62443 provides industrial security but says nothing about model drift. ISO/IEC 27001 covers information security but not AI-specific risk.

Used together they cover the ground, and the paper maps how. Annex A.3 of ISO 42001 carries impact assessment, structured as a safety case. Annex A.5 carries competency and the latency budgets. Annex A.6 carries lifecycle management with explicit retraining and rollback triggers. Annex A.7 carries data governance, extended to physical layer integrity and to validating that synthetic training data obeys mass and energy balance before anyone trains on it.

That last point catches people out. Generative models used to augment scarce training data can produce samples that look statistically plausible and are physically impossible.

Why this matters in New Zealand and Australia

This is not a European problem arriving slowly.

Australia’s Security of Critical Infrastructure Act already mandates risk management programs across eleven critical infrastructure sectors, and its all-hazards risk assessment requirement is wide enough that AI in a control environment is already inside scope. No new instrument is needed to make this a live obligation. The EU AI Act classifies AI used in the management and operation of critical infrastructure as high-risk under Annex III, with conformity assessment, technical documentation, post-market monitoring and human oversight obligations. For operators with European exposure, that is a live compliance question rather than a future one.

An operator who cannot show how an AI system was assessed, what its failure modes are and who authorises its decisions is going to struggle with both.

Where to start

For most operators the first step is not technology. It is an inventory. You cannot govern what you have not listed, and in my experience the list is always longer than the control room expects, because anomaly detection and predictive maintenance arrive inside vendor products rather than through a procurement decision.

After that: classify each system through the five gates, structure impact assessments as safety cases rather than risk registers, set drift and false alarm thresholds before deployment rather than after the first incident, and make sure the fallback to deterministic control is written down and tested.

None of this requires the newest model. It requires being able to answer, in front of an auditor, how you know the system is behaving.

The paper

The full paper covers the control-by-control mapping, the quantitative thresholds and their derivations, the adversarial taxonomy and the SWaT case study. It has been submitted for peer review.

Cianaa audits AI management systems against ISO/IEC 42001, and conducted the audit behind New Zealand’s first ISO/IEC 42001 certificate. If you are deploying AI in an operational environment and want to understand what an auditor will ask for, we are happy to talk.