Abstract
The deployment of AI and machine learning systems into critical national infrastructure introduces a class of risk that existing governance frameworks were not designed to address. This paper examines the gap between enterprise AI governance models and the operational realities of CNI environments, and proposes a practitioner-led framework for AI security governance that is proportionate to consequence, not just probability.
1. The Problem with Enterprise AI Governance in CNI
Enterprise AI governance frameworks, whether derived from NIST AI RMF, ISO/IEC 42001, or internal risk management processes, are built around assumptions that do not hold in critical infrastructure contexts. They assume reversibility. They assume that a failure mode is primarily reputational or financial. They assume that the system under governance is logically isolated from physical consequence.
In CNI environments, power generation, water treatment, transport control, industrial manufacturing, these assumptions fail. An AI system making an incorrect inference in a SCADA-adjacent context does not produce a wrong recommendation in a dashboard. It may produce a physical outcome that cannot be undone in the time available to respond.
The governance gap is not a documentation problem. It is a consequence-modelling problem. Existing frameworks ask “how likely is this to go wrong?” CNI requires frameworks that ask “what happens if this goes wrong, and can we recover?”
2. AI Attack Surface in Operational Technology
The integration of AI into OT environments creates attack surfaces that are qualitatively different from those in IT. Three categories are of particular concern:
Model Poisoning via Sensor Manipulation
AI systems trained on sensor data can be manipulated by adversaries who control or can influence sensor inputs. In industrial environments, this may be achievable through physical access, supply chain compromise, or protocol-level attacks on field devices.
Adversarial Inference Attacks
AI systems used for anomaly detection or predictive maintenance can be probed to understand their decision boundaries, enabling adversaries to operate within the envelope of "normal" while conducting malicious activity.
Automation Chain Exploitation
Where AI outputs feed into automated control decisions, an adversary who can influence the model output can potentially influence physical actuation without direct access to control systems.
3. A Practitioner's Governance Framework
The framework proposed here is built on four principles derived from operational experience rather than theoretical risk modelling:
Consequence-First Classification
AI systems should be classified by the worst-case physical consequence of a failure or adversarial manipulation, not by the probability of that outcome. Classification drives the level of governance, testing, and monitoring applied.
Human Override Architecture
Any AI system with a path to physical actuation must have a clearly defined, tested, and regularly exercised human override mechanism. The override must be operable under degraded conditions, including network isolation.
Adversarial Testing as Standard
AI systems in CNI should be subject to adversarial testing, red-teaming of model inputs, sensor manipulation exercises, and inference attack simulations, before deployment and on a regular cadence thereafter.
Explainability as a Safety Requirement
In high-consequence environments, black-box AI is not acceptable for decisions that affect physical systems. Explainability is not a nice-to-have; it is a prerequisite for safe operation and incident investigation.
4. Regulatory Landscape and Emerging Standards
The regulatory landscape for AI in CNI is evolving rapidly but unevenly. The EU AI Act classifies certain AI applications in critical infrastructure as high-risk, imposing conformity assessment requirements. The UK's approach, through the DSIT AI Safety Institute and sector-specific regulators, is more principles-based but increasingly specific in its expectations for high-consequence deployments.
NIST AI RMF provides a useful structural foundation but requires significant augmentation for OT contexts. IEC 62443, the industrial cybersecurity standard, does not yet have mature AI-specific guidance, though working groups are active. Practitioners should expect the standards landscape to consolidate over the next 24–36 months, but should not wait for standards to mature before implementing governance.
The practical recommendation: implement the consequence-first framework described in Section 3 now, document it against NIST AI RMF and IEC 62443 where applicable, and build in the flexibility to map to emerging standards as they mature.
Conclusion
AI governance in critical infrastructure is not a compliance exercise. It is a safety-critical engineering discipline. The frameworks that work in enterprise environments are necessary but not sufficient. Practitioners operating in CNI contexts need governance models that are built around consequence, that assume adversarial intent, and that treat explainability and human override as non-negotiable requirements, not optional features. The cost of getting this wrong is not a regulatory fine. It is a physical outcome.