OSRA

The gap

Governance frameworks audit what should happen. OSRA audits what the system actually depends on, and where a dependency, a failure nobody would notice and a trust signal nobody has verified land on the same point.

Before writing OSRA I went through twelve of the major AI governance frameworks and regulations, NIST AI RMF, ISO 42001, the EU AI Act and DORA among them, and rated each one on five things: whether it requires the infrastructure to be mapped, whether it requires failure modes to be analysed at that level, whether it follows dependency chains, whether it requires trust signals to be verified rather than self-assessed, and whether it looks at where those converge. Most of them cover a piece. DORA asks for asset documentation, NIST CSF has supplier management, ISO 27001 has cloud service controls. None of the twelve requires all of it, none goes below the process level to the regions, chips, data feeds and contracts the system runs on, and none of them looks at the convergence. I also surveyed fifteen existing risk methodologies, MITRE ATLAS, the CSA controls matrix, the cloud providers’ own resilience guidance and the supply chain literature among them, looking for one that already did this so I would not have to write it. The closest is the ARMOR framework from WWT and NVIDIA, which covers operational resilience for enterprise AI and stops short of substrate mapping, trust auditing and convergence. Each of the fifteen holds a slice. The gap matrix and the methodology survey are published clause by clause, so you can argue with the ratings.

Meanwhile the regulatory landscape has started creating enforceable liability for AI operational failures: EU AI Act penalties reaching 7% of global turnover, mandatory resilience testing under DORA, personal accountability under the UK’s SM&CR, and AI liability law emerging at US state level. An organization can now be fined for an AI failure it had no method of seeing coming.

OSRA is that method. It sits underneath the governance you already have rather than replacing it. Governance frameworks tell you what has to exist, security frameworks tell you what can attack the system, and OSRA tells you what the system actually depends on, what happens when a dependency fails, whether anyone would notice, and where the organization’s own trust turns that failure into a liability.

The model is one of those dependencies, and in the worked example it is the top finding, but the method does not stop being useful when the model changes. Swap the model out and the identity, API, cloud, data pipeline, monitoring, vendor and contractual dependencies are all still there, and so are the human approval steps everyone assumes are working. OSRA audits the operational system the AI sits inside, which is why it should hold up longer than a framework written around one class of model.

Four phases, four artefacts, one convergence risk summary

Phase 1, Substrate Mapping. What does this AI system actually depend on to function? The artefact is the Substrate Map.

Phase 2, Failure Surface Analysis. For each dependency, what does failure look like, and who would notice? The artefact is the Failure Surface Register.

Phase 3, Trust Surface Audit. Where is the organization trusting a signal it has not verified? The artefact is the Trust Surface Register.

Phase 4, Convergence Mapping. Where do substrate risks, undetected failures and unverified trust overlap? The artefact is the Convergence Risk Summary.

The order matters. Phases 2 and 3 ask different questions of the same map, what happens when this fails and who notices, and what are we trusting without checking, and Phase 4 is only possible because both have been answered separately first.

One methodology, three altitudes

I wrote each artefact for a different reader, because a board, a CISO and a CTO do not need the same document.

Boards and non-executive directors get the convergence risk summary: the three to five points where the AI deployment is most exposed, the regulatory liability at each point, and what to do about it. It is the document that answers the DORA Article 15 question.

CISOs get the full four-phase methodology with integration into existing risk management, vendor assessment and compliance evidence. It reads as a programme of work: what to fix first, and how to report it upward.

CTOs get infrastructure dependency maps, detection gap analysis and engineering remediation priorities. What to build, what to monitor, what to renegotiate, and in what order.

That makes the convergence layer a translation as much as a finding. A CTO does not need the regulatory liability model and a board does not need the dependency graph. The convergence summary is where one becomes the other, and it is the same set of facts all the way down.

Categories and scoring

A finding’s category comes from three yes-or-no questions about one dependency: does its failure carry high severity, would that failure be silent, and is the organization’s confidence in it resting on a trust signal nobody has verified. The category sets the clock. Two people with the same registers in front of them should land on the same category before they discuss a score.

• Critical Convergence, all three conditions met, remediation within 30 days.
• Convergence Point, two of three conditions met, remediation within 90 days.
• Concentration Risk, a single point of dependency at high severity, exit strategy within six months.
• Monitored Risk, one of three conditions or low severity, handled on the standard risk cycle.

The score orders findings within a category so that remediation has a sequence, and the category was settled before any number was written down. Six factors, each on a five-point scale with a published anchor for every point: regulatory exposure weighted at 1.5, detection deficit, trust depth, blast radius weighted at 1.5, remediation complexity, and materialisation horizon. Trust depth, for example, runs from a signal the organization verified itself to a chain of vendors it cannot even enumerate. Detection deficit runs from automated alerting on the failure itself to monitoring that measures something else and stays green while the failure runs.

The 1.5 weights are a judgement that regulatory consequence and reach matter more than the other four, and I would rather say so than dress it up as a constant. What I have checked is whether they change the result. Rescoring every point in the five sector calibration with the weights set anywhere from 1.0 to 2.0 changes no ranking in any scenario. The sensitivity check is a script in the repository, so you can rerun it with your own weights and see what happens to your own ranking before you commit to them.

The remediation catalogue

Findings map to 22 structured remediation actions in four categories: detection gap actions D1 to D6, trust verification actions V1 to V6, substrate resilience actions R1 to R5, and governance integration actions G1 to G6. An assessment ends with a sequenced set of actions drawn from a published catalogue, which is a different thing from a list of problems.

Integration with existing frameworks

OSRA does not replace existing governance. It supplies the substrate layer they are missing.

• NIST AI RMF. Phase 1 feeds the Map function. Phase 2 extends Measure to the infrastructure level.
• ISO 42001. Phase 1 provides the infrastructure detail Annex A.9 requires but does not specify. Phase 3 strengthens certification evidence.
• EU AI Act. Phase 1 fulfils Annex IV documentation at genuine depth. Phase 4 provides Article 9 risk management evidence.
• DORA. Phase 1 extends Article 6 asset documentation. Phase 2 provides Article 25 and 26 resilience scenarios. Phase 4 answers Article 15 directly.
• MITRE ATLAS. Phase 2 incorporates the ATLAS threat model and extends it from adversarial to operational resilience.

Evidence base

OSRA rests on work you can inspect rather than on counts. Twelve governance frameworks rated clause by clause on five dimensions. Fifteen existing methodologies surveyed to test whether OSRA needed writing at all. Regulatory liability mapped across the EU, the US at federal and state level, the UK, Japan and APAC. Eight incident case studies and three enforcement actions, each laid out along the dependency, failure, detection and trust chain the method is built to expose. And 24 convergence points scored across five sector scenarios to calibrate the model, with the sensitivity check above run over all of them. The gap matrix, the incident table and the methodology survey are published as appendices in the repository.

Take it and run it

OSRA v1.2. The methodology, the fillable practitioner templates for all four phases, the action catalogue, the scoring calibration and the evidence appendices are published in full on GitHub under CC BY-SA 4.0. Use it, adapt it, improve it, and give credit. Version 1.2 is the response to an external review of the first public release: the novelty claim is stated precisely, every scoring factor has anchors for the whole scale, the weights are described as what they are, and the evidence behind the counts is published.

The worked example is a complete run of all four phases against EuroBank Sentinel, a fraud detection system at a DORA-regulated bank, with every convergence point scored factor by factor and mapped to the catalogue, including the findings the quarterly board report would have kept showing as green.

What OSRA still needs is people who did not write it running it. If you take the same system through all four phases and your map, your scores and your remediation order come out close to mine, that is evidence it is a methodology rather than one person’s way of thinking. If they come out different, that shows where the specification is still loose, and I want to know either way.