RailAgents for a safety-critical workflow

Requirement compliance platform

Answering one customer requirement document took 30 to 50 days: senior engineers reading hundreds of numbered clauses and cross-referencing each against the applicable regulation, the product specification and every previous certification. The platform now produces a first full analysis in under five minutes, and every position it produces arrives with the evidence behind it and an engineer's signature on it.

30–50 daysRequirement review per document, before
Under 5 minFirst full analysis now
Every clauseCarries its regulation, specification and precedent
Engineer signsNo position reaches the customer unsigned
Requirement review screen showing classified clauses with compliance positions and the evidence behind a flagged clause
Redrawn from the product. Clause text, standards references and product names are illustrative.
Client
An industrial manufacturer in the EU building safety-critical systems
My role
Tech lead. I owned the overall architecture design and contributed the AI engineering: the document pipeline, retrieval design, the supervisor and worker structure, and the evidence model.
Team
Six engineers: two data, two AI, two full stack.
Timeline
First release two months after kickoff, then a release every six weeks. Final signoff in January 2026.
Stack
Azure AI Document Intelligence, Azure OpenAI, pgvector on Azure Database for PostgreSQL, LangChain, FastAPI

The problem

A requirement specification arrives as hundreds of numbered clauses, and every one needs a compliance position before anything is built.

Senior engineers found the applicable standard, the product specification and the precedent by hand. A pass took 30 to 50 days, almost all of it document retrieval rather than judgement.

Scrolls sideways.

Before and after workflow for the requirement compliance platform
How a requirement document was answered before, and how it is answered now. Open full size

The solution

In a regulated domain the deliverable is not the answer, it is the answer plus its provenance. An auditor asks why a position is defensible, which makes the citation the product and the generation step the least important part.

The platform extracts clauses with their numbering intact, classifies each by region, train type and standard, and sends it to three workers scoped one each to the regulation, the specification and the precedent.

A position without citations is never produced. The clause is held for an engineer instead, and an engineer signs every position that leaves the system.

Scrolls sideways.

Railway compliance architecture: document ingestion through Azure AI Document Intelligence, clause splitting and classification, three retrieval workers over regulation, precedent and specification, evidence binding, engineer review, and an audit trail
System architecture. Each worker can read only its own index, so a regulation answer cannot be contaminated by specification text, and the evidence binding step is the only path to an output. Open full size

How it works

Document ingestion

Azure AI Document Intelligence reads layout, tables and clause boundaries. Section numbering is preserved, because a clause without its number cannot be answered or audited.

Classification

Each clause is tagged with region, train type, standard and discipline. This decides which regulation edition applies, so it is confirmed by the engineer at review rather than trusted silently.

Three scoped workers

One finds the applicable rule and its edition. One finds what the product specification states. One finds how the clause was answered before and whether the customer accepted it. No worker can read another's index.

Evidence binding

The conclusion is assembled only from what came back. A position that cannot name its regulation, specification reference and precedent is not produced at all.

Engineer review

Every position is accepted, edited or rejected by a qualified engineer. There is no path from the model to the customer that does not pass a signature.

Audit trail

Source and page, rule and edition, specification reference, precedent, model and prompt version, reviewing engineer, the edit and the signing time, all recorded per clause and immutable once signed.

Evaluation

Engineer edits are captured as labelled corrections. Every edited clause is a recorded statement that the draft was wrong and what correct looks like, written by someone qualified to say so.

Release gate

Regulation editions change and prompts change. A fixed set of previously signed clauses, re-run after each change, is what turns an opinion about a release into a measurement.

Stack

Document ingestion

Azure AI Document IntelligenceLayout and table extraction

AI and models

Azure OpenAIClause classificationResponse generation

Retrieval

pgvector on Azure Database for PostgreSQLSeparate index per source typeAzure AI Search (POC)

Orchestration

LangChainSupervisor and worker agents

Application

FastAPIEngineer review interface

Product screens

Clause analysis with the evidence behind a flagged position

1 of 3

Redrawn from the product. The originals contain customer requirement text and standards material that cannot be published. Clause text, standards references and product names here are illustrative; the fields, states and controls are as built.

Results

Before

30 to 50 days

Per requirement document. Senior engineers read every clause and cross-referenced it against the regulation, the specification and the certification history by hand.

After

Under 5 minutes

Machine time to a first full analysis, with every clause carrying the regulation, specification and precedent behind its position. Engineers review and sign, starting from a drafted answer instead of a blank page.

What did not go to plan

The first drafts read beautifully and were unusable, assembling confident compliance language out of thin sources. Making citation structural rather than instructed fixed it, and the count of clauses the system declines to answer became a quality measure rather than a shortfall.

Under five minutes is machine time to a first analysis, not the end of the process. Engineers still review. They just start from a drafted position with its sources attached.

What this demonstrates

  1. 01Evidence-first generation in a regulated domain: no citation, no answer
  2. 02One index per source type, so a regulation answer cannot be contaminated by specification text
  3. 03Human sign-off as a structural step in the pipeline, not a policy
  4. 04Evaluation built from engineers' own corrections, and a release gate re-run on every change
  5. 05Document ingestion that preserves clause numbering, because an unnumbered clause cannot be audited

Next caseData quality portal and analytics platform