Defence & AI2024

Project SCRIBE

Multilingual AI Transcription & Translation

On-premise AI transcription and translation system supporting 12 Indian languages for classified government briefings and intercepts.

Headline outcome

91.4% WER improvement · 12 Indian languages · 2-hour audio in 8 minutes

A corridor of compact rolling shelving in a records archive — Illustrative photography. No imagery on this site is from a client environment.

Whisper Fine-tuningMultilingual NLPPyTorchAir-gapped12 LanguagesFastAPI

Overview

A government intelligence unit needed a secure system to transcribe and translate audio from 12 Indian languages — including dialects — without any data leaving their secure facility.

Challenge

Commercial ASR services (Google, AWS, Azure) were prohibited. Existing open-source models had poor accuracy for Indian language dialects, particularly regional variants of Hindi, Tamil, and Telugu. The system needed to process both clear audio and low-quality field recordings.

Solution

We fine-tuned OpenAI's Whisper model on a proprietary dataset of 200 hours per language, with particular focus on regional dialects. A custom preprocessing pipeline improved accuracy on low-SNR recordings by 18%. Translation used a locally-deployed multilingual LLM. Deployed on air-gapped servers.

Outcome

91.4% average word error rate improvement over base Whisper across 12 languages. Processes a 2-hour briefing in under 8 minutes. 12 language pairs supported. Fully air-gapped — zero external API calls.

Measured

91.4%

word-error-rate improvement

2 hours → 8 min

audio turnaround

Indian languages

Air-gapped

no external API path

The constraint that shaped it

No audio may leave the building. That rules out every commercial speech API, so the accuracy had to be earned inside the perimeter instead: an open-weight model fine-tuned across twelve languages, sized to a node we could specify and procure, evaluated on a held-out set we built ourselves, and updated over controlled media. One rule about data movement decided the model choice, the throughput target, the evaluation method and the update procedure.

How it was built

How the figure is stated, and what it is not

91.4% is a word-error-rate improvement over the base model, not an absolute accuracy score, and we state it that way deliberately — an absolute number on transcription means little without the audio it was measured on. Evaluation ran language by language on held-out recordings drawn from the same conditions as the work, including poor field audio rather than only clean capture. Twelve languages is twelve evaluation problems: an average across them would hide the weakest, which is the only one worth arguing about.

The operator keeps the audio

Nothing is presented as authoritative on its own. A transcript is delivered segment by segment against the audio it came from, so a reader can select a line and hear it, and a translation sits beside its source-language segment rather than replacing it. Operator corrections are kept. That is the same rule we apply to every generated output on a sovereign deployment: provenance back to a named source, so the person accountable for the briefing can verify a sentence without having to trust the model.

Updates without an inbound path

Two hours of audio in eight minutes is a throughput property of the deployed node, so sizing was proven against sustained batch load before hardware was procured. Acoustic and language models are versioned and shipped as signed offline packages over controlled media; no inbound network route is opened to make an update convenient, and no outbound telemetry leaves. Adding or improving a language is a package, a documented procedure and a re-run of the evaluation set — not a service call to a vendor.

What was hardest to get right

Code-switching. In real speech an Indian language and English alternate inside a single sentence, sometimes inside a single clause, and a model told in advance which language it is listening to will mis-handle the other half. Dialect variation does the same thing at the acoustic level: the written language is one thing, the spoken register in the room is another. Both were handled in the fine-tuning data and the preprocessing chain rather than by asking the operator to pick a language and hope.

Stack, by concern

Speech & language

Whisper fine-tuningMultilingual NLPPyTorch

Serving

FastAPIOn-premise GPU nodeBatch transcription pipeline

Deployment discipline

Air-gappedZero-egressSigned offline model updates

More work

Project SENTINEL

Acoustic Intelligence & Threat Classification

All case studies Start a project