> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kair.is/llms.txt
> Use this file to discover all available pages before exploring further.

# Transcription

> Local whisper.cpp and Scaleway remote inference — how they differ and when to use each.

## Clients

Two transcription backends are supported, selected via `transcription.client` in `config.yml` (switchable at runtime via the admin UI):

| Client | Value | How |
| - | - | - |
| Local whisper.cpp | `local` | Runs inference in-process; loads model at worker startup |
| Scaleway | `scaleway` | Posts audio to Scaleway Generative APIs `/audio/transcriptions` |

See [Configuration](https://docs.kair.is/platform/configuration.md) for all config keys.

## Silence hallucinations (local whisper.cpp)

Over silence or room noise, whisper.cpp can emit stock subtitle phrases ("Thank you", "Thanks for watching") and repeat them. The transcription worker mitigates this in layers:

1. **Decode-time** (`transcription.local` in config): `whisper_no_context` (default on) disables conditioning on previous text; optional Silero **VAD** (`local.vad`, model on the `ai_models` volume) skips non-speech audio; `suppress_non_speech_tokens`, logprob/entropy/no-speech thresholds, and temperature fallback match whisper.cpp defaults unless overridden.
2. **Post-decode** (`transcription.hallucination_filter`): drops segments with very low confidence or high `no_speech_prob`; removes known stock phrases when the segment window is quiet or low-confidence. A segment that is only back-to-back repeats of one or more stock phrases ("Thank you. Thank you.") counts as that phrase, matched on whole words; segments starting with a prefix such as "Untertitel" (German subtitle credits) also match. Genuine short "thank you" with strong confidence is kept.
3. **Collapse** (`kair_voice_core::repeat_collapse`): consecutive identical segments (run ≥ 3) and intra-segment comma-clause loops are collapsed before persistence.

The same `hallucination_filter` block applies to Scaleway responses (confidence from `avg_logprob`; no `no_speech_prob` on the remote path).

## Switching behaviour

* `local` → `scaleway`: takes effect per-job, no restart needed.
* `scaleway` → `local`: requires a worker restart — whisper.cpp models are only loaded when the worker boots with `client: local`.

## Audio decoding

Audio is decoded in-process with symphonia (WAV, MP3, AAC/ADTS, FLAC, Vorbis). If symphonia rejects the file — multi-frame ADTS AAC, truncated streams, MP4/M4A containers — the worker re-decodes it through `ffmpeg -err_detect ignore_err` into a 16 kHz mono WAV and retries. The job fails with `Failed to load audio` only when both decoders fail; the message names both errors. Both worker images ship `ffmpeg`.

S3 downloads retry transient failures (3 attempts) unless the object is missing or access is denied.

## Upload size limits

The Scaleway backend uploads the normalised 16 kHz mono WAV (\~1.9 MB per minute). When that exceeds `transcription.scaleway.max_upload_mb` (default 25), it is re-encoded as 32 kbit/s mono MP3 (\~14 MB per hour) first. Only audio still over the limit after compression fails, with `Remote transcription upload too large`.

Separately, the API caps single-file session audio uploads at 200 MiB; long sessions must use chunked upload.

Config resolver: `api/crates/transcription/src/settings.rs`.

## Scaleway

Credentials live in `ai_inference.scaleway` (`base_url` + `api_key`); model names live in `transcription.scaleway`. The worker still POSTs to `{base_url}/audio/transcriptions`. If Scaleway rejects `verbose_json`, the client retries with `json` (untimed single segment if no `segments` array is returned).

Dedicated Whisper deployments should use `model: whisper-large-v3` and a `base_url` ending in `/v1` (not `/audio/transcriptions`).

## Name redaction (NER)

Person names are redacted at transcript time, before segments are stored, so summaries, strands and the topic graph only ever see `<PERSON>`. The regex redactor (emails, phones, IBANs, ...) runs first, then an ONNX NER model (`transcription.pii.ner`, see [configuration](https://docs.kair.is/platform/configuration.md)) runs over the whole transcript in overlapping 256-token windows. Names it detects are also replaced wherever they recur in the same transcript, including German genitives ("Annas Vorschlag"). Locations and organisations are opt-in (`kinds`).

* Fail-closed: a model that cannot load stops the worker; an inference error fails the transcription job. There is no silent skip.
* Pasted summaries get the same two passes (crate `kair-voice-pii`): the rag worker redacts a paste before asking the model to split it into topics. It reads `transcription.pii`, always runs NER on CPU (the main image ships CPU ONNX Runtime; `device` only applies to the transcription worker), and fails the job rather than prompt on unredacted text. Uploaded documents are not redacted.
* The same step runs for remote providers (staging/dev), after the provider returns text.
* Live chunks are redacted one chunk at a time, so a name only spoken in a chunk of its own can be missed there; the final pass sees the whole recording.
* Summaries are told that `<PERSON>` is a redacted name and must become a role or "a participant". A stray `<PERSON>` in a summary is rewritten to `[participant]`.
* Run `device: cuda` only after checking the ONNX Runtime GPU build has kernels for your card (`cuobjdump --list-elf libonnxruntime_providers_cuda.so | grep sm_61` for the GTX 1080); otherwise use `device: cpu`, which is usually fast enough.

## GPU notes (CUDA image)

`api/Dockerfile.transcription-cuda` builds whisper.cpp for one architecture (`--build-arg CUDAARCHS=61` for the GTX 1080) and bundles Microsoft's ONNX Runtime GPU build for the NER model (`ORT_DYLIB_PATH`). Whisper tuning lives under `transcription.local` (`use_gpu`, `gpu_device`, `flash_attn`, `beam_size`, `warm_up`); `nr_of_threads_for_transcription` still sets the CPU threads whisper.cpp uses beside the GPU, so keep it low (1-2) on the GPU worker. Quantised ggml models (`q8_0`, `q5_0`) cut VRAM; compare word error rate on invented audio before switching.

## Speaker labels

The transcription worker does not diarise. Whisper segments (local, Scaleway) carry no speaker label. Labels exist only for sources that supply them: Recall (`P{id}` labels reconciled onto final-pass segments) and phone/track names. Speaker claims & relabelling still operate on the labels present on segments.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.