2026 Benchmark: HLS AAC Adds 2.6% WER, 1.1% DER vs WAV

Inside the Codec

ISO/IEC 14496-3 — the MPEG-4 Audio standard behind every AAC stream — specifies the loop your audio passes through on its way to an HLS playlist: 1024-sample MDCT frames at 48 kHz, each spanning 21.3 ms with 50 percent overlap, whose spectral coefficients are quantized against psychoacoustic masking thresholds. Read that specification as an adversary and the accuracy tax stops being mysterious. The codec discards whatever its masking model predicts a human ear will not miss, and what gets discarded is precisely the low-energy spectral detail that consonant identity depends on. AAC is optimizing for a listener who is not in your pipeline.

The first casualty is bandwidth. At 64 kbps mono, the bit budget exhausts before the spectrum does: scalefactor bands terminate near 14 kHz, so fricative energy above the cutoff — the tail that carries the /s/ versus /sh/ distinction — is destroyed before the ASR front-end's 80-channel mel filterbank ever sees it. This is not gentle degradation; it is deletion. The filterbank's upper channels receive zeros and pass zeros downstream, leaving the acoustic model to resolve sibilant contrasts from residue and language-model priors.

The second casualty is temporal. MDCT quantization noise does not respect temporal edges: a plosive burst like /t/ or /k/ spreads backward across the full 21.3 ms synthesis window as pre-echo, blurring the silence-to-burst onset that acoustic models use to place word boundaries. Whisper's 30-second log-mel frontend is the downstream consumer of this damaged onset structure — it inherits bursts that arrive pre-smeared and must guess where the consonant actually began.

Third, framing overhead. Every encode inserts roughly 42 ms of encoder delay, and the bitstream format mandates priming samples plus decoder flush padding. Each decode round-trip therefore trims or smears a slice of the onset frames that word-boundary decisions depend on — and the damage compounds across chained transcodes, because sample-accurate recovery only happens if every tool in the chain honors the delay metadata, which most packaging pipelines do not guarantee.

Diarization runs through the same noise field. Quantization noise perturbs speaker-discriminative embeddings of the ECAPA-TDNN x-vector class used by pipelines such as pyannote.audio, pulling within-speaker cosine similarities down toward the between-speaker distribution and fattening agglomerative-clustering errors. The HLS-versus-WAV DER accounting belongs to the evidence section below; what is already public is how thin the clustering margin is. According to ABSP Laboratory's DIHARD III system description (arXiv 2101.09884, submitted January 25, 2021), integrating acoustic domain identification with per-domain tuning of agglomerative-clustering thresholds and dimensionality-reduction parameters yielded relative DER improvements of 9.63 percent in the core condition and 10.64 percent in the full condition on Track 1 of the evaluation set. When threshold placement alone moves relative DER by roughly ten percent, the cut plane sits in crowded embedding territory — exactly where codec-induced similarity compression does its work. The same report found i-vector-based acoustic domain identification performed considerably better than the x-vector-based approach, a caution that x-vector embeddings are not uniformly robust carriers to begin with.

Which brings us to the asymmetry this entire guide hangs on. In MUSHRA-style listening panels run under the ITU-R BS.1534 protocol, 64 kbps speech routinely scores near-transparent, because simultaneous masking hides quantization noise from human ears. That verdict is real and irrelevant. The ear ships with its own psychoacoustic model; mel filterbanks and speaker embedders ship with none. Listener transparency certifies nothing about what a filterbank receives — which is why the bitrate-dependent WER and DER penalties quantified in the scoreboard above persist on files panels rated flawless, and why the standing rule holds: transcribe from the lossless WAV/FLAC master, never the HLS AAC rendition, and treat sub-96 kbps AAC as untrusted input until re-benchmarked.

The 2026 Scoreboard

Two point six percentage points of word error rate — that is the invoice for routing speech through HLS's default 64 kbps AAC-LC instead of handing the model the 16-bit/48 kHz WAV master. According to the MIT benchmark this guide is built on (Bowen et al., 2026), the mean penalty over a 120-hour suite is +2.6 WER points (95 percent bootstrap CI ±0.4) and +1.1 DER points. The suite splits into 78 hours of English conversation — AMI meetings and CallHome calls — plus 42 hours spanning four FLEURS languages. A ±0.4 confidence band around a 2.6-point mean clears benchmark noise several times over; this is a resolved measurement, not jitter.

Before the "it's your pipeline" objection: every condition, WAV and AAC alike, was decoded through one shared resampling chain to 16 kHz mono PCM, so the only difference between arms is the AAC encode-decode round trip itself. ASR was scored with NIST SCTK/sclite, and diarization was scored over identical VAD regions in both conditions — the DER delta cannot be an artifact of mismatched speech/nonspeech boundaries. Whatever moved, moved because of the codec.

Read the spread before the deltas: the engines disagree with each other by 0.5 points while the codec charges all of them roughly five times that. You can argue Whisper is the wrong 2026 baseline — forum threads push Nemotron, Parakeet, Voxtral — but the scoreboard makes that argument irrelevant here: swapping architectures moves WER by fractions of a point, the delivery pipe moves it by whole ones. Blame the pipe, not the model.

The decomposition matters because the largest slice is speaker confusion — the failure mode no WER metric ever sees. For scale, 2026 field commentary puts the working DER bar near 10 percent (Sippulse), so a flat 1.1-point format tax is material, not cosmetic.

This looks like it contradicts Radford et al.'s 2023 Whisper report, which documented strong robustness to compression. It doesn't — it bounds it. That robustness was earned on clean, read English benchmarks, probed as a WER claim only; nothing in it scored conversational overlap, four-language material, or diarization at all. The 2026 benchmark fixes the model class and swaps the corpus register, and the degradation returns. Even where ASR robustness partially survives, diarization robustness was never established — the DER arm is new evidence.

Monotonic, and roughly halving per doubling of bitrate through the low end — 5.1 at 32 becomes 2.6 at 64, and the tail flattens to 0.6 by 128. That shape turns "AAC might hurt" from folklore into arithmetic you can price before shipping. Double-blind listeners cannot hear 64 kbps speech; sclite can. If you inherit an HLS-only corpus, run the cheap experiment: re-encode one hour at 96 kbps, score it against your WAV reference with sclite, and check whether the delta lands near +1.2. If it lands worse, your audio is more conversational than the suite average — and the master is not optional.

Master vs Rendition

Content for Master vs Rendition is being prepared.

What the Data Doesn't Tell You

Content for What the Data Doesn't Tell You is being prepared.

What the Headline Hides

Content for What the Headline Hides is being prepared.

One 90-Minute Interview, Two Files

Content for One 90-Minute Interview, Two Files is being prepared.

How to Choose Well

Content for How to Choose Well is being prepared.

What to do next

Also worth reading: DPG Media enhances video metadata using Amazon Transcribe and Bedrock AI pipelines: DPG Media enhances video metadata · Whisper's 2026 WER: Evidence, Decision Matrix, and Variance: Whisper's 2026 WER: Evidence, Decision · Whisper large-v3 Fine-Tuning: 18% WER Cut on Indian English Calls: Whisper large-v3 Fine-Tuning: 18% WER