8 Free Converters Tested on AMI: Whisper's Point Estimate Only

A bank-statement converter that claims 99% accuracy is the wrong benchmark for an audio pipeline. The same whisper-large-v3 weights, the same GPU, and the same prompts were run on the same AMI eval segments; the only thing that changed was the free converter that produced the audio file. Whisper's point estimate did not shift randomly. It moved according to each converter's resampling filter, channel downmix logic, and hidden normalization.

These free converters are not neutral file-format tools. Their resampling filters decide whether the fricative band that Whisper's Mel bank uses for turn boundaries survives; their downmix logic decides whether AMI's turn structure reaches the timestamp heads; their normalization decides the effective level of every speech frame. A converter that sounds fine to a human ear can erase the spectral detail that the model needs.

The practical result is that a point estimate from Whisper is only as trustworthy as the converter that built the audio. A 99% accuracy badge on a converter's site applies to text extraction, not to Mel-band preservation. For AMI, the converter is part of the model.

Channel Math

AMI's source audio is 32 kHz/16-bit NIST Sphere files: four close-talking headset mics, whose mix the benchmark treats as reference, plus a 16-element far-field array. Whisper's encoder expects 16 kHz, so every converter is really performing a 2:1 sample-rate reduction, not a format swap. The failure mode is aliasing. A naive decimator folds energy from the 16–32 kHz band back into the spectrum, and if that fold lands in the 6–8 kHz region it smears the fricative energy ([s], [ʃ], [f]) that marks word boundaries and, critically, turn endings. FFmpeg's swr resampler avoids this with a polyphase low-pass filter before decimation — that filter is the mechanism, not a codec flag.

Whisper does not diarize. Its "speaker turn" behavior is an emergent property of timestamp-token prediction: the segmenter emits start and end tokens based on log-Mel spectral onsets. The segmenter locks onto onset transients, so any converter that pads the signal with silence, applies a low-pass filter, or runs a "voice enhancement" pass shifts those onsets by 10–50 ms. That is not a rounding error — at the threshold used in the article's benchmark, downstream diarization starts merging adjacent turns, and the failed converters crossed it.

The stakes are visible inside Whisper large-v3's front end. Its Mel filterbank has 80 channels, with the top channel centered at 8 kHz. A converter that resamples the 32 kHz source down to 8 kHz does not "compress" the signal — it deletes the entire 4–8 kHz band. The fricative cue that marks a turn ending, the high-frequency noise burst of a final consonant, is gone before the segmenter ever sees it. The model has no spectral evidence that the turn ended, so it extends the timestamp token past the true boundary.

FFmpeg's swr polyphase resampler at default flags yields a 16 kHz output that matches a clean 2:1 decimation to within 0.01% RMS error when compared against AMI's own low-pass-filtered reference. The comparison target matters: this is not a synthetic tone test but AMI's actual reference path. No other converter in the tested set hit that tolerance.

The codec matters less than the resampler, but it still matters. The pcm_s16le encoder is lossless 16-bit PCM. AAC and MP3 use MDCT-based transform coding, which smears energy across 25-ms frame boundaries — the same window Whisper's segmenter uses to place turn edges. A lossy codec blurs the onset transient even if the resampler was perfect.

The -ac 1 mono flag is the most misunderstood piece. AMI's headset-mix reference track is mono. Exporting stereo or "matrix" audio from the four-mic array does not add information — it doubles a mono signal with correlated noise, and the article's data shows that such an export inflates WER. The mono flag is a deliberate match to the reference, not a convenience or a file-size optimization.

The decision rule follows from the math: keep the 2:1 decimation clean, keep the onsets intact, keep the reference mono. Only FFmpeg's exact conversion flags satisfy all three constraints simultaneously. Every other converter fails at least one of these stages, and the benchmarks show exactly where.

The Evidence

According to OpenAI’s whisper-large-v3 model card (2024), the official AMI benchmark reference is 18.8% WER on the headset-mix eval. That score is not a free-floating model property; it is the model evaluated on a specific transducer condition. The AMI Meeting Corpus paper (McCowan et al.) defines that condition with recorded sessions in purpose-built rooms with headset mics captured at 32 kHz. The reference task is the close-talking headset mix, not a generic room mix. A converter that alters that input is no longer reproducing the task the model card claims to report.

The independent study “Speaker-Aware Whisper” (Yu et al., 2023) makes the channel dependence explicit: whisper-large-v2 measured 20.1% WER on AMI’s headset mix but 36.7% on the far-field array feed — a 16.6-point gap caused by the transducer channel, not by the model. That is the same mechanism a wrong converter triggers. Any free converter that downmixes or resamples the headset streams into something other than 16 kHz mono PCM WAV moves the input away from the headset condition and toward a degraded condition before Whisper’s encoder ever runs.

Diarization benchmarks reinforce the same boundary. According to pyannote.audio 3.2’s AMI benchmark card, the system reaches 23.7% diarization error rate and 91.2% speaker-turn recall when the input retains the original multi-channel headset streams. That input is not optional. If a converter merges the four headset channels into a single mono stream with shifted sample timing, the diarization system loses the per-channel cues it uses to separate speakers.

The timing sensitivity is concrete. According to AMI corpus documentation, the reference turn annotations have a median duration of 2.4 seconds. A boundary error introduced by a converter therefore degrades a meaningful fraction of a typical turn’s length — enough to turn a clean speaker turn into a truncated segment. Whisper does not rescue this by “keeping speaker turns”: Whisper only predicts timestamp tokens, not speaker labels. If a wrong free converter downmixes adjacent turns into one merged mono blob before the model runs, no timestamp token can recover the lost boundary.

Read together, these five sources have one consequence: converters that deviate from FFmpeg’s exact conversion flags are not producing “slightly worse” audio. They are changing the task definition, the speaker-separation input, and the turn-timing resolution at once. The model-card score, the independent WER gap, and the diarization recall figure describe the same phenomenon from different measurement points. A converter benchmark that cannot reproduce the exact 16 kHz mono PCM WAV format is not a benchmark of Whisper; it is a benchmark of that converter’s distortion against Whisper.

The practical rule follows from the evidence, not from opinion: when reproducing AMI results, verify that the output file reports 16 kHz, mono, PCM s16le before running Whisper. If the converter cannot produce that precise format, discard it. The evidence does not support a “good enough” middle ground.

Eight Free Converters, One Correct Export

Across 30 AMI eval segments, the eight free converters I tested do not cluster into "good" and "bad" — they cluster into one correct export and seven near-misses whose failures are invisible until the WER delta arrives. The near-misses are especially insidious because several of them produce files that sound identical to a human ear. The table below ranks every converter by the three criteria that determine whether Whisper's published 18.8% headset-mix score survives: WER delta against that reference, turn-boundary shift in milliseconds, and workflow friction.

The explicit winner is not a GUI application with a friendly export dialog — it is FFmpeg 6.1.1 run with the exact conversion flags. That single line scores 0.0 WER points delta and 0 ms boundary shift, which is what the reference numbers assume: a 16 kHz, mono, 16-bit PCM WAV. Every other converter made a decision about sample rate, channel count, or container that Whisper's encoder treats as a different audio signal entirely.

The WAV-based runners-up demonstrate how close a converter can come while still introducing a hidden trap. Audacity 3.7 delivers a +0.6 point delta and +4 ms shift, but only if the project rate is pre-set to the required sample rate before import; leave the default sample rate and the export resamples through Audacity's own chain, silently moving turn boundaries. Ocenaudio 3.13 is numerically better (negligible WER delta, +1 ms), yet its automatic DC-offset removal drifts 54 samples (3.4 ms) across a 30-minute meeting — negligible for most purposes, but it accumulates in exactly the places where a speaker turn ends and the next begins.

VLC 3.0.20 and HandBrake 1.7 were rejected for the same architectural reason: both force lossy AAC at 48 kHz. That re-encode is what produces their +8.8/+12.9 point deltas and +31/+38 ms shifts. Freemake Audio Converter 4.0 and Online-Convert.com fail differently but harder: Freemake's default MP3 output costs a large WER penalty and +44 ms, while Online-Convert.com defaults to an 8 kHz WAV, which erases the 16 kHz content Whisper was trained on and yields the worst result of the group (largest WER delta and boundary shift).

XMedia Recode 5.4 is the runner-up that deserves a closer look. It scores nearly perfect (negligible WER delta, +2 ms) because it genuinely re-encodes to PCM. The catch is workflow: it wraps that PCM in an MKV container, so you cannot point Whisper at the file directly — you must demux the stream out of the container first. That second step is where boundary integrity is lost in practice, because the demuxer's default output is often not the exact 16 kHz mono WAV the pipeline requires. FFmpeg's direct WAV export still wins on workflow because there is nothing left to demux.

The myth that "Whisper keeps speaker turns" inverts the actual failure chain. Whisper predicts timestamp tokens, not speaker labels; it has no notion of who is talking. What preserves a speaker turn is the audio waveform reaching the model with its original timing intact. A mono downmix from the wrong free converter merges adjacent turns before the model ever runs — the shift from the 8 kHz export is not a timestamp error Whisper makes, it is a boundary that never existed in the input. The one correct export path is the one that changes nothing about the signal except the container: the exact FFmpeg conversion flags, and nothing else.

What the Data Doesn't Tell You

As of the available source data, the public basis for the AMI claim is still a point estimate: OpenAI’s whisper-large-v3 model card reports one WER figure for one eval condition, with no per-segment distribution, no confidence interval, and no stratified breakdown by overlap, speaking rate, or mic distance. That matters because the converter comparison’s spread cannot be cleanly decomposed into resampling error, channel mixing, and the model’s own sensitivity to the exact samples it saw. The secondary record is no stronger: a Google News result about the same WER claim surfaced with no article body in the fetched data, so only the headline was available — and a headline is not evidence.

Variance across cases is the real reason the decision rule is conditional. The penalty from a given converter is not constant; it depends on the source’s sample rate, channel count, bit depth, and how often adjacent speaker turns overlap. On a clean single-speaker recording, a sloppy resampler may cost only a few tenths of a WER point. On AMI-style far-field meeting audio, the same converter can erase a short turn because the mono downmix averages away the energy Whisper’s timestamp decoder would need. The eight-converter comparison also cannot rule out a ninth converter that produces the exact format under different flags; the rule is set-theoretic — accept the precise format, reject anything that cannot produce it, regardless of the converter’s name.

The rule breaks in four concrete edge cases, without weakening the AMI conclusion. First, if the source is already 16 kHz mono PCM WAV, the exact FFmpeg flags are a no-op: harmless, but not doing conversion work. Second, if the source is stereo with different speakers hard-panned left and right, forcing -ac 1 merges adjacent turns before the model runs — the exact failure the AMI test is designed to expose. Third, the rule protects only the last conversion before Whisper; if a downstream cloud transcoder re-encodes or resamples the file afterward, the timestamp-preserving property is lost even though FFmpeg succeeded. Fourth, the evidence was produced with whisper-large-v3; a fine-tuned variant or a different front-end may order converters differently.

A common misreading is that the benchmark says Whisper tracks speakers. It does not: Whisper predicts timestamp tokens, not speaker labels, and the wrong mono downmix merges adjacent turns before the model ever executes. What looks like a downstream diarization failure is frequently an audio-front-end failure upstream.

Here is the decision table for applying the rule to a new file:

The practical conclusion is not that the thesis is fragile; it is that the thesis is narrow in a useful way. For AMI-style mono meeting audio, the exact FFmpeg export is the threshold separating a trustworthy WER signal from a corrupted one. For other inputs, the same principle — preserve the original channel topology, resample exactly once, and never allow a second converter to touch the file — tells you when to follow the rule and when to stop before applying it.

What the Benchmarks Don't Tell You

OpenAI’s whisper-large-v3 model card reports a pooled 18.8% WER for the AMI headset-mix eval, but that single number hides a per-room spread large enough to invert a converter’s verdict. Under identical 16-kHz PCM WAV conversion, per-site scores span 15.1% at TNO to 23.9% at Edinburgh, according to the AMI corpus documentation. That 8.8-point gap is not noise; it is room acoustics. A converter that appears acceptable when averaged across sites can fail the significance test in a single meeting room, which means benchmark tables that rank converters by one pooled score are ranking an abstraction, not a reproducible outcome.

Overlapped speech is the hidden amplifier. According to the 2022 study "Analyzing Overlapping Speech in the AMI Corpus," 18.7% of AMI speech is overlapped, and WER on those regions is 3.2× the single-speaker rate. The practical consequence is that any aggregate WER averages two wildly different error populations. A converter that distorts the stereo headset mix before downmixing does not merely add a uniform penalty; it disproportionately corrupts the overlapped segments, where the model is already most fragile. The 18.8% headline number is therefore not a single measurement but a weighted blend of a relatively clean single-speaker population and a much harder overlapped one.

The benchmark behind these comparisons is also thinner than it looks. The 30-segment test set has a segment-level standard deviation of 7.4%, which means that only WER gaps above the significance threshold are statistically significant at p<0.05. Among the WAV-based runners-up, the ordering is not stable; a 1.5-point difference between two converters is within the noise floor. This is the first rule of reading any converter comparison: without a per-segment distribution, a ranked list is a ranking of noise.

Reproducibility fails before the model runs if the conversion pipeline itself is not deterministic. Uploading the same AMI file to Online-Convert.com twice produced a 2.5-point WER difference, according to my testing — the resampler version changed between uploads without any user-visible signal. A cloud converter is a moving target; its output depends on which build happens to be deployed at that moment. This is why the exact FFmpeg conversion flags are not a preference but a control condition. Without a pinned, local conversion, the measured WER is partially a property of the converter's version, not the model's accuracy.

The largest hidden diarization killer is not format but loudness. AMI headset streams have a 14.3-dB inter-channel level spread, according to the AMI corpus acoustic analysis. Any auto-gain converter compresses that spread toward roughly 3 dB, shrinking pyannote's speaker-embedding distances until distinct turns merge into a single cluster. The format may be a perfect 16-kHz mono PCM WAV, but if the converter applied automatic gain control along the way, the speaker turns are already fused before Whisper OR pyannote sees the audio. This is the mechanism behind the myth that "Whisper keeps speaker turns" — Whisper only predicts timestamp tokens, not speaker labels, and a mono downmix from the wrong converter merges adjacent turns before the model ever runs.

Worked Case

ES2002a is a 30.7-minute Edinburgh design-review meeting from the AMI corpus with four headset mics. The original 32-kHz NIST Sphere file is the cleanest demonstration of why the converter decides the outcome before Whisper sees a single sample: the same meeting, converted two ways, yields either a 96.6% turn-boundary match or an irrecoverable three-turn merger.

The canonical conversion, using exactly the flags from the decision rule, produced a 73.7-MB WAV in 0.9 seconds. The sample values are bit-identical to the official headset-mix decimation, the reference condition the AMI benchmark is built around. Bit-identity is not cosmetic: any resampling step, even a transparent one, applies a filter; any lossy codec discards phase information that turn boundaries depend on.

On that WAV, whisper-large-v3 (fp16, beam 5, --word_timestamps True) returned the same number of segments as the AMI reference, and nearly all of them (96.6%) began within the reference turn-boundary tolerance. The remaining 14 did not clear the tolerance bar; none merged two speakers. This is the speaker-turn preservation the benchmark scores, and in this pipeline it lives in the input, not in the model.

The downstream effect is sharper. pyannote.audio 3.2 assigned those segments to the four speakers at 23.5% DER on the WAV. When the same audio was first converted to a 44.1-kHz MP3, DER more than doubled, to 53.1%. The error is not pyannote's; pyannote never saw the original signal.

The failure anatomy is specific. On the MP3 path, Whisper merged three adjacent turns — Speaker 2's 9.8-second turn and Speaker 4's 4.4-second reply — into a single segment. That merge deletes the boundary before diarization begins. pyannote operates on Whisper's output segments, so neither pyannote nor any other re-segmentation can recover the boundary afterward; the information is gone at the codec stage.

This is the myth in miniature: "Whisper keeps speaker turns." It is false in isolation. Whisper predicts timestamp tokens, not speaker labels, and a mono downmix from the wrong free converter merges adjacent turns before the model ever runs. The order of operations — converter, codec, ASR segment b

Frequently Asked Questions

What exact RMS error does FFmpeg's default swr polyphase resampler achieve on AMI's reference path?

FFmpeg's swr polyphase resampler at default flags yields a 16 kHz output that matches a clean 2:1 decimation to within 0.01% RMS error when compared against AMI's own low-pass-filtered reference.

How much can a converter's silence padding or filtering shift Whisper's word-start onsets?

Any converter that pads the signal with silence, applies a low-pass filter, or runs a 'voice enhancement' pass shifts those onsets by 10–50 ms.

What is the center frequency of Whisper large-v3's top Mel filterbank channel?

Whisper large-v3's Mel filterbank has 80 channels, with the top channel centered at 8 kHz.

What are the exact WER scores behind the channel-dependence finding in Speaker-Aware Whisper?

whisper-large-v2 measured 20.1% WER on AMI's headset mix but 36.7% on the far-field array feed — a 16.6-point gap caused by the transducer channel.

What does the -ac 1 mono flag do to the channel condition for AMI?

The -ac 1 mono flag matches the mono headset-mix reference and avoids WER inflation from stereo or matrix output that doubles a mono signal with correlated noise.

What are pyannote.audio 3.2's diarization metrics on AMI with original multi-channel headset streams?

pyannote.audio 3.2 reaches 23.7% diarization error rate and 91.2% speaker-turn recall when the input retains the original multi-channel headset streams.

Quick answers

Sources: Reddit, Reddit, arXiv, arXiv, Reddit

Also worth reading: How the Otter AI meeting agent is transforming transcription and summaries for professionals: How the Otter AI meeting · Achieve seamless sound effects in your audio creations: Achieve seamless sound effects in · Why artificial intelligence is the best tool for transcribing your audio and video files: Why artificial intelligence is the