More benchmarks
Fixed public tests of sound recognition, image classification, video understanding and English transcription.
Run status
Sound, images & video
Video checks
The same 32 questions, with no frames, one middle frame or all eight frames. A second run allows a longer answer. This helps separate visual understanding from question clues and answer-format problems.
| Model | No frames 16 output tokens | One frame 16 output tokens | Eight frames 16 output tokens | Eight frames 128 output tokens | No frames 128 output tokens |
|---|
A random choice averages 25%. Only 32 questions are used; small differences are uncertain. These checks do not change the main benchmark scores.
English speech recognition
Compare all speech-to-text models →Fluid Speech Mini & common baselines
Words that were missed, replaced or added, compared with the written transcript. The overall result averages three English tests equally, using the same 400 recordings. German recordings are excluded.
| Model | English WER ↓ | Recordings | Pi 5 time per minute of audio ↓ |
|---|
Timing uses short recordings with the model already loaded.
What these tests measure
ESC-50: 100 five-second clips, two from each of 50 sound categories. This tests sound recognition, rather than transcription. Fold 5 only; the two church-bell clips share an original recording.
CIFAR-10: 100 test images across ten object categories. The original images are only 32 × 32 pixels and are enlarged to 512 pixels. This tests recognition of small images; it is separate from the Vision score.
MVBench: 32 questions about action order and what happens next. Models see eight ordered frames from each clip, without sound. This does not test continuous video or live streaming.
How to read the results
Accuracy is the percentage of correct choices. An answer may be a letter, number, exact label, or an option followed by its matching label. Conflicting answers and unfinished output count as incorrect. A response has a 16-token limit and an 8,192-token input limit.
Pi 5 timings use three repetitions of the same small set of examples: three images, three videos or five sound clips. Memory is the highest engine process memory observed during that run. Missing results remain blank while the queue runs.
These are public, small subsets; close scores can change with a different sample. They are kept separate from the main capability score. Dataset versions, file checksums, prompts and raw answers are saved for repeat runs. Source media stay local; ESC-50 has a non-commercial licence.
Fluid Speech Mini uses its publisher’s mixed 2/4-bit ONNX model and greedy decoder. Its English results are compared with the same recordings used by the existing speech models.
Results table · 19 configurations
Correct choices on 100 ESC-50 sound clips, 100 CIFAR-10 images and 32 MVBench video questions. These small public selections test different skills and are kept separate from the main capability scores. A dash means no matching measurement is available.
| Model | Sound recognition · % ↑ | Image recognition · % ↑ | Video questions · % ↑ |
|---|---|---|---|
| Qwen3.5-0.8B | — | 88 | 16 |
| Qwen3.5-2B | — | 96 | 81 |
| Qwen3.5-4B | — | 93 | 78 |
| Qwen3.5-9B | — | 97 | 72 |
| Gemma 4 E2B IT | 3 | 85 | 41 |
| Gemma 4 E4B IT | 6 | 86 | 41 |
| Ministral 3 3B Instruct | — | 79 | 59 |
| Ministral 3 8B Instruct | — | 78 | 38 |
| Qwen3-VL 4B Instruct | — | 92 | 72 |
| Qwen3-VL 8B Instruct | — | 96 | 72 |
| MiniCPM-V 4.7 1B | — | 88 | 78 |
| LFM2.5-VL-1.6B | — | 91 | 66 |
| LFM2.5-VL-3B | — | 93 | 66 |
| SmolVLM2 256M | — | 26 | 41 |
| SmolVLM2 2.2B | — | 94 | 63 |
| Voxtral Mini 3B | 2 | — | — |
| Qwen2.5 Omni 3B | 39 | — | — |
| Ultravox 0.5 1B | 2 | — | — |
| Qwen2.5 Omni 7B | 45 | — | — |