W
Whisper API
Navigation

The Whisper API uses GGML-format model files (.bin) compatible with whisper.cpp. Files live under MODELS_DIR (default models/). The server exposes a fixed registry in code (model_names in app/utils/constant.py): GET /v1/models lists registry entries whose matching .bin file exists on disk. On startup, missing registry models can be downloaded automatically (see app/utils/checks.py).

Available Model Sizes

| Model | Parameters | Disk Size | Relative Speed | Best For | |-------|-----------|-----------|----------------|----------| | tiny | 39M | ~75 MB | Fastest | Quick prototyping, low-resource devices | | tiny.en | 39M | ~75 MB | Fastest | English-only, highest speed | | base | 74M | ~142 MB | Fast | Good balance for English | | base.en | 74M | ~142 MB | Fast | English-only with better accuracy | | small | 244M | ~466 MB | Medium | Multi-language, good accuracy | | small.en | 244M | ~466 MB | Medium | English-only, recommended for production | | medium | 769M | ~1.5 GB | Slow | High accuracy, multi-language | | medium.en | 769M | ~1.5 GB | Slow | English-only, near-best accuracy | | large-v2 | 1550M | ~3.1 GB | Slowest | Maximum accuracy, all languages | | large-v3 | 1550M | ~3.1 GB | Slowest | Latest full multilingual model | | large-v3-turbo | 809M | ~1.6 GB | Fast | Pruned decoder (large-v3 quality at ~4× speed) | | large-v3-turbo.q5 | 809M | ~574 MB | Very Fast | Recommended multilingual model (q5_0) | | large-v3-turbo.q8 | 809M | ~874 MB | Fast | High-precision quantized large-v3-turbo (q8_0) | | silero-v6.2.0 | VAD | ~885 KB | Instant | Silero Voice Activity Detection (vad=true) |

Quantization Formats

GGML models support multiple quantization levels to trade accuracy for speed and memory:

| Format | Bits | Size Reduction | Quality | Recommended | |--------|------|---------------|---------|-------------| | f32 | 32-bit | 1x (baseline) | Highest | No (too large) | | f16 | 16-bit | ~0.5x | Very High | Research only | | q8_0 | 8-bit | ~0.25x | High | Yes, if memory allows | | q5_1 | 5-bit | ~0.16x | Good | Best overall | | q5_0 | 5-bit | ~0.16x | Good | Alternative to q5_1 | | q4_1 | 4-bit | ~0.125x | Fair | Max speed priority | | q4_0 | 4-bit | ~0.125x | Fair | Smallest size |

Downloading Models

Using the Built-in CLI (app.cli)

You can list all registered models and download any model directly via the CLI without manual curl commands:

# List all registered models, aliases, and installation status
python -m app.cli models

# Download large-v3-turbo (q5_0) or Silero VAD
python -m app.cli download large-v3-turbo.q5
python -m app.cli download silero-v6.2.0

From Hugging Face

Official GGML models are available on the whisper.cpp Hugging Face repository:

# Download tiny.en (q5_1 quantized)
curl -L -o models/ggml-tiny.en-q5_1.bin \
  "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny.en-q5_1.bin"

# Download small.en (q5_1 quantized)
curl -L -o models/ggml-small.en-q5_1.bin \
  "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en-q5_1.bin"

# Download base.en (full precision)
curl -L -o models/ggml-base.en.bin \
  "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin"

From the Setup Script

The setup_whisper.sh script automatically downloads the tiny.en model if none exists:

./setup_whisper.sh

Adding a New Model

    1. Download the .bin file from Hugging Face or another source

    2. Move it to the models/ directory:

      mv ggml-small.en-q5_1.bin models/

    3. Register the model in app/utils/constant.py (model_names, and optionally model_info / model_urls) so the API knows the model query id and filename.

    4. Restart the API server:

      # Stop the running server (Ctrl+C)
uvicorn app.main:app --host 0.0.0.0 --port 7860

    5. Verify the model appears (no auth required):

      curl -X GET 'http://localhost:7860/v1/models'

Model File Naming

The API uses the model filename (without ggml- prefix and .bin suffix) as the model identifier in API requests:

| File Name | API model Parameter | |-----------|----------------------| | ggml-tiny.en.bin | tiny.en | | ggml-model-whisper-small.en-q5_1.bin | model-whisper-small.en-q5_1 | | ggml-base.en.bin | base.en |

Model Directory Configuration

The model directory is configured via the MODELS_DIR environment variable:

# .env
MODELS_DIR=./models

Only registered models (see model_names) with a present file under MODELS_DIR are listed and accepted by /v1/listen.

← Previous

Live Streaming (WebSocket)

Next →

Docker Deployment