evilfreelancer/whisper-server repository overview

⁠Whisper.cpp API Webserver in Docker

Whisper.cpp HTTP transcription server with OAI-like API in Docker.

This project provides a Dockerized transcription server based on whisper.cpp⁠.

Русский⁠ | 中文⁠ | English

⁠Features

• Dockerized whisper.cpp HTTP server for audio transcription
• Configurable via environment variables
• Automatically converts audio to WAV format
• Automatically downloads required model on startup
• Can quantize any Whisper model to the required type on startup

⁠Requirements

Before you begin, ensure you have a machine with an GPU that supports modern CUDA, due to the computational demands of the docker image.

• Nvidia GPU / Intel Arc
• CUDA / oneAPI
• Docker
• Docker Compose
• Nvidia Docker Runtime (Nvidia only)

For detailed instructions on how to prepare a Linux machine for running neural networks, including the installation of CUDA, Docker, and Nvidia Docker Runtime, please refer to the publication "How to Prepare Linux for Running and Training Neural Networks? (+ Docker)⁠" on Russian.

⁠Installation

1. Clone the repo and switch to sources root:

   git clone https://github.com/EvilFreelancer/docker-whisper-server.git
   cd docker-whisper-server
   

   Copy

2. Copy the provided Docker Compose template:

   cp docker-compose.dist.yml docker-compose.yml
   

   Copy

Example for Intel Arc cards:

x-shared-logs: &shared-logs
   logging:
      driver: "json-file"
      options:
         max-size: "10k"

services:
  whisper-intel:
    restart: "unless-stopped"
    build:
      context: ./whisper
      dockerfile: Dockerfile.intel  
      args:
        - WHISPER_VERSION=v1.7.4
    devices:
       - /dev/dri
    volumes:
      - ./models:/app/models
    ports:
      - "127.0.0.1:9000:9000"
    environment:
      WHISPER_MODEL: large-v3-turbo
      WHISPER_MODEL_QUANTIZATION: q4_0
    <<: *shared-logs

1. Build the Docker image:

   docker-compose build
   

   Copy

2. Start the services:

   docker-compose up -d
   

   Copy

3. Navigate to http://localhost:8080⁠ in browser:

   

⁠Endpoints

⁠/inference

Transcribe an audio file:

curl 127.0.0.1:9000/inference \
  -H "Content-Type: multipart/form-data" \
  -F file="@<file-path>" \
  -F temperature="0.0" \
  -F temperature_inc="0.2" \
  -F response_format="json"

⁠/load

Load a new Whisper model:

curl 127.0.0.1:9000/load \
   -H "Content-Type: multipart/form-data" \
   -F model="<path-to-model-file-in-docker-container>"

⁠Environment variables

Basic configuration

⁠Links

• whisper.cpp⁠
• server example⁠ of whisper.cpp

⁠Citing

[Pavel Rykov]. (2024). Whisper.cpp API Webserver in Docker. GitHub. https://github.com/EvilFreelancer/docker-whisper-server

@misc{pavelrykov2024whisperapi,
  author = {Pavel Rykov},
  title  = {Whisper.cpp API Webserver in Docker},
  year   = {2024},
  url    = {https://github.com/EvilFreelancer/docker-whisper-server}
}