dockerbuildcom/ai-llama-cpp repository overview

# Run llama.cpp with a model from your local machine
docker run -it --rm \
  -v /path/to/your/models:/models \
  dockerbuildcom/ai-llama-cpp:latest \
  -m /models/your-model.gguf -p "Hello, my name is"
Benefits

Optimized build with OpenBLAS for better performance
Supports both AMD64 and ARM64 processors
Based on distroless container for smallest possible footprint
Regular rebuilds for latest llama.cpp improvements

Model Management
Mount your models directory to /models within the container:
bash-v /path/to/your/models:/models
Example: Interactive Chat
bashdocker run -it --rm \
  -v /path/to/models:/models \
  dockerbuildcom/ai-llama-cpp:latest \
  -m /models/mistral-7b-instruct-v0.2.Q4_K_M.gguf \
  --interactive-first \
  --color
Example: API Server
bashdocker run -it --rm \
  -p 8080:8080 \
  -v /path/to/models:/models \
  dockerbuildcom/ai-llama-cpp:latest \
  -m /models/mistral-7b-instruct-v0.2.Q4_K_M.gguf \
  --server --host 0.0.0.0 --port 8080
More AI Docker images at dockerbuild.com