alexakhbar/llmsnarf repository overview

⁠LLM Snarf

LLM Snarf is a lightweight proxy that sits between the LLM provider and the client application. You configure the client's base URL to use point to LLM Snarf instead of the target provider.

Collect and Inspect traces (request / responses) for debugging you agent Provides web_search and web_fetch if using a local provider like llama.cpp. Requires you to also deploy OpenSerp. Shout out to the OpenSerp team: https://hub.docker.com/r/karust/openserp⁠ Allows automatic routing of models to different providers Management UI where you can configure LLM Snarf. Track costs / savings configure token prices and track how much you are saving from running local models, or how much you spend on your API subscriptions

⁠Example Docker Compose stack

services:
  llm-snarf:
    build: .
    container_name: llmsnarf
    hostname: llmsnarf
    image: alexakhbar/llmsnarf:latest
    env_file:
      - .env
    ports:
      - "${PORT}:${PORT}"
    networks:
      - llmsnarf
    environment:
      - LLMSNARF_CONFIG=/app/config
    volumes:
      - ./config:/app/config
      - ./traces:/app/traces
      - ./logs:/app/logs
    depends_on:
      - openserp

  openserp:
    image: karust/openserp:latest
    container_name: openserp
    hostname: openserp
    command: serve -l
    ports:
      - "7000:7000"
    networks:
      - llmsnarf
    environment:
      - PORT=7000

networks:
  llmsnarf:
    driver: bridge