sayanmohsin/go-feather-route repository overview

⁠Go Feather Route

⁠A fast, featherweight, OpenAI-compatible model-routing gateway written in Go

Go Feather Route is a small Go 1.27.1 gateway for routing chat and embeddings requests to OpenAI-compatible provider endpoints. It gives applications one authenticated boundary for provider credentials, model aliases, request limits, retries, timeouts, and streaming responses.

It is designed for small VMs, edge services, homelabs, containers, and teams that want a focused, standard-library-oriented routing layer without a database, queue, dashboard, or large platform runtime.

Documentation⁠ · API reference⁠ · Benchmarks⁠ · Docker Hub⁠

⁠Capabilities

• OpenAI-compatible Chat Completions and embeddings requests and errors.
• Configurable provider base URLs and model aliases.
• Ordered configured fallbacks, retryable-status routing, and provider cooldowns.
• Non-streaming and Server-Sent Events streaming responses.
• Bounded request bodies, timeouts, retries, and concurrent work.
• Liveness, readiness, status, model status, Prometheus-style metrics, and opt-in private runtime diagnostics.
• Provider credentials supplied only at runtime through environment injection.
• Optional sanitized token, cost, latency, and routing usage events for Cloud.
• Static, non-root, multi-architecture Docker images.
• Planned Thingd MCP integration as a separate capability boundary, never a core dependency.

⁠Architecture

flowchart LR
    client[OpenAI-compatible client] -->|Bearer token| router[Go Feather Route]
    router -->|model alias and policy| providerA[Provider endpoint A]
    router -->|model alias and policy| providerB[Provider endpoint B]
    router -. planned, separate boundary .-> mcp[Thingd MCP connector]

The router does not embed Thingd and does not access databases directly. The planned connector will use the authenticated Thingd MCP boundary when added.

⁠Quickstart

Prerequisites: Go 1.27.1 and provider credentials.

cp .env.example .env
export GOFEATHERROUTE_API_KEY=gateway-key
export OPENAI_API_KEY=your-key
go run ./cmd/go-feather-route

Check health and models:

curl http://127.0.0.1:4000/health/liveliness
curl -H 'Authorization: Bearer gateway-key' \
  http://127.0.0.1:4000/v1/models

Send a chat request:

curl http://127.0.0.1:4000/v1/chat/completions \
  -H 'Authorization: Bearer gateway-key' \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hello."}]}'

Try streaming:

curl -N http://127.0.0.1:4000/v1/chat/completions \
  -H 'Authorization: Bearer gateway-key' \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Tell me a short story."}]}'

For local stalled-stream or goroutine investigations, enable the separate loopback-only diagnostics listener:

GOFEATHERROUTE_PPROF_ADDR=127.0.0.1:6060 go run ./cmd/go-feather-route
curl 'http://127.0.0.1:6060/debug/pprof/goroutineleak?debug=1'

Diagnostics are disabled by default and are not available through port 4000. See the health and operations guide⁠ before enabling them.

⁠Docker

docker run --rm -p 4000:4000 \
  -e GOFEATHERROUTE_API_KEY=gateway-key \
  -e OPENAI_API_KEY=your-key \
  sayanmohsin/go-feather-route:0.1.0

Use Doppler, your deployment secret manager, or an equivalent runtime injector for production credentials. Never put real credentials in a Dockerfile, .env.example, committed Compose file, or image layer.

⁠Configuration model

Configuration precedence is supported CLI flags (-config, -addr) → environment → YAML file → safe defaults. Non-secret defaults belong in config/defaults.yaml; credentials belong in environment variables and can be injected by Doppler or CI. See the environment guide⁠ and configuration guide⁠. Model names and provider mappings belong in model_list and route_rules, not in application code. The same OpenAI-compatible SSE contract can therefore route DeepSeek, Ollama, OpenAI, or a future provider by configuration.

⁠Resource profile

The benchmark harness compares the Go gateway with a pinned LiteLLM image against the same deterministic fake provider. It records latency, throughput, CPU, memory, I/O, process count, cgroup peaks, and OOM state where the host exposes those measurements. The homepage shows a concise summary; the full benchmark methodology and results⁠ explain platform and architecture context before the numbers are interpreted.

A preliminary Thingd Cloud canary returned 10/10 successful chat requests through both gateways and showed lower Go Feather Route p95 latency and streaming time-to-first-byte in that sample. Re-run the documented comparison after changing routes or provider models; the canary is not a substitute for production monitoring. See the Cloud canary results⁠.

⁠Long-term direction

Go Feather Route is intended to grow into a small operational boundary for model access: more OpenAI-compatible providers, multimodal requests, per-tenant quotas, usage metrics, health-aware routing, graceful degradation, memory-aware deployment profiles, and a planned Thingd MCP data boundary. The standalone router remains useful without Thingd.

⁠Documentation

• Documentation site⁠
• Getting started⁠
• API reference⁠
• OpenAPI contract⁠
• Providers and routing⁠
• Environment configuration⁠
• Streaming⁠
• Health and operations⁠
• Docker deployment⁠
• Benchmarks⁠
• LiteLLM compatibility contract⁠
• Compatibility gap matrix⁠
• Planned Thingd MCP boundary⁠
• Contributing⁠
• Security⁠

⁠Development

make tools
make check
make bench

See coding standards⁠ for the project contract.

⁠License

Go Feather Route is available under the Apache License 2.0⁠.