GoModel

GoModel - The last AI gateway you will ever need

GoModel is the fastest and the most resource-efficient AI Gateway (the self-reproducible benchmarks). It's an alternative to LiteLLM (which was hacked recently) and Portkey (which is no longer maintained on GitHub).

(click on the animation ↑ to see the live demo)

GoModel saves you money and nerves.

Money - because you can remember the responses on this layer (caching), track your spending and do tricks like prompt compression and intelligent routing.

Nerves - because we strive to achieve good quality and reliability. Our ambition is to be the last AI gateway you will need - the most reliable, resource-optimal, feature-rich and fast.

Quick Start

Step 1: Install and start GoModel

Docker

docker run --rm -p 8080:8080 \
  -v gomodel_data:/app/data \
  enterpilot/gomodel

macOS / Linux

curl -fsSL https://gomodel.enterpilot.io/install.sh | sh
# OPENAI_API_KEY="your-openai-key" # (optional)
gomodel

Windows (PowerShell)

irm https://gomodel.enterpilot.io/install.ps1 | iex
# $env:OPENAI_API_KEY = "your-openai-key" # (optional)
gomodel

ℹ️ Configure GoModel with .env, a config.yaml file, or manage the most important settings directly in the dashboard.

ℹ️ See .env.template for the complete list of environment variables, including all available providers.

Step 2: Open the dashboard and add a provider

http://localhost:8080/admin/dashboard

Step 3: Make an API call

curl http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5-chat-latest",
    "input": "Hello!"
  }'

GoModel and official SDKs

GoModel accepts requests in two compatible formats:

• OpenAI-compatible at /v1
• Anthropic-compatible at /v1/messages

The official SDKs therefore work unchanged. Configure their base URLs as follows:

• OpenAI SDK: http://localhost:8080/v1
• Anthropic SDK: http://localhost:8080 (the SDK appends /v1/messages)

List of Supported LLM Providers

• OpenAI
• Anthropic
• xAI (Grok)
• Google Gemini
• Cohere
• Vertex AI
• DeepSeek
• Groq
• Fireworks AI
• Meta (Muse Spark)
• OpenRouter
• Z.ai
• Alibaba Cloud Model Studio (Bailian)
• Kilo AI
• Eden AI
• MiniMax
• Xiaomi MiMo
• OpenCode Go
• Azure OpenAI
• Oracle
• Ollama
• SGLang
• vLLM
• llm-d
• Amazon Bedrock Runtime and Bedrock Mantle
• ChatGPT (the Codex backend) and Claude
• ElevenLabs (text-to-speech and speech-to-text)
• Jev (TypeSafe System One decision API) and self-hosted Kev, also behind OpenAI's Decisions API
• All OpenAI-compatible providers

See the Providers Overview for the full per-provider feature matrix.

Docker Compose

Infrastructure only (Redis, PostgreSQL, MongoDB, Adminer - no image build):

cp .env.template .env
# Add your API keys to .env
docker compose up -d
# or: make infra

Full stack (adds GoModel + Prometheus; builds the app image):

docker compose --profile app up -d
# or: make image

API docs

• API Endpoints
• Admin API Endpoints

Gateway Configuration

GoModel resolves configuration in the following order, with each source overriding those to its left:

Good defaults → config.yaml → .env → exported environment variables

See the Configuration reference for the full list of settings.

Features

• Caching - exact and semantic response caching, so repeated prompts cost nothing
• Cost tracking - per-request cost estimates, usage analytics, and spending breakdowns in the dashboard
• Budgets - hard spend limits per user, team, or key
• Rate limits - requests, tokens, and concurrency caps per user path, provider, or model
• Usage API - clients check their own usage, remaining budget, and rate-limit headroom with the key they already use for inference
• Virtual models - aliases and load balancing (round-robin or cost-based) behind stable model names
• Session keeping - detect a client session and pin it to one target and provider key, so provider prompt caches stay warm and audit logs read as threads
• Failover - automatic rerouting to backup providers, with retries and circuit breakers
• Labelling - tag requests from HTTP headers or API keys and break down usage by label
• User paths - hierarchical scoping of keys, model access, budgets, usage, and audit logs
• Model access control - per-group, per-user, and per-key model allowlists that intersect down the user-path tree
• MCP gateway - aggregate your MCP servers behind one authenticated endpoint
• Passthrough API - provider-native APIs under /p/{provider}/..., with GoModel auth and tracking
• Audio and image APIs - OpenAI-compatible text-to-speech, transcription, and image generation and editing with the same access rules, budgets, and cost tracking as chat
• Provider replay state - preserves Gemini thought signatures and Anthropic thinking blocks across turns, APIs, and providers
• Guardrails - request and response policies enforced at the gateway
• Plugins - one contract for guardrails, response and stream filters, header edits, and routing strategies; built in, compiled in, or loaded from a .so at startup
• Workflows - versioned per-request policies that scope cache, budgets, audit logging, guardrail phases, and failover by user path, provider, or model
• Provider key rotation - round-robin over multiple API keys to lift per-key rate limits
• Observability - Prometheus metrics, OpenTelemetry traces, audit logs, and live request streaming in the dashboard
• Playground - try any model or virtual model from the dashboard and inspect the exact request and response JSON

GoModel Pro

GoModel Pro is the commercial build: the same gateway, configuration, and dashboard, with licensed extensions.

• Prompt compression - remove repeated and structural context before it reaches the provider, without changing the request shape
• Intelligent routing - classify each request as easy or hard, then pick the healthiest and cheapest provider in that tier
• OIDC single sign-on - protect the dashboard with your identity provider using Authorization Code flow with PKCE

More in the documentation...

Roadmap

See the roadmap for GoModel Pro and the upcoming 0.2.0 release.

Sponsors

Community

We are on Discord. Feel free to stop by and tell us what you think about GoModel.