Local AI MCP

Local AI MCP

One MCP server for every local model runtime on your machine. It knows what fits your GPU, and it never sends a prompt to the cloud unless you tell it to.

Local AI MCP gives your agent (Claude, Cursor, VS Code, and others) one control plane for Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM and Foundry Local. Your agent can see what's installed and loaded, check what will actually fit in your VRAM, get models ready, and hand work off to the best local model with a single delegate call.

# illustrative session
You:    Summarise these 40 pages of meeting notes locally, I don't want them in the cloud.
Agent:  → delegate { task: "summarize", prompt: "...", contextLength: 32768 }
        ← routing: ollama / qwen3:14b — already loaded (no load cost); general model; fits: gpu
          local: true · 412 tokens · 38 tok/s

Why use it

• It knows your hardware. fit_check sizes the KV cache from the model's actual layer architecture, adds up VRAM across GPUs, models partial GPU/CPU offload, and understands Apple Silicon unified memory. It answers "will this model run, and where?" with numbers rather than guesses.
• It works across runtimes. If you run Ollama and LM Studio, each call goes to the runtime that actually has the model. Lifecycle actions such as pull, load and unload affect exactly one runtime.
• It keeps data local by default. Hosted providers such as Moonshot are opt-in, and a call without a provider argument never uses one. Every result reports provider, host and local, so you can see where inference ran.
• It handles operations, not just chat. Pull with progress, load with a context size, evict least-recently-used models to make room, benchmark time-to-first-token and tokens per second, and search Hugging Face for GGUF quantizations that fit your machine.

Things to ask your agent

• "What can I run for coding on this laptop?" → suggest_model, then ensure_model
• "Free up my GPU." → list_loaded, then unload_model (or use the free-vram prompt)
• "Benchmark my installed coder models and tell me which is fastest." → benchmark { models: [...] }
• "Find a newer Gemma build that fits my 12 GB card." → search_models { source: "huggingface" }
• "Extract the invoice fields from this text as JSON, locally." → delegate { task: "extract", responseFormat: { type: "json_schema", ... } }
• "Will Llama 3.1 70B run here with 32K context?" → fit_check { model, contextLength: 32768 }

Quick start

Install at least one runtime (Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM or Foundry Local), then add the server to your client. No configuration is needed for runtimes on their default ports.

Claude Code

claude mcp add local-ai -- npx -y @tmhs/local-ai-mcp

Claude Desktop. Download the .mcpb file from the latest release and open it. You get a settings screen for hosts and no JSON to edit. Alternatively, add this to claude_desktop_config.json:

{
  "mcpServers": {
    "local-ai": { "command": "npx", "args": ["-y", "@tmhs/local-ai-mcp"] }
  }
}

Cursor / VS Code. Use the install buttons above, or add the same command/args to .cursor/mcp.json or .vscode/mcp.json (VS Code uses a top-level "servers" key instead of "mcpServers").

Windsurf, Cline, Zed and other clients. Any stdio MCP client works: the command is npx, the args are ["-y", "@tmhs/local-ai-mcp"].

Windows: a few clients can't launch npx directly. If the server doesn't start, use "command": "cmd", "args": ["/c", "npx", "-y", "@tmhs/local-ai-mcp"].

Adding a hosted provider (optional). Put MOONSHOT_API_KEY in the server's env. It is only used when a tool call names provider: "moonshot", unless you also set LOCAL_AI_ALLOW_HOSTED_FALLBACK=1.

Supported runtimes

You can run more than one instance of a runtime, for example a second Ollama box on your LAN, with LOCAL_AI_PROVIDERS='[{"type":"ollama","name":"ollama-gpu2","host":"http://192.168.1.20:11434"}]'. Use LOCAL_AI_DISABLE=lmstudio,dockermodelrunner to skip probing runtimes you don't use.

Privacy model

• Each provider is either local or hosted. The kind is inferred from its host: loopback, private LAN ranges, *.local, Tailscale CGNAT and host.docker.internal count as local. You can override it per instance in LOCAL_AI_PROVIDERS.
• With no provider argument, complete, embed, delegate, benchmark, load_model and the other model-scoped tools only consider live local providers that have the model. If the model exists only on a hosted provider, the call fails and tells you how to opt in.
• Hosted providers are never probed on routine calls. Only health_check contacts them.
• search_models with source: "huggingface" queries the Hugging Face Hub, and only when you ask it to. Set LOCAL_AI_DISABLE_NETWORK_SEARCH=1 to forbid it.
• The server uses stdio only. It never opens a network listener.

Tools (19)

Every tool declares MCP annotations (read-only, destructive, open-world) and an output schema, and returns structured content.

Resources: local-ai://system, local-ai://loaded (subscribable; updates when models load or unload), and local-ai://models/{provider}. Prompts: setup, free-vram, benchmark-my-models, pick-model.

Configuration

Every setting is an environment variable, and blank means the default. See .env.example for the full annotated list.

Troubleshooting

• A provider isn't detected. Run health_check. Check the host and port. From WSL, use the Windows host IP. From a container, use host.docker.internal. For Docker Model Runner, enable host-side TCP support.
• LM Studio pull/load fails with "requires the lms CLI". Install the lms CLI as described in the LM Studio CLI docs so it is on PATH, then restart your MCP client.
• llama.cpp embed fails. Start llama-server with --embeddings and an embedding-capable model.
• Answers about long inputs look truncated (Ollama). Pass contextLength (it sets num_ctx). The result includes a warning when truncation is likely.
• "was not found on any live local provider". The model is either not installed (check list_models) or only available on a hosted provider. Name that provider explicitly to use it.
• macOS memory numbers look low. system_resources reports available memory (free, inactive and purgeable pages) and the Metal working-set limit, not total RAM.

Development

npm install
npm run build       # tsc -> dist/
npm test            # vitest, fully offline (mocked HTTP, stubbed hardware)
npm run lint        # Biome
npm run typecheck   # src and tests
npm run gen:tools   # regenerate mcp-tools.json after changing tools
npm run smoke       # boot the built server over stdio

CI runs on Linux, Windows and macOS with Node 22 and 24, plus a Node 20 runtime check. A nightly workflow tests the adapters against real Ollama and llama.cpp. See CONTRIBUTING.md, including Adding a runtime adapter.

Support

Questions and ideas go in Discussions. Report bugs with the issue templates, and security problems through private reporting. If this saves you time, consider sponsoring.

License

CC-BY-NC-ND-4.0. See LICENSE. Contributions are accepted under the DCO and the inbound grant described in CONTRIBUTING.md.

Built by TMHSDigital