English Français العربية

MCP Turboquant

Community AI and agents

LLM quantization: convert models to GGUF, GPTQ and AWQ, recommend quantization settings, evaluate quality and push to Hugging Face Hub.

Source code Documentation

Install

The installation steps are in the documentation below. Documentation

Documentation

From the project's README on GitHub, refreshed weekly. Links open on GitHub.

Contents
  1. mcp-turboquant
  2. Install
  3. Configure
  4. Tools
  5. Examples
  6. How it works
  7. Run directly
  8. License

mcp-turboquant

Self-contained Python MCP server for LLM quantization. Compress any HuggingFace model to GGUF, GPTQ, or AWQ format in a single tool call.

No external CLI required -- all quantization logic is embedded.

Install

pip install mcp-turboquant

Or run directly with uvx:

uvx mcp-turboquant
Optional backends

The info, check, and recommend tools work out of the box. For actual quantization, install the backend you need:

# GGUF (Ollama, llama.cpp, LM Studio)
pip install mcp-turboquant[gguf]

# GPTQ (vLLM, TGI)
pip install mcp-turboquant[gptq]

# AWQ (vLLM, TGI)
pip install mcp-turboquant[awq]

# Everything
pip install mcp-turboquant[all]

Configure

Claude Code

Add to ~/.claude/settings.json:

{
  "mcpServers": {
    "turboquant": {
      "command": "mcp-turboquant"
    }
  }
}

Or with uvx (no install needed):

{
  "mcpServers": {
    "turboquant": {
      "command": "uvx",
      "args": ["mcp-turboquant"]
    }
  }
}
Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "turboquant": {
      "command": "uvx",
      "args": ["mcp-turboquant"]
    }
  }
}

Tools

ToolDescriptionHeavy deps?
infoGet model info from HuggingFace (params, size, architecture)No
checkCheck available quantization backends on the systemNo
recommendHardware-aware recommendation for best format + bitsNo
quantizeQuantize a model to GGUF/GPTQ/AWQYes
evaluateRun perplexity evaluation on a quantized modelYes
pushPush quantized model to HuggingFace HubNo

Examples

Once configured, ask Claude:

"Get info on meta-llama/Llama-3.1-8B-Instruct"

"What quantization format should I use for Mistral-7B on my machine?"

"Quantize meta-llama/Llama-3.1-8B to 4-bit GGUF"

"Check which quantization backends I have installed"

"Evaluate the perplexity of my quantized model at /path/to/model.gguf"

"Push my quantized model to myuser/model-GGUF on HuggingFace"

How it works

Claude / Agent  <-->  MCP Protocol (stdio)  <-->  mcp-turboquant (Python)  <-->  llama-cpp-python / auto-gptq / autoawq

All quantization logic runs in-process. No external CLI tools needed.

Run directly

# As a command
mcp-turboquant

# As a module
python -m mcp_turboquant

License

MIT

Frequently asked questions

What is MCP Turboquant?

LLM quantization: convert models to GGUF, GPTQ and AWQ, recommend quantization settings, evaluate quality and push to Hugging Face Hub.

How do I install MCP Turboquant?

Follow the installation steps in its documentation, shown on this page and in its repository.

Is MCP Turboquant open source?

Yes. Its code is on GitHub under the MIT license.

Who maintains MCP Turboquant?

It is published by ShipItAndPray. AI.tn lists it but does not maintain or endorse it.

From the community awesome-mcp-servers lists and the project's GitHub repository. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.