dompruner-mcp

dompruner-mcp

한국어 | English

DOM AST middleware for LLM web pipelines — strips layout noise (nav, scripts, sidebars) and passes original text directly. Add a query to filter to relevant sections with BM25.

When an LLM uses a web fetch tool, the raw page HTML is converted to text and passed into the model's context — often tens of thousands of tokens of navigation, sidebars, and boilerplate the model doesn't need. DomPruner strips that at the DOM level and passes only the original content directly to the model.

> [DomPruner] docs.python.org
> | Raw HTML  | 44,316 tokens |
> | DomPruner |  1,328 tokens |
> | Reduction |        97.0%  |
> Fetch: 194ms · Parse: 11.2ms

93.5% fewer context tokens than WebFetch on average. 45% faster end-to-end. → Full benchmark

Quick Start

No installation, no API key:

npx -y dompruner-mcp

Claude Code

{
  "mcpServers": {
    "dompruner": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "dompruner-mcp"]
    }
  }
}

Add to .mcp.json in your project root, or ~/.claude/.mcp.json for global. Run /mcp to verify.

Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "dompruner": {
      "command": "npx",
      "args": ["-y", "dompruner-mcp"]
    }
  }
}

Cursor / Windsurf / other MCP clients

{
  "mcpServers": {
    "dompruner": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "dompruner-mcp"]
    }
  }
}

Remote HTTP (no install, always up to date)

For clients that support HTTP transport — no Node.js install required, always runs the latest version:

{
  "mcpServers": {
    "dompruner": {
      "url": "https://dompruner-mcp.vercel.app/api/mcp"
    }
  }
}

LangChain / LangGraph

langchain-mcp-adapters wraps any MCP stdio server as LangChain tools automatically:

from langchain_mcp_adapters.client import MultiServerMCPClient

client = MultiServerMCPClient({
    "dompruner": {
        "command": "npx",
        "args": ["-y", "dompruner-mcp"],
        "transport": "stdio",
    }
})
tools = await client.get_tools()

Ensuring Your AI Always Uses DomPruner

DomPruner's tool description already tells clients to prefer dompruner_fetch over WebFetch. If your client still falls back, add this to its instruction file:

When retrieving a URL, always use dompruner_fetch instead of WebFetch.
- URL known → dompruner_fetch(url, query?)
- URL unknown → search for the URL first, then dompruner_fetch(url)

Tools

→ Full tool reference

Benchmark Summary

vs WebFetch (live comparison, Claude Haiku)

Measured using Anthropic's web_fetch_20260209 server tool under identical conditions (same model, system prompt, tool definitions). → Full methodology

BM25 Section Accuracy (snapshot-based, deterministic)

headingBoost=2.0 was selected by running a parameter sweep (1.0–3.0) on the tuning set. The validation set uses five sites not present in the tuning set (react.dev, typescriptlang.org, expressjs.com, doc.rust-lang.org, nextjs.org) to confirm the result is not overfit. The 2 %p gap is within the known heading-bias limitation.

→ Full benchmark · Architecture

vs Crawl4AI

Crawl4AI is a crawler framework — browser automation, JS rendering, click/scroll interaction, JSON extraction, deep crawl. DomPruner is LLM context middleware — a single-purpose layer that reduces token cost when an agent already knows the URL.

They solve different problems. But both can be used to read a web page inside an MCP agent, so the comparison is real.

Setup

Held-out validation: 5 sites not in DomPruner's tuning set

The most defensible comparison: five sites that were not used to tune DomPruner's parameters (react.dev, typescriptlang.org, expressjs.com, doc.rust-lang.org, nextjs.org). Both tools use the same query.

On sites DomPruner was not tuned on: 1.9× fewer tokens with the same accuracy (96%).

Additional reference: 10-page tuning set (MDN, docs.python.org)

On DomPruner's own tuning pages, vs Crawl4AI PruningContentFilter: 13.2× smaller, 8 pp more accurate (90% → 98%). Crawl4AI BM25ContentFilter with the same query returns near-empty output on 5 of 10 pages — a structural issue with <dl>/<dt>/<dd> definition-list pages where a threshold sweep (0.0–0.10) shows a hard cliff at 0.01 with no intermediate optimum.

→ Full comparison with all tables and threshold sweep data

Related

• dompruner-py — Python port. DomPrunerLoader, DomPrunerSitemapLoader, DomPrunerFetchTool for LangChain. pip install dompruner.
• LangChain integrations — dompruner-py listed as a third-party web loader.

Glama Score

License

MIT