English Français العربية

CRW Web Scraper

Official registry Search and web

Open-source web scraper for AI agents with scrape, crawl, and map tools

Source code Website Documentation

Install

Add this to your client's MCP configuration (Claude Desktop: claude_desktop_config.json; Cursor: .cursor/mcp.json; VS Code and others use the same "mcpServers" shape).

OCI package ghcr.io/fastcrw/crw:latest
Claude Code 
claude mcp add crw-web-scraper -- docker run -i --rm ghcr.io/fastcrw/crw:latest crw-mcp
JSON configuration (Claude Desktop, Cursor, VS Code) 
{
    "mcpServers": {
        "crw-web-scraper": {
            "command": "docker",
            "args": [
                "run",
                "-i",
                "--rm",
                "ghcr.io/fastcrw/crw:latest",
                "crw-mcp"
            ]
        }
    }
}

Documentation

From the project's README on GitHub, refreshed weekly. Links open on GitHub.

Contents
  1. fastCRW
  2. One-command install
  3. What it does
  4. Why fastCRW
  5. Choose how you use it
  6. Choose where it runs
  7. Learn more
  8. Contributing
  9. Star History

fastCRW

Turn URLs into clean markdown or structured JSON with one engine for search, scrape, map, crawl, and extract.

Run it locally as a small Rust binary or use the managed API.

Get 1000 free credits → · Install · Docs

No credit card. Continue with GitHub.

One-command install

curl -fsSL https://fastcrw.com/install | sh

Runs local and free, no account needed. To use the Cloud, paste your key into the same command and it installs the binary, connects the key, and registers the MCP server with the AI coding tools you already have:

curl -fsSL https://fastcrw.com/install | CRW_API_KEY=crw_live_... sh
crw search "rust tutorials"

Claude Code, Cursor, Codex, Gemini CLI, OpenCode and Windsurf are picked up automatically when they are already set up; nothing else is touched, and your key stays in ~/.config/crw/config.toml rather than being copied into each tool. Add CRW_NO_AGENTS=1 to skip that step, or run crw setup on its own to choose interactively.

1000 free credits, no credit card. Managed proxies, JS rendering and search, with nothing to run or keep up to date. Get my free key →

macOS and Linux, Intel and ARM. More install options →

What it does

OperationOutcome
ScrapeOne URL to markdown, HTML, links, screenshots, or schema JSON
CrawlFollow a bounded site crawl and collect its pages
MapDiscover URLs without scraping every page
SearchSearch the web and optionally scrape selected results
ExtractProduce structured fields from one or many URLs

See the full API →

Why fastCRW

On Firecrawl's own public 1,000-URL dataset, fastCRW recovered more truth than Crawl4AI and Firecrawl, matched the fastest median latency, and idled at ~14 MB RAM.

Methodology, full numbers, and how to reproduce it

On a different benchmark entirely, answer accuracy rather than scrape recall, fastCRW answers 90.0% of the 600 AA-Omniscience questions correctly. Every product listed on the Artificial Analysis Search Index sits below it.

The full 14-product comparison, the control run, and how to reproduce it

Choose how you use it

CLI
crw https://example.com            # scrape, works right after install
crw search "rust async runtime"    # search, after `crw setup`
Python SDK

Using Cloud? Get an API key, then export it once:

export CRW_API_KEY="crw_live_..."
pip install crw
from crw import CrwClient

client = CrwClient()
page = client.scrape("https://example.com", formats=["markdown"])

print(page["markdown"])

Node.js

npm install crw-sdk
import { CrwClient } from "crw-sdk";

const client = new CrwClient();

const page = await client.scrape("https://example.com", {
  formats: ["markdown"],
});

console.log(page.markdown);

Local mode and more SDK examples → · REST API →

MCP for AI agents
npx -y crw-mcp@latest install

Installs the CRW skill and MCP server in your detected AI tools. crw setup can also do this step, so either path is enough. Manual setup →

Choose where it runs

Managed APILocal / self-hosted
Best forZero infrastructure and managed scalingData control, private networks, or custom infrastructure
StartCreate an API key, then crw setupInstall and run crw <URL>
OperationsManaged proxies, billing, and hosted capabilitiesYou choose renderers, search, auth, proxies, and capacity

Capabilities and response shapes can differ by deployment: /v1/capabilities · response shapes

Self-hosting guide →

Learn more

Contributing

The workspace requires Rust 1.85 or newer:

git clone https://github.com/fastcrw/crw
cd crw
make check-fast

Read the contributor guide →

Contributors

crw is better because of these people. Thank you, all of you:

Engine and MCP server: AGPL-3.0. Python and TypeScript SDKs: MIT. Embedding license: hello@fastcrw.com.

Star History

Please respect website policies. Crawl and map follow robots.txt by default.

Frequently asked questions

What is CRW Web Scraper?

Open-source web scraper for AI agents with scrape, crawl, and map tools

How do I install CRW Web Scraper?

It runs on your machine. In Claude Code: claude mcp add crw-web-scraper -- docker run -i --rm ghcr.io/fastcrw/crw:latest crw-mcp. For other clients, paste the JSON configuration from this page into their MCP settings.

Does CRW Web Scraper need an API key?

Its registry entry lists no required key or secret. Some features may still need an account with the service it connects to.

Is CRW Web Scraper open source?

Yes. Its code is on GitHub under the AGPL-3.0 license.

Who maintains CRW Web Scraper?

It is published by fastcrw. AI.tn lists it but does not maintain or endorse it.

From the official MCP Registry. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.