English Français العربية

DeepSift Search

Official registry Search and web

Live Google web search for AI agents that returns only the passages that answer the question.

Source code Website Documentation

Install

Add this to your client's MCP configuration (Claude Desktop: claude_desktop_config.json; Cursor: .cursor/mcp.json; VS Code and others use the same "mcpServers" shape).

Remote (streamable-http)
Claude Code 
claude mcp add --transport http deepsift-search 'https://aadstra--deepsift-search.apify.actor/mcp' --header 'Authorization: <Bearer followed by your Apify API token (https://console.apify.com/settings/integrations)>'
JSON configuration (Claude Desktop, Cursor, VS Code) 
{
    "mcpServers": {
        "deepsift-search": {
            "type": "http",
            "url": "https://aadstra--deepsift-search.apify.actor/mcp",
            "headers": {
                "Authorization": "<Bearer followed by your Apify API token (https://console.apify.com/settings/integrations)>"
            }
        }
    }
}

Documentation

From the project's README on GitHub, refreshed weekly. Links open on GitHub.

Contents
  1. DeepSift
  2. Don't want to run it yourself? Use DeepSift Search on Apify
  3. What Claude Code gets
  4. Install (about 1 minute)
  5. How it compares to Tavily
  6. Optional: a local Tavily-compatible API
  7. Settings
  8. How it works
  9. Tests
  10. License

DeepSift

A free, self-hosted alternative to Tavily for Claude Code. Ask one question; DeepSift searches the web, reads every page, and hands back only the paragraphs that matter. It runs on your laptop and uses the Claude subscription you already pay for.

Tavily is a paid web search API built for AI agents. You send one question; it searches the web, opens the best pages, cuts out the passages that answer the question, and sends it all back in one tidy reply. That's great, but every call costs credits.

DeepSift does the same job on your own computer, in three steps:

  1. Find the right websites, using Claude's web search on your existing Claude subscription (with free DuckDuckGo as a backup).
  2. Read every page itself: HTML and PDFs, with menus, ads and clutter stripped out.
  3. Pick the passages that best match your question, and give them back in one reply.

No per-call bill, no API key to buy.

Don't want to run it yourself? Use DeepSift Search on Apify

DeepSift Search is the hosted version: real Google results, the same passage picking, nothing to install, and it works for apps and teams, not just your own laptop.

  • Tavily drop-in: same /search, /extract, /crawl and /map requests and responses. Change one line:
    from tavily import TavilyClient
client = TavilyClient(api_key="<YOUR_APIFY_TOKEN>", api_base_url="https://aadstra--deepsift-search.apify.actor")
  • MCP server for Claude, Cursor and others: https://aadstra--deepsift-search.apify.actor/mcp with your Apify token as a Bearer header.
  • Claude Code plugin: /plugin marketplace add pavanmulka/deepsift, then /plugin install deepsift-search@deepsift. Claude Code asks for your Apify token once.
  • Pay per use, no subscription: $4 per 1,000 searches, $1 per 1,000 pages read. Failed requests are free, and a free Apify account includes $5 of credit every month.

The Apify version lives in .actor/ and deepsift/actor.py; it never uses your Claude subscription.

What Claude Code gets

After installing, every Claude Code session has these tools:

ToolWhat you'd askWhat it does
web_search"web_search Databricks Unity Catalog pricing"One topic → many websites → the best passages from each, in one reply
web_read"read these 3 pages and focus on pricing"Full text of up to 20 pages or PDFs, or only the parts about a topic
web_map"list all pages on docs.example.com"A site's page addresses
web_crawl"read the pricing pages on example.com"Reads many pages of one site, keeps the relevant ones
web_research"research the top 5 vector databases with sources"A multi-search investigation that returns a written report with numbered sources (takes minutes)

The same abilities work from any terminal:

deepsift search "Databricks Unity Catalog pricing" --advanced
deepsift read https://example.com/page --focus "pricing"
deepsift map https://docs.example.com
deepsift crawl https://docs.example.com --instructions "pricing"
deepsift research "Which vector databases support hybrid search?"

Install (about 1 minute)

You need a Mac or Linux computer with Claude Code and Python 3.10+.

git clone https://github.com/pavanmulka/deepsift.git
cd deepsift
./install.sh
claude auth login        # only if you aren't signed in to Claude Code's command line yet

Open a new Claude Code session and ask it to use web_search. That's it.

./uninstall.sh removes it from Claude Code again.

How it compares to Tavily

TavilyDeepSift
CostPay per creditFree (uses your Claude plan)
Speed per search1–2 s~10–30 s through Claude; ~1–3 s on the DuckDuckGo backup
Search, extract, crawl, map, research✓✓
Same request/response format—✓ (optional local HTTP API, see below)
Runs whereTheir serversYour laptop

Limits, honestly:

  • Searches through Claude count toward your Claude plan's usage limits (3 at a time at most). Repeat searches within 24 h come from a local cache.
  • DuckDuckGo pauses you after a burst of quick searches; the tool then rests it for 15 minutes.
  • Pages that only show their text after JavaScript runs come back thin.
  • It's meant for your own personal use with your own Claude subscription. To power an app for other people, use a proper paid API.

Optional: a local Tavily-compatible API

./run.sh starts an HTTP server on 127.0.0.1:8787 with Tavily's paths and fields (/search, /extract, /crawl, /map, /research, GET /research/{id}, /usage). Code written for Tavily can point its base URL there. Every reply reports usage.credits: 0.

Settings

Copy .env.example to .env to change engines, models, cache time, or set an API key for the HTTP server.

How it works

question ─► find sites (Claude WebSearch via `claude -p`, or DuckDuckGo)
         ─► download pages in parallel (httpx) ─► readable text (trafilatura, pypdf)
         ─► split into passages ─► rank by relevance to the question (BM25) ─► one reply

claude -p always runs on your subscription sign-in. ANTHROPIC_API_KEY is removed before it starts, so it never bills an API account by accident.

Tests

.venv/bin/python -m pytest -q tests

The tests run offline, using a stand-in for claude.

License

MIT

Frequently asked questions

What is DeepSift Search?

Live Google web search for AI agents that returns only the passages that answer the question.

How do I install DeepSift Search?

It runs as a remote server. In Claude Code: claude mcp add --transport http deepsift-search 'https://aadstra--deepsift-search.apify.actor/mcp' --header 'Authorization: <Bearer followed by your Apify API token (https://console.apify.com/settings/integrations)>'. For other clients, paste the JSON configuration from this page into their MCP settings.

Does DeepSift Search need an API key?

Yes. It expects Authorization, which you set in its configuration.

Is DeepSift Search open source?

Yes. Its code is on GitHub under the MIT license.

Who maintains DeepSift Search?

It is published by pavanmulka. AI.tn lists it but does not maintain or endorse it.

From the official MCP Registry. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.