io.github.shulex-voc/smart-crawler
E-commerce competitor intelligence, VOC sentiment, Amazon ASIN analysis, Reddit growth playbooks.
Install
Add this to your client's MCP configuration (Claude Desktop: claude_desktop_config.json; Cursor: .cursor/mcp.json; VS Code and others use the same "mcpServers" shape).
claude mcp add --transport http smart-crawler 'https://smartcrawler.io/mcp/' --header 'Authorization: <API Key in the form '\''Bearer sck_...'\''. Generate a key in the smart-crawler console under '\''API 接入'\''.>'{
"mcpServers": {
"smart-crawler": {
"type": "http",
"url": "https://smartcrawler.io/mcp/",
"headers": {
"Authorization": "<API Key in the form 'Bearer sck_...'. Generate a key in the smart-crawler console under 'API 接入'.>"
}
}
}
}Documentation
From the project's README on GitHub, refreshed weekly. Links open on GitHub.
smart-crawler MCP
E-commerce competitor intelligence, VOC review sentiment, Amazon ASIN analysis, Google Shopping landscape, on-demand URL scraping, and Reddit subreddit growth playbooks — exposed to AI agents as a hosted Model Context Protocol server.
- Endpoint:
https://smartcrawler.io/mcp/(streamable-http) - Auth:
Authorization: Bearer sck_...— generate a key in the smart-crawler console - Registry:
io.github.shulex-voc/smart-crawler
This repository hosts the public MCP manifest for the smart-crawler service. It is the discovery/connection entry point — the service itself is hosted.
Connect
Claude Code
export SMARTCRAWLER_API_KEY=sck_...
claude mcp add --scope user --transport http smart-crawler \
https://smartcrawler.io/mcp/ \
--header "Authorization: Bearer $SMARTCRAWLER_API_KEY"--scope user makes the server visible in every project. Without it,
claude mcp add defaults to local scope (only the current directory), a common
"I added it but /mcp is empty" pitfall. Verify with claude mcp list or /mcp.
Codex (~/.codex/config.toml)
[mcp_servers.smart-crawler]
url = "https://smartcrawler.io/mcp/"
bearer_token_env_var = "SMARTCRAWLER_API_KEY"Any MCP client
Point your client at https://smartcrawler.io/mcp/ over streamable-http and send
your API key as a Bearer token. See mcp.json for the full tool list
and server.json for the registry manifest.
Tools (highlights)
| Tool | What it does |
|---|---|
reddit_subreddit_playbook | Full growth playbook for a subreddit's top contributors |
search_competitor_products | Search competitor catalog by brand / country / keyword / price |
get_voc_reviews | Consumer VOC reviews from Trustpilot / Reviews.io / Google Maps |
amazon_voc_report | Amazon ASIN reviews + AI VOC analysis (pain points, listing tips) |
competitor_landscape | Google Shopping merchant-share ranking for a keyword |
scrape_url | One-URL scrape with short-lived agent memory |
query_warehouse | Warehouse-first natural-language query (0 credits) |
Full list in mcp.json.
License
MIT
Frequently asked questions
What is io.github.shulex-voc/smart-crawler?
E-commerce competitor intelligence, VOC sentiment, Amazon ASIN analysis, Reddit growth playbooks.
How do I install io.github.shulex-voc/smart-crawler?
It runs as a remote server. In Claude Code: claude mcp add --transport http smart-crawler 'https://smartcrawler.io/mcp/' --header 'Authorization: <API Key in the form '\''Bearer sck_...'\''. Generate a key in the smart-crawler console under '\''API 接入'\''.>'. For other clients, paste the JSON configuration from this page into their MCP settings.
Does io.github.shulex-voc/smart-crawler need an API key?
Yes. It expects Authorization, which you set in its configuration.
Is io.github.shulex-voc/smart-crawler open source?
Its code is public on GitHub, but the repository does not declare a standard license.
Who maintains io.github.shulex-voc/smart-crawler?
It is published by shulex-voc. AI.tn lists it but does not maintain or endorse it.
From the official MCP Registry. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.