ActableSite AI Crawler Monitor
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
What can you do with ActableSite AI Crawler Monitor?
audit_public_websitechecks public identity, offer evidence, action paths, discovery files, metadata, and structured data.check_ai_crawler_policyevaluates eight OpenAI, Anthropic, Perplexity, and Google crawler tokens.get_crawler_watch_offerreturns the $9 monthly price, one-site scope, 15-minute cadence, confirmation rule, delivery, cancellation, and synthetic-check limitations. It cannot open checkout or purchase anything.get_full_report_offerreturns explicit metadata for an optional $19 one-time report. It cannot open checkout or purchase anything.get_practice_radar_offerreturns the current weekly-edition receipt, public sample, $39 monthly price, delivery, cancellation, and material NPI limitations. It cannot open checkout or purchase anything.
Install
Add this to your client's MCP configuration (Claude Desktop: claude_desktop_config.json; Cursor: .cursor/mcp.json; VS Code and others use the same "mcpServers" shape).
claude mcp add --transport http actablesite-ai-crawler-monitor 'https://actablesite.com/mcp/'
{
"mcpServers": {
"actablesite-ai-crawler-monitor": {
"type": "http",
"url": "https://actablesite.com/mcp/"
}
}
}
Documentation
From the project's README on GitHub, refreshed weekly. Links open on GitHub.
Contents
ActableSite MCP
A remote, read-only MCP server for public website evidence, AI crawler policy, and explicit Crawler Watch, report, and Practice Radar offer metadata.
Tools
It exposes five tools over Streamable HTTP:
audit_public_websitechecks public identity, offer evidence, action paths, discovery files, metadata, and structured data.check_ai_crawler_policyevaluates eight OpenAI, Anthropic, Perplexity, and Google crawler tokens.get_crawler_watch_offerreturns the $9 monthly price, one-site scope, 15-minute cadence, confirmation rule, delivery, cancellation, and synthetic-check limitations. It cannot open checkout or purchase anything.get_full_report_offerreturns explicit metadata for an optional $19 one-time report. It cannot open checkout or purchase anything.get_practice_radar_offerreturns the current weekly-edition receipt, public sample, $39 monthly price, delivery, cancellation, and material NPI limitations. It cannot open checkout or purchase anything.
Connect
Claude Code:
claude mcp add --transport http actablesite https://actablesite.com/mcp/
For another MCP client, add a remote server named actablesite with:
- URL:
https://actablesite.com/mcp/ - Transport: Streamable HTTP
- Authentication: none
For clients that require a local stdio command, clone this repository and run the dependency-free bridge:
git clone https://github.com/unitedideas/actablesite-mcp.git
cd actablesite-mcp
npm run start:stdio
The bridge contains the visible MCP implementation source in src/stdio.mjs. Distribution version 1.4.2 exposes the same five read-only tools and sends requests only to ActableSite's bounded public JSON APIs. It has no file, shell, browser, account, or write capability.
The same stdio process is available as a public, non-root container:
docker run --rm -i ghcr.io/unitedideas/actablesite-mcp:1.4.2
The hosted server implements MCP protocol version 2025-06-18 and is published in the official registry as com.actablesite/ai-crawler-robots-monitor version 1.4.2.
Discovery metadata
Directory scanners can read either machine-readable discovery surface:
/.well-known/mcp.jsonidentifies the official registry name, repository, remote endpoint, transport, authentication mode, and tool names./.well-known/mcp/server-card.jsonprovides static tool descriptions and JSON input schemas for scanners that cannot complete a live MCP handshake.
Both files describe the same unauthenticated endpoint and five-tool inventory verified by this repository.
Ongoing crawler monitoring
The MCP tools report current public evidence and do not create background jobs. Crawler Watch is the separate $9/month service for one public website: it checks crawler policy, synthetic homepage responses, sitemap.xml, and llms.txt every 15 minutes and emails after a changed state appears twice.
Its synthetic requests do not authenticate provider IP ranges or prove crawling, indexing, citation, ranking, or traffic.
Safety contract
- Public HTTP and HTTPS websites only; private, local, loopback, and reserved network destinations are rejected.
- No browser control, file access, account access, write, or mutation tools.
- No authentication header or API key is requested.
- Tool calls are limited to 60 per caller per hour.
- All three offer tools are informational and require explicit user confirmation before any separate checkout or purchase action.
- Practice Radar records do not prove licensure, active operation, demand, or buying intent and must be verified before contact.
- Results do not guarantee crawling, indexing, citation, ranking, recommendation, traffic, legal compliance, or revenue.
Verify the live server
Node.js 20 or newer is required.
npm test
npm run verify:live
The verifier initializes the live server, checks its protocol and version, and confirms the exact read-only tool inventory. It does not call an audit tool or create a funnel event.
Repository scope
This repository contains a runnable stdio implementation, public registry manifest, connection contract, live verifier, and security policy for the hosted ActableSite MCP server. The production runtime is operated at https://actablesite.com/mcp/.
License
MIT
Frequently asked questions
What is ActableSite AI Crawler Monitor?
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
How do I install ActableSite AI Crawler Monitor?
It runs as a remote server. In Claude Code: claude mcp add --transport http actablesite-ai-crawler-monitor 'https://actablesite.com/mcp/'. For other clients, paste the JSON configuration from this page into their MCP settings.
Does ActableSite AI Crawler Monitor need an API key?
Its registry entry lists no required key or secret. Some features may still need an account with the service it connects to.
Is ActableSite AI Crawler Monitor open source?
Yes. Its code is on GitHub under the MIT license.
Who maintains ActableSite AI Crawler Monitor?
It is published by unitedideas. AI.tn lists it but does not maintain or endorse it.
From the official MCP Registry. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.