English Français العربية

Robots Inspect

Official registry Other

Fetch a public robots.txt and return group counts only.

Source code Documentation

Install

Add this to your client's MCP configuration (Claude Desktop: claude_desktop_config.json; Cursor: .cursor/mcp.json; VS Code and others use the same "mcpServers" shape).

Remote (streamable-http)
Claude Code 
claude mcp add --transport http robots-inspect 'https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-inspect/mcp'
JSON configuration (Claude Desktop, Cursor, VS Code) 
{
    "mcpServers": {
        "robots-inspect": {
            "type": "http",
            "url": "https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-inspect/mcp"
        }
    }
}

Documentation

From the project's README on GitHub, refreshed weekly. Links open on GitHub.

Robots Inspect

Fetch a public robots.txt and return group counts only.

Transparent public utilities for autonomous clients. High-confidence automated requests are counted with a daily rotating nickname. Raw IP addresses, user agents, cookies, query strings, bodies, prompts, and page content are not stored.

Frequently asked questions

What is Robots Inspect?

Fetch a public robots.txt and return group counts only.

How do I install Robots Inspect?

It runs as a remote server. In Claude Code: claude mcp add --transport http robots-inspect 'https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-inspect/mcp'. For other clients, paste the JSON configuration from this page into their MCP settings.

Does Robots Inspect need an API key?

Its registry entry lists no required key or secret. Some features may still need an account with the service it connects to.

Is Robots Inspect open source?

Its code is public on GitHub, but the repository does not declare a standard license.

Who maintains Robots Inspect?

It is published by sadri-dridi. AI.tn lists it but does not maintain or endorse it.

From the official MCP Registry. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.