English Français العربية

io.github.bch1212/modelwatch

Registre officiel IA et agents

Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.

Code source Site web Documentation

Installation

Ajoutez ceci à la configuration MCP de votre client (Claude Desktop : claude_desktop_config.json ; Cursor : .cursor/mcp.json ; VS Code et les autres utilisent la même forme « mcpServers »).

NPM package modelwatch-mcp
Claude Code 
claude mcp add modelwatch -e 'MODELWATCH_API_KEY=<your modelwatch_api_key>' -e 'MODELWATCH_API_BASE=https://api.modelwatch.app' -- npx -y modelwatch-mcp@0.1.3
Configuration JSON (Claude Desktop, Cursor, VS Code) 
{
    "mcpServers": {
        "modelwatch": {
            "command": "npx",
            "args": [
                "-y",
                "modelwatch-mcp@0.1.3"
            ],
            "env": {
                "MODELWATCH_API_KEY": "<your modelwatch_api_key>",
                "MODELWATCH_API_BASE": "https://api.modelwatch.app"
            }
        }
    }
}

Variables d'environnement

  • MODELWATCH_API_KEY obligatoire secret API key from https://modelwatch.app (free tier: 5 specs, 500 runs/mo).
  • MODELWATCH_API_BASE Override the API base URL for self-hosted deployments.

Documentation

Tirée du README du projet sur GitHub, actualisée chaque semaine. Les liens ouvrent GitHub.

ModelWatch

Continuous behavioral drift monitoring for LLM-powered applications. Define a spec (input and expected behavior), ModelWatch replays it on a schedule against your model endpoint, and alerts when the output drifts from the baseline.

Customers bring their own OpenAI or Anthropic key. Auth is a workspace key minted at POST /api/auth/signup (Authorization: Bearer mw_...).

  • API: https://api.modelwatch.app
  • App: https://modelwatch.app
  • HTTP reference: frontend/docs.html
pip install -r requirements.txt
pytest tests/ -q

Tool monitoring

ModelWatch also watches the tools an agent calls. A decorator in your process records latency, success, and a fingerprint of the response shape. The API compares that fingerprint with the dominant shape from the last 24 hours and alerts on schema drift. Response values are not uploaded.

pip install -e sdk
export MODELWATCH_API_KEY=mw_...
from modelwatch import monitor

@monitor(tool_name="search_web", agent_id="researcher")
async def search_web(query: str) -> dict:
    ...

Calls go to https://api.modelwatch.app/api/tools/ingest. Read recent calls at GET /api/tools and schema changes at GET /api/tools/schema-drift. The same mw_ key and the same email/Slack settings are used. Full SDK notes, including the TypeScript wrapper, are in sdk/README.md.

Quotas for tool calls are separate from spec runs: Free 5,000/month, Pro 100,000, Team 1,000,000, Enterprise unlimited.

This feature replaces a standalone ToolPulse deployment for the same buyer. Point clients at ModelWatch (MODELWATCH_API_KEY, https://api.modelwatch.app). The decorator does not call api.toolpulse.io.

Questions fréquentes

Qu'est-ce que io.github.bch1212/modelwatch ?

Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.

Comment installer io.github.bch1212/modelwatch ?

Il s'exécute sur votre machine. Dans Claude Code : claude mcp add modelwatch -e 'MODELWATCH_API_KEY=<your modelwatch_api_key>' -e 'MODELWATCH_API_BASE=https://api.modelwatch.app' -- npx -y modelwatch-mcp@0.1.3. Pour les autres clients, collez la configuration JSON de cette page dans leurs réglages MCP.

Faut-il une clé API pour io.github.bch1212/modelwatch ?

Oui. Il attend MODELWATCH_API_KEY, à renseigner dans sa configuration.

io.github.bch1212/modelwatch est-il open source ?

Son code est public sur GitHub, mais le dépôt ne déclare pas de licence standard.

Qui maintient io.github.bch1212/modelwatch ?

Il est publié par bch1212. AI.tn le référence sans le maintenir ni le recommander.

Issu du registre MCP officiel. Être listé n'est pas une recommandation : vérifiez le code et les permissions d'un serveur avant de lui donner accès à vos données.