English Français العربية

Backtest Overfitting Audit

Official registry Productivity

Probability of Backtest Overfitting (CSCV), Deflated Sharpe Ratio, and purged CV splits.

Source code Documentation

Install

Add this to your client's MCP configuration (Claude Desktop: claude_desktop_config.json; Cursor: .cursor/mcp.json; VS Code and others use the same "mcpServers" shape).

Remote (streamable-http)
Claude Code 
claude mcp add --transport http backtest-overfitting-audit 'https://overfitting-audit-mcp.mcpize.run/mcp'
JSON configuration (Claude Desktop, Cursor, VS Code) 
{
    "mcpServers": {
        "backtest-overfitting-audit": {
            "type": "http",
            "url": "https://overfitting-audit-mcp.mcpize.run/mcp"
        }
    }
}

Documentation

From the project's README on GitHub, refreshed weekly. Links open on GitHub.

Contents
  1. overfitting-audit-mcp
  2. The problem this solves
  3. Tools
  4. Use it
  5. Part of the AlgoForge suite
  6. License

overfitting-audit-mcp

An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.

The problem this solves

Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.

Tools

probability_of_backtest_overfitting

Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.

deflated_sharpe_ratio

Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.

minimum_backtest_length

The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.

purged_cv_split

Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.

Use it

Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.

Self-host:

npm install
node server.js

Part of the AlgoForge suite

Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.

License

MIT

Frequently asked questions

What is Backtest Overfitting Audit?

Probability of Backtest Overfitting (CSCV), Deflated Sharpe Ratio, and purged CV splits.

How do I install Backtest Overfitting Audit?

It runs as a remote server. In Claude Code: claude mcp add --transport http backtest-overfitting-audit 'https://overfitting-audit-mcp.mcpize.run/mcp'. For other clients, paste the JSON configuration from this page into their MCP settings.

Does Backtest Overfitting Audit need an API key?

Its registry entry lists no required key or secret. Some features may still need an account with the service it connects to.

Is Backtest Overfitting Audit open source?

Yes. Its code is on GitHub under the MIT license.

Who maintains Backtest Overfitting Audit?

It is published by tylerscomic-lab. AI.tn lists it but does not maintain or endorse it.

From the official MCP Registry. Listing is not an endorsement: review a server's code and permissions before giving it access to your data.