Routemind

RouteMind

A routing table for an agent, written and kept by people. You split a body of documents into areas. Each area describes, in one sentence, when someone should come to it. An agent reads that list of sentences first, chooses, and walks down a tree of short lines to the document. The map shows you the tree, and one click shows you the exact text the agent is given at each step.

The domain is not in the code. The areas, the lines and the documents are data, kept in your own git repository. Every change is a commit.

The bet

RouteMind does not replace retrieval. It makes one bet:

Where documents resemble each other — revisions of one rule, rows that differ by a code, policies that differ by one condition — a routing table that people write and maintain finds the right document more reliably than similarity search. The price is several times the cost per question, plus the labour of keeping the table true.

The bet is lost if any of these turns out to be true:

• Retrieval is already good enough on real documents. Then the walk costs more and buys nothing.
• Keeping the map costs more than the wrong answers it prevents. The map is human work, every week.
• A large map loses the agent. Measured once, small: 5 areas against 30, with confusable neighbours, did not change which area was chosen (eval/scale/). Dozens of real areas or tens of thousands of documents have not been tried.

None of these has been settled on real data. Everything measured so far is listed below, together with what it does not show.

How it works

The idea is borrowed from routing on a network. An area advertises when it is relevant, not everything it holds. That keeps the first list the same size whether the areas hold 80 documents or 8,000.

   BACK-BONE ─ the whole map. Hop 0: one row per area, nothing else,
       │       and the only place "it is not here" may be said
       │
       ├── AS  expense    "what to do with a receipt · whether the corporate card
       │        │          may be used here · how much a business trip pays"
       │        ├── AS  corp-card   ← an area holds areas: the same thing one level
       │        │    └── card-limit   down, describing itself the same way
       │        └── AS  evidence
       ├── AS  approval   "whom to put in the approval chain · …"
       └── AS  payroll    "what this payslip line means · …"

• use_when is the only text the agent reads before it chooses an area. An area with a title and no reason is an area nobody picks.
• Each table describes only its own area. An area's table says what that area holds, never what the whole map lacks. Only someone who has read hop 0 may conclude that something is not here.
• Every walk starts at hop 0, and every step records why it was taken. The MCP server enforces this. The map replays any walk from the last six hours.
• Writing is a set of decisions. When an agent files a document, it may create a holder and move siblings under it in the same change. Every line whose table changes must be either kept or rewritten, or nothing is written. (docs/CHANGE.md)

Two containers and no database. ontology is the API and the only process that touches your git repository. web serves the map and acts as a proxy. mcp/knowledge_mcp.py is the agent's door: a single stdlib-only file with four tools — read a table, read a document, place a document, read another RouteMind.

What was measured — and what it does not show

The setting. 700 questions over one frozen corpus of 1,126 documents, in four arms, with one fresh agent per question:

All three columns are scored the same way: a question that needs two documents counts only when both are found, and an accepted alternative document counts for the one it stands in for. Until 2026-10-10 the retrieval columns counted any one of the documents, which was more lenient than the walk's scoring (0.516 / 0.541 overall). An independent audit found the difference (15 of 1,400 rows; issue #2, finding F-RM-01), and bench/attribution.py re-derives these numbers from the saved results.

Read it as a stress test, not a benchmark.

• The corpus is synthetic and was built so that retrieval fails. Each answer has many near-identical rivals: other revisions, neighbouring codes. eval/COLLAPSE.md describes how that was constructed. On questions phrased in the documents' own words, retrieval scores 0.99 and the walk adds nothing.
• One domain, one map, one author side. The corpus is a fictional back office. The 700 questions are templated by the project (bench/crowd.py, bench/temporal.py), not written by outsiders. The map's sentences were written by people who knew the corpus. No real organisation's documents have been measured. The question of who labels the ground truth is still open (eval/PREREGISTRATION.md).
• It costs far more. A walk takes 6.7 tool calls, 23–111 seconds and $0.12–$0.28 per question, against one sub-second embedding call. That is 20–100× the cost. For 320 of the 700 questions, retrieval already found the answer first time.
• The map is a single point of failure. The one miss in 700 came from a wrong sentence in the map, not a wrong document, and the agent followed it. An agent that trusts the map inherits the map's errors, and nothing flags them.

Flat or folded (eval/depth/, 2026-10-09/10). The same 73 questions on two trees: area tables of 15–23 rows behind holders, against the same documents with every holder removed (61–244 rows per area). With a Sonnet-class walker both trees scored 65/73, with Opus 67/73 (the same questions missed on both); the flat tree cost 2.3–2.4× the input. With a Haiku-class walker the flat tree scored higher — 66 vs 61 — because every holder is one more decision a weak model can get wrong. Folding buys tokens, not accuracy, and costs a weak model accuracy.

Behaviour (eval/philosophy/, 2026-10-10). Twelve questions through the real MCP door, Sonnet and Haiku: every walk started at hop 0 with a reason on every step; every question with no answer in the documents was answered "not here", with no invented rule; deliberately false lines were answered around by reading the document. But the agents did not report the false lines until one instruction asked them to (1/4 → 4/4), nor a false hop-0 sentence until the server pointed out that the walk had switched areas (0/2 → 2/2). The census-style miss was reproduced on purpose: a false hop-0 sentence into an area whose plausible page does not point onward trapped 3 of 4 walks; one more instruction brought that to 2 of 4, and the remaining case — a partial table that looks complete — is not fixed. Twelve questions, one run each — read it as "did not fail", not as a rate.

Everything is in the repository: the corpus generator (bench/), the gold sets (eval/gold/), and every run including the failures (eval/runs/). Re-running needs an API key — bench/README.md.

When not to use it

• Your documents do not resemble each other. If retrieval already finds the right page, a walk only adds cost and latency.
• The cost of each question matters more than its accuracy. A walk takes several model calls and tens of seconds.
• Nobody will own the map. A table that no one maintains goes stale. A stale table is worse than none, because the agent follows it confidently.
• You need it hosted, multi-tenant or hardened today. See below.

Known limits

Where it stands

A research prototype with one maintainer and no outside team known to be running it.

Worth trying if your team keeps answering from the wrong revision, or retrieval keeps returning the neighbouring row. Reports of where it fails are the most useful thing you can send: GitHub Issues.

Quickstart

Docker with docker compose. The containers include Python and git.

git clone https://github.com/CSP911/routemind.git routemind && cd routemind
./install.sh --port 9000

→ http://localhost:9000**. On a first install it asks whether to start from the example back office (examples/back-office: five areas, 74 documents) — say yes unless you have data of your own ready. You can run it again safely: an existing .env is kept. An optional LLM adds a **✨ Suggest button that drafts a line for you to edit; nothing else depends on it. docs/INSTALL.md · already running one: docs/UPGRADE.md.

An agent. Claude Code finds the server in .mcp.json, which reads the port from .env:

claude          # in the same directory; approve the "knowledge" server once, then /mcp lists four tools

For any other MCP client, run the same server — it finds the install the same way, or take --api:

python3 mcp/knowledge_mcp.py      # or: --api http://localhost:9000/api/knowledge

The area list reaches the agent in the server's instructions, so you do not have to name RouteMind in the question. docs/AGENTS.md.

More

Layout: ontology/ the API · web/ the map and proxy · mcp/ the agent's door · static/ the screen · check/ every check · data/repo your map, the only thing to back up.

Contact

qct8377@gmail.com · linkedin.com/in/cspark911 · GitHub Issues

License

MIT — see LICENSE.