CompareGraph

CompareGraph

Compare many code repositories feature by feature: which one has what, with the file and lines that show it.

Get started · Usage · Commands · MCP tools · FAQ · Troubleshooting · Reports

You keep clones of the projects you care about, your own and other people's, and you write down the features you want to compare. CompareGraph reads every clone into one graph and helps you, or your coding agent, fill in a table of features against repositories. Each answer is has, partial or none, and each points at the code, or at the search that found nothing. It is free and open source (Apache-2.0), one program on your machine; it reads clones and never runs them, and the engine never calls an AI model or the network.

What you will see in ten minutes: the same kind of table for a small sample, on your screen, after installing and six commands. On your own clones, add the build: 0.82 s for 85 files to 6.20 s for 797 files on our host (timing report).

---

Get started

One binary, no runtime. Linux x64 and arm64, macOS arm64 and x64, Windows x64; anywhere else, build from source.

1. Install the binary

The install script downloads one file, the comparegraph binary for your platform, checks it against the release's SHA256SUMS, and copies it to ~/.local/bin. It does not use sudo and changes nothing else. Read it first if you like:

# macOS / Linux: fetch the script from the v0.1.1 tag, read it, then run it
curl -fsSLO https://raw.githubusercontent.com/MassRelayAI/comparegraph/v0.1.1/scripts/install.sh
less install.sh
COMPAREGRAPH_VERSION=v0.1.1 sh install.sh

# the same in one line, script from main, latest release
curl -fsSL https://raw.githubusercontent.com/MassRelayAI/comparegraph/main/scripts/install.sh | sh

# Windows x64 (PowerShell)
irm https://raw.githubusercontent.com/MassRelayAI/comparegraph/main/scripts/install.ps1 | iex

Windows: written to %LOCALAPPDATA%\Programs\comparegraph and added to your user PATH; open a new terminal. COMPAREGRAPH_INSTALL_DIR moves the binary. The scripts check the checksum only. Releases from v0.1.2 also carry a build-provenance attestation, which the scripts do not check; check it yourself with gh attestation verify <asset> -R MassRelayAI/comparegraph (release notes).

No pipe to a shell: Homebrew, npm, by hand, from source

Homebrew, macOS and Linux: brew install massrelayai/tap/comparegraph.

npm, any of the five platforms with Node 18 or newer (the package wraps the same release binaries): npx -y @MassRelayAI/comparegraph --version, or npm i -g @MassRelayAI/comparegraph.

By hand: comparegraph-<platform> and SHA256SUMS from the latest release; check with sha256sum -c --ignore-missing SHA256SUMS. From source, anywhere Rust 1.85 or newer runs: cargo install --locked --git https://github.com/MassRelayAI/comparegraph comparegraph (--locked matters: the Perl grammar's newest release on crates.io is yanked, and only Cargo.lock resolves).

2. See it work on the sample

The sample repository and the example list of features live in this repository, not in the binary, so the first command downloads a copy of it. Run these in any folder; they make two folders side by side, comparegraph and cg-try.

git clone --depth 1 https://github.com/MassRelayAI/comparegraph   # the sample lives here
mkdir cg-try && cd cg-try                                           # your workspace
comparegraph add ../comparegraph/fixtures/typescript --name tsfix   # read, never run
comparegraph build tsfix                                            # parse it
comparegraph family add ../comparegraph/families/example-code-graph-tools
comparegraph map tsfix example-code-graph-tools                     # places to read
comparegraph claim tsfix symbols.extraction has --author me --at src/util.ts:11-11
comparegraph matrix example-code-graph-tools                        # the table

The table shows has for symbols.extraction and - (not yet answered) for the other nine features. The commands take under a second in all. Usage shows the real output of every step.

3. Compare your own repositories

The example pack describes code-graph tools. For your own field you write the list of features yourself, as a small rows.md; three rows are enough to start.

comparegraph add ../your-lib --name mine --as-root   # any clones on your disk
comparegraph add ../their-lib --name theirs
comparegraph build mine && comparegraph build theirs
comparegraph family propose libs --repos mine,theirs # optional draft

family propose drafts a list of shared words (name, file, extract, ...), not features; keep the ones that matter, define them, and save the file as families/libs/rows.md:

---
family: libs
description: the three features I want to compare
---
| id | group | feature | definition | counts_as_has | tier | weight |
|---|---|---|---|---|---|---|
| search.fulltext | search | Full-text search | Search over names or text through an index | a query entry point over a full-text index | basics | 2 |
| search.embeddings | search | Embedding search | Search by vector similarity | code that embeds text and ranks by similarity | extra | 1 |
| graph.communities | graph | Module communities | Groups of related code found from the graph | a clustering pass over the graph | extra | 1 |

Then comparegraph family add families/libs, map, claim and matrix libs as in step 2. map gives places to read, not verdicts: you, or your agent, open them and write each claim. The column rules are in Usage.

4. Wire up your agents (optional)

comparegraph setup --apply      # omit --apply for the plan; --scope project for an entry you commit
comparegraph doctor             # each client's state: aligned, stale, conflict, missing, plugin

Registers the MCP server with every agent it finds on the machine (twelve: Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Hermes, Windsurf, Antigravity, Kiro, Copilot, Copilot CLI, Qwen Code), writes a short rules block into the workspace's instruction file, and for Claude Code a session-start hook. It only wires agents; it indexes nothing. Already installed? comparegraph doctor tells you what is wired.

Claude Code as a plugin, or your agent's own command

Claude Code as a plugin: the seven skills, the server and the hook in one install (Claude Code 2.1.275 or newer; claude plugin install … does the same from a shell):

/plugin install comparegraph --marketplace MassRelayAI/comparegraph

With the plugin enabled, setup --apply leaves Claude Code's server and hook to the plugin (their rows read plugin) and still writes the rules block.

Your agent's own command, run in the workspace; each writes the user-scope entry setup would (Claude Code's default scope, local, is one doctor does not read), so doctor reads it as aligned:

The server is listed in the MCP Registry as io.github.MassRelayAI/comparegraph; the table has every client's file.

To uninstall, see Agents, platforms, uninstall. Free and open source, with no paid tier; questions and bugs go to GitHub issues, security reports to SECURITY.md.

---

Why CompareGraph

Measured on six study copies and 30 features, binary 0.1.1: numbers in docs/assets/cost.json, method and caveats in the answer-size report.

You have many repositories and one question: who has what? Reading each by hand does not scale, a summary without a line number cannot be checked, and notes go stale when the code moves. Five properties, each one you can test:

1. Deterministic. The same clone and version give the same store_hash. comparegraph check rebuilds and compares, and an incremental build must equal build --full (equal on both large trees in the Plan 6 report).
2. An evidence class on every answer. Each envelope says observed, derived, inferred or unknown, plus a limitations[] list of what the engine cannot see.
3. A claim needs a citation that resolves now. has and partial need --at path:start-end matching a stored node; otherwise the engine refuses. A moved citation shows as unknown* after check.
4. none needs proof of absence. Either --searched terms that find nothing, the same terms with every hit named in --read-hits as opened and rejected, or the graph check: four channels (symbols, paths, community terms, node kinds) all empty. A none with --searched must name as read the top 10 ranked candidates that absent-check --searched returns (read_set); candidates beyond the cap are recorded as a read_capped limitation. An unread candidate in the top 10 refuses the claim and names it.
5. No model in the engine. Scores, bands, descriptions and candidates are computed from stored facts. An agent may do the reading; the engine only stores, checks and refuses.

Against a single-repo code graph

For: teams comparing their product with the field, due diligence on a set of projects, a newcomer mapping a field (family propose drafts the taxonomy), and coding agents that need compact facts over MCP.

---

How it works

The three rules

1. The engine never calls a model. Every score, band, description and candidate is computed from stored facts.
2. A claim needs a citation that resolves now. has and partial need --at path:start-end matching a stored node. none needs --searched terms that find nothing, or the graph check. Otherwise the engine refuses or writes unknown.
3. Claims and decisions are append-only, enforced by the database. A change is a new row naming the one it supersedes. The latest wins, and the trail stays.

A candidate from search or map is labelled candidate, never has. Design: docs/design.md.

The flow and the code rings as diagrams: docs/design.md.

A family in the graph

A family is a taxonomy: groups of features, each with a definition and a test for what counts as having it. A cell is one clone against one feature, holding a verdict (has, partial, none) and, for has and partial, a citation. The picture draws the family code-graph-tools with the agreed cells of the mapping gold; comparegraph export vault shows the same graph live in Obsidian.

Plugins and the second engine

An external provider is any executable that speaks JSON lines on stdin and stdout: the host writes one request line, closes stdin, and reads answer lines until the first ok answer. Everything it prints is data, parsed into our own types. Register one in comparegraph.toml ([providers.mine] command = "tools/my-provider") or as comparegraph-provider-<name> on PATH, with a comparegraph-plugin.toml manifest beside it (kind, name, protocol = "1", languages, network). Without a manifest it runs only with --allow-unverified. The manifest is a declaration, not a signature: no hash of the executable is checked, and a provider runs as its own program with your permissions. Prefer a path in comparegraph.toml, which is looked up first, to a name found on PATH. A manifest's network value is passed to the provider as COMPAREGRAPH_NETWORK, not enforced.

Full protocol, guards, a 25-line Python provider and the code-review-graph adapter (a second engine behind build --providers reference,crg): docs/dev/plugin-protocol.md.

Families and the seven skills

A family is a directory with a rows.md; two packs ship, families/example-code-graph-tools (10 features, the quick start) and families/code-graph-tools (30 features, the mapping gold). The column rules, synonyms and family propose are in Usage.

The method is seven plain-markdown skills in skills/ that any agent can read. Each gives when to use the verb, the exact commands, what the engine refuses, and how to check. In Claude Code the skills and the MCP server arrive together as one plugin (Get started); other agents read the skills/ directory from the clone.

---

Using it

Four pages under docs/ hold the reference; each is a few clicks from here.

What you get

Four levels, each answering how it knows: the repo surface, module communities, symbols (callers, callees, blast radius, tests, entry points, dead-code candidates) and claims (verdict, citation, author, trail). Every answer is one JSON envelope with an evidence class. Usage has the table and a real show answer.

CLI reference

Global options --root and --pretty, then verbs in four groups: build and inspect, graph queries, families and claims, exports and agents. Commands lists every verb with the flags that matter.

MCP tools

comparegraph mcp serves eighteen tools over stdio (seventeen read-only, one write: comparegraph_claim); there is deliberately no decide tool. MCP tools has the table; docs/catalog.json is the machine-readable source and catalog --check guards it in CI.

The vault, configuration, library

comparegraph export vault --out ./vault writes an Obsidian-style markdown vault of notes with generated descriptions. Settings are --root and COMPAREGRAPH_* environment variables, and the crate exposes each verb as a function. All three are in Usage.

---

How good is it

Every number below is copied from a report in this repository; read its caveats before quoting one. All were measured on 2026-10-13 on one shared host.

On the hand-labelled gold of call sites, CompareGraph reaches precision 0.995 and recall 0.834 pooled, against 0.956 / 0.736 for code-review-graph through its adapter and 0.708 / 0.506 for codegraph. It builds a 38,258-file repository in 499.6 s, and impact on a 1.29M-edge graph answers in 2.5 s; the tables below carry the caveats. Data: accuracy.json, scale.json; parity report: engine parity.

Call edges against a hand-labelled gold standard

The gold: 150 call sites per project (750 in all) drawn at random from each project's own source, labelled by two readers who worked independently under one rubric (both are language-model agents; none was run blind to the engines). final keeps only the rows both agree on. How the gold is made: fixtures/gold/README.md. Precision is over answered sites, recall over repo-target sites (shown in brackets). Kappa is Cohen's kappa on the call target between the two readers.

Pooled counts: CompareGraph 196 correct of 197 answered, 196 of 235 sites; codegraph 119 of 168 answered, 119 of 235. Sources: docs/reports/2026-10-12-engine-parity.md, sections "After Plan 6" and "codegraph head-to-head"; kappa from "After Plan 5", inter-reader agreement (pooled is_call kappa 0.94). Reproduce: scripts/gold-score.sh <project-copy> fixtures/gold/<project>/calls.jsonl --binary <comparegraph>, scripts/compare-engines.sh (all engines), scripts/h2h-codegraph.sh (codegraph).

Read it with these limits. Sites per project are 36 to 53, so per-project noise is about 0.1. caddy 1.000 means "every labelled site" (48 sites drawn before the Go work), and hono precision rests on 24 answered sites. codegraph is run in its --graph-only mode through one get_callees call per site; where it lists several same-named targets our scorer counts the site correct if the true one is among them. On hono none of its sites is answered, because the tightest node it finds there is an anonymous arrow function with no call edges; a lookup through the nearest named ancestor was not run. The floors in fixtures/gold/FLOORS.json are asserted by crates/comparegraph/tests/gold.rs and are only ever raised.

Build and query time

The numbers are in docs/assets/scale.json, from the proof report (last section) and the Plan 6 report.

Caveats from the reports: build times ran at load 6 to 15 on 40 cores, so read the ratios, not the seconds. The hermes-agent build is 1.40x slower than at the end of Plan 6 lane B1, a regression that was measured and not diagnosed. Query bounds are one hub symbol (3,156 callers), at defaults, one run each. A 4.4k-file private product monorepo builds in 103 s (53k nodes) and its three harness comparators in 25 s, 81 s and 436 s (proof report, last section); the ranked map over 12 features takes 1.2 s on the product and 12.1 s on the 454k-node comparator. Reproduce: scripts/compare-engines.sh, scripts/h2h-codegraph.sh, and COMPAREGRAPH_PROFILE=1 comparegraph build <repo>.

Overlap with code-review-graph (symbols and edges)

Jaccard over nodes and resolved edges, on three projects (parity report, "After the review fixes"; symbols and extends from "After the precision and recall pass"):

Test links are the weak spot on all three projects. Reproduce: scripts/compare-engines.sh.

Conformance fixtures (reference engine)

comparegraph conform reference runs the golden fixtures in fixtures/; a provider below the floor (nodes 0.95, edges 0.85, recall and precision) is refused. Output at this commit:

Five fixtures are five small repositories: they prove the extractors against a hand-checked graph, not every language feature.

Feature mapping gold

A second gold standard scores map (feature to code) and the graph-backed absent check: 30 features of families/code-graph-tools/rows.md against six study copies of code-graph tools, 180 pairs read by two independent readers, 121 agreed and 59 disagreements left out of the scores (13 agreed locations have no node in the engine graph) (method and kappa: fixtures/gold-map/README.md). The ranking was tuned on three of the clones and measured once on the other three; the held-out column is the honest number for a repository the tool has never seen.

The target (held-out recall@5 at or above 0.50 at precision@5 0.30) is open: a gold location is among the first 200 candidates for only a third of the held-out features, because the store holds symbol names, paths and community keywords and no doc text; the next lever is in the roadmap. relate bands are a known weak signal: repositories outside the family still score above 0.5 (docs/decisions.md, "Plan 7 IMP"); until relate has been run, the matrix reads not run for that clone and says so in limitations[] (matrix --relate scores the missing ones first). Sources: docs/reports/2026-10-14-mapping-gold-first-measurement.md and the parity report's "Feature-mapping baselines". Reproduce: scripts/map-score.sh, scripts/map-baselines.sh.

Answer size

For 30 features on six study copies, the map answers total 336 KB (about 84 k tokens, an estimate) against 106.1 MB of source (about 26.5 M tokens): 316 to 1. They are ranked places to read, not verdicts (held-out recall@5 0.149, above), so the reader still opens files. Method, per-clone numbers and the caveat: docs/reports/2026-10-14-answer-size.md. Reproduce: scripts/answer-size.sh, then node scripts/readme-assets.mjs.

Languages

29 languages are parsed. Five are resolved deeply. Four have a measured gold standard. Read the tiers, not the headline. Every grammar (32 ids: tsx, jsx and mjs count apart) has a fixture with a committed graph digest in fixtures/languages/, so a grammar bump that changes any graph fails CI and is re-golded on purpose.

Limits, known gaps and skipped files

The registry holds 32 identifiers (tsx, jsx, mjs count with their language). Unresolved languages bind a call only by name, at confidence 0.5, and only inside the same language family. Known gaps: a Java new T() has no call site, a Rust trait method and its impl share one qualified name, and Go calls on a method result, range variables and generics bind nothing yet (Plan 7 candidates). Skipped on every build: the directories node_modules, target, dist, build, .git, vendor, __pycache__, .venv, venv; files that .gitignore or .ignore exclude; minified files. Skips are counted in the run's limitations.

---

Agents, platforms, uninstall

comparegraph setup writes the entry; comparegraph doctor reports each client's state (aligned, stale, conflict, missing, unverified, plugin). --client all acts only on clients that look installed. plugin: left to the Claude Code plugin when comparegraph@comparegraph is enabled and --client all is used; name --client claude to write anyway. A bare comparegraph command that resolves on PATH to the registered binary counts as aligned. setup --apply is additive and leaves a backup named <file>.bak-<unix-seconds>; a differing entry is left alone unless you pass --force.

setup also writes a rules block into the agent's instruction file and, for Claude Code, a SessionStart hook; Usage has the details.

The twelve clients, their files and what to remove

To uninstall the rules block, delete the lines from to; to uninstall the hook, delete the SessionStart entry whose command is comparegraph doctor --brief.

Platforms: every release builds linux-x64, linux-arm64, darwin-arm64, darwin-x64 and win32-x64 on their own runners and ships each as a release asset listed in SHA256SUMS; the Homebrew tap follows the release. The test suite runs on linux-x64 only. Anywhere Rust 1.85+ runs, build from source. Local only: the engine makes no network call (how to check it yourself, and what is not yet enforced: FAQ), the MCP server speaks over stdio, the store is one SQLite file you own.

Uninstall: remove the entry above (in Claude Code with the plugin, /plugin uninstall comparegraph), then delete the binary (~/.local/bin/comparegraph, brew uninstall comparegraph or cargo uninstall comparegraph), and rm -r .comparegraph in each workspace.

---

FAQ and troubleshooting

• FAQ: how it differs from code-review-graph and codegraph (and what they have that it does not), whether it uses a model (no), whether it adopts code for you (no), why some call edges are 0.5, whether it is free and who supports it, how to check that it sends nothing, what it does with a repository you do not trust, and what it does not do.
• Troubleshooting: twelve symptoms with their causes, from no workspace and a refused none claim to a silent session-start hook.
• Start with comparegraph doctor: it reports the store, every provider and each client's registration.

Roadmap

Nothing below exists yet, and the maintainer decides the order. Full list with the measurement behind each item: docs/roadmap.md.

• Next measured lever: a text channel in the store (doc-comment and signature tokens) and per-family synonyms, for map on unseen clones.
• Port-kit export for an adopted feature; feature stats export; claim import and family import/export.
• Graph-parity queries, a C/C++/C# resolver, a per-file analysis cache, more clients (Zed, Continue).

---

Documentation

---

Contributing, licence, third-party code

Security reports go to support@massrelay.ai or a private advisory (SECURITY.md); conduct reports to the same address (CODE_OF_CONDUCT.md).

Read CONTRIBUTING.md and AGENTS.md first. A bug fix starts with a failing test, extraction changes extend a conformance fixture, and no model or network call enters the engine. CI runs clippy, the suites, the conformance and language digests, and cargo-deny (licences, advisories, sources; deny.toml). The vendored parser layer comes from graphmind (MIT); see docs/THIRD_PARTY_NOTICES.md. Licence: Apache-2.0, see LICENSE.