madeleine

madeleine

Involuntary memory for AI agents. A place, an hour, a sound, a photo or a sensor reading brings back a related memory on its own, without the agent having to ask. It works the way the taste of a madeleine brought back Proust's childhood.

· Python 3.11–3.13 · Apache-2.0 · local-first (SQLite) · MCP server + Python library

Most agent memory is voluntary: the agent decides to search, writes a query, and gets results. madeleine adds the other pathway. Every memory is stored with the sensory fingerprint present when it was encoded: where, when, what was said, seen or heard. Incoming cues from webhooks, phones, smart homes and wearables are matched against those fingerprints all the time. When enough channels line up, the memory is queued as an involuntary recollection, and the agent receives it with its next prompt. This is encoding specificity applied to agents (concepts).

60-second quickstart

# 1. See the whole loop, offline (nothing is downloaded, nothing leaves your machine)
uvx madeleine-mcp demo

The demo remembers three moments. Then a cue arrives that carries only a place and a time of day: 30 m from grandmother's house, on a Sunday afternoon. The matching memory surfaces by itself, is delivered through the MCP tool pending_recollections, and does not fire again for a repeated cue:

[madeleine 0.80 via geo+time] Sunday afternoons in grandmother's kitchen: lime-blossom tea and a madeleine (2019-04-14)
  channel similarity geo=0.97, time=1.00; salience x1.16, age decay x0.70

# 2. Give Claude Code the tools
claude mcp add madeleine -- uvx madeleine-mcp serve

1. Let involuntary recollections reach every prompt. Add this to .claude/settings.json:

{"hooks": {"UserPromptSubmit": [{"hooks": [
  {"type": "command", "command": "uvx madeleine-mcp hook claude-code", "timeout": 10}
]}]}}

The hook works even if your client ignores MCP notifications. See examples/claude_code for options such as --sense-prompt, which treats what you type as a cue too.

Semantic text needs the [text] extra. The base install is small and offline. Without the extra, text is matched lexically by a hashing embedder, so "car" and "automobile" count as unrelated. For real sentence embeddings (this pulls in PyTorch):

claude mcp add madeleine -- uvx --from 'madeleine-mcp[text]' madeleine-mcp serve

Images ([image], open_clip) and sound ([audio], LAION CLAP) are opt-in the same way. [all] installs everything. Models download lazily on first use, never at import.

How it works

flowchart LR
    subgraph Sources
      HA[Home Assistant] -->|webhook| HTTP
      IOS[iOS Shortcuts] -->|webhook| HTTP
      AG[Agent] -->|MCP tools| MCP
    end
    HTTP[HTTP API /cues] --> ENG
    MCP[MCP server] --> ENG
    subgraph ENG[engine]
      direction TB
      CH["channels<br/>adapter → embedder → similarity"] --> FU["fusion<br/>base · coverage · salience · decay"]
      FU --> RF{"push?<br/>threshold + refractory"}
    end
    ENG <--> DB[(SQLite + sqlite-vec)]
    RF -->|enqueue| Q[(recollection queue)]
    Q -->|drain| HOOK[Claude Code hook]
    Q -->|drain / notify| MCP

• Channels are pluggable senses. Each one has an adapter (raw signal → normalised observation), an embedder (observation → vector) and a similarity function (→ [0, 1]). Built in: text, time (hour, weekday and season, all cyclical), geo (distance on the sphere), biometric, image, audio. smell is a documented stub for future e-noses.
• Recall combines per-channel similarity over the channels the cue and the memory share, adjusts for how much of the cue is covered, boosts salient memories, and decays with age down to a floor, so decades-old memories stay reachable. The formula is short and explainable; every result says which channels matched (ADR 0001).
• Two modes. Pull: the agent asks (recall). Push: a cue arrives (sense, POST /cues) and memories that clear the threshold are queued without anyone asking.
• Habituation. A memory that fired stays quiet for a refractory period (default 6 h). Walking past the same café all afternoon reminds you of it once.
• Context alone never fires. "It's 9 am" matches every memory made at 9 am, so time (and biometric) can strengthen a match but never cause one.

Sending cues

madeleine serve --transport http          # REST + MCP (streamable HTTP at /mcp/) on 127.0.0.1:8765
curl -s localhost:8765/cues -H 'content-type: application/json' \
  -d '{"lat": 48.30195, "lon": 1.24650, "signals": [{"channel": "text", "value": "smell of lime-blossom tea"}]}'

Ready-made integrations: Home Assistant (arriving home, media starts playing, wearable sensors), iOS Shortcuts ("Arrive at location" personal automation), curl.

MCP surface:

• Tools: remember, recall, sense, pending_recollections, forget.
• Resource: recollections://pending, with resources/updated notifications when something surfaces. These notifications also work across processes that share the database.

Privacy and security defaults

• Local-first. Everything lives in one SQLite file (~/.madeleine/madeleine.db, owner-only permissions). There is no telemetry and no cloud.
• Raw media is not kept. Image and audio bytes are embedded and then discarded. Only vectors and metadata are stored, unless you pass store_raw=True (per call or MADELEINE_STORE_RAW=1).
• No local file access by default. Clients must send media bytes. Allowing file paths takes madeleine serve --allow-file-uris.
• HTTP binds to 127.0.0.1. Host and Origin checks block DNS rebinding. Set MADELEINE_API_TOKEN to require a bearer token. Binding to another interface without a token is refused unless you pass --insecure-no-auth.
• forget deletes everywhere: memory, vectors, raw bytes, trigger history and queued recollections, with SQLite secure_delete on.
• Recollections are data, not instructions. The hook wraps them in delimiters and labels them as such. Anyone who can write memories can still put text in your agent's context, so keep write access to yourself. See SECURITY.md.

Benchmark (and where it fails)

madeleine bench runs an offline synthetic benchmark: 500 memories across 3 cities, 300 cues with known ground truth, distractors and hard negatives. The numbers below are for the offline hash text backend. Full tables, methodology and blind spots are in docs/benchmarks.md.

Latency with 500 memories: sense p50 11 ms and p95 23 ms, in pure Python.

Honest limitations

• Involuntary recall is tuned to be conservative: no negative cue fired, but only about half of positive cues fire their memory.
• With the hash backend, text alone rarely intrudes (21 %) even when it is findable (R@10 = 1). Semantic embeddings ([text]) should help, but those numbers are not reported here yet.
• Altered media never matches with the hash backend. That needs the CLIP and CLAP extras, and their similarity rescaling has only been calibrated on synthetic inputs.
• Place + hour cues saturate (3 fires per cue), and only 56 % of those fall in the strict "same place, same hour ±1 h" ground truth.
• The fusion is a hand-designed formula, not a learned one. The data is synthetic English text. There is no multi-user auth and no UI. The olfactory channel is a stub.
• Hour-of-day uses each timestamp's own UTC offset. Send local offsets (+02:00), or "evening" means UTC evening.

Configuration

CLI: madeleine serve | remember | sense | recall | pending | forget | demo | bench | hook claude-code (madeleine --help).

As a Python library

from madeleine.config import load_config
from madeleine.core.serde import context_from_fields
from madeleine.engine import create_engine

engine = create_engine(load_config(db_path="memories.db", text_backend="hash"))
now = engine.now()
engine.remember("Ana proposed at the Eiffel Tower",
                context=context_from_fields(now=now, lat=48.8584, lon=2.2945, salience=0.9))
cue = engine.build_cue(context=context_from_fields(now=now, lat=48.8585, lon=2.2946))
for rec in engine.sense(cue):          # involuntary: fires on place (+ time) alone
    print(rec.to_prompt_line())

Write your own sense with an Adapter, an Embedder and a similarity function (docs/channels.md). ImageBind is deliberately not a dependency because its license is CC-BY-NC, but you can plug it in yourself as an Embedder.

Development

See CONTRIBUTING.md. Run uv sync once, then before every commit: uv run pytest, uv run ruff check src tests, uv run mypy. The architecture is in ARCHITECTURE.md and the design decisions are in docs/adr.

License

Apache-2.0. See LICENSE.