Playwright E2E MCP

playwright-e2e-mcp

The test doctor for AI agents.

Your Playwright test fails. Your AI finds out why, fixes it, and proves the fix works.

What it is

An MCP server and agent skill that gives your AI assistant (Claude, Cursor, VS Code Copilot, Gemini, Codex) a full Playwright toolkit: a test runner, a failure detective and a fixer that checks its own work.

Other tools let an AI click around a browser. This one lets it keep your test suite green.

run-test ──▶ get-failure ──▶ suggest-fix (apply + re-run) ──▶ green
   │              │                 └─ refuses when the element is really gone (app bug)
   │              └─ DOM at failure, failed requests, console errors
   └─ background runs, progress ──▶ analyze-history (flaky / broken / regressed)

What it does

Why it stands out

• Fixes are proven, not guessed. Every fix is re-run before it's kept.
• Honest. It won't paper over a real bug to make a test pass.
• Lean. In the benchmark, finding and fixing a broken locator took about 5,300 tokens here versus about 7,400 with Playwright MCP plus a shell.
• Teaches as it works. The bundled skill carries a guide to writing tests that stay green (locators, waiting, auth, test data, mocking, flake patterns, review checklist).
• Ready in one line. npx -y playwright-e2e-mcp init sets up Claude Code, Cursor and VS Code.

Free and open source, for good

• MIT licensed. Use it, change it, ship it in commercial work; no fees, no paid tier, no feature held back.
• No account, no API key, no sign-up. npx -y playwright-e2e-mcp init is the whole setup.
• No telemetry. It never phones home; your tests, traces and run history stay on your machine.
• Free dependencies only: the MCP SDK (Apache-2.0), zod (MIT), axe-core (MPL-2.0) and the Playwright your project already has (Apache-2.0).
• Free to host yourself, too: the HTTP bridge deploys to a free Vercel account (see Hosted endpoint).

The technical details

• Structured runs. Stats plus each failure's file:line, kind (assertion, timeout, dead dev server, syntax error, crash, full disk…) and a concrete fix hint. Failures that share one cause are grouped. Long suites run in the background with live progress.
• Diagnosis from the trace. The DOM at the moment of failure, the failed action, the 4xx/5xx requests and console errors, read from Playwright's own trace.
• Verified fixes. suggest-fix matches a broken locator against the DOM at failure, swaps in a Playwright locator proven unique on the page (getByRole('button', { name: 'Proceed to checkout' })), re-runs the test and reverts the edit if it is still red. When nothing on the page plausibly matches, it says the element is missing and does not patch the test.
• Flakiness with evidence. diagnose-flaky repeats failing tests in one process and gives a verdict per test; isolate: true re-runs them alone to tell a shared-state bug from a race inside the test, and order: true runs files in random orders to catch an order dependency, naming the test and the seed that reproduces it. analyze-history separates flaky, regressed and broken tests from the local run record.
• Accessibility. audit-accessibility runs axe-core against WCAG 2.2 AA on any page, signed in or after steps, with offending elements as verified locators.
• Pages behind a login. Every page tool accepts a Playwright storageState, extra headers, and click/fill steps to reach a modal or a later step.

What it does not do

It doesn't drive an exploratory browsing session step by step, and it doesn't replace Playwright's own Test Agents inside VS Code. Use Playwright MCP for interactive browsing and this server for running, diagnosing and fixing the suite. They work side by side.

Demo

Hosted endpoint — what an initialize + tools/list round-trip against https://playwright-e2e-mcp.vercel.app/api/mcp returns for a client that sends the bearer token (without one, only list-tests and get-failure are listed):

A real test run — run-test served over stdio by npx -y playwright-e2e-mcp against the bundled examples/sample-test.spec.ts (actual output, unedited):

Images are rendered with node scripts/gen-demo-images.mjs: the run-test card is real captured output; the endpoint card is an illustration of the authenticated listing.

If this saves you a debugging session, a ⭐ on the repo helps other Playwright users find it.

Install

Works with Claude Desktop, Claude Code, Cursor, Windsurf, Codex, Gemini CLI, Freebuff and every other MCP client — pick whichever route fits:

Let your agent do it. Paste this into Claude Code, Cursor, Codex, Gemini CLI or any other coding agent:

Install the playwright-e2e-mcp server and skill by following https://raw.githubusercontent.com/xsignon-secure/E2E/main/llms-install.md

or run the installer yourself from the project root. It merges the server into each client's project config it finds (Claude Code, Cursor, VS Code, Gemini CLI, Codex) and installs the agent skill, leaving your other servers alone:

npx -y playwright-e2e-mcp init            # --client all | cursor,vscode … · --dry-run

One line for the common clients:

claude mcp add playwright-e2e -- npx -y playwright-e2e-mcp
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","playwright-e2e-mcp"]}'

Only need the fix loop? --tools=core (or PW_MCP_TOOLS=core) exposes six tools and halves the tool-definition context; see Configuration.

Details and per-client config: Installation · MCP client configuration.

---

Tools

Two prompts package the main workflows for clients that show them as slash commands: fix-failing-test (optional test) and triage-flaky-tests.

Page state: signed-in pages, headers and steps

inspect-page, validate-selector and compare-visual-state accept the same page-state arguments (suggest-fix accepts storageState and headers for a live url):

A failing step reports which one failed and how many ran. When the page lands on a login screen, the response says so and suggests storageState.

run-test

Progress: clients that send a progress token get a notification as each test finishes (3/12 tests done, 1 failed). Every run is appended to the local history used by analyze-history.

Flakiness handling: by default the server injects --retries=1 (unless the config already sets retries), so a test that passes on the retry is reported as flaky, not failed. Traces are captured automatically (--trace=retain-on-failure) so get-failure can show the DOM at the moment of failure.

Example result:

## Playwright run — ❌ FAILED

**Command:** `playwright test --config playwright.config.ts tests/checkout.spec.ts --reporter=json`
**duration 4.2s · exit 1 · config `playwright.config.ts`**

| passed | failed | flaky | skipped | duration |
| ---: | ---: | ---: | ---: | ---: |
| 0 | 1 | 0 | 0 | 1.1s |

### ❌ 1 failing test(s)

### 1 of 1. checkout.spec.ts › pays with card
**File:** `checkout.spec.ts:5`  |  **failed · server-unreachable**

### ⚠️ SERVER_NOT_RUNNING
Your app (dev server) does not appear to be reachable. Start it in another terminal
(e.g. npm run dev / npm start), keep it running, then retry — or configure `webServer`
in playwright.config.* so Playwright starts it automatically.

get-failure

Returns the message/code frame, expected vs actual, stack, failure kind with a diagnosis, the test's console output, the DOM snapshot from the Playwright trace (plus the failed action, its selector, and the action log leading up to it), the network requests that failed (4xx/5xx, dead endpoints, no-response — with method, URL, status and resource type), the console errors/warnings the page logged before the failure, and numbered next steps (re-run this single test by file:line, headed/debug mode, validate-selector when the message mentions a locator, …).

inspect-page

Each element comes with Playwright locators that were checked on the page to resolve to exactly that element, best first (role + accessible name, test id, label, placeholder, text, then CSS), plus its unique CSS selector, visibility, box and text. Console messages are captured, errors first. A locator map looks like:

1. `getByRole('button', { name: 'Proceed to checkout' })` — button "Proceed to checkout" · also `getByTestId('checkout-btn')`
2. `getByLabel('Search')` — searchbox "Search"

list-tests

Uses playwright test --list when Playwright works, and falls back to a source scan (keeping the reason) when the install or a spec file is broken.

validate-selector

Locator expressions are parsed into a whitelisted call chain (getByRole, getByText, getByLabel, getByPlaceholder, getByAltText, getByTitle, getByTestId, locator, first, last, nth, filter) and are never evaluated as code.

Verdicts: ✅ VALID — N matches with a sample of matches and a unique locator for each, a strict-mode warning when more than one element matches, a hint when a CSS selector is brittle (:nth-child, long descendant chains, XPath), ✅ VALID — 0 matches with debugging advice, or ❌ INVALID with the parse error.

generate-e2e-test

Reads the agent's recent changes (git status, falling back to git diff HEAD~1, then recent mtimes), extracts the locators those files actually declare (data-testid, getByRole, aria-label, placeholder, id, name, element text), ranks verified-live selectors first, writes a spec built from them, and reports each selector with its source file:line. When the source has no usable selectors, it uses the verified locators from the live page that best match the description.

compare-visual-state

The first call saves a baseline under .pw-mcp/visual/ (add that to .gitignore, or commit it for CI comparisons). Later calls report changed-pixel counts, merged regions ((x, y) 120×40 — 1,200 px), the average color shift ("blue → red"), and write a red-highlighted diff image for review.

diagnose-flaky

All repetitions run in one Playwright process with --repeat-each and --retries=0, so each attempt is honest evidence and the browser and workers start once instead of once per run (about 25% faster on the integration fixture, more for suites with expensive setup). A small extra reporter records every attempt, because Playwright's JSON report keeps only one result per test.

The verdict is per test: a table of passes, failures, duration spread and first error for each test, plus the verdict block for the worst one:

• ❌ CONSISTENTLY FAILING — failed every run (same error → reproducible bug, different errors → still broken, just noisy). Fix it; it is not flaky.
• ⚠️ FLAKY — some runs passed. Includes N of M counts and whether the failures share one signature (real intermittent bug) or vary (timing/environment instability).
• ✅ NOT REPRODUCING — passed every re-run; the original failure was one-off.

With isolate: true a second table compares each failing test in parallel and alone:

With order: true a third section runs the spec files (not just the failing lines) in random orders with one worker and --fully-parallel, so every test is scheduled on its own. A test that fails in some orders is replayed in a failing order; if it fails again it is read as 🔗 order dependent, otherwise as flaky regardless of order:

| `tests/cart.spec.ts:10` checkout lists the book | 2/3 | `4` | 🔗 order dependent … |

**checkout lists the book**
- Relies on: `adds a book to the cart` (ran before it in every passing order, after it in a failing one)
- Reproduce: `npx playwright test tests/cart.spec.ts --workers=1 --fully-parallel --retries=0 --shuffle=4`

This is the failure that otherwise reads as "passes alone, fails in the suite". Set PW_MCP_SHUFFLE_SEEDS=4,7 on the server to replay given orders, for example to check a fix.

Each repetition is recorded in the run history, and the last failing run is stored, so analyze-history and get-failure work right afterwards.

audit-accessibility

axe-core (the engine behind Lighthouse's accessibility score) is bundled and evaluated inside the page, so a Content-Security-Policy does not block it. Example output:

## ♿ 3 accessibility violations (2 critical, 1 serious)

### 1. critical · `button-name`: Buttons must have discernible text (WCAG 4.1.2)
1 element · https://dequeuniversity.com/rules/axe/4.14/button-name
- `getByTestId('go')`: Element does not have inner text that is visible to screen readers

### 3. serious · `color-contrast`: Elements must meet minimum color contrast ratio thresholds (WCAG 1.4.3)
- `getByText('Low contrast')`: Element has insufficient color contrast of 2.32 (foreground color: #aaaaaa, …). Expected contrast ratio of 4.5:1

It ends with an @axe-core/playwright check for the spec, so the page stays fixed.

suggest-fix

Two kinds of fix:

• Locator drift. The broken locator is read from the error, its words are matched (with synonyms such as btn → button, checkout → proceed to checkout) against every element in the DOM at failure, and the best match's verified-unique locator replaces it. Confidence is high when the match is clear, medium otherwise.
• Changed copy. For toHaveText, toContainText, toHaveValue, toHaveTitle, toHaveURL and toHaveAttribute, it proposes the value the page actually showed. It is only applied with allowExpectationUpdate: true.

The response is a unified diff. With apply: true the file is written only if it has not changed since the failure, the single test is re-run (same project, no retries), and the edit is reverted when it is still red. If nothing on the page shares the locator's distinctive words, it refuses and reports that the element is most likely missing.

get-run-status

While running it reports tests done / total and failures so far; once finished it returns the full run-test result, which also becomes the last run for get-failure and suggest-fix. The 20 most recent runs are kept in memory.

analyze-history

Every run-test, diagnose-flaky and suggest-fix run appends a line to <project>/.playwright-e2e-mcp/history.jsonl (the folder carries its own .gitignore; nothing leaves the machine; PW_MCP_HISTORY=0 turns it off). The flaky view separates tests that flip between pass and fail, tests that started failing and stayed red (regressed), and tests that failed every run (broken), each with a PFFP~ timeline.

---

Installation

Requirements:

• Node.js ≥ 20 (the server is built on MCP SDK v2 — the 2026-07-28 spec line)
• A project with @playwright/test installed and browsers available (npx playwright install chromium)

No npm account needed — install straight from GitHub (the prepare script builds dist/ automatically on install). Pin a release tag: an unpinned github:xsignon-secure/E2E runs whatever is on the default branch at that moment.

npx -y github:xsignon-secure/E2E#v0.4.2
npm install -D github:xsignon-secure/E2E#v0.4.2 @playwright/test   # or as a project dependency
npx playwright install chromium

Or grab the packaged tarball from the repo's GitHub Releases page and install it locally:

npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.4.2/playwright-e2e-mcp-0.4.2.tgz

Listed in the official MCP Registry as io.github.xsignon-secure/E2E — registry-aware clients discover it there, and every v* release tag republishes the entry from CI via server.json.

MCP client configuration

Claude Code / generic (project-scoped):

{
  "mcpServers": {
    "playwright-e2e": {
      "command": "npx",
      "args": ["-y", "github:xsignon-secure/E2E#v0.4.2"],
      "env": { "PW_MCP_PROJECT_ROOT": "/absolute/path/to/your/project" }
    }
  }
}

Claude Desktop / Cursor / Windsurf / Cline: add the same block to their MCP config file (Cline: MCP Servers → Configure). Zed and other clients that take a command: run npx with the arguments -y playwright-e2e-mcp. The server uses its working directory as the project root; set PW_MCP_PROJECT_ROOT when the client launches it somewhere else (e.g. your home directory).

Codex / VS Code / Copilot CLIs:

codex mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.2
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","github:xsignon-secure/E2E#v0.4.2"]}'

Codex's defaults fight this server: the first launch clones the repo and runs tsc (measured 30 s on a cold npx cache, against a 10 s startup_timeout_sec default), and a Playwright run with retries beats the 60 s tool_timeout_sec default. Raise both in ~/.codex/config.toml:

[mcp_servers.playwright-e2e]
command = "npx"
args = ["-y", "github:xsignon-secure/E2E#v0.4.2"]
startup_timeout_sec = 60
tool_timeout_sec = 600

Claude Desktop (one-click): download and double-click the .mcpb Desktop Extension attached to the latest release — the bundle ships its own dependencies, so no Node setup is required. On install it asks you to pick your project root (required, no default: choose the project folder, not your home directory) and wires it into PW_MCP_PROJECT_ROOT, so the tools point at a real project from the first call.

Claude Code:

claude mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.2

Gemini CLI / Qwen Code: paste the mcpServers block above into .gemini/settings.json (Qwen Code: .qwen/settings.json) — both speak the same MCP settings format.

Freebuff / Codebuff (project-scoped): this repo ships a committed .agents/mcp.json, so opening the checkout in Freebuff attaches the server workspace-wide — no global config needed. Your own projects can do the same: drop an mcp.json with the block above into their .agents/ directory. Freebuff asks you to trust a repository's .agents/ on first run.

All tools ship MCP tool annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint), so clients can show accurate safety prompts before running anything.

From a local checkout:

{
  "mcpServers": {
    "playwright-e2e": {
      "command": "node",
      "args": ["/path/to/playwright-e2e-mcp/dist/index.js"],
      "env": { "PW_MCP_PROJECT_ROOT": "/path/to/your/project" }
    }
  }
}

Hosted endpoint (ChatGPT & remote clients)

Some clients — ChatGPT custom connectors especially — only accept remote HTTPS MCP servers and refuse to spawn a local npx process. This repo ships a Streamable HTTP bridge for exactly that case:

The bridge runs the same createServer() as the stdio transport; the SDK serves every request with a fresh server instance, which is what a serverless function wants. test/http-bridge.test.mjs drives the real Node adapter over node:http so a broken bridge fails in CI, not in ChatGPT.

This demo deployment is token-protected. https://playwright-e2e-mcp.vercel.app runs on the maintainer's account and has PW_MCP_HTTP_TOKEN set, so a request without Authorization: Bearer <token> (or an OAuth sign-in with it) gets 401 {"error":"unauthorized"}. Nothing is held back: the same bridge is free to host yourself on a free Vercel account, with your own token:

A serverless function has no browser, so for running and fixing your own tests the free local install (npx -y playwright-e2e-mcp init) is the better route.

Open vs. token-protected. With PW_MCP_HTTP_TOKEN set (at least 16 characters; use a random value) every tool is served to clients that send Authorization: Bearer <token>, and every other request gets 401. Without that variable anyone who reaches the URL can call the bridge, so it degrades to list-tests and get-failure in restricted mode: list-tests scans sources instead of running playwright test --list (which would execute the project's config), callers cannot pick another projectRoot, and nothing spawns a process, drives a browser or writes a file. Child processes started over HTTP get only an allowlisted environment. PW_MCP_ALLOWED_HOSTS (comma separated, * for any) limits the accepted Host header; without a token and without that variable, only localhost names and the deployment's own Vercel hostnames are accepted (DNS-rebinding protection).

Add it to ChatGPT: Settings → Connectors → turn on Advanced → Developer mode → Create custom connector → paste the endpoint above. Authentication None only works against a bridge with no token (read-only tools), so for this deployment choose the connector's API key authentication and send Authorization: Bearer <token> — or point the connector at your own deployment with PW_MCP_HTTP_TOKEN unset.

Add it to claude.ai (or Claude Desktop's connectors): Settings → Connectors → Add custom connector → paste the endpoint above and leave the OAuth fields empty. Claude registers itself and opens a sign-in page; type the deployment's access key (the value of PW_MCP_HTTP_TOKEN) and press Allow access. The bridge is its own small OAuth 2.1 server (discovery, dynamic client registration, authorization code with PKCE S256, refresh tokens) and keeps no state: client ids, codes and tokens are signed with a key derived from PW_MCP_HTTP_TOKEN, access tokens last an hour and refresh tokens 30 days, and changing PW_MCP_HTTP_TOKEN signs every connected client out. The sign-in page shows which site will receive access, so only type the key when that is the app you are connecting. Source: src/oauth.ts; the /.well-known/oauth-* and /oauth/* paths reach the function through the rewrites in vercel.json.

To self-host the open, read-only variant, deploy this repo with PW_MCP_HTTP_TOKEN unset; nothing else changes.

Codex can also take the remote transport instead of spawning npx, if you'd rather not ship Playwright to every machine. Point it at the token through an environment variable so the secret stays out of the config file:

export PW_MCP_HTTP_TOKEN=…   # ask the operator for the value
codex mcp add playwright-e2e-remote --url https://playwright-e2e-mcp.vercel.app/api/mcp \
  --bearer-token-env-var PW_MCP_HTTP_TOKEN

What to expect: list-tests works and reports the specs bundled with the deployment. Even with a token, tools that spawn a browser (run-test, inspect-page, validate-selector, diagnose-flaky, …) cannot download Chromium in a serverless function, so they return NO_PLAYWRIGHT with a note that points to the local install. Use the stdio install for real runs; the hosted endpoint is for discovery and for clients that cannot run local processes.

# verify the handshake without any client (the live deployment needs the token;
# drop the authorization header and it answers 401 instead)
curl -X POST https://playwright-e2e-mcp.vercel.app/api/mcp \
  -H 'content-type: application/json' \
  -H 'accept: application/json, text/event-stream' \
  -H "authorization: Bearer $PW_MCP_HTTP_TOKEN" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1.0"}}}'

Redeploy after a change: merge to main. Vercel's Git integration deploys every push, so there is no token to manage and no CLI step — watch the Vercel commit status for the deployment result.

CI: GitHub Action and ci command

The same diagnosis runs without an agent. playwright-e2e-mcp ci runs the suite (auto-retry once, so flaky tests are reported as flaky, not failed), optionally drafts suggest-fix diffs and a flaky verdict for the failures, and writes one Markdown report to stdout, to --output, and to the GitHub job summary when GITHUB_STEP_SUMMARY is set. Exit code 0 passed, 1 tests failed, 2 the run itself failed.

npx -y playwright-e2e-mcp ci --flaky-runs 3 --suggest 2 --output report.md
npx -y playwright-e2e-mcp ci --help

As a GitHub Action, it adds the report to the job summary and keeps one PR comment up to date when the run fails:

permissions:
  contents: read
  pull-requests: write   # only for the PR comment
steps:
  - uses: actions/checkout@v6
  - uses: actions/setup-node@v6
    with: { node-version: 24 }
  - run: npm ci && npx playwright install --with-deps chromium
  - uses: xsignon-secure/E2E@v0.4.2
    with:
      flaky-runs: 3      # 0 skips the flaky verdict
      suggest: 2         # suggest-fix diffs for the first 2 failures
      # args: --browser chromium tests/checkout

Inputs: project-root, args, flaky-runs, suggest, comment, fail-on-failure, version, github-token. Outputs: exit-code, report.

Docker

docker build -t playwright-e2e-mcp .
docker run -i --rm -v "$PWD:/work" playwright-e2e-mcp            # stdio MCP server
docker run --rm -v "$PWD:/work" playwright-e2e-mcp ci            # CI report

The image is based on mcr.microsoft.com/playwright:v1.63.0-noble, so its browsers match Playwright 1.63. Mount your project at /work; its own node_modules provide @playwright/test.

Configuration

Logs always go to stderr — stdout is reserved for the MCP protocol. Unrecognized values fall back to the default, so a typo in LOG_LEVEL logs at info and one in LOG_FORMAT logs as text rather than silencing the server. With LOG_FORMAT=json each line is a single object (time, level, message plus context):

{"time":"2026-10-07T08:00:00.000Z","level":"info","message":"server ready","name":"playwright-e2e-mcp"}

Typical workflow

1. run-test { "testFiles": ["tests/checkout.spec.ts"] }, or { "background": true } for a long suite and get-run-status { "waitSeconds": 30 } until it finishes.
2. get-failure { "index": 1 } to read the code frame, expected/actual, the DOM snapshot at failure, the failed network requests and the page's console errors.
3. suggest-fix { "index": 1 } for a locator or copy change: review the diff, then { "index": 1, "apply": true } to write it and verify with a re-run. If it reports the element is missing, fix the app, not the test.
4. For anything it cannot patch: inspect-page { "url": "…", "view": "locators" } (add storageState or actions to reach the right screen), then validate-selector to prove the new locator before editing.
5. If a failure looks intermittent: diagnose-flaky { "runs": 3 } for fresh evidence, and analyze-history to see how the test behaved across earlier runs.
6. After changing CSS or components: compare-visual-state { "url": "…", "name": "checkout" }.
7. Re-run only what failed with run-test { "lastFailed": true } until green.

To write a new test, start with generate-e2e-test { "description": "checkout with a saved card" }.

Benchmark

bench/token-benchmark.mjs breaks a checkout button's test id on a storefront page and fixes it two ways: with this server (run-test → suggest-fix → apply) and with Playwright MCP plus a shell (playwright test, browser_navigate + browser_snapshot, edit, re-run). Latest result (full output):

Tokens are characters ÷ 4. The Playwright MCP route is a lower bound: it assumes the agent picks the right locator from the snapshot on the first try, which the server does for it here.

Edge cases handled

Security notes

• No shell: Playwright is spawned as node <playwright/cli.js> … with an argument array — no command interpolation. Extra args are limited to an allowlist of Playwright flags, each with a checked value (--flag=value), so a caller cannot swap in another --config or --output or hand git an option through --only-changed. Paths with a segment starting with - are rejected, so a test file name cannot be read as a flag.
• Path sandbox: user paths must stay inside the project root, checked lexically and again with symlinks resolved. A caller-supplied projectRoot must sit inside PW_MCP_PROJECT_ROOT (or a PW_MCP_ALLOWED_ROOTS entry).
• Generated code: generate-e2e-test only writes *.spec.* / *.test.* files, only overwrites specs whose header it wrote itself, never writes through a symlink, and escapes every value it puts into strings or comments.
• HTTP bridge: read-only tools unless PW_MCP_HTTP_TOKEN is set; child processes started over HTTP get only an allowlisted environment (PATH, HOME, temp dirs, locale, CI, PLAYWRIGHT_*, npm_config_* without embedded credentials, plus anything in PW_MCP_PASSTHROUGH_ENV), at most PW_MCP_MAX_CHILDREN run at once (a disconnected client frees its slot immediately), and internal error messages are not returned to clients. The allowlist only covers the child's own environment: test code runs as the same OS user, so on Linux it could still read the server's startup environment from /proc. That is why only token holders can run project code; keep other secrets out of the bridge's environment, or run it under a separate user.
• SSRF guard: on the HTTP bridge (or with PW_MCP_BLOCK_PRIVATE_URLS=1) the URL tools refuse hosts that resolve to loopback, private, link-local/metadata or reserved addresses, and route the browser through a local proxy that applies the same check to every redirect hop and subresource; IPv6 forms that embed an IPv4 address (mapped, NAT64, 6to4, Teredo) are blocked too, and WebRTC UDP is disabled so a page cannot reach the network around the proxy. It is off for stdio by default because opening http://localhost dev servers is what these tools are for.
• Cleanup: temp report/script files are written to the OS temp dir and removed; child processes are tracked and killed on shutdown. Trace decompression has one size budget per archive, and PNGs are capped at 16,384 px per side and 50 M pixels.

Development

src/
├── index.ts            # bin entry point (--version/--help, main-module guard)
├── server.ts           # McpServer setup, tool registration, shutdown handling
├── tools/              # the twelve tools, the probe script + shared plumbing
├── prompts.ts          # fix-failing-test, triage-flaky-tests
├── utils/              # playwright-runner, report-parser, project-detector, path-utils,
│                       # logger, trace-reader (trace.zip → DOM/network/console),
│                       # image-diff (PNG codec + pixel diff), change-analyzer,
│                       # locator-expr, run-history, run-registry
└── types/              # shared interfaces and the ErrorKind taxonomy

Built on @modelcontextprotocol/server v2 (the 2026-07-28 MCP spec line) with Zod v4 standard schemas; every tool declares spec tool annotations.

npm install
npm run build   # tsc → dist/ (zero errors)
npm test        # build + test/run-tests.mjs (unit tests, any Node ≥20)
npm run e2e     # build + e2e/run.mjs: live MCP ↔ Playwright integration suite
npm run mcpb    # build + bundle dist/ into playwright-e2e-mcp-<version>.mcpb (+ .sha256)
npm run mcpb:smoke   # ...then install that bundle in a temp dir, launch it and assert the handshake
npm run mcpb:sign    # ...then sign it with MCPB_CERT/MCPB_KEY → <version>.signed.mcpb
npm run skill:zip    # just the archive: dist-skill/playwright-e2e-mcp.zip
npm run skill:package # the uploadable folder: ../agensi-skill-<version>/ (skill copy, README, zip)
npm run skill:check  # build, then fail if a packaged copy has drifted from skill/

Tests cover the report parser (sample Playwright JSON, trace attachments), path utils (Windows and macOS paths, sandboxing), the project detector (temp-dir fixtures: config discovery, multiple configs, missing install, test-file scanning), the shared tool helpers, the trace reader (synthetic trace.zip: error, failed action, DOM snapshot, *.network failed-request parsing, console error/warning events), the image diff (PNG round-trip, regions, color shift, dimension changes), the change analyzer (selector extraction, git + mtime paths), the flaky verdict logic (failure signatures, CONSISTENTLY FAILING / FLAKY / NOT REPRODUCING / NO TESTS RAN, per-test grouping of repeat attempts and the parallel-vs-alone reading), the ci argument parser and tool profiles, the HTTP bridge (token, restricted mode, Host allowlist), the security regressions (sandbox escapes, argument smuggling, symlink writes, code injection, SSRF ranges, env scrubbing, decode limits), the .mcpb manifest, the .mcpb zip extractor, the skill package (byte-identical rebuilds, a CRLF checkout hashing the same, refusal of junk/binary/symlinks, and drift detection between skill/ and a packaged copy), and the MCP config files.

Integration suite (npm run e2e)

Unit tests prove the logic; the integration suite proves the loop. It boots the real server over stdio against a live fixture app and a Playwright project under e2e/fixture/, then drives it exactly like an MCP client and asserts ~40 behaviours that only appear end-to-end:

• initialize handshake, 12 tools, 2 prompts, spec tool annotations and object input schemas,
• verified locators, locator validation, signed-in pages (storageState) and page steps,
• background runs with progress, run history views, and suggest-fix healing a drifted locator and verifying it with a re-run (and refusing when the element is missing),
• live DOM inspection, CSS selector validation (matches, zero matches, engine syntax, parse errors), dead-server detection,
• visual regression: baseline → unchanged compare → blue → red diff detection,
• run-test pass/fail/lastFailed stats and meta lines,
• get-failure trace diagnostics: DOM at failure, the 404 network request, the console.error message, diagnosis and next steps,
• auto-retry turning a first-run failure into PASSED (1 flaky),
• diagnose-flaky verdict FLAKY (2 of 3 runs) with retries disabled, in one --repeat-each process, and isolate reading a shared-slot test as "fails only in parallel",
• --tools= profiles trimming tools/list, and the ci command's exit code, report, --output file and GitHub job summary,
• generate-e2e-test writing its scaffold, plus error paths (missing test path, unknown tool, unreachable server).

First run needs the browser once: npx playwright install chromium. CI runs the suite on Ubuntu and Windows (see .github/workflows/ci.yml).

Signing the bundle

MCPB_CERT=~/signing/cert.pem MCPB_KEY=~/signing/key.pem npm run mcpb:sign

Signing appends a PKCS#7 signature block to the bundle, so the signed file is the reproducible bundle plus that block. That ordering is the point: the bytes CI proved identical across platforms stay inside the signed file, and mcpb unsign returns them exactly. scripts/sign-mcpb.mjs asserts that round trip on every run and deletes its output if it fails, which is what keeps <version>.signed.mcpb tied to <version>.mcpb.

The private key is read from a path you pass in (--cert/--key or MCPB_CERT/MCPB_KEY) and never from the repository — keep it outside version control, and back it up: the same key across releases is what makes the signer recognisable.

What signing does and does not buy you is decided by the certificate, not by this script. mcpb verify and mcpb info accept a signature only when the certificate chains to something in the OS trust store, so with a self-signed certificate a host still reports Not signed — the signature is real integrity and key continuity, verifiable by the fingerprint the script prints, but not third-party identity. A certificate issued by a public CA, or an internal one installed on your machines, is what makes hosts report it as signed.

Assembling a release

npm run mcpb && npm pack                  # the bundle, then the npm tarball
npm run release:assemble -- --tag v0.2.3  # sign, checksum and write the notes
npm run release:publish -- --tag v0.2.3 --assets dist-release/assets \
  --title-file dist-release/RELEASE_TITLE.txt \
  --body-file dist-release/RELEASE_BODY.md --download dist-release/downloaded
npm run release:verify -- v0.2.3 --dir dist-release/downloaded
npm run release:publish -- --tag v0.2.3 --publish

Pushing a v* tag runs exactly that sequence in .github/workflows/release.yml. The release used to be assembled by hand, and the notes were written from what those commands printed: every hash in them was a copy, and nothing compared the bundle hash printed there with the bundle that shipped.

scripts/assemble-release.mjs stages the whole release into dist-release/assets/: the bundle and its sidecar, the signed bundle and the signing certificate when a key is configured, the npm tarball, the skill package built by the same writer as npm run skill:zip, and SHA256SUMS.txt covering every file it staged. It empties that directory first, so a file left over from an earlier run cannot be hashed and published as if this run had produced it.

The notes are half written, half generated. release-notes/<tag>.md holds the prose — its H1 becomes the release title and has to name the tag — and the script appends the asset table, the reproduction recipe carrying the real bundle hash, the signer and fingerprint, and the commit list. The prose is held to the same rule: a 64-character hex string in it has to be a hash this release publishes, and writing an ### Assets section of your own fails the assembly rather than shipping a second, staler table. See release-notes/README.md.

scripts/publish-release.mjs fills the release as a draft, uploads the assets, downloads every one of them back and compares it byte for byte, and only --publish makes it public — after release:verify has re-derived the release from the tag. Re-running is safe: a draft this workflow assembled is refreshed in place, while a release it did not assemble, or one already published, is refused by name instead of overwritten.

Signing is optional and its absence is stated rather than hidden: without the MCPB_KEY and MCPB_CERT repository secrets the release is assembled and published unsigned, with a notice in the run and a line saying so in the notes. Add both — the private key stays out of the repository and out of the logs:

gh secret set MCPB_KEY  --repo xsignon-secure/E2E < ~/signing/key.pem
gh secret set MCPB_CERT --repo xsignon-secure/E2E < ~/signing/cert.pem

Creating the release and uploading its assets needs contents: write, which the job requests for itself; if it is refused, the run names the setting to change.

Verifying a published release

git checkout v0.4.2 && npm ci && npm run mcpb
npm run release:verify -- v0.4.2

The release notes promise that the tag rebuilds the published bundle byte for byte. scripts/verify-release.mjs checks that promise from a checkout of the tag: the published bundle against your fresh build, every checksum the release publishes (and that nothing shippable is missing from SHA256SUMS.txt), a signed bundle against the bytes its signature wraps, the skill package against a rebuild from the same tag, and that the published archive still extracts with that version's manifest. The skill package is required of a tag whose tree carries the publishing workflow; older tags predate it, so there it is checked when present and skipped when not, like the signing certificate. .github/workflows/verify-release.yml runs it on demand for any past tag, and by itself for a release published by hand. A release assembled by .github/workflows/release.yml runs the same check on its own assets before it is published — GitHub does not raise release: published for a release created with the repository's own token, so the check is a step there rather than a trigger. The .tgz is checked for download integrity only: npm records the packer's line endings, so a tarball packed locally on another OS is legitimately different.

Packaging the agent skill

npm run skill:package                    # ../agensi-skill-<version>/, ready to upload
npm run skill:zip                        # only the archive: dist-skill/playwright-e2e-mcp.zip
npm run skill:check                      # build, then compare every copy
npm run skill:check -- ../agensi-skill-0.3.0

skill/ is an Agent Skill — the instructions an agent follows to drive this server — and the marketplace gets the same Markdown as a zip with playwright-e2e-mcp/ at its root. scripts/package-skill.mjs produces that zip, so the uploaded copy is built rather than assembled by hand, and it refuses what the marketplace rejects: a missing SKILL.md, a non-text file, a symlink, Thumbs.db.

skill:package writes the whole folder a release would otherwise assemble by hand — the skill copy, skill/README.md and the archive — to ../agensi-skill-<version> (or --out <dir>), then verifies it with the same comparison check uses, so a successful exit means the folder really is the skill tree. It is idempotent: a folder that already matches is left untouched. A folder that has drifted is refused instead of overwritten, because it may already be published — bump version for a new package, or pass --force to replace that one.

The archive comes from the same deterministic writer as the .mcpb, so the same tree always hashes the same, and text payloads are LF-normalised first: a CRLF checkout builds the bytes an LF one does. CI runs skill:check, generates a package folder, and uploads the built zip as an artifact (.github/workflows/ci.yml).

Every release publishes the package too. Assembling a release builds it from the release tag and attaches it as <name>-<version>-skill.zip, and the .github/workflows/release.yml run downloads that asset back off the draft and byte-compares it with the build before anything is published. The marketplace upload is that asset, not a folder someone assembled — a deterministic archive means "the same build as the tagged release" is an exact statement, and release:verify checks it again against a rebuild from the tag. .github/workflows/publish-skill-package.yml stays for the repair path: run it by hand to attach or rebuild the package for a release that is missing it.

skill:check is the other half of that, and the part that matters when a package has already been uploaded. It builds twice (the bytes must agree), reads the archive back and compares its contents with skill/ file by file, then compares a packaged copy — ../agensi-skill-<version> by default, or a path you pass: the package folder, the bare playwright-e2e-mcp/ folder, or the zip on its own. Comparisons are by content, never by archive bytes, because a zip written by a platform zip tool carries real timestamps that say nothing about whether the Markdown inside it is current. Any file that differs, is missing or is extra is named on stderr and the command exits 1 — the repository copy and the uploaded copy are the same document, so an edit to either one should not be invisible.

Try the example

With Playwright installed in your project:

npx playwright test examples/sample-test.spec.ts

or ask your agent to call run-test with "testFiles": ["examples/sample-test.spec.ts"] — it hits the public example.com page, so it verifies browsers, network and the MCP pipeline in one shot.

License

MIT