ai tag debugger

English | Türkçe

AI Tag Debugger

An open-source AI agent that QA-tests your website's analytics and ad tracking.

An AI model drives a real browser through your key journeys (product → cart → checkout, lead forms) like a real visitor. While it browses, ai-tag-debugger records every tracking hit (GA4, Google Ads, Meta Pixel, TikTok and about a dozen more platforms) and every dataLayer push. It then checks them against your measurement plan and Consent Mode setup and writes a plain-language report of what broke and how to fix it, in English or Turkish.

Use it in two ways:

• From your terminal or CI, with your own API key for Anthropic (Claude), OpenAI, Google Gemini, or any OpenAI-compatible API (Kimi, GLM, DeepSeek, OpenRouter, a local Ollama and more).
• From your AI chat app (Claude, ChatGPT, Gemini and other apps that support MCP). The app's own model drives the audit, so you don't need a separate API key.

Built by Analitik İşler. MIT licensed.

▶ checkout (mobile)
  Accept cookies
  Open the first product
  Add to cart
  Open the cart
  Go to checkout
  …
  Place the order
  ✓ All scripted steps completed. · 20 tracking hits captured

2 broken · 1 risk · 1 warning · 4 ok
Report: reports/demo/localhost/runs/2026-10-16_0800/report.html

Why

Tracking breaks silently. A redesign renames a button, a new template forgets a field, a consent banner update lets pixels fire too early. Nobody gets an alert. You find out weeks later, after ad platforms have optimized on bad conversions.

Manual tag QA catches this, but it's tedious, so it rarely happens after every release. ai-tag-debugger makes it a command you can run in CI.

What it checks

Platforms: GA4 (including server-side GTM endpoints and Google tag gateway paths on the site's own domain; hits to a server container are marked server-side, and Stape and Elevar requests are recognised), Google Ads conversions, Meta Pixel, TikTok Pixel, basic X Pixel events, LinkedIn Insight Tag, Microsoft Advertising UET, Criteo OneTag, and Pinterest, Snapchat and Klaviyo events. Adobe Analytics, Matomo and Piwik PRO events are read too (page views, links, goals, site search and ecommerce orders), and Microsoft Clarity and Hotjar tag loads name the project or site id.

When GA4 hits go to a server container on another host, or its loader is served from one, the report adds an info finding: the events it forwards to ad platforms never pass through the browser, so they can't be seen here.

With a gtm_container export, each broken event also names the GTM tag that sends it and the trigger it fires on, or says that no tag sends it. Each tag then says whether its trigger fired in that journey and, when it did not, why: the tag is paused, no dataLayer push carried the trigger's event, a trigger condition failed (with the value the push had, read from the cumulative dataLayer the way GTM's Data Layer Variables read it), an exception trigger blocked it, or the consent it needs was denied when the event fired. When everything fired and the hit is still wrong, it says so, pointing at the tag's own setup. Click, form, visibility and timer triggers are evaluated by GTM in the page, so they are reported as not evaluable unless their gtm.click-style event reached the dataLayer. All of this is evaluated from the captured dataLayer, with no AI model.

Choose the platforms to check

By default an audit checks GA4, Google Ads, Meta and TikTok, and also any other platform the site turns out to use. That platform is checked from its first request, with no setting needed. Choosing platforms turns this detection off. Pick a fixed list with --platforms ga4,meta,x, "providers" in the config, or providers on the MCP start_audit and run_audit tools. You can choose from ga4, google_ads, meta, tiktok, x, linkedin, microsoft_ads, pinterest, snapchat, criteo, klaviyo, adobe, matomo, piwik_pro, clarity, hotjar, doubleclick and adsense. Events are parsed for GA4, Google Ads, Meta, TikTok, X, LinkedIn, Microsoft Ads, Pinterest, Snapchat, Criteo, Klaviyo, Adobe Analytics, Matomo and Piwik PRO. Clarity and Hotjar show their tag loads, and the others show their network requests. --providers still works as the old name of --platforms for one more minor version and prints a note.

Requests to platforms you didn't choose are still answered locally, so they never reach those accounts either. They are left out of the timeline, the CSV and the checks, and only counted: the report lists them with their request counts (for example "61 out-of-scope requests were skipped: Microsoft Clarity, DoubleClick"), and each step says how many it skipped. Without --platforms, any platform your measurement plan names or the site sends to is added to the default list. Planned events for platforms you did leave out are listed in one info finding instead of being reported as missing. Tag manager and consent platform requests are always shown.

Consent scenarios

Each audit also opens the start page once per consent choice, each time in a fresh browser: accept all, reject all and analytics only. The choice goes through the consent platform's own API (OneTrust, Cookiebot, Usercentrics or Didomi) when it has one, or else by clicking the banner's buttons. The report then shows how many hits each tag sent after each choice. Analytics only works with OneTrust and Cookiebot; with other platforms that column says "Not supported here". A tag that sends after reject all is flagged, except cookieless Consent Mode pings. An advertising tag that sends after analytics only is flagged unless it was sent with ad storage denied. The table is at the top of the Journeys tab and above the findings in report.md. It is skipped when the page shows no banner the tool can answer, and when a saved site is replayed. No AI model is called, but the start page is visited three more times. Set "consent_scenarios": false in the config to turn it off.

How it works

1. Browse: the AI model gets a goal in plain language ("add any product to the cart and complete checkout with the test payment") and a text view of the page. It acts through two tools, browser_action (goto, click, fill, select, scroll, wait) and finish, one step at a time with Playwright.
2. Capture: every request to a tracking endpoint is parsed into normalized hits and tagged with the step that caused it. Every dataLayer push is recorded through an init script, including gtag('consent', ...) commands. By default, the collection requests of every platform the tool can parse (GA4, Google Ads, Meta, TikTok, X, LinkedIn, Microsoft Ads, Pinterest, Snapchat, Criteo, Klaviyo, Adobe Analytics, Matomo and Piwik PRO), plus Google's conversion, remarketing and consent pings, are answered locally, so test traffic doesn't reach those real accounts. Tag loads and requests to vendors the tool only recognises by host (such as Clarity and Hotjar) still go out; the network part of the timeline shows each call and whether it was answered locally.
3. Check: deterministic rules compare the hits with your measurement plan. This part needs no API calls, so results are reproducible.
4. Explain: the model reads the findings, the hits per step and (optionally) your GTM container export. It writes the likely cause and fix for each problem, naming the tags and triggers involved.

AI providers

Bring your own API key. ai-tag-debugger uses the official SDK of each provider:

Without --provider, the first setting found decides, in this order: OPENAI_COMPATIBLE_BASE_URL, Vercel AI Gateway (AI_GATEWAY_API_KEY), Cloudflare Workers AI, Anthropic, OpenAI, Gemini. Choose explicitly with --provider anthropic|openai|gemini|vercel|cloudflare or "provider" in the config, and pick another model with --model. --model cheap picks each provider's small, inexpensive model for routine runs in CI: claude-haiku-5-5, gpt-5.5-mini, gemini-flash-latest, or anthropic/claude-haiku-5.5 on Vercel. Cloudflare has one model in its list, so cheap and the default are the same there. Navigation asks for low reasoning effort to keep runs cheap; explanations ask for high effort and structured JSON output.

Any OpenAI-compatible API: Kimi, GLM, DeepSeek, OpenRouter, Ollama and more

Many providers offer an API that works like OpenAI's. Point ai-tag-debugger at it with its address and a model id:

export OPENAI_COMPATIBLE_API_KEY=...        # leave unset for a local Ollama
ai-tag-debugger run --site https://yourshop.com --journey add_to_cart \
  --provider openai-compatible --base-url <API address> --model <model id>

Use the model id from your provider's documentation. You can also set OPENAI_COMPATIBLE_BASE_URL and OPENAI_COMPATIBLE_MODEL instead of the flags, or "provider", "base_url" and "model" in the config. When the API can't return strict JSON schemas, the explanations fall back to plain JSON mode.

Vercel AI Gateway

One key for many models. Set AI_GATEWAY_API_KEY from your Vercel dashboard and pick a model as <provider>/<model>:

export AI_GATEWAY_API_KEY=...
aitd run --site https://yourshop.com --journey purchase --model google/gemini-3.6-flash

Without --model it uses anthropic/claude-opus-5.5. Usage and costs show in Vercel's AI Gateway dashboard and are billed by Vercel. To see which AI a run will use, run aitd: the last line says, for example, AI: vercel (anthropic/claude-opus-5.5) · also set: anthropic (pick with --provider).

Cloudflare AI Gateway

To see logs, costs and rate limits for every AI call, send them through Cloudflare AI Gateway. Your provider key stays the same:

export AI_GATEWAY_URL=https://gateway.ai.cloudflare.com/v1/<account id>/default
export AI_GATEWAY_TOKEN=...   # only if the gateway has authentication on

This works with anthropic, openai and gemini. Leave caching off in the gateway so each run sees your live site. To run models on Cloudflare itself, see Cloudflare Workers AI.

A word on model choice. The AI browses by calling tools, one step at a time, for 5 to 30 steps. Models differ a lot in how reliably they do that: smaller and local models may click the wrong thing, loop, or stop early. The checks stay correct whatever the model does, because they come from the captured hits, but a journey that doesn't finish can't show its later events. If a model struggles, try a larger one, a ready-made journey with a short goal, or --max-steps. The Anthropic, OpenAI and Gemini defaults are the tested ones.

Cloudflare Workers AI

Open models on Cloudflare's network, billed per use by Cloudflare. Create an API token with Workers AI permission, and find your account ID in the Cloudflare dashboard:

export CLOUDFLARE_API_TOKEN=...
export CLOUDFLARE_ACCOUNT_ID=...
aitd run --site https://yourshop.com --journey purchase

Without --model it uses @cf/meta/llama-3.3-70b-instruct-fp8-fast. When both variables are set, Cloudflare is picked before Anthropic, OpenAI and Gemini, so pass --provider to use one of those instead.

Adding a native provider means implementing two functions in src/providers.js: a tool-calling conversation and a structured JSON call.

Quick start

Requires Node 20+.

npm install -g @analitikisler/ai-tag-debugger
npx -y playwright@1.56.1 install chromium   # the browser version this release uses
export ANTHROPIC_API_KEY=...                # or OPENAI_API_KEY, or GEMINI_API_KEY

ai-tag-debugger run --site https://staging.yourshop.com --journey add_to_cart --headed

The global install also adds aitd, a short alias: aitd run --site … --journey add_to_cart does the same.

Or run it without installing: npx -y @analitikisler/ai-tag-debugger run --site … --journey add_to_cart.

Run aitd --help to see the commands, and aitd run --help (or plan, replay, mcp) for each command's options. If something does not work, aitd doctor checks Node, Chromium, your AI key and the data folder, and says what to fix.

Try the demo shop (from source)

The repository includes a demo shop with three deliberate tracking bugs:

git clone https://github.com/analitikisler/ai-tag-debugger.git
cd ai-tag-debugger
npm install
npx playwright install chromium   # run again after upgrading, so the browser matches Playwright
npm run demo:serve                # in one terminal: http://localhost:4321
node src/cli.js run --config examples/demo.config.json --data-dir reports/demo
open reports/demo/localhost/runs/*/report.html

No API key yet? Replay the fixed steps without any AI model:

node src/cli.js run --config examples/demo.config.json --scripted --no-ai --data-dir reports/demo

Just ask

You don't have to learn the options first. Put a plain sentence in quotes and aitd turns it into the matching command, prints that command and asks Run this? (Y/n) before it runs anything:

aitd "check my google analytics tags"
# aitd run --platforms ga4
# Run this? (Y/n)

Add -y to skip the question: aitd -y "check my google analytics tags".

If a sentence leaves something out, aitd asks one short question instead of guessing: plan or run, which site, or what to test. Reports can't be filtered from a sentence yet ("Show me only the errors from the last report" says so). Without an AI key, a small keyword matcher handles the common phrases. With one, the AI reads any wording, but it can only pick options that exist, so it never invents a flag.

Plan, then audit

A run checks what a site sends against a plan: the events, parameters and user properties you expect. Three steps:

# 1. Draft a plan. The AI explores the site and proposes what should be tracked.
aitd plan --site https://yoursite.com

# 2. Optional: edit it in a spreadsheet, then save the edited copy as a new plan.
aitd plan export latest --site https://yoursite.com     # writes plan.xlsx next to the plan
aitd plan import plan.xlsx --site https://yoursite.com  # saves a new plan; the old one is untouched

# 3. Audit the site against the plan.
aitd run --site https://yoursite.com --against latest

Drafting a plan needs no tracking code. The AI explores one page of each type: the home page, a category, a product, the cart, the first checkout step, search, and any sign-up or contact forms. It never pays, and it never submits a form with personal details. --provider, --model and --max-steps (default 25, the most AI steps) work as they do for run. The output prints "Page types found" and "Not present on this site", so you can see which page types the AI reached. A small site, with only 2 pages, is fine.

To draft a plan for one kind of event only, add --topic: custom-events, ecommerce, forms or consent. The plan then covers just those events, and the rest of the plan format is unchanged:

aitd plan --site https://yoursite.com --topic ecommerce

The plan is saved in a new folder, aitd/<domain>/plans/<id>/, as measurement-plan.json and a readable plan.html. For each event it lists:

• name and platform. Events specific to your site, such as a size guide or a store locator, are marked custom: true.
• description and trigger: the page type and the visible label of the element that fires the event.
• parameters: each with a description and an example value. Custom parameters are marked. required_params lists the ones that must be present.
• user_properties: suggested ones such as customer_type, each with a description and an example.

Events your site already sends are marked "seen": true.

Review the plan before you use it. The examples and suggestions come from the pages the AI saw, so check that each event fits your business. The extra fields are ignored by the checks.

Spreadsheet import and export. aitd plan export <plan> --site <url> writes plan.xlsx next to the plan, with one row per event parameter on an events tab (event, platform, parameter, required, example, trigger, description) and one row per user property on a user_properties tab. Add --format csv to get events.csv and user_properties.csv instead; CSV needs no extra package. .xlsx uses one small library, fflate, which is installed with the tool as an optional dependency. aitd plan import <file> --site <url> reads the spreadsheet back (from Google Sheets, use File > Download > Microsoft Excel (.xlsx), or download each tab as CSV: Shop plan - events.csv is imported together with Shop plan - user_properties.csv beside it, and checkbox cells work as yes/no) and saves it as a new plan folder. Import never edits an existing plan. Unknown columns are ignored with a warning, and an error names the tab, row and fix, for example events, row 7: "required" must be yes or no (found "maybe").

measurement-plan.json looks like this:

{
  "consent_mode": true,
  "events": [
    { "platform": "ga4", "name": "add_to_cart", "required_params": ["currency", "value", "items"] },
    { "platform": "ga4", "name": "purchase", "journeys": ["checkout"], "required_params": ["transaction_id", "currency", "value", "items"] },
    { "platform": "meta", "name": "Purchase", "journeys": ["checkout"], "required_params": ["value", "currency"] }
  ]
}

platform is ga4, google_ads, meta or tiktok. journeys limits where an event is expected (default: every journey). Set allow_multiple: true for events that may legitimately fire more than once per action.

Event names are matched loosely, so dl_add_to_cart (Elevar), addToCart and Add To Cart count as add_to_cart, and transaction or order_completed as purchase. Add your own with "aliases": ["my_event"] on a planned event. Required parameters are also found under their usual other names: products for items, revenue for value, order_id for transaction_id.

Auditing against a plan. run --against <plan> takes latest (or last), first, a plan ID, or a path to a plan file or its folder. When --site is not given, the site comes from the plan. Give the run a journey as well and it walks only that flow, checking only the plan events expected there; the rest of the plan is reported as not checked, never as missing. Used alone, --against derives up to six journeys from the plan: it groups the planned events by the page they are triggered on and makes one journey per page, at most six (the smallest groups share the last journey), with no extra AI call. The chosen journeys are printed before the browser starts. Every report header then says Checked against plan <id>.

run needs something to test: --against, --journey, --goal, or journeys in your config file. Without one, it stops before opening the browser with Nothing to test. Add --against latest, --journey <id> or --goal "<text>"., so it can't wander and spend API usage. If a site has no plan yet, run --against says No plan yet for example.com. Draft one with: aitd plan --site https://example.com.

aitd plan list --site <url> lists the site's plan IDs, oldest first.

Learn the plan from the first run. When a site has no saved plan, a run without --against or --out lists the events the site already sends and asks whether to save them as its plan:

No plan yet for this site. It already sends 3 events: ga4:page_view, ga4:purchase, meta:PageView
Save them as the site's plan? (Y/n)

Say yes, and the next runs can check against it with --against latest. A parameter that every hit of an event carries becomes one of its required parameters. No values are copied, so nothing a visitor typed ends up in the plan. In a script or CI job nothing is saved unless you add --yes; the run then prints the aitd plan --site command for drafting a fuller plan.

Plan vs. site

When a run uses --against, the report has a Plan vs. site section: at the top of report.md, and in the Coverage section of report.html. Each planned event lands in one bucket:

The coverage score is implemented divided by the planned events that were checkable (all except not checked), shown overall and per platform, for example Coverage 7 of 11 (64%), 2 events not checked. The same entries are in plan-check.json, one per planned event with its bucket. Exit codes don't change: --fail-on still looks at finding severity. Without --against, the report is exactly as before.

When the plan is not the site's first, the header compares it with the version saved before it: Plan changed since 2026-10-01_0900: 1 added, 0 removed, 2 changed, or Plan unchanged since 2026-10-01_0900. The header in report.html and report.md then lists each change: + for an added event, − for a removed one and ~ for a changed one, followed by what changed (required parameters, parameters or journeys). The first version has nothing to compare with, and capture.json keeps the same comparison under planChanges.

Tag inventory

Every report also has a tag inventory: each vendor the site sent requests to, marked first party when its requests went to the site's own domain, third party when they went to the vendor, or both. It shows how many pages the vendor was seen on and the cookies tied to it. Cookies that no vendor owns are listed apart, as first or third party. Only cookie names are recorded, never their values. It is the Inventory section of report.html and a table in report.md.

Folder layout and plan IDs

Plans and runs are kept in aitd/ in the folder you run the tool from:

aitd/
  example.com/                      the site's host, without www.
    plans/2026-10-16_0900/          measurement-plan.json, plan.html, plan.xlsx (after an export)
    runs/2026-10-16_0912/           report.html, report.md, timeline.csv, capture.json, steps.json,
                                    junit.xml, summary.json and plan-check.json (with --against)
    history.json                    a small summary of every saved run, for the trend (see below)

An ID is the local time, YYYY-MM-DD_HHMM. When that minute is already taken, seconds are added (_HHMMSS). Nothing is ever replaced: each draft, import and run gets a new folder. Use --data-dir <dir> to keep the aitd/ folder somewhere else. replay and --save-site folders are not affected.

Each report has a "Changed since last run" section that compares the run with the site's previous one in runs/, for example 2 events fixed, 1 new problem, and lists each one: what was fixed (with what it was), what is new, and what is still open. When both runs used --against, it compares plan coverage (an event that went from Missing to Implemented is fixed); otherwise it compares the health findings. The first run for a site says there is nothing to compare yet. There is no flag to set. The comparison is also saved under changes in capture.json.

Trend over runs

From the second saved run on, the tool also looks at the site's last 30 runs together. After the counts, the CLI prints the verdict line and up to three trend lines:

2 broken · 1 risk · 1 warning · 0 info · 4 ok
Verdict: Fail · purchase tracking is broken on mobile · score 25/100
Trend over 5 runs (2026-10-01 → 2026-10-07): improving. Now 0 broken · 1 risk · 2 warning; 3 runs ago 2 broken · 1 risk · 0 warning.
Open longest: Meta PageView before_consent in 4 places since 2026-10-01 (5 runs); GA4 add_to_cart duplicate in 4 places since 2026-10-03 (3 runs).
Journeys that do not finish every time: checkout (mobile) 80%.

The same lines, with a text sparkline of the runs (one character per run, · for a run whose journeys did not all finish), make the Trend section of report.md and report.html. A problem counts as open while the last run that completed its journey still showed it; a run whose journey stopped early proves nothing, so it neither fixes nor reintroduces anything. Flapping problems (seen, gone, back again) are named, because they usually depend on timing or on the product the AI picked rather than on a fix. The direction compares the last few complete runs with the ones before them.

aitd history --site <url> prints the trend on its own: one row per run with its counts and score (3·broken + 2·risk + warning, lower is better), the sparkline and the lines above. --limit <n> reads only the newest n runs (default 30), --lang tr switches the lines to Turkish, and --json prints the trend series, the per-finding timeline (first seen, last seen, open since, fixed in, flapping) and the journey completion rates for scripts.

The summaries live in aitd/<domain>/history.json, about 2 KB per run, which every saved run updates after writing its capture.json. The file is a cache: a run folder that is not in it is read from its capture.json, a folder that was deleted drops out on the next update, and deleting the file is always safe. Runs sent to --out and replays are not part of the history.

Choose what to test

You don't need a config file to start. Pick ready-made journeys, describe your own in plain language, check against a plan, and narrow the check to the events you care about:

# Check the site against its latest saved plan.
aitd run --site https://staging.yourshop.com --against latest

# Ready-made journeys come with a goal and the GA4 and Meta events to expect.
ai-tag-debugger run --site https://staging.yourshop.com --journey purchase --journey sign_up

# Describe a journey in your own words. The events you name are expected on it.
ai-tag-debugger run --site https://yoursite.com --goal "Subscribe to the newsletter in the footer" --events newsletter_signup

# Only check some events.
ai-tag-debugger run --config ai-tag-debugger.config.json --events purchase,add_to_cart

--journey also picks journeys from your config by id. With a config, ready-made journeys add their events to your plan.

Not sure what to test? aitd plan --site https://yoursite.com drafts a measurement plan, and --against latest checks the site against it.

Keeping API usage down

Each browsing step is one model call, so usage grows with journeys × viewports × steps, plus one call at the end to explain the findings. The checks themselves never call the model.

• Run only the journeys you need. This is the biggest saving: --journey add_to_cart instead of the full purchase journey takes a few steps instead of ten or more.
• Run one viewport. --viewport mobile halves the calls compared with desktop and mobile.
• Cap the steps. --max-steps 15 stops a journey that wanders.
• Expect bigger pages to cost more. Each step sends the page view (up to 250 elements and 3,000 characters of text) to the model, so busy pages use more tokens per step than simple ones.
• Repeat runs start from the last route. Once a journey has completed on a site, the next run gives the AI the pages that run went through, in order, so it goes straight to the goal instead of exploring. The route is reused only for the same goal, from runs made with 0.14.0 or later, and only for AI runs. It isn't used with --out or replay. If the site changed and the route no longer fits, the AI takes another path.
• Use --events to keep the report focused. It doesn't shorten browsing, so it barely changes usage.
• Replay without AI. Every completed run saves the steps it took as steps.json in its run folder, and --scripted --no-ai repeats the last completed run's steps for a journey that has no fixed steps of its own. Run a journey once with the AI, then replay it on every deploy. See Save the steps and replay them without AI.
• Use a chat app. Over MCP, the chat app's own model does the browsing, so no API key is used at all.
• Transient failures don't cost a run. A rate limit, an overloaded answer or a dropped connection from the model API (429, 529, 5xx and the like) is tried again up to 3 times with a growing wait, honoring retry-after, and a page load the network dropped or that timed out is tried once more after 2 seconds. Other errors, such as a bad key or a refused page, fail at once.

Running time is another matter: --concurrency 3 runs up to three journeys at once, each in its own browser, so a run with several journeys and viewports finishes much sooner. It makes the same model calls, only closer together, so stay at 1 or 2 when the provider rate-limits you.

Check a new version against saved sites

--save-site <dir> also saves every response the site sent and the steps taken. replay then audits the same pages again offline, with no network and no AI model, so it costs nothing. It lists findings that are new or gone compared with the saved run:

npx @analitikisler/ai-tag-debugger run --site https://www.yourshop.com --journey purchase --save-site saved/yourshop
npx @analitikisler/ai-tag-debugger replay saved/*
✗ https://www.yourshop.com (saved/yourshop): 0 new, 1 gone
    - warning duplicate purchase desktop ga4 page_view

The first run's findings become the expected ones. When a change is right, for example a false alarm that is fixed, replay --update makes the new findings the expected ones. Use it to check a new version of the tool against sites you know, or a site change against its last good run. A saved site is usually a few MB. It leaves out tracking requests, images and video, HttpOnly and login cookies and the cookies the browser sent, authorization headers, form data, and requests with an email or other user data in their address. It still holds the site's pages and scripts, so keep sites you don't own out of public repositories.

Save the steps and replay them without AI

Every completed run saves the steps each journey took in its run folder, aitd/<domain>/runs/<id>/steps.json. A --scripted --no-ai run then repeats them for every journey that has no steps of its own: it takes the site's newest run that completed the journey, on the same viewport when there is one and on another otherwise, and says which run it is replaying under the journey's ▶ line. So the AI is needed once per journey, and every run after that costs nothing:

ai-tag-debugger run --site https://www.yourshop.com --journey purchase
ai-tag-debugger run --site https://www.yourshop.com --journey purchase --scripted --no-ai

When the site changes and a saved step no longer finds its element, the journey ends as incomplete with a note that says the steps of that run no longer fit; run it with the AI once more and the next replay uses the new steps. A journey that has never completed on the site stops the run before the browser starts. Only runs saved in aitd/ are replayed from: a run sent elsewhere with --out or made over MCP still writes steps.json, but the next run doesn't look there. The steps are stored as they were taken, typed values included (an email address typed into a form is in steps.json as typed, since a replay has to type it again), so use test data on the journeys and keep the run folder out of version control.

To keep the steps in your repository instead, --save-steps <file> writes them into journeys[].steps of a config file. It creates the file if it's missing, and it also saves the run's plan when the config has none. The same config then repeats the journey with no AI model and no API key:

ai-tag-debugger run --site https://www.yourshop.com --journey purchase --save-steps ai-tag-debugger.config.json
ai-tag-debugger run --config ai-tag-debugger.config.json --scripted --no-ai

When a journey ran on both viewports, only the first viewport's steps are kept. Run the second command on every deploy to check the same journey without any model calls.

Accept known findings

--fail-on fails the build on every finding at a level. When a finding is known and won't be fixed soon, for example a vendor pixel that fires before consent, a baseline lets later runs fail only on new findings. Run once with --update-baseline to write every current finding into a file, look through it, and commit it:

ai-tag-debugger run --config ai-tag-debugger.config.json --scripted --no-ai --baseline baseline.json --update-baseline
ai-tag-debugger run --config ai-tag-debugger.config.json --scripted --no-ai --baseline baseline.json --fail-on warning

The file holds one entry per finding, with a key the run matches on (the check, platform, event, journey, viewport and, for missing parameters, which ones) and the rest for whoever reads it. Delete an entry to start failing on it again, or add a "note" saying why it's accepted; notes survive the next --update-baseline. An accepted finding stays in the report with an "Accepted" label, is left out of the counts and the AI explanation, and doesn't count toward --fail-on. After the counts, the run prints how many findings were accepted and how many baseline entries are stale, that is no longer seen, so the file can be trimmed. A problem accepted on some viewports or journeys only is reported as partially accepted and still counted, and a baseline written for another site prints a warning. "baseline": "baseline.json" in the config, relative to the config file, saves passing the option; the MCP run_audit tool takes baseline too.

Configure your site

ai-tag-debugger.config.json:

{
  "site": "https://staging.yourshop.com",
  "plan": "measurement-plan.json",
  "gtm_container": "GTM-XXXX_workspace.json",
  "viewports": ["desktop", "mobile"],
  "journeys": [
    {
      "id": "checkout",
      "goal": "Accept cookies, add any product to the cart and complete checkout using the test payment method."
    }
  ]
}

The config and the plan are checked as soon as they are read, before the browser starts. An error names the field and, for a fixed choice, the allowed values, for example viewports[1]: must be one of desktop, mobile (got "tablet") or journeys[0].steps[2].action: must be one of goto, click, fill, select, scroll, wait. A key the tool does not know is ignored with a warning that suggests the closest one (Unknown key "viewport" ... Did you mean "viewports"?), so an extra note in the file does not stop a run. Plans saved by aitd plan and imported from a spreadsheet get the same check.

Alerts

After each saved run, notify sends the "Changed since last run" diff to your team, so a change on the site reaches you without opening a report:

"notify": {
  "slack": "https://hooks.slack.com/services/...",
  "email": ["team@example.com"]
}

• Slack takes the webhook URL. "slack": true reads it from the SLACK_WEBHOOK_URL environment variable instead, so the URL stays out of the file.
• Email takes one address or a list. Mail goes through the server in SMTP_URL, for example smtps://user:password@mail.example.com:465. SMTP_FROM sets the sender (default: the first recipient). Use smtps://, which encrypts the connection from the start. A plain smtp:// URL sends the login and the mail unencrypted, and STARTTLS isn't supported.

An alert goes out only for a saved run (not one sent to --out) that has an earlier run to compare with, and only when something changed since. The subject carries the verdict first, then the counts and the headline: shop.example: Fail · 2 new problems · purchase tracking is broken on mobile. The body repeats the verdict line, then lists what was fixed (with what it was) and what is new, each new item with its severity and owner, then the trend line with its sparkline, the flapping problems, and "Still open after N runs" when a problem reaches 5, 10, 20 runs open (then every 10), so the same open list does not repeat in every message:

shop.example
Verdict: Fail · purchase tracking is broken on mobile · score 25/100
1 problem fixed, 1 new problem. Compared with run 2026-10-15_1000, by health findings (2026-10-16_1000).
- Fixed: GA4 page_view (was: Sent before consent)
- New: Meta Purchase · Broken · "Purchase" was sent without "value". · Developer
Trend over 5 runs (2026-10-12 → 2026-10-16): worsening. Now 2 broken · 1 risk · 0 warning; 3 runs ago 0 broken · 1 risk · 0 warning. `▃▃·▅█`

The trend never triggers an alert on its own. If a channel fails, the run prints why and carries on. The exit code still follows --fail-on.

CLI

aitd "<sentence>" [-y]
aitd plan --site <url> [--topic <name>] [--headed] [--provider <name>] [--model <id>] [--max-steps <n>] [--data-dir <dir>]
aitd plan list --site <url> [--data-dir <dir>]
aitd plan import <file> --site <url> [--data-dir <dir>]
aitd plan export <plan> --site <url> [--format xlsx|csv] [--data-dir <dir>]
aitd run --site <url> (--against <plan> | --journey <id> | --goal "<text>") [--config <file>] [--events <names>]
         [--max-steps <n>] [--viewport desktop|mobile] [--concurrency <n>] [--platforms <list>] [--data-dir <dir>]
         [--provider anthropic|openai|gemini|openai-compatible] [--model <id>] [--lang en|tr] [--headed]
         [--fail-on broken|risk|warning|never] [--baseline <file>] [--update-baseline]
         [--scripted] [--no-ai] [--save-site <dir>] [--save-steps <file>]
aitd replay <dir>... [--update] [--out <dir>]
aitd mcp [--http] [--port <n>] [--host <addr>] [--token <secret>] [--public-url <https url>] [--reports <dir>]
         [--allow-private-network]
aitd doctor [--site <url>] [--data-dir <dir>]
aitd history --site <url> [--limit <n>] [--json] [--lang en|tr] [--data-dir <dir>]

aitd is a short alias for ai-tag-debugger and takes the same commands and options. Run aitd <command> --help for the essential options of each command.

All options

Removed in 0.10.0: plan --from, plan --out and plan --force. Drafting a plan always saves a new folder, and a removed flag prints what to run instead.

The exit code is 1 when findings reach --fail-on (default broken), so it can gate a deploy. Findings accepted in a baseline don't count. See examples/github-actions/ai-tag-debugger.yml for running it after every deployment.

Test results in CI. Every run also writes junit.xml, where each journey is a test suite and each finding a test case, failed when it reaches --fail-on, so GitHub Actions, GitLab and Jenkins show the findings as test failures. In GitHub Actions, upload it and hand it to a JUnit reporter action:

      - run: aitd run --config ai-tag-debugger.config.json --data-dir tracking-report
      - uses: actions/upload-artifact@v6
        if: always()
        with:
          name: junit
          path: tracking-report/**/junit.xml
      # Optional: any JUnit reporter action turns the file into annotations on the run.
      - uses: dorny/test-reporter@v2
        if: always()
        with:
          name: tracking
          path: tracking-report/**/junit.xml
          reporter: java-junit

In GitLab, point artifacts: reports: junit: at the same path. Next to it, summary.json holds the counts, the verdict (status: fail, attention or pass, and the headline key such as fail_broken), the data quality score (0 to 100), the trend direction over the saved runs, the journeys, every finding (check, severity, platform, event, journey, viewport, title, message), the run folder's name as run and the names of the other files as files, relative to that folder, so a script can read the result without parsing capture.json, and nothing in it depends on the machine the run was made on:

jq '.counts.broken' aitd/example.com/runs/*/summary.json
jq -r '.verdict.status + " " + (.score | tostring)' aitd/example.com/runs/*/summary.json

Scheduled runs. To audit on a timer instead of after each deploy, use examples/github-actions/scheduled-run.yml (weekly by default, with a cron line you can change). It saves the history between runs in a cache, so "Changed since last run" keeps working on a fresh runner. On your own server, examples/cron/ has a script and a crontab line that do the same.

Output:

• report.html: one file, one screen at a time. A collapsible rail on the left lists seven sections; every section, filter and card has its own URL you can bookmark or share, and a link from one section opens the right card in another (a finding's "Where" chip opens that hit in its step).

  • Summary: the data quality score as a ring, coloured red below 50, amber to 89 and green from 90, the verdict and its headline, up to three lines saying what cost the points (none for a perfect score), status chips that open the findings filtered by severity, and six cards about the run: vendors checked, journeys with the platforms seen on each, events checked, consent, the AI provider and the run itself.
  • Findings: a filter bar (severity, owner, platform, journey, viewport, search) over three groups: failed, unverified and passed. Every row starts collapsed and opens into Evidence, Fix, Where and Verify tabs: the impact, the likely cause, the evidence with missing parameters marked, numbered fix steps, the steps where it happened, and the tools to check the fix in, such as Tag Assistant, GA4 DebugView, Meta Test Events or TikTok Pixel Helper. Changed since last run sits above the groups when the site has an earlier run.
  • Journeys: one journey and viewport at a time. The steps on the left, named after the pages ("Open Home", "Add to cart") and grouped by page, with the hits each step produced; the selected step on the right with Hits, Data Layer and Requests tabs. Each hit shows its event, key parameters and consent state, and opens into the decoded parameters, item cards and the raw request. A batched GA4 request shows each of its events. It opens on the first step with a problem.
  • Consent: one line per platform with what it received before a choice, after accept and after reject, the Consent Mode default and updates with their timing, and the consent scenarios table.
  • Coverage: whether each planned event was sent on each journey, with the parameters it still needs, conversion integrity, and, with --against, the Plan vs. site buckets.
  • Inventory: the tag inventory, one line per platform, opening into its hosts, pages and cookies.
  • Run: the facts of the run (site, version, AI provider and model, plan, journeys, viewports, platforms, timing, files) and, when the site has earlier runs, the Trend.

  Every card starts collapsed except the Summary; each section has Expand all and Collapse all. The page carries one small script, allowed by its hash in the page's Content Security Policy, for the rail, the filters, the tabs and the deep links; without JavaScript the sections stack into one document, which is also the print layout. Keyboard: [ folds the rail, 1–7 jump to a section. The logos are embedded, so nothing is loaded from the internet. --no-branding leaves out the Analitik İşler logo and credit line.

• timeline.csv: the same timeline as one row per hit, dataLayer push and network call, ready for Excel or Google Sheets.
• report.md: the report in Markdown for a pull request comment, a Slack message or a mail, around 100 lines and under 8 KB for a four-journey run. Its title is the verdict (# shop.example: purchase tracking is broken on mobile), then one meta line, the status with the data quality score, and the counts. Then, in this order: Changed since last run and Trend (when the site has earlier runs), Plan vs. site (with --against), Priorities (the to-do list per owner: Developer, Tag Manager, Marketing), Findings (up to five full blocks with Where, Impact, Cause, Evidence, Also fine on, Owner, Fix, How to verify and a link into report.html; the rest one line each under Also), Consent (the Consent Mode timing sentence and the scenario table with a reason in every cell), What passed, Journeys (one row per run, then only the steps that carry a problem), Not checked (what the run could not judge, with the config key that enables it) and the Tag inventory. The per-hit timeline is not in the file; timeline.csv has it.
• capture.json: everything captured, for your own analysis, including the tool version, the ai provider and the computed metrics (verdict, score, plan coverage, funnel, platform parity, consent compliance, conversion integrity, timing, priorities).
• plan-check.json: with --against, one entry per planned event with its bucket.
• junit.xml: the findings as JUnit test results, one suite per journey and viewport, for CI systems to show as test failures (see above).
• summary.json: the counts, verdict, score, trend direction, journeys, findings and file names of the run in one small file for scripts.
• plan.html (in the plan's folder): a readable page of the plan's events, parameters and user properties.

Reports end with a credit line, "Generated by AI Tag Debugger · analitikisler.com", where the domain is a plain link (no images, no tracking, and no referrer, so report URLs aren't leaked). --no-branding removes it along with the logo.

Use --lang tr for a Turkish report. Finding texts, headings, and the model's explanations and fixes are all written in Turkish, and the agent describes its steps in Turkish too.

Use it from your AI chat app (MCP)

aitd mcp starts a Model Context Protocol server. Your chat app's model then runs the audit itself: you ask "check the tracking on the checkout of staging.myshop.com" and it opens the browser, clicks through, and explains the findings in the chat. No API key is needed, because the app's own model does the thinking.

Tools the server offers:

against takes the same values as the CLI's --against. Over HTTP it takes only latest, first or a plan ID, never a path. Plans are the ones saved by aitd plan --site <url> or by make_plan. Ask for a plan when you want to know what to track, and for an audit when you want to know what the site sends.

The browser runs on the computer where the server runs, so it can reach your staging and localhost sites. Tracking requests are answered locally as usual.

Apps that start a local server: Claude Desktop, Gemini CLI, Cursor, VS Code, Claude Code

Add the server to the app's MCP config. For Claude Desktop that is Settings → Developer → Edit Config (claude_desktop_config.json):

{
  "mcpServers": {
    "ai-tag-debugger": {
      "command": "npx",
      "args": ["-y", "@analitikisler/ai-tag-debugger", "mcp"]
    }
  }
}

The browser (Chromium) downloads itself the first time you run an audit, about a minute, one time. To get it ahead of time: npx -y playwright@1.56.1 install chromium.

One-click and one-line setup:

• Claude Code: claude mcp add ai-tag-debugger -- npx -y @analitikisler/ai-tag-debugger mcp
• Cursor: Add to Cursor
• VS Code: Install in VS Code, or code --add-mcp '{"name":"ai-tag-debugger","command":"npx","args":["-y","@analitikisler/ai-tag-debugger","mcp"]}'
• Claude Desktop, Gemini CLI and others: the JSON block above. Gemini CLI uses the same mcpServers block in ~/.gemini/settings.json.

Restart the app, and the ai-tag-debugger tools appear. If the app can't find npx, use its full path (which npx, for example /opt/homebrew/bin/npx on a Mac with Homebrew). Reports are written to ~/ai-tag-debugger-reports.

Running from a clone instead? Use "command": "node" and "args": ["/path/to/ai-tag-debugger/src/cli.js", "mcp"].

Apps that connect to a URL: ChatGPT, Gemini

These apps connect to MCP servers over HTTPS (in ChatGPT, through developer mode connectors, depending on your plan). Start the server in HTTP mode and expose it with a tunnel:

cloudflared tunnel --url http://localhost:8787     # or: ngrok http 8787; note the https address it prints
npx -y @analitikisler/ai-tag-debugger mcp --http --port 8787 --public-url https://<your-tunnel-address>

The server prints a URL with a random secret path, like https://<your-tunnel-address>/mcp/<secret>. Add that URL as a connector in the app. Anyone with the URL can use the browser on your machine, so keep it private and stop the server when you're done. A secret of your own (--token) must be at least 16 characters.

In HTTP mode, the server is more locked down than locally:

• It doesn't read measurement plans or configs from your disk (pass the plan as JSON text in the chat, or name a saved plan with against: latest, first or its ID), and run_audit isn't offered. It only serves the reports it wrote itself.
• The browser opens only http and https pages, and refuses localhost and private network addresses (such as 192.168.x.x or 169.254.169.254), so the URL can't be used to look into your network. All browser traffic, WebSockets included, goes through a small built-in proxy that resolves each host once and connects only to the address it checked, so a DNS answer that changes mid-audit can't get around the check. To audit a local staging site this way anyway, start the server with --allow-private-network.
• Audits left open for 15 minutes are closed, and request bodies are limited to 1 MB.

A hosted server, so you don't need a tunnel, is part of the Analitik İşler Cloud version.

Safety

• The agent is told to use obvious test data and never to enter real payment details. If a journey needs a real payment or account, it stops and reports that instead.
• Tracking requests to the platforms listed in How it works are answered locally by default, so those analytics and ad accounts don't receive test conversions.
• The browser stays on the audited site and its subdomains, plus any allowed_hosts. It opens only http and https pages.
• The model is told that page content is data, never instructions. A prompt rule can't guarantee a model is never misled, so the navigation limits above contain what a misled model can do.
• The browser runs without your API keys and other secrets in its environment.
• User data sent for ad matching is redacted before it is stored in capture.json or shown in the report: emails, phone numbers, names, addresses and user ids in GA4 and Google Ads enhanced conversions (em, ph, uid, …), Meta advanced matching (ud[*], udff[*]), TikTok context.user, and the user_data the site pushes to the dataLayer for enhanced conversions. Any value that looks like an email address is redacted whatever its field is called, except the audited site's own address in a mailto: link or a link field (link_url, link_text, gtm.elementUrl, …), such as a contact link in a GTM click event, which is not visitor data. Each value becomes a marker such as [redacted: 64 chars, looks hashed], and a value that wasn't hashed is reported as a personal data risk. --keep-bodies ("keep_bodies": true) stores requests as sent, for debugging.
• Stored request bodies are capped at 32 KB per request and 2 MB per journey. Past that, the report says the body wasn't stored.
• Only run it against sites you own or are authorized to test.
• Hosting the HTTP server for other people? Also block private and cloud metadata address ranges at the network level (an egress firewall or an isolated network), as defense in depth.

Roadmap

• Consent Check: deeper audits of cookie banners and Consent Mode v2 against GDPR/KVKK.
• Measurement Plan: draft a plan and dataLayer spec from a URL.
• Hosted version with scheduled runs, deploy webhooks, a hosted MCP server and multi-site dashboards for agencies at analitikisler.com.

Development

npm run setup   # checks Node, installs the dependencies and Chromium, says whether an AI provider key is set
npm test        # parsers, checks, each AI provider and the MCP server, end to end against the demo shop
npm run test:unit   # only the tests that need no browser, in about ten seconds
npm run lint    # ESLint, the recommended rules only: undefined names, unused imports and the like

The repository also has a dev container (.devcontainer/devcontainer.json). Open it in VS Code with the Dev Containers extension, or in GitHub Codespaces, and it runs npm run setup for you, Chromium's system libraries included. To use an AI model there, add your key as a Codespaces secret or export it in the container's terminal.

The end-to-end tests use fake model clients, so they run without an API key.

Hosted version

For hosted runs, scheduling and agency dashboards, see analitikisler.com.

License

MIT © Analitik İşler