webfetch

webfetch

The license-first image layer for AI agents and humans.

One MCP server, one CLI, and one HTTP server that federate across 25 image providers, rank results license-first, and reject UNKNOWN results by default. Any agent that speaks MCP (Claude Code, Cursor, Cline, Continue, Roo Code, Codex) wires up from one config line. Landing page, pricing, and hosted usage live at getwebfetch.com.

Install

The curl | bash installer also wires webfetch into Claude Code's ~/.claude/settings.json idempotently. Re-run any time to update.

Surfaces

Package-level API notes live in packages/core/README.md, packages/browser/README.md, and the other package READMEs under packages/.

30-second usage

CLI:

webfetch search "drake portrait" --limit 5
webfetch artist "Taylor Swift" --kind portrait --min-width 1200
webfetch download <url> --out ./portrait.jpg
printf "drake portrait\nradiohead album\n" | webfetch batch --jsonl --continue-on-error

MCP (from inside any MCP-speaking agent):

search_images({ query: "drake portrait", limit: 5 })
search_artist_images({ artist: "Taylor Swift", kind: "portrait" })
download_image({ url: "..." })

TypeScript library:

import { searchArtistImages, pickBest, downloadImage } from "webfetch-core";

const { candidates } = await searchArtistImages("Drake", "portrait");
const best = pickBest(candidates, { minWidth: 1200 });
if (best) {
  const { cachedPath, sha256 } = await downloadImage(best.url);
  console.log(best.attributionLine, "->", cachedPath);
}

What problem this solves

Manually sourcing an image has four failure modes:

1. You don't know the license, so you can't safely ship the result.
2. You can't script it — every new site means another afternoon.
3. Google's Image Search API is retired; scraping is brittle and ToS-grey.
4. No shared cache — you re-download the same file dozens of times.

webfetch fixes all four by federating across direct-source APIs that have stable terms and structured license metadata, ranking candidates license-first, and exposing the result as a single MCP tool.

Providers

See docs/PROVIDERS.md for gotchas, rate limits, and docs/PROVIDER_TUNING.md for per-use-case picks.

Local and cloud modes

The CLI is local-first: by default webfetch search, artist, album, download, probe, license, and batch call webfetch-core in-process and use provider API keys from your environment. Pass --cloud or set WEBFETCH_MODE=cloud to call https://api.getwebfetch.com/v1/* with WEBFETCH_API_KEY or webfetch config set apiKey wf_live_....

Use local mode when you want direct provider calls and a local cache. Use cloud mode when you want hosted auth, pooled provider keys, managed browser fallback, usage accounting, or team controls.

Why license-first

The only outcome we reject by default is an image we can't justify. A marginally-better photo under an unknown license is worthless to a pipeline that needs to ship without human review. Relevance ties are easy to break; provenance is not.

The ranker sorts by: license tag -> metadata confidence -> resolution -> provider priority. UNKNOWN is rejected by default (Berne Convention: most of the web is all-rights-reserved unless proven otherwise). See docs/LICENSE_POLICY.md.

Migration: CC0 stock providers

Older webfetch builds treated Unsplash, Pexels, and Pixabay as CC0. Current builds expose their platform terms explicitly:

Most callers should keep licensePolicy: "safe-only" because it still allows open, platform, editorial, and press-kit categories while rejecting UNKNOWN. Pipelines that require only Creative Commons or public-domain assets should use licensePolicy: "open-only" and update type guards to handle the three platform tags separately.

webfetch vs alternatives

Architecture

                             +------------------+
                             |  webfetch-core  |
                             |  (ranker, cache, |
                             |   license coerce)|
                             +---------+--------+
                                       |
          +----------------+-----------+-----------+----------------+
          |                |                       |                |
  +-------v------+  +------v-------+       +-------v------+  +------v-------+
  | webfetch     |  | webfetch-mcp |       | webfetch-    |  | browser      |
  | CLI          |  | (stdio)      |       | server (HTTP)|  | extensions   |
  +-------+------+  +------+-------+       +-------+------+  +------+-------+
          |                |                       |                |
          |                |                       |                |
          +----------------+-----------+-----------+----------------+
                                       |
                 +---------------------v---------------------+
                 |              provider adapters            |
                 |  wikimedia  openverse  unsplash  pexels    |
                 |  pixabay    itunes     mb-caa    spotify   |
                 |  youtube    brave      bing      serpapi   |
                 |  flickr     nasa       met       europeana |
                 |  loc        wellcome   rawpixel  burst     |
                 |  browser + managed-browser + archival opt-in|
                 +-------------------------------------------+

Every surface shares ~/.webfetch/cache/ keyed by SHA-256, so a download from the CLI is instantly available to the MCP server and vice versa.

Safety defaults

• licensePolicy: "safe-only" — open, platform-license, and editorial/press categories are allowed; UNKNOWN is rejected.
• safeSearch: "strict".
• Opt-in providers (youtube-thumb, bing, serpapi, browser, managed-browser, europeana-archival) off by default.
• 20 MB per-download cap, content-type guard, host blocklist.
• robots.txt respected on generic page probes.

Roadmap

• webfetch watch — daemon mode for repeated queries / incremental refresh.
• Bring-your-own-provider plugin API.
• Hosted tier at getwebfetch.com — pooled provider keys, managed browser fallback, team usage dashboard.

Contributing

Issues and PRs welcome. Run bun install && bun test to get started. See docs/ for per-area reference docs.

License

MIT.