FitLLM

FitLLM Engine

FitLLM is an open-source, zero-dependency engine that checks whether a local LLM fits on a GPU or Apple Silicon Mac using architecture-aware memory math.

Live: https://fitllm.run · Bilingual · Free · No ads · No login

Open engine: fitllm-engine (MIT · npm fitllm-engine · npx fitllm)

Zero dependencies. One readable file: engine.js. Conformance-vector tested. MIT.

Install

# self-contained binary — no Node required
curl -fsSL https://fitllm.run/install.sh | sh

# Homebrew (tap) — the trust line is required for any third-party tap
brew tap click6067-ship-it/fitllm
brew trust click6067-ship-it/fitllm
brew install fitllm

# Windows (Scoop)
scoop install https://raw.githubusercontent.com/click6067-ship-it/fitllm-engine/master/packaging/fitllm.json

# already have Node? nothing to install
npx fitllm "Gemma 4 12b" --gpu "RTX 4090"

Binaries for macOS (arm64/x64), Linux (x64/arm64, glibc and musl) and Windows x64 are attached to every release with a .sha256 next to each one; install.sh verifies the checksum and installs nothing if it does not match.

Two things stated plainly rather than buried: each binary embeds its JavaScript runtime, so it is 60–94 MB — much larger than an equivalent Rust or Go tool, which is the price of not forking the engine into a second language. And this is a Homebrew tap (click6067-ship-it/homebrew-fitllm), not homebrew-core: core's bar for a self-submitted project is 225 stars / 90 forks / 90 watchers, which this project does not meet yet.

Quick start

npx fitllm "Gemma 4 12b" --gpu "RTX 4090"     # one line, exit 0 fits / 1 won't — run before you download
npx fitllm --top --gpu 4090                    # what can this hardware run?
npx fitllm --top --detect --json               # detect every supported local GPU; agent-ready JSON
npx fitllm Qwen/Qwen3-32B --detect --json      # public Hugging Face ID or URL; unsupported configs fail closed
npx fitllm "Gemma 4 12b" --detect --json --why # include architecture, evidence, assumptions, and exact memory inputs
npx fitllm scan                                # what you already downloaded — does it fit this machine?
npm install fitllm-engine                      # use the same engine as a library (see Usage)

fitllm scan — start from what you already have

Instead of typing a model name, read the ones already on disk:

$ fitllm scan
Hardware: RTX 4090 (24GB) · quant Q4_K_M · ctx 8,192

ollama (3)
  ✓ FITS  qwen3:0.6b  →  Qwen3-0.6B
          1.2 GB used, 22.8 GB free  (on disk: 0.5 GB · Q4_K_M)
  ?       some-finetune:latest  —  not in the catalog, no verdict  (on disk: 6.7 GB · Q5_K_M)

It reads Ollama through its loopback API and LM Studio through lms ls --json. Both are read-only: no pull, no load, no settings written, and nothing is sent anywhere. A non-loopback OLLAMA_HOST is refused rather than queried.

A model the catalog does not know gets no estimate — only the size and quant the runtime itself reported. Guessing from a similar name would hand you another model's numbers, which is worse than no number.

Verification status, stated plainly: the Ollama path is exercised end-to-end in CI against a real Ollama install (verify-install-paths.yml). The LM Studio path is covered by fixtures only — there is no LM Studio on the machine this was written on, so it has not been run against the real CLI.

Remote MCP server

Connect any Streamable HTTP MCP client to https://fitllm.run/api/mcp:

{
  "mcpServers": {
    "fitllm": {
      "url": "https://fitllm.run/api/mcp"
    }
  }
}

Tools

• check_llm_fit — check one model against a GPU, multi-GPU rig, or Mac and return the verdict, memory breakdown, and a fix when it does not fit.
• what_fits_on_hardware — rank the supported local models that fit the given GPU, multi-GPU rig, or Mac.
• list_supported — list the built-in model and hardware names accepted by the fit checker.

The server is read-only, stateless, and requires no authentication.

npx fitllm "GLM-4.7-Flash" --gpu 4090     # ✓ FITS — 21.9/24 GB, free 2.1 GB
npx fitllm "gpt-oss-120b" --mac 64        # ✗ WON'T FIT → what to change to make it fit
npx fitllm "Qwen 3.6 35B" --gpu "5090 + 3090"   # multi-GPU rig — VRAM pools (56GB), even mixed cards
npx fitllm --top --detect --json          # detect hardware + what fits, for people or agents
npx fitllm "Gemma 4 12b" --detect         # one verdict on this machine

--detect reads all nvidia-smi adapters, uses Apple Silicon unified memory on arm64 macOS, and can resolve an exact catalog GPU name through Windows/WSL PowerShell. It never uses Win32_VideoController.AdapterRAM, serials, PNP IDs, or full environment dumps. Intel-only, ambiguous, and unsupported adapters stop with exit 2 instead of borrowing a nearby GPU's memory.

Why a CLI? The "will it run?" question is born in the terminal — one line before ollama pull. No install, no tab-switching, and it reads your actual hardware with --detect instead of asking you to know your VRAM. Exit code 0/1 makes it a pre-download guard:

# in your model-pull script — stop BEFORE the 40 GB download:
npx fitllm "gpt-oss-120b" --detect || { echo "won't fit — aborting pull"; exit 1; }

Put the guard in the download path

The CLI exit contract composes directly with model runtimes. The download or launch runs only after a FITS or TIGHT verdict; invalid inputs stop with exit 2.

npx fitllm "Gemma 4 12b" --detect && ollama pull gemma4:12b
npx fitllm "Llama-3.1-8B-Instruct" --detect && llama-cli -m ./Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf

For CI, use the repository's composite Action. It needs no secret and returns the full --json --why result as steps.preflight.outputs.result:

permissions:
  contents: read
steps:
  - name: Check model memory before download
    id: preflight
    uses: click6067-ship-it/fitllm-engine@v2.21.5
    with:
      model: Gemma 4 12b
      gpu: RTX 4090
      ctx: '8192'

Set exactly one of gpu or mac. The Action preserves the CLI contract: exit 0 means fits/tight, 1 means it will not fit, and 2 means the request is invalid. Inputs cross the GitHub expression boundary through environment variables and are passed to Node as a Bash argument array.

Agents can collect a typed measurement without uploading anything:

npx fitllm measure "Qwen 3.6 27B" --detect --measured 15.3 --kind system_total_peak --unit GiB --runtime "llama.cpp b6400"

The command validates the conditions and prints a candidate JSON object plus a prefilled GitHub issue URL. Submission remains a human action.

If you run Ollama, you do not have to measure anything yourself — it already reports the resident size of a loaded model:

ollama run qwen3:0.6b        # in one shell
npx fitllm measure --from-ollama   # in another

This reads GET /api/ps and GET /api/version over loopback. It is read-only: it does not load, unload, pull, or change anything, and it submits nothing — you get the candidate and the issue link, and you decide.

What it verifies, and what it does not. The number is idle_resident: weights plus KV resident in VRAM with the model loaded and idle. It is not peak memory during generation, so it checks part of a prediction rather than all of it, and the candidate says so.

When it refuses. If size_vram differs from size, or is zero, no candidate is produced. Ollama's documentation does not define what those fields mean when they disagree, and partial residency is something this engine deliberately does not model — a number nobody can interpret is worse than no number. Models outside the catalog are skipped too: with no prediction, there is nothing to compare against. Public Hugging Face IDs (org/model) are fetched with a bounded config/index reader and accepted only when parseHfConfig() supports the architecture.

This is the open calculation core of FitLLM. The math is open so you can audit it.

Ask an LLM "does Qwen 3.6 fit my GPU?" and it pattern-matches to an architecture from its training cutoff — and usually says no. Catalog-based calculators lag new releases. The CLI, API, and MCP use a curated catalog pinned to official configs. The web calculator can additionally inspect a pasted Hugging Face ID's official config.json live, so supported architectures work on day-one releases — including the hybrid / sliding-window / MoE structures that naive formulas get wrong.

Covers Apple Silicon unified memory (M1–M6, Pro/Max/Ultra — up to the 512GB Mac Studio), NVIDIA GPUs (RTX 20/30/40/50, workstation RTX 6000 Ada / RTX PRO Blackwell 2000–6000, datacenter A100/H100/H200/B200), NVIDIA GB10 unified-memory systems (DGX Spark 128GB and the same-spec ASUS / Dell / HP / Lenovo / MSI / Acer / GIGABYTE boxes — a partner name resolves only as the full name with its capacity, e.g. ASUS Ascent GX10 128GB; linked 2× / 4× clusters via --count), AMD Radeon (RX 7000/9000, PRO W7900) and multi-GPU presets (2×3090, 2×4090, 4×3090) — with GGUF Q-tier weight quantization kept separate from KV-cache quantization. Hardware entries carry their source URLs per-value in engine.js; new entries require ≥2 independent sources (CONTRIBUTING).

---

Why most LLM memory calculators are wrong

Almost every "can I run this LLM?" calculator estimates the KV cache with the textbook formula:

KV ≈ 2 × num_layers × num_kv_heads × head_dim × context_length × bytes

That assumes every layer keeps a full-context KV cache with one uniform head shape. True for Llama-1/2 — wrong for most 2025–2026 models:

An 11× error flips the verdict: a naive calculator says Gemma 4 31B won't fit in 64 GB at long context, when it fits comfortably.

The six things they ignore

1. Sliding-window attention (Gemma 2/3/4, gpt-oss): most layers only keep the last N tokens, so their KV stops growing. Only the global layers scale with full context.
2. Hybrid / linear attention (Qwen 3.6 / 3.8, many 2026 models): linear-attention layers use a fixed-size recurrent state, not a growing KV cache. That state is modeled too, as its own component (linearState) — it is a constant per sequence, so it never inflates the context curve.
3. MLA — Multi-head Latent Attention (GLM-5.2, GLM-5.3, GLM-4.7-Flash, DeepSeek-V2/V3 family): the cache is a single low-rank latent (kv_lora_rank + RoPE dims) shared across all heads — per-head "2 × heads × head_dim" formulas over-count by an order of magnitude. Verified against the DeepSeek-V2 paper (arXiv:2405.04434) and the official DeepSeek-V3 inference code.
4. Heterogeneous head dims + MoE: global layers can use a different head_dim (Gemma 4: 512 vs 256). MoE keeps every expert in memory while activating only a few per token.
5. PLE — Per-Layer Embeddings (Gemma 4 e2b/e4b): the pinned llama.cpp/GGUF path assigns the per_layer_token_embd input-layer tensor to CPU/host buffers instead of accelerator memory, so only the non-PLE weights are counted against VRAM for the verified Gemma 4 e2b/e4b entries. Counting all 5.1B params against a GPU over-predicts e2b's resident weights by ~1.9× and flips small-card verdicts. This deduction is conditional and rests on that placement alone: the host memory the tensor needs is not budgeted by the GPU verdict, a runtime that loads PLE tensors onto the accelerator (vLLM, for example) invalidates the estimate, and unverified families keep their full weights resident. On Apple Silicon unified memory the full weights stay counted. The exact premise and its pinned sources are listed under Structural premises; a direct measurement on a Gemma 4 GGUF is welcome in issue #7.
6. Compressed sparse attention (DeepSeek-V4-Flash / V4-Pro, V4.1-Flash): one shared 512-dim KV head per layer, a 128-token sliding window everywhere, and a compressed cache (one entry per compress_ratio tokens) plus indexer keys only on the layers that compress their own KV — V4.1 shares one cache across six or more layers (CSA2). The engine counts the cache buffers DeepSeek's reference inference code allocates, plus a bounded indexer prefill workspace; V4.1's 1,600 cached elements per token are the 890 bytes/token its tech report states once DeepSeek's FP4 formats are applied. Configs that deviate from the official layout fail closed.

This engine models each layer type separately, verified against official HuggingFace config.json files.

---

What it computes

Total = Parameters (quantization-adjusted)
      + KV cache (per layer kind: sliding / global / linear / dense)
      + Runtime overhead (quant metadata + KV block padding + activations + fixed)
      + OS reserve (macOS base on Apple Silicon · DGX OS share on GB10 · driver/display reserve on discrete GPUs)

GGUF weights never below the published file. A GGUF tier (Q4_K_M … F16) is a per-tensor recipe, not one bit width: llama-quantize keeps some tensors at higher types (tied embeddings at the output type, rows that are not a multiple of 256 fall back to Q5_0/Q8_0), and publishers add their own overrides — ggml-org keeps the attention and linear-attention tensors of the Qwen3.5/3.6 family at Q8_0 even in Q4_K_M, bartowski raises attention and ships the MTP layer, poolside keeps eight expert layers of Laguna S Q4_K_M in BF16. The same tier name therefore differs by up to 15% between publishers (Qwen3.6-27B Q4_K_M: 15.40 / 16.74 / 17.77 GiB). Where the largest same-name file from unsloth, bartowski, ggml-org, lmstudio-community, mradermacher or the model's own organization is more than 1% above the tier average, the weights are counted at that file size (ggufFileBytes, repository@revision next to each value in engine.js); every other reference file is within 1% of the estimate. The October 2026 sweep found such rows in 22 catalog models (61 model-tier pairs) — e.g. Nex-N2.5-Pro Q5_K_M 263.5 → 284.6 GiB, Gemma 4 31B Q4_K_M 17.49 → 18.25 GiB, Qwen3-0.6B (most GGUFs store the tied lm_head separately) +27–33%. The floor applies only at the exact tier, never on Apple MLX bit widths, and not to the GPU-resident part of Gemma 4 PLE models (the per_layer_token_embd tensor stays on the host; the remaining tensors are below the estimate in every reference file). A Hugging Face checkpoint inherits a row's floors only when its tensor list matches that row exactly.

Unified memory (GB10 / DGX Spark). There is no dedicated VRAM: DGX OS, CPU processes and the GPU share one 128 GB LPDDR5x pool, so — like the Mac path — the OS and runtime share is subtracted from the same pool instead of using a discrete-card environment preset. The reserve is three sourced or labelled parts: the memory the OS cannot see (128 − 121.69 GiB MemTotal, which equals the CUDA total on DGX OS 7.6), about 5 GiB of idle DGX OS use (MemTotal − MemAvailable), and the same 2 GiB runtime allowance the Apple path uses (an engine estimate, not a GB10 measurement) — 13.31 GiB in total, with the Mac-style 20% headroom for the shared pool. On one unit the GGUF weights count at their file size: since 2.21.4 the 12% weight overhead that the discrete-GPU and Mac paths add is not applied, because public GB10 allocation logs show no such share (calculated vs. measured). Gemma 4 PLE is never deducted here (host buffers are the same pool), and a GB10 system cannot be pooled with other GPUs (fails closed). Per-value source URLs are in engine.js.

GB10 clusters (2× / 4× DGX Spark). Several identical GB10 units add up for one model only when a distributed runtime splits it across them over ConnectX-7 — NVIDIA documents vLLM with Ray and TensorRT-LLM with tensor parallel equal to the unit count, two units by a direct QSFP cable and four through a RoCE switch (DGX OS from April 2026). npx fitllm <model> --gpu "DGX Spark 128GB" --count 2 (or --gpu "2× DGX Spark 128GB") counts each unit's own reserve and activation buffers, and the KV cache as replicated when the model has fewer KV heads than units or uses an MLA latent cache or DeepSeek's compressed cache (vLLM / TensorRT-LLM tensor-parallel behaviour; a layer split would need less). Three units (ring), more than four, or a mix with other GPUs fail closed. NVIDIA's "two units, up to 405B" is a weights-only FP4 figure; the calculator answers at your context instead.

Plus a parseHfConfig() that turns configs from verified, modeled Hugging Face architecture families into the model shape above; unsupported structures fail closed instead of returning a guess. (No token/s prediction — deliberately: speed depends on runtime/backend in ways a static model can't claim honestly. Fit is a verifiable claim; speed is not.)

Structural premises

Some verdicts are valid only under a structural premise about the artifact or the runtime path. The engine derives the active premises from the same normalized model fields the calculation uses and attaches them to affected results as structuralAssumptions, an array of { id, statement }. Unaffected results, such as plain GQA models, carry no such key, so legacy JSON shapes are unchanged. The CLI prints each active premise as a premise [id]: line in text mode and includes the array in --json (also per row in --top --json); --why and the composite Action forward the same array. There is no runtime selector, no confidence score, and no speed claim: a premise tells you what the memory math assumes, not how fast the model runs.

• mla-compressed-latent-cache — KV memory assumes a compressed-latent MLA artifact or mode; legacy non-MLA GGUF or an explicitly uncompressed mode invalidates this estimate. Sources: DeepSeek-V3 inference model.py · llama.cpp convert_hf_to_gguf.py · llama.cpp llama-kv-cache.cpp.
• ple-llamacpp-non-gpu-residency — GPU weight memory excludes the verified Gemma 4 PLE tensors only because the pinned llama.cpp/GGUF path assigns the per_layer_token_embd input-layer tensor to CPU/host buffers instead of accelerator memory; that host memory is not budgeted here, and a runtime that loads PLE onto the accelerator invalidates this estimate. Sources: Gemma 4 E2B config.json (text body model_type is exactly gemma4_text) · pinned llama.cpp 8b4b3558f1459c13e4aa38d5c94d306a00dc6acd: input-layer device assignment · PER_LAYER_TOKEN_EMBD as an input-layer tensor · Gemma 4 construction · loader header · loader implementation. The deduction applies only to the catalog entries Gemma 4 e2b and Gemma 4 e4b (pleOffloadVerified: true) and to parsed configs whose text body is exactly gemma4_text and whose full profile (layer count, hidden_size, intermediate_size and both PLE dimensions) matches a pinned official E2B/E4B config with a checkpoint that reconciles against the config's own dense body. A look-alike field on any other family, an altered dimension or layer count, or an inconsistent checkpoint keeps the full weights resident on the GPU, so an unverified config cannot flip a verdict toward "fits".
• native-fp4-expert-weights — attached to models whose release ships its expert weights in FP4 (gpt-oss-20b / 120b in MXFP4, DeepSeek-V4-Flash / V4-Pro / V4.1-Flash-1017): 4-bit E2M1 values plus one E8M0 scale per 32 values, 4.25 bits. At any weight precision above 4.25 bits those tensors are counted at their shipped size and only the other weights follow the chosen tier, because GGUF has a native MXFP4 type and the public GGUFs keep them at every level. Where a model's public GGUF files are known (gpt-oss), the weights are never counted below the largest file of the chosen tier — gpt-oss-120b Q4_K_M 58.53 GiB, Q8_0 59.03 GiB, F16 60.88 GiB against the 60.77 GiB release (the tier average said 66.55 / 115.59 / 217.56 GiB); the F16 file is 0.18% above the release because GGUF stores biases and norms in F32. Where no same-tier public file is known (DeepSeek-V4, and Hugging Face checkpoints that are not a tensor-for-tensor match of a catalog row), the weights are never counted below the released checkpoint itself (plus 0.25% at F16 for those F32 copies). Builds that re-expand the experts to 8 or 16 bits (an MLX 8-bit build is 115.6 GiB) are larger without adding precision and are not what this counts; below 4.25 bits (Mac 4-bit) the experts follow the chosen precision. FP8 (E4M3) weights are not in this class: GGUF has no FP8 type, so Q8_0 and BF16 GGUFs re-encode them and the tier sizes already match the published files (GLM-5.3 Q8_0 746.3 GiB, BF16 1,404.4 GiB; FP8 release 703.7 GiB). Sources: ggml GGML_TYPE_MXFP4 · llama-quantize keeps already-quantized tensors · unsloth/gpt-oss-120b-GGUF file sizes · ggml-org/gpt-oss-120b-GGUF · bartowski/openai_gpt-oss-120b-GGUF · openai/gpt-oss-120b release · unsloth/DeepSeek-V4-Flash-GGUF · unsloth/GLM-5.3-GGUF.
• deepseek-compressed-sparse-kv — KV memory follows the cache buffers of DeepSeek's reference inference code: a sliding-window cache on every layer, and a compressed cache plus indexer keys only on layers that compress their own KV; counted at the chosen KV precision (16-bit = the reference code's bf16 buffers) with the compressors' fp32 state, V4.1's Engram token history and a 512 MiB indexer prefill workspace (vLLM's default VLLM_SPARSE_INDEXER_MAX_LOGITS_MB), in full on every linked GB10 unit. Whole-prompt prefill as in the reference generate.py (its indexer score tensor grows with prompt × context) and runtimes that cache uncompressed KV need more and are not modeled. V4.1's 196.6B Engram tables are never counted below their FP8 storage (8.25 bits with scales), whatever the weight quantization; the vision tower and MTP/DSpark draft weights count as resident weights. Sources: DeepSeek-V4-Flash inference model.py · DeepSeek-V4.1-Flash-1017 inference model.py · engram.py · DeepSeek-V4.1-Flash tech report (890 bytes/token global KV) · vLLM DeepSeek V4 blog (9.62 GiB bf16 KV at 1M for V4-Pro) · vLLM envs.py (indexer logits budget) · vLLM indexer prefill chunking.
• mtp-ordinary-generation — KV memory assumes ordinary non-speculative generation; an MTP draft context is not included. Sources: llama.cpp llama-hparams.cpp · llama.cpp llama-model.cpp · llama.cpp convert_hf_to_gguf.py · vLLM qwen3_next_mtp.py.
• gb10-cluster-tensor-parallel — attached to 2× / 4× GB10 verdicts: the units add up only under a runtime that splits one model across them (vLLM with Ray or TensorRT-LLM, tensor parallel = units); Ollama and LM Studio use one unit; per-unit reserve and activation buffers and tensor-parallel KV replication are counted; network/NCCL buffers and uneven splits are not. Sources: NVIDIA DGX Spark clustering · NVIDIA: up to four nodes (2026-03-16) · NVIDIA vLLM playbook · vLLM decode context parallelism — MLA KV replicated per TP rank · TensorRT-LLM parallel strategy · ServeTheHome GB10 cluster.
• gb10-cluster-tensor-parallel-incompatible — attached when the model's attention, KV or linear-attention head counts do not divide across the units (vLLM / TensorRT-LLM refuse that tensor-parallel size); the verdict is no and no cluster fit is claimed. Source: vLLM llama.py head-divisibility asserts.
• gb10-unified-memory-reserve — attached to every GB10 (DGX Spark-class) verdict: the reserve subtracts the memory DGX OS cannot see, idle DGX OS use and a 2 GiB runtime allowance; on one unit the GGUF weights count at file size with no extra percentage, plus 15% of the KV cache and a context-scaled activation allowance — calibrated on llama.cpp/Ollama runs with flash attention on (both default to --flash-attn auto: llama.cpp common.h · Ollama llama_server.go); flash attention off (llama.cpp then holds the full attention scores) and vLLM (CUDA graphs, a pre-allocated KV pool) need more and are not budgeted; other running apps, the file page cache, llama-server's host-side prompt cache (--cache-ram, 8 GiB by default) and context checkpoints are not budgeted, and weights are assumed resident once (a loader that keeps an mmap page-cache copy can need up to the weight size again while loading). Sources: NVIDIA DGX Spark porting guide — no fixed GPU carve-out · NVIDIA DGX Spark known issues — nvidia-smi memory "Not Supported", page-cache flush · MemTotal/CUDA total measurements: bloomery #13 · NVIDIA developer forum · Back2Gaming · idle use: AAIF · Techno Tim.

Resident scope. SSD/NVMe streaming, expert paging, swap and general CPU/RAM offload stay outside ordinary FitLLM fit verdicts. The one disclosed structural exception is a placement fact, not storage paging: for the verified Gemma 4 e2b/e4b entries the discrete-GPU verdict excludes the per-layer-embedding (PLE) tensor because the pinned llama.cpp/GGUF path deterministically assigns that input-layer tensor to CPU/host buffers instead of accelerator memory (premise ple-llamacpp-non-gpu-residency, shown with the verdict). That host/system memory is not budgeted by the discrete-GPU verdict, a runtime that loads PLE onto the accelerator invalidates the PLE estimate, and on Apple unified memory — and on GB10 (DGX Spark) unified memory — the full weights stay counted.

npx fitllm "GLM-4.7-Flash" --gpu "RTX 4090"          # text adds:  premise [mla-compressed-latent-cache]: KV memory assumes …
npx fitllm "GLM-4.7-Flash" --gpu "RTX 4090" --json   # adds "structuralAssumptions": [{ "id": "…", "statement": "…" }]
npx fitllm "Llama-3.1-8B-Instruct" --gpu "RTX 4090" --json   # plain GQA: no structuralAssumptions key at all

Usage

// from npm:  npm install fitllm-engine
import { simulate, structuralAssumptions, LOCAL_MODELS, GPUS, gpuDevice, parseHfConfig } from 'fitllm-engine';
// …or vendored single-file:
// import { simulate, structuralAssumptions, LOCAL_MODELS, GPUS, gpuDevice, parseHfConfig } from './engine.js';

const model = LOCAL_MODELS.find((m) => m.name === 'Gemma 4 31b');
const sim = simulate(model, /*ram*/ 64, /*ctx*/ 131072, /*bits*/ 8);
// → { used, free, verdict: 'yes'|'tight'|'no', param, kv, rt, os, maxContext, ... }

// affected results carry the active structural premises; plain GQA results have no such key:
const gpu = gpuDevice(GPUS.find((g) => g.name === 'RTX 4090'));
const mla = simulate(LOCAL_MODELS.find((m) => m.name === 'GLM-4.7-Flash'), gpu, 8192, { weightBpw: 4.8944, kvBits: 16 });
mla.structuralAssumptions; // → [{ id: 'mla-compressed-latent-cache', statement: '…' }]
structuralAssumptions(model, gpu); // → [] for an unaffected model (pure; same inputs as the calculation)

// a config from a modeled Hugging Face architecture family:
const m = parseHfConfig('Qwen/Qwen3-32B', configJson, totalSizeBytes);

Verification

From a repository checkout, recompute the typed measured-vs-predicted ledger with npm run benchmark:accuracy (or add -- --json). Run the pinned estimator differential with npm run benchmark:differential. To recapture its uncut source output from a separately downloaded official llmfit release artifact:

npm run benchmark:capture:llmfit -- --binary /path/to/llmfit --artifact /path/to/llmfit-v1.1.12-x86_64-unknown-linux-gnu.tar.gz

The differential is labeled architecture_differential_not_runtime_accuracy: it demonstrates how estimators treat GQA, sliding-window attention, and MLA, but it is not runtime measurement evidence. The methodology, competitor input format, checksums, and precommitted kill conditions are in benchmarks/README.md. The current evidence does not clear the comparative accuracy claim gate.

• Architecture values checked against official HuggingFace config.json.
• Gemma 4 31B full-context KV reproduces 20.78 GiB, matching the published architecture analysis. Reproduce it by hand:

global: 10 layers × 2(K,V) × 4 heads × 512 dim × 2 B × 262,144 = 21,474,836,480 B
local:  50 layers × 2(K,V) × 16 heads × 256 dim × 2 B × 1,024  =    838,860,800 B
total = 22,313,697,280 B ÷ 1024³ = 20.78 GiB

• MLA per-token cost: GLM-4.7-Flash = (512 + 64) × 2 B × 47 layers = 54,144 B/token — pinned by conformance vectors.

GB10 (DGX Spark class): calculated vs. measured in public logs

Checked 2026-10-20. Logged is a floor of the run (allocation_floor), scoped per row. Qwen rows: the sum of the buffers the runtime listed when it loaded the model (weights, KV cache, recurrent state, compute, output); it leaves out the 248 MiB CPU compute buffer of the vision encoder that the same log lists, the encoder's own weights, the CUDA context and caches that grow with use. Gemma row: the per-process nvidia-smi figure, which includes the CUDA context but no host memory. Calculated = weights + KV cache + recurrent state + runtime buffers, excluding the 13.3 GiB GB10 reserve (6.3 GiB the OS cannot see, ~5 GiB idle DGX OS, 2 GiB runtime allowance). The entries, with raw-log hashes and the checks behind them, are in fixtures/measured.json; test/gb10-measured.test.mjs keeps the calculation above every floor and the weights within 3% of the logged model buffers.

Where 2.21.3 was off: it added 12% of the weights as runtime overhead — 7.8 GiB for the 35B BF16 file. The log shows the GGUF loading at its file size (66,221.98 + 970.00 MiB = 65.62 GiB) and compute buffers of 0.5–0.7 GiB (llama.cpp) or 1.7 GiB (Ollama's engine). 2.21.4 drops that 12% on one GB10 unit; the 15% KV padding and the context-scaled activation allowance stay. Adding the 2 GiB runtime allowance that sits inside the reserve, 2.21.4 is 1.5–6.0% above every row — headroom for the memory the floors leave out, not an error by itself.

Limits: four runs on two machines, all with flash attention on (the llama.cpp and Ollama default, --flash-attn auto). With it off, llama.cpp holds the full attention scores — about 24 GiB for a 48-head model at 262K tokens and batch 512, growing with batch size — which no version budgets; vLLM on one unit (CUDA graphs, a pre-allocated KV pool) is not covered either. Host memory that grows with use is not in any row: llama-server's prompt cache (up to 8 GiB by default) and up to 32 context checkpoints. ¹ Ollama 0.24.0 reports a KV cache 2.5× the hand count, which the log does not explain — that row is a floor check only (community_unverified). Discrete GPUs, GB10 clusters and Macs keep the 12%: none has a comparable runtime-allocation check yet (the Mac entries in the ledger are idle resident-weight readings). The same table is on fitllm.run/data.

All figures are estimates — real usage varies with the runtime (MLX/Ollama/llama.cpp), OS state, and quantization scheme.

Conformance vectors

vectors/fit-vectors-v1.json pins 49 language-neutral test vectors (exact KV bytes, per-token costs, fit verdicts) derived by hand from official config.json values — e.g. "Gemma 4 31B at 262,144 ctx, bf16 = exactly 22,313,697,280 bytes". Any implementation in any language conforms if every vector passes — run ours with node vectors/run.mjs.

Why this matters: the formulas are easy to copy; a verified answer key is not. If you port this engine to Python, Rust or Go, you don't become an untrusted fork — pass the vectors and you're a conformant implementation of the same standard. Port the engine, keep the vectors.

The Fit Census — every model × every device, one truth table

census/ holds 13,804 verdicts (34 models incl. draft tier × 104 GPUs/Macs × quant tiers) computed by this engine — as CSV/JSON you can import, chart or cite, plus a starter matrix ("biggest model that fits comfortably per device"). Regenerate it yourself: npm run census. Real-world measurements land next to predictions via fixtures/ PRs — predicted vs. measured, in public.

Embed a fit badge

Show whether a model runs on given hardware — live from the engine, one line in any README or model card:

![fits](https://img.shields.io/endpoint?url=https%3A%2F%2Ffitllm.run%2Fapi%2Fbadge%3Fmodel%3DGLM-4.7-Flash%26gpu%3D4090)

Params: model (name, fuzzy), gpu (name, fuzzy) or ram (GB, Apple unified memory), optional quant (GGUF tier / 4|8|16), ctx, kv. Verdict color: green fits · yellow tight · red won't fit.

Why embed it? The #1 question under every model card and local-AI tutorial is "will it run on my machine?" The badge answers it live from the engine — recomputed when the data updates, not a stale claim frozen into your README. If you publish models or write guides: one line replaces a whole FAQ paragraph and cuts the "it OOM'd on my 8GB card" issues before they're filed.

Ask your AI assistant (MCP)

The engine runs as a public MCP server at https://fitllm.run/api/mcp — connect it once and your assistant answers "can I run X on my Y?" with this engine's math instead of guessing from stale training data (LLMs routinely get KV-cache math wrong — see the 17.8× table above).

• Claude (web / desktop / mobile): Settings → Connectors → Add custom connector → paste https://fitllm.run/api/mcp
• Claude Code: claude mcp add --transport http fitllm https://fitllm.run/api/mcp
• Cursor / Windsurf: add to mcp.json → { "mcpServers": { "fitllm": { "url": "https://fitllm.run/api/mcp" } } }
• ChatGPT: Settings → Apps → Advanced → Developer mode → add MCP server (Plus/Pro)

Tools: check_llm_fit (verdict + full memory breakdown + fix suggestion — supports multi-GPU rigs like "RTX 5090 + RTX 3090"), what_fits_on_hardware (ranked list for your machine), list_supported. Resources: fitllm://models, fitllm://hardware, fitllm://census, fitllm://engine. Intentionally open: read-only, stateless, no auth, no secrets — every call is a pure function of public data.

Listed on: official MCP registry (run.fitllm/fitllm) · Glama · mcp.so · Smithery

For agents & scripts — plain HTTP API

No MCP client? One GET, no auth, no key — JSON by default, plain text for curl:

curl 'https://fitllm.run/api/check?model=gemma%204%2031b&gpu=4090'
# multi-GPU rigs: gpu=5090%2B3090 · Mac: ram=64 · usage: curl https://fitllm.run/api/check

Open data: the full Fit Census (13,804 verdicts, CC0) at fitllm.run/data and on Hugging Face Datasets. Try the engine in-browser: HF Space demo.

Principles

No ads. No login. No affiliate links. Output is never for sale. Fit is a winnable, verifiable claim; raw tok/s is not — so this engine refuses speed predictions rather than dress a guess as precision.

Help calibrate

Ran a model and measured real peak memory? Report a measurement — it improves the estimates for everyone.

Built by

yongha — GitHub. Powers fitllm.run.

License

MIT © click6067-ship-it