Your WebGPU Benchmark Is a Lie — Why Dispatch Measurement, Not Peak Throughput, Decides Browser Inference Performance

The headline numbers are real. The methodology behind them is not.

Every “WebGPU is N× faster” claim you have read this year — the 159× fused kernel over PyTorch, the 10–15× client-side inference gap over WebAssembly — is a real number produced by a real machine. But almost none of them generalize to the laptop or phone your users actually carry. The reason is not insufficient hardware or a slow shader. It is that the way most of these benchmarks measure WebGPU’s fixed per-operation cost is wrong by an order of magnitude, and the resulting figure is then treated as if it were a property of the API rather than of one developer’s workstation.

If you are about to commit engineering time to shipping a WebGPU inference path, the single most important thing you can do is stop trusting published throughput numbers and start interrogating how they were measured. This article shows why naive dispatch benchmarks overestimate WebGPU’s overhead by roughly 20×, why dispatch cost varies by up to 50× across real devices, why the “right” backend flips with task shape — and how to calibrate your own decision on the hardware that matters.

The 20× mirage: how naive microbenchmarks inflate WebGPU’s true cost

A paper from February 2026 — “Characterizing WebGPU Dispatch Overhead for LLM Inference” (arXiv:2604.02344) — set out to answer a deceptively simple question: what does a single WebGPU dispatch actually cost? The authors measured across four GPU vendors (NVIDIA, AMD, Apple, Intel), two native implementations (Dawn and wgpu-native), three browsers, and two model sizes (Qwen2.5-0.5B and 1.5B). Their headline finding is uncomfortable for anyone who has benchmarked WebGPU by wrapping a single operation in a timer (per the paper’s abstract): naive single-operation benchmarks overestimate dispatch cost by roughly 20×.

The mechanism is a measurement artifact the authors call sync conflation. WebGPU’s queue.submit() is asynchronous; unless you insert a fence and wait for GPU completion, your timer is dominated by synchronization, not dispatch. A single Dawn “operation” that appears to cost ~497 µs is in fact ~24 µs of genuine dispatch plus ~450 µs of synchronization. Measure sequentially — dispatch many operations back-to-back and amortize the sync — and the true per-dispatch cost drops to 24–36 µs on Vulkan and 32–71 µs on Metal.

Concretely, the naive version measures one dispatch with a fence around it:

const t0 = performance.now();
encoder.setPipeline(pipe);
encoder.setBindGroup(0, bindGroup);
encoder.dispatchWorkgroups(1, 1, 1);
device.queue.submit([encoder.finish()]);
await device.queue.onSubmittedWorkDone();   // sync dominates the timer
const naiveCost = performance.now() - t0;   // ~497 µs: mostly sync

The sequential variant issues many dispatches into one command buffer and times the whole batch, then divides:

const t0 = performance.now();
const enc = device.createCommandEncoder();
for (let i = 0; i < 100; i++) {
  const pass = enc.beginComputePass();
  pass.setPipeline(pipe);
  pass.setBindGroup(0, bindGroup);
  pass.dispatchWorkgroups(1, 1, 1);
  pass.end();
}
device.queue.submit([enc.finish()]);
await device.queue.onSubmittedWorkDone();
const perDispatch = (performance.now() - t0) / 100; // ~24 µs: dispatch only

That is the difference between believing WebGPU’s overhead is half a millisecond per operation and knowing it is closer to two dozen microseconds. Choose the second number when you decide whether fusion or a smaller model is worth the engineering.

The distinction matters because it separates a property of the API from a property of your stack. Per-dispatch cost is API-inherent: it is the fixed validation and command-buffer submission WebGPU imposes on every operation regardless of host language. Per-operation overhead (~95 µs) adds the Python interpreter and framework marshaling on top — which is stack-specific and, crucially, responds to the same fix as the API cost: fewer dispatches.

The device lottery: dispatch cost swings 50× across the fleet

The 20× measurement error is compounded by an even more consequential truth: the true dispatch cost is wildly device-dependent. Ayoob AI’s engineering write-up on self-calibrating GPU dispatch measured a minimal single-workgroup dispatch across consumer hardware and found a spread of roughly 50×: about 0.02 ms on an NVIDIA RTX 4060, up to 1.1 ms on a budget Android GPU (Mali-G57). On a 0.5 ms compute operation, that overhead ranges from 4% of the total on a discrete NVIDIA GPU to more than 100% on a phone — the phone is slower with the GPU enabled than without it.

This is why a hardcoded dispatch threshold like if (data.length > 100000) { useGPU() } is, in the author’s words, “wrong. Not approximately wrong. Fundamentally wrong.” The number encodes an assumption about hardware that is true for exactly one device: whichever one the developer benchmarked on. The crossover point for element-wise work ranges from ~50,000 elements on an RTX 4090 workstation to ~400,000 on an M2 MacBook Air to ~3,000,000 on a Mali-G57 Android tablet. A threshold tuned on a workstation dispatches small batches to the GPU on a phone, where it is 4× slower than CPU.

The combined effect is stark. At one million elements, the GPU beats a 4-core CPU by 6.8× on an RTX 4060 and 6.4× on an M2 — but runs at 0.64× CPU speed on a Mali-G57. Raw compute alone is a 64× swing (15,110 GFLOPS on the RTX 4060 vs 100 on the Mali-G57), and bandwidth adds another 8.3× spread. The performance number you quote in a blog post is not a property of WebGPU; it is a joint property of WebGPU, the GPU, the driver, the memory subsystem, and the phase of the moon.

Task shape flips the answer: WASM beats WebGPU at small models

Dispatch overhead does not merely vary across devices — it changes which backend wins for a given workload. SitePoint’s February 2026 WebGPU vs WebAssembly benchmark is a useful counterweight to the “GPU is always faster” narrative. For a small embedding model (all-MiniLM-L6-v2, 22M params) on an M2 MacBook Air, WASM wins: 8–12 ms versus WebGPU’s 15–25 ms, because the GPU marshaling overhead — buffer upload, shader dispatch, readback — exceeds the computation itself. The GPU only pays off when computation time dominates transfer time.

The picture inverts hard for autoregressive generation. Generating 128 tokens with TinyLlama-1.1B, WebGPU on a discrete RTX GPU produces 25–40 tokens/sec versus WASM’s 2–5 tokens/sec — a 10–15× advantage. On integrated hardware (M2) the gap narrows but remains: 15–25 TPS vs 3–6 TPS. And there is a third curve: WebGPU pays a 1–5 second shader-compilation penalty on first token, while WASM streams its first token immediately.

Layer these three findings together and the “is WebGPU faster?” question dissolves into a decision that depends on model size, task shape, hardware tier, and whether cold-start latency is a hard constraint. The backend that is strictly correct on a developer workstation is silently wrong on a third of the fleet.

When fewer dispatches actually help: the fusion caveat

The most common “fix” for dispatch overhead is kernel fusion — fusing the entire autoregressive loop into a single dispatch. It is genuinely powerful, but it is not a universal win, and the same measurement discipline applies. The arXiv paper’s fusion experiment is the cleanest causal evidence available: on Vulkan, fusing reduced dispatches from 876 to 564 and improved throughput by 53% with unchanged kernels — proving per-operation overhead is the actionable bottleneck at batch size 1. But CUDA fusion provided no benefit, and the paper notes backend choice (Vulkan vs Metal) is the dominant factor in whether fusion helps at all. Kernel fusion is a tool for a specific overhead profile, not a blanket optimization.

Separately, the webgpu-transformer-fusion project reports that at small hidden dimensions, 92% of wall-clock time is dispatch overhead, not computation — the fused kernel’s win is real but concentrated exactly where dispatch dominates. The lesson is the same as everywhere else in this article: before you trust a fusion speedup, ask what fraction of the measured time was actually dispatch, on which device, and whether your production batch size matches the benchmark’s.

Read the bench, not the number: a calibration checklist

The goal is not to abandon benchmarks — it is to read them correctly. Here is the discipline that separates a decision you can stand behind from a headline you cannot. The three questions to ask of any WebGPU performance claim:

Question	Why it matters	Red flag
Was dispatch measured sequentially or as a single op?	Naive single-op timing conflates sync with dispatch, inflating cost ~20×	A single-operation timer with no amortized sync
What device (and GPU/driver/browser) was it measured on?	Dispatch cost swings ~50× and compute ~64× across the fleet	A workstation-only result presented as universal
What task shape and batch size?	WASM wins small/single-pass; WebGPU wins large/sustained; fusion only helps on some backends	One workload generalized to all workloads

Do not ship a hardcoded dispatch threshold tuned on your development laptop. Do run a short self-calibrating probe at session start — Ayoob’s approach measures adapter capability, effective memory bandwidth, and median dispatch overhead in under 200 ms, then derives a device-specific crossover ratio. Do not decide WebGPU vs WASM from a single benchmark. Do benchmark the actual model, task, and target hardware tier, and record browser and driver versions alongside the numbers. Do not assume kernel fusion transfers across backends. Do verify fusion’s benefit on the backend you actually deploy (Vulkan yes, Metal and CUDA, often no).

The honest take: WebGPU is real, but it is not a number

None of this is an argument against WebGPU. Browser inference is a genuine breakthrough — the ability to run LLMs and other models on-device with no server cost and no data leaving the user’s machine is exactly the kind of capability that made production browser inference viable in 2026. The argument is that the performance envelope is a distribution, not a point. The 159× and 10–15× figures are real measurements of real machines running real workloads — but they describe a narrow slice of a very wide distribution.

The engineering teams that ship browser inference successfully are not the ones who found the biggest headline number. They are the ones who built a calibration loop: measure the crossover on the hardware their users actually have, pick the backend and fusion strategy for the task shape that actually occurs, and re-measure when the driver or browser updates. That is more work than copy-pasting a benchmark. It is also the difference between a feature that feels fast on a workstation and slow on a phone, and one that is fast everywhere it ships.

Treat every WebGPU benchmark as a hypothesis about your specific users’ devices — then verify it. The measurement methodology, not the peak throughput, is what decides whether browser inference is a win in production. And if you are weighing whether on-device inference is worth the build cost at all, the same discipline applies to the decision framework: measure the workload, not the marketing.