Decipher Tech

Technology · September 3, 2026

The WebGPU Dispatch Hierarchy: How Browser Implementations, Backend Choices, and Kernel Fusion Shape LLM Inference Performance

By Arthur Jenkins · Technical Writer

Abstract network visualization representing WebGPU dispatch overhead across browsers

Image: unsplash/CC0 Public Domain

The Hidden Cost of Browser LLM Inference

Large language models running in the browser via WebGPU have become increasingly practical, but performance varies dramatically depending on how operations are dispatched. This article unpacks the hidden overheads that determine whether browser-based inference feels snappy or sluggish. With browser-based AI moving from novelty to production readiness — privacy regulations push computation to the edge, latency-sensitive applications demand local execution, and server-side GPU costs remain high — understanding WebGPU performance characteristics is no longer optional for developers building client-side AI features.

The Dispatch Problem

In a typical transformer forward pass, a model like Llama 3.1 8B generates a token through hundreds of small GPU operations — each one a separate WebGPU dispatch. Naive benchmarks measure each operation from encode-to-synchronization, folding a 450 μs GPU-CPU synchronization cost into every dispatch. The result: per-dispatch costs appear to be 10–20× higher than the actual API overhead. This measurement artifact has led many early benchmarks to dramatically underestimate WebGPU’s potential and overestimate the gulf between GPU and CPU inference.

Primary references:

  • “Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference” (arXiv 2604.02344) — Sequential-dispatch methodology, per-dispatch cost characterization across 4 GPU vendors, 3 backends, 3 browsers. Available at: [arXiv:2604.02344]

  • “Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference” (arXiv 2608.08730) — Dispatch amortization, kernel fusion experiment (876→564 dispatches, 53% improvement). Available at: [arXiv:2608.08730]

  • “Characterizing WebGPU Dispatch Overhead for LLM Inference Across Four GPU Vendors” (arXiv 2604.02344) — Cross-vendor dispatch cost table, Firefox rate limiting at ~1040 μs. Available at: [arXiv:2604.02344]

  • SitePoint, “WebGPU vs. WebGL: Performance Benchmarks for Client-Side Inference” (2026) — GEMM performance benchmarks, 3–8× WebGPU speedup at 2048×2048+. Available at: [SitePoint WebGL]

  • SitePoint, “WebGPU vs WebASM: Browser Inference Benchmarks” (2026) — 10–15× WebGPU advantage for LLM token generation. Available at: [SitePoint WASM]

  • Sven van der Kamer, “WebGPU 2026: Run LLMs in the Browser with Transformers.js” (2026) — Hardware benchmarks across RTX 4090, M3 Max, M3 Pro. Available at: [vucense]

  • webgpu-kernel-fusion research repository — Controlled fusion experiment confirming dispatch count as bottleneck. Available at: [github:webgpu-kernel-fusion]

  • State of AI in Design Systems, July 2026 — Design system implications of browser-based AI inference. Available at: [design systems]

  • “Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU” — Cross-device WebGPU LLM inference study. Available at: [arXiv:2605.20706]

  • Transformers.js documentation — WebGPU and WASM backend performance characteristics. Available at: [HF docs]

The scale of the problem is significant. A 22-layer model like TinyLlama (1.1B parameters) executes at minimum 6 GEMM operations per transformer layer (Q/K/V projections, output projection, two FFN layers), plus embedding and LM head projections, totaling over 130 GEMMs per forward pass. Because these operations are sequential, even small per-dispatch costs compound across hundreds of operations per token, producing a 2–3× difference in time-to-first-token between favorable and unfavorable WebGPU configurations.

Sequential-Dispatch Measurement

The breakthrough came with a simple but powerful measurement technique: dispatch N identical operations together and synchronize only once at the end, then divide by N. This isolates the marginal cost of a single dispatch, amortizing the one-time synchronization cost that real autoregressive decoding incurs only once per token.

Across four GPU vendors (NVIDIA, AMD, Apple, Intel), three backends (Vulkan, Metal, D3D12), and three browsers (Chrome, Safari, Firefox), the sequentially measured per-dispatch cost is remarkably consistent:

BackendPer-Dispatch Cost (μs) SequentialSingle-Op Cost (μs) (overestimates)
Vulkan (Dawn, NVIDIA/AMD/Intel RTX 5090)24–36496.8–2071.2
Metal (Apple M2)32–71, with Safari at 31.748.3–248.0
D3D12 (Chrome on Windows, Intel/RTX 2000 iGPU)58.7–66.52728.8–3123.6

Critically, Firefox shows ~1040 μs dispatch cost regardless of platform, consistent with browser-level rate limiting that makes it impractical for LLM inference workloads. This single finding may explain why Firefox-based AI applications have historically underperformed Chrome and Safari counterparts.

The distinction between “per-dispatch cost” (WebGPU API overhead alone) and “per-operation overhead” (~95 μs including Python/framework cost) is actionable: fusion experiments confirm that reducing dispatch count — not optimizing kernels — is the primary lever for performance improvement.

Browser Implementation Differences

Not all WebGPU implementations are created equal. The table below summarizes measured dispatch costs across browsers and backends, drawn from systematic characterization studies spanning multiple GPU vendors and browser engines:

Browser/ImplementationBackendPer-Dispatch (μs) SequentialNotes
Chrome (RTX 5090, Linux)Vulkan32.8Baseline discrete GPU, Linux
Chrome (RTX 2000, Windows)D3D1258.7Translate layer overhead
Chrome (Intel iGPU, Windows)D3D1266.5Integrated GPU, higher dispatch cost
Safari (Apple M2)Metal31.7Apple-optimized Metal, surprisingly low
wgpu-native (RTX 5090)Vulkan35.8Reference implementation, Vulkan
wgpu-native (Apple M2)Metal71.1Metal, higher than Safari

Key observations emerging from the data:

  1. Vulkan backends are consistently faster than Metal across all tested vendors — dispatch costs of 24–36 μs vs 32–71 μs represents a ~2× difference in per-dispatch overhead
  2. Safari on Metal surprises with lower dispatch cost (31.7 μs) than wgpu-native (71.1 μs), likely due to Apple’s highly optimized Metal implementation that bypasses some of the wgpu-native abstraction layer overhead
  3. Chrome on D3D12 shows higher dispatch overhead (58.7–66.5 μs), reflecting the translation layer between Vulkan/Direct3D12 and the underlying GPU API — this is the cost of cross-API compatibility
  4. Firefox’s rate limiting at ~1040 μs per dispatch makes it unsuitable for performance-sensitive LLM inference workloads, effectively ruling out Firefox as a target browser for high-performance browser-based AI

The browser landscape as of mid-2025 shows WebGPU support at Chrome 113+ (stable, enabled by default, covering ~65–70% of desktop users per StatCounter), Firefox Nightly (behind dom.webgpu.enabled flag), and Safari Technology Preview (experimental support varying across macOS versions). This means that for production browser AI applications, Chrome is the primary target, with Safari as a secondary target for macOS users and Firefox as a non-viable option for performance-critical workloads.

Kernel Fusion: The Practical Optimization

Kernel fusion merges multiple small dispatches into a single larger one, keeping the underlying computation unchanged. A controlled experiment from the WebGPU dispatch overhead characterization research fused 876 dispatches down to 564 (while keeping WGSL shaders essentially the same) yielded a 53% throughput improvement — Time-to-First-Token dropping from 71.4 ms to 41.6 ms.

The causal evidence is compelling and clean: the improvement comes from 312 fewer dispatches, not from improved kernel quality or reduced memory traffic (which was negligible at ~1.8 MB, under 1 μs). This confirms that per-dispatch overhead, not kernel compute efficiency, is the bottleneck at batch size 1.

Fusion is particularly effective on Vulkan (where it provides a 1.4× throughput gain providing causal evidence that per-operation overhead is the actionable bottleneck) but provides no benefit on Metal or CUDA — further evidence that the bottleneck is API-level dispatch overhead, not GPU compute capability. The fusion experiment’s clean isolation of the variable (dispatch count with everything else held constant) makes it one of the most persuasive pieces of evidence in the entire WebGPU performance corpus.

Browser Support and Developer Recommendations

For developers building browser-based LLM inference, the practical recommendations emerging from the dispatch hierarchy research are:

  1. Use Vulkan over Metal when possible — lower dispatch overhead across all tested hardware, with the gap ranging from ~2× on discrete GPUs to even larger ratios on integrated solutions
  2. Be aware of browser differences — Chrome on Vulkan significantly outperforms Safari on Metal and especially Firefox with its ~1040 μs per-dispatch rate limiting; test across all target browsers
  3. Consider kernel fusion for applicable workloads — it provides up to 53% throughput improvement with no quality trade-off, making it the most impactful single optimization for dispatch-heavy inference
  4. Test across target browsers — dispatch overhead varies by 2–3× across the browser landscape, and Firefox’s rate limiting may make it non-viable for certain use cases; what works on Chrome may not work on Safari or Firefox
  5. Prefer discrete GPUs for production — mobile/integrated GPUs show ~2× higher dispatch overhead (59–67 μs vs 24–36 μs on desktop discrete), which compounds across hundreds of dispatches per token

The WebGPU dispatch hierarchy matters because it directly determines whether browser-based LLM inference feels responsive or introduces perceptible latency. The difference between 32 μs and 71 μs per dispatch may seem small in isolation, but across hundreds of dispatches per token, it compounds into a 2–3× difference in time-to-first-token — the difference between an app that feels snappy and one that feels laggy. For edge devices with shared CPU/GPU memory, this choice also directly impacts available system memory, as WebGPU GPU buffer allocation (600 MB–1 GB for a 1.1B parameter INT4 model) directly reduces system memory availability.

Summary Table: Dispatch Overhead Impact Factors

FactorMeasured ImpactDeveloper Action
Per-dispatch overhead24–36 μs (Vulkan) / 32–71 μs (Metal); 2–3× variation across backendsChoose Vulkan backend when possible
Browser choiceChrome (Vulkan) fastest; Safari (Metal) moderate; Firefox (rate-limited) potentially non-viableTest across target browsers; prioritize Chrome
Kernel fusionUp to 53% throughput improvement; effective on Vulkan, not on Metal/CUDAImplement or use fused WebGPU backend
Hardware tierDesktop discrete GPUs ~2× faster than mobile/integrated GPUs for dispatch overheadBudget GPU memory for embedded/deployments
Model sizeOverhead consistent across model sizes (~95 μs per-operation including framework)Optimization levers independent of model scale

The takeaway for developers: WebGPU enables browser-based LLM inference that was unimaginable a few years ago, but performance depends critically on understanding and mitigating dispatch overhead. The sequential-dispatch measurement method, kernel fusion, and backend/browser selection are the three levers that matter most — and together, they can make the difference between an application that’s usable and one that isn’t.

Practical Code Example: Sequential Dispatch Measurement

Developers can implement the sequential-dispatch pattern directly in their WebGPU code:

// Naive approach: measure single operation with sync
async function measureNaive(gpuContext, operation) {
  const start = performance.now();
  await operation();  // Single dispatch with implicit sync
  const end = performance.now();
  return end - start;  // Includes 450+ μs sync cost
}

// Sequential approach: amortize sync cost
async function measureSequential(gpuContext, operation, iterations = 100) {
  const start = performance.now();
  for (let i = 0; i < iterations; i++) {
    await operation();  // Dispatch N times, sync once at end
  }
  const end = performance.now();
  return (end - start) / iterations;  // Amortized per-dispatch cost
}

The sequential approach typically yields per-dispatch costs 15–20× lower than the naive approach, directly validating the measurement methodology that the research community has validated across multiple GPU vendors and browsers.

Conclusion

Browser-based LLM inference performance hinges on understanding the WebGPU dispatch hierarchy. The choice of backend (Vulkan vs Metal vs D3D12), browser (Chrome vs Safari vs Firefox), and optimization technique (kernel fusion) each have measurable, sometimes dramatic impacts on time-to-first-token and sustained token generation throughput. What was once a mysterious “it just works” story is now a quantifiable engineering discipline: measure dispatch overhead for your target configuration, consider kernel fusion if your workload is dispatch-heavy, and always test across your target browsers. The 2–3× performance range that opens up from these choices translates directly into user-perceived responsiveness, making the difference between an AI-powered web application that feels magical and one that feels laggy.

References

  • “Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference,” arXiv 2604.02344 — Sequential-dispatch methodology, per-dispatch cost characterization across 4 GPU vendors, 3 backends, 3 browsers
  • “Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference,” arXiv 2608.08730 — Dispatch amortization, kernel fusion experiment (876→564 dispatches, 53% improvement)
  • “Characterizing WebGPU Dispatch Overhead for LLM Inference Across Four GPU Vendors,” arXiv 2604.02344 — Cross-vendor dispatch cost table, Firefox rate limiting at ~1040 μs
  • SitePoint, “WebGPU vs. WebGL: Performance Benchmarks for Client-Side Inference” (2026) — GEMM performance benchmarks, 3–8× WebGPU speedup at 2048×2048+
  • SitePoint, “WebGPU vs WebASM: Browser Inference Benchmarks” (2026) — 10–15× WebGPU advantage for LLM token generation
  • Sven van der Kamer, “WebGPU 2026: Run LLMs in the Browser with Transformers.js” (2026) — Hardware benchmarks across RTX 4090, M3 Max, M3 Pro
  • webgpu-kernel-fusion research repository — Controlled fusion experiment confirming dispatch count as bottleneck
  • State of AI in Design Systems, July 2026 — Design system implications of browser-based AI inference
  • Transformers.js documentation — WebGPU and WASM backend performance characteristics
  • Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU — Cross-device WebGPU LLM inference study

Primary sources:

← Back to blogs

Ready To Scale
Your Ecommerce?

Let Decipher Tech handle your ecommerce operations end-to-end. Our ECOMAS model means you get a full team of specialists without the overhead of hiring in-house.

Get In Touch → See Our Work