WebGPU Kernel Fusion Is 159× Faster Than PyTorch — But Full Fusion Isn't the Answer

The Browser GPU Compute Revolution Arrived Without Most Developers Noticing

WebGPU is no longer experimental. In mid-2026, five independent research projects running in browser tabs — a Phi-3-mini chatbot at ~40 tokens per second, a quantum statevector simulator, a full Geant4-DNA radiolysis simulation, and a benchmark platform with 400+ submissions from real hardware — are all doing their compute on the same GPU API, with no server, no CUDA install, no Python runtime, and no WASM overhead. The question is no longer whether browser GPUs can do real compute work. It is how to structure the compute for best performance, and the answer is more nuanced than the headline numbers suggest.

The headline number deserves attention: a March 2026 preprint by independent researcher Ahmet Baris Gunaydin reports that a WebGPU compute shader running in Chrome achieves 159× the throughput of PyTorch MPS on the same Apple M2 Pro GPU for a fused sequential fitness evaluation, and 720× over PyTorch CUDA on an NVIDIA Tesla T4. These are not synthetic microbenchmarks — they run standard optimization workloads (Rastrigin, Acrobot-v1, MountainCar-v0, CartPole-v1) as real WGSL compute shaders. The paper confirms the technique across four GPU APIs: CUDA, WebGPU, JAX/XLA, and Triton.

But there is a catch. When the same fusion approach is scaled to a production-sized language model (Phi-3-mini, 3.6 billion parameters), full fusion performs 18× worse than a carefully decomposed 10-kernel design. The architectural decision that actually matters is not “fuse or don’t” — it is “where do you stop fusing.”

What Kernel Fusion Actually Does to GPU Dispatch Overhead

The core insight is simple but counterintuitive: GPU dispatch overhead is large enough that it dominates the runtime of short sequential workloads, yet small enough that it is not the bottleneck in transformer inference. Understanding where your workload sits on this curve determines the optimal factoring.

GPU compute shaders work by dispatching workgroups to the GPU. Each dispatch carries overhead — API calls, command buffer submissions, GPU scheduler work, cache warm-up. On WebGPU, a June 2026 characterization across four GPU vendors (NVIDIA, AMD, Apple, Intel), three browsers (Chrome, Safari, Firefox), and two native implementations finds the true per-dispatch API overhead is 36–71 µs, with total per-operation overhead including runtime/framework cost at ~95 µs. The study notes that synthetic benchmarks overestimate dispatch cost by roughly 20× compared to real inference pipelines — a finding that matters later when we evaluate the trade-offs.

Kernel fusion eliminates this overhead by combining multiple sequential steps into a single dispatch. Instead of dispatching 500 separate shader invocations for a 500-step simulation, you write one shader that loops internally:

// Fused sequential step — single dispatch per simulation
@compute @workgroup_size(64)
fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
    let idx = gid.x;
    var pos = load_position(idx);    // initial state
    var vel = load_velocity(idx);

    for (var step: u32 = 0u; step < 500u; step++) {
        let acc = compute_gravity(pos, masses[idx]);
        vel += acc * DT;
        pos += vel * DT;
    }

    store_position(idx, pos);
    store_velocity(idx, vel);
}

The preprint measures the effect precisely. On a 1,500-timestep financial simulation:

Runtime	Throughput (gen/s)	vs WebGPU
WebGPU fused shader	46.2	1× (baseline)
JAX GPU (lax.scan + vmap)	6.43	7.2× slower
PyTorch CUDA	0.49	94× slower
PyTorch MPS	0.29	159× slower

The fusion advantage scales with episode length L. On Acrobot-v1 (500 timesteps), the gap over JAX narrows to 1.29×. On embarrassingly parallel Rastrigin (no sequential dependency), JAX dominates WebGPU 1,164 vs 170 gen/s — confirming the speedup is specific to sequential workloads. The paper also shows that torch.compile fails to fuse at L ≥ 1,000, while the hand-written WGSL shader handles arbitrary lengths.

For more context on how WebGPU compares to WebGL for inference workloads, a February 2026 SitePoint benchmark found WebGPU delivers 3–8× faster matrix multiplication at 2048×2048+, translating to 3.8× faster LLM token generation (85 ms/token vs 320 ms/token for Phi-3-mini on M2).

The 18× Gap: Full Fusion Meets a Real LLM

Kernel fusion sounds like a universal win. Fuse everything into one shader, eliminate all dispatch overhead, achieve maximum throughput. This is what the webgpu-fusion-max project set out to test: can you run Phi-3-mini (3.6B parameters, D=3072) entirely inside a single fused WGSL compute shader — all 32 transformer layers, attention, FFN, everything — dispatched once per token?

The answer is yes, you can — and it runs at 2.2 tokens/second on an M2 Pro. That is an 11× speedup over the unfused baseline (0.2 tok/s). But here is the part that matters: the project’s sibling zero-tvm runs the same model at ~40 tok/s with 228 dispatches per token. That is an 18× gap, and it flips the intuition about fusion entirely.

Strategy	Dispatches/token	Phi-3-mini throughput	Bundle size
Full fusion (1 kernel)	~1	2.2 tok/s	—
10-kernel decomposition	228	~40 tok/s	157 kB / 33 kB gz
WebLLM (85 autotuned kernels)	342	~51 tok/s	5.9 MB / 2.1 MB gz

The author’s own analysis in the fusion-max README is blunt: “At D=3072, dispatch overhead is real but not dominant. Once you fuse everything into one shader, you give up the GPU’s ability to pick optimal launch geometry for each stage (Q/K/V projection, attention, and FFN want different parallelism), and that loss is bigger than the dispatch-overhead saving.”

This is a concrete illustration of the arXiv characterization’s finding: at ~95 µs per dispatch and 228 dispatches, total overhead is ~22 ms — against a per-token decode time of ~25 ms (at 40 tok/s), dispatch overhead accounts for under 10% of runtime at production scale. The 90% is kernel computation, and a single fused shader cannot allocate GPU resources optimally across operations with fundamentally different parallelism profiles.

The Zero-TVM Architecture: 10 Kernels, 27 Files, 80% of a Compiler

zero-tvm — live in the browser — is the most instructive data point in the current WebGPU inference landscape. Where WebLLM ships 85 autotuned WGSL kernels (12,962 lines, generated by Apache TVM’s compiler) driven from a 5.9 MB WASM runtime, zero-tvm uses exactly 10 kernel roles across 27 hand-written WGSL files (3,078 lines) and ~2,000 lines of TypeScript — no compiler, no WASM scheduler, no runtime.

Dimension	WebLLM (TVM)	Zero-TVM	Transformers.js v4
Unique kernels	85 (autotuned)	10 roles / 27 files	ORT-generated
WGSL lines	12,962 (generated)	3,078 (hand-written)	0
Runtime	TVM → WASM	Plain TypeScript	C++ via ONNX Runtime
JS bundle (excl. weights)	5.9 MB / 2.1 MB gz	157 kB / 33 kB gz	~10 MB
Decode speed (M2 Pro)	~51 tok/s	~40 tok/s	—

The key insight from zero-tvm is not that hand-written kernels are better — they are not, at 40 vs 51 tok/s they are ~22% behind the autotuned compiler. The insight is that the gap is only 22%. The project demonstrates that the compiler’s entire complexity budget (85 kernel variants, optimizer passes, WASM runtime, 12,962 generated lines) buys a 1.22× speedup over a stack you can read end-to-end in a single sitting. As the project puts it, “Every FLOP the model executes is in a file you can open. Every GPU buffer has a human label.”

The 10 kernel roles themselves are instructive:

• qkv_fused.wgsl — Q/K/V projection + RoPE + paged-KV append (1 dispatch vs TVM’s 3)
• attention.wgsl — paged attention with page-table read
• fused_ffn.wgsl — gate + up + SiLU in one pass
• add_norm.wgsl — residual add + RMSNorm fused
• Subgroup variants (_sg.wgsl) for Apple GPUs

This is not academic. The project ships a working chat interface at zerotvm.com where you can type a prompt and watch Phi-3-mini generate text, entirely in the browser, on hand-written WGSL. It is a proof that browser LLM inference does not require a compiler stack — just careful kernel design at the right granularity.

What 32,604 Real-World GPU Benchmarks Tell Us

The research ecosystem described above is running on real hardware — not idealized benchmarks. The Volume Shader 2026 browser GPU report analyzed 32,604 production submissions and found that WebGPU now accounts for 86.2% of all GPU-accelerated browser sessions, with WebGL2 at 13.8%. Desktop: 92.9% WebGPU. Mobile: 84.1% WebGPU.

The mobile figure is the one that matters: 73.6% of all submissions came from mobile devices. The benchmark’s center of gravity is not the RTX 5060 — it is the mass-market phone GPU (Mali-G57 MC2 had 3,151 submissions alone).

Preset	WebGPU avg FPS	WebGL2 avg FPS	Ratio
Ultra Low	56.0	35.2	1.59×
Balanced	35.7	23.3	1.53×
High	23.4	24.4	0.96×
Extreme	14.5	16.1	0.90×

At lighter workloads, WebGPU wins by a wide margin. At heavier shader loads, the gap narrows and reverses — WebGL2’s simpler driver stack sometimes wins on older hardware. This matches the dispatch-overhead research: GPU-level optimizations matter most at the lighter end of the workload spectrum, which is also where most browser-based ML inference will land (quantized 2–4B parameter models, not 70B clusters).

For teams evaluating WebGPU for production workloads, the data supports a pragmatic starting point: the runtime path is ready; the fallback strategy (WebGL2 or CPU) still matters for the 13.8% of sessions and the heaviest preset tiers.

The Decision Framework: When to Fuse, When to Decompose

Based on the evidence from these five research projects, here is a decision framework for factoring WebGPU compute:

Workload characteristic	Fusion strategy	Why	Reference
Sequential simulation, long episodes (L ≥ 500)	Full fuse — 1 dispatch per simulation	Dispatch overhead dominates; fused 94× faster than PyTorch CUDA at L=1,500	Fusion preprint
Sequential simulation, short episodes (L < 100)	Partial fuse — fuse per-simulation-step group	Gap over JAX is only 1.3× at L=500; extra granularity costs little	Fusion preprint
Transformer LLM inference (D ≥ 768)	10–15 kernel roles — fuse QKV, fuse FFN, decompose by parallel profile	Full fusion 18× slower; 10 kernels get 80% of compiler’s speed	fusion-max, zero-tvm
Embarrassingly parallel (Rastrigin, Monte Carlo)	Don’t fuse — let GPU scheduler distribute	JAX GPU does 1,164 gen/s vs WebGPU fused 170 gen/s; JAX unfused also faster	Fusion preprint
Small model inference (D < 256)	Full fuse or near-full fuse — try both	Dispatch overhead a larger fraction; gap between strategies narrows	webgpu-transformer-fusion
Production deployment with fallback	Decomposed + WebGL2 fallback	13.8% of sessions still on WebGL2; mobile-heavy audience needs both paths	Volume Shader report

The single most actionable takeaway: start with the decomposed design, then fuse only the stages that profiling shows are dispatch-bound. Do not start with full fusion and expect to decompose your way back to performance. The full-fusion-to-reality gap is 18×, and you cannot recover that by un-fusing later — the fused shader’s workgroup geometry is baked into its WGSL.

Browser GPUs Are First-Class Compute Targets — If You Respect the Architecture Math

The research momentum in the WebGPU compute space over the last six months is remarkable. A single independent researcher has published two preprints, an SDK, and a live benchmark platform that collects data from 92 unique GPU configurations across 7 vendors — all running in a browser tab. The gpubench.dev platform aggregates 400+ runs from real devices showing Apple Metal-3 silicon averaging score 440, NVIDIA Blackwell at 334, and mobile Qualcomm Adreno GPUs achieving 15,000 tokens/second average on fused transformer inference.

The message for engineering teams is threefold.

First, the browser is now a viable target for compute-intensive ML workflows, as long as you stay within the 2–4B parameter range with quantization. Transformers.js v4, WebLLM, and zero-tvm all exist right now. The infrastructure is mature enough for production.

Second, kernel fusion is the right technique for the right problem, but it is not a universal hammer. The 159× number is real and replicable for sequential fitness evaluations. For transformer inference, the optimal strategy is 10–15 carefully chosen kernel roles — not 1, and not 85.

Third, the mobile-first reality of the web shapes every decision. With 73.6% of WebGPU sessions on mobile devices and the Mali-G57 being the single most common GPU, browser-based compute must optimize for thermal constraints and shared-memory bandwidth, not peak desktop FLOPS. The Volume Shader report’s conclusion is the right one: “WebGPU is the main road now” — but the road runs through mobile, and the potholes (backend fill rate at 88.3%, WebGL2 fallback for 13.8% of sessions) are real.

The browsers caught up. The APIs shipped. The benchmarks are public. The stack that matters in 2026 is not about choosing a runtime or a compiler — it is about understanding the architecture of your compute, choosing the right granularity for your kernels, and measuring against real hardware. The tools to do all of that now run in <script type="module">.