AI
284 billion parameters on your desk: running DeepSeek V4 Flash on the NVIDIA GB10 Superchip
Vlado Balko

This weekend I ran DeepSeek V4 Flash (284B MoE, 13B active) on a single ASUS Ascent GX10 — the ASUS-branded GB10 Superchip workstation. The model fits, runs at 15–19 tokens/sec decode at 128K context, and is genuinely usable as a backend for coding agents.
My motivation was three-fold:
See what GB10 can actually do with a frontier-class open model
Push the silicon to the limit — try to wring out every percent
Touch the recent wave of LLM-optimization techniques that have emerged in the last few weeks: MTP (multi-token prediction), TurboQuant KV compression, EAGLE-3, custom Flash Attention kernels for unusual head dims, etc.
I tried each of them. Some worked, most didn’t (for reasons I’ll get into), and the cumulative weekend speedup ended up around +10.5% over Saturday’s baseline. Here’s the technical breakdown — every attempt, every failure, every “huh, that’s interesting.”
Setup — hardware and model
The box: ASUS Ascent GX10 — ASUS’s productization of NVIDIA’s GB10 design (same Superchip as NVIDIA’s own DGX Spark, just a different OEM enclosure):
GB10 Grace Blackwell Superchip:
20 ARM cores (10× Cortex-X925 + 10× Cortex-A725)
Blackwell GPU integrated (~1 PFLOP FP4, native MXFP4 tensor cores)
128 GiB LPDDR5X unified memory @ 273 GB/s — shared between CPU and GPU
Compute capability 12.1 (sm_121a)
Comes preloaded with Ubuntu 24.04 + CUDA 13.0 + NVIDIA driver 580
This is not server-class hardware. It’s a quiet workstation-class node, ~$4000 retail, designed for personal AI work. The unified memory is the whole pitch: 128 GB of GPU-addressable memory at less than half the latency of full discrete GPU memory. The model: DeepSeek V4 Flash, released April 24 2026 by DeepSeek-AI:
284 billion total parameters, 13B active per token (MoE, 256 experts, top-K=6) 1 million token native context
Hybrid attention: Compressed Sparse Attention (CSA) + Heavily Compressed Attention (HCA)
MTP head (Multi-Token Prediction) trained jointly as 1 extra layer
MIT license, weights on HuggingFace
V4 Flash’s native checkpoint is ~160 GB (FP4-FP8 mixed). For a single 128 GB box you need community quants.
I used antirez (Salvatore Sanfilippo)’s quantization: DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf — 87 GB.
What’s “IQ2_XXS” and why does it work?
A 2-bit-per-weight quantization that isn’t awful needs to be smart about which bits go where. The recipe:
Routed experts (the bulk of V4’s parameters, 256 of them per layer, of which only 6 fire per token): IQ2_XXS — "i-quant" extra extra small. 2 bits per weight nominally, ~2.06 bits-per-channel effective. Uses a codebook lookup (E8 lattice centroids) instead of straight per-tensor scaling. The "i" prefix means importance-matrix calibrated — antirez ran ~1M tokens through the FP16 model, recorded which weights matter most, and used that to bias quantization toward preserving the high-impact channels.
Attention LoRA Q/B projections, shared experts, output projection: Q8_0 — 8-bit, basically lossless. These tensors are active on every token (no MoE gating), so the quality hit from aggressive quantization would compound badly.
Output unembed: also Q8_0.
The result: 86.72 GB total, fits with 30+ GiB free for KV cache and runtime buffers. Quality, by my measurement at temperature=0, is indistinguishable from FP8 for chat and code prompts — V4 was designed with aggressive quantization in mind (the official native checkpoint mixes FP4 experts with FP8 attention).
There’s also a separate 3.8 GB MTP weights file ( DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf ). This is the trained MTP head — one extra transformer layer that DeepSeek trained jointly with the main 43-layer stack. Its job at inference: given the parent model's hidden state at position t and the predicted next token, predict token t+2 in parallel. We'll come back to this in Attempt 1.
The first plot twist: V4 is already well-optimized

What I expected: huge model on small-ish memory bandwidth box (273 GB/s) → bandwidth-bound decode → low t/s, room for tricks.
What I measured with llama.cpp (the Fringe210 fork of antirez’s V4-specific llama.cpp branch):
ContextMemory usedDecode @ 8K promptDecode @ 128K context fully filled16K84 GiB14.78 t/s–128K (production)87 GiB15.4 t/s–1M94 GiB (+9 vs 16K)14.82 t/s10.6 t/s @ 8K prompt
Three surprises:
Decode is essentially the same at 16K vs 1M context (14.78 vs 14.82 t/s). On any normal Transformer, going from 16K to 1M would 64× the KV cache and crater the decode — meaning per-token generation slows to a crawl, because attention has to read more and more cached keys/values from memory for each new token. At 1M context with f16 KV, you’re moving hundreds of GB through memory bandwidth per token. V4 doesn’t crater because of how its attention works:
CSA (Compressed Sparse Attention): compresses chunks of past KV state by 4× into “summary tokens,” then does query-dependent top-K selection over those summaries. So instead of attending to all 1M cached positions, the model attends to a few hundred summaries.
HCA (Heavily Compressed Attention): compresses even harder — by 128× — and runs dense attention over the resulting short sequence. Because the sequence is now ~8K tokens of summaries for a 1M context, dense attention is cheap.
V4 interleaves CSA and HCA layers through the stack, getting both fine-grained query-dependent attention (CSA) and broad global context (HCA) at extremely low cost.
Net effect: the per-token attention cost stays roughly constant as context grows. No crater.
The 1M KV cache is only ~7 GB, not 50+ GB. That compression is real, and it’s the whole point of V4’s “Flash” variant.
Prefill peaks at ~334 tokens/sec for ~256-token prompts, then degrades to ~220 t/s at 9K — a totally non-pathological curve. Long-context prefill is workable. Visualization 1 — V4 doesn’t crater
Decode tokens/sec by context size (small prompt, V4-Flash baseline)
15 ┤●────●────●────●────● ← Hybrid CSA+HCA keeps decode flat14 ┤13 ┤12 ┤11 ┤10 ┤└────┬────┬────┬────┬────┬──
Compare: a standard Transformer with f16 KV would drop ~80% from 16K to 1M on this hardware.
Visualization 2 — prefill vs prompt size
Prefill t/s vs prompt tokens (V4-Flash @ 128K ctx, baseline)
350 ┤ ● (285 tok: 330 t/s peak)300 ┤ ╱ ╲250 ┤ ╱ ╲● (1138)200 ┤ ╱ ●─────● (4552 → 8890, ~220 t/s)150 ┤100 ┤●● (72: ~170 t/s)50 ┤30 ┤● (18: 30 t/s — first-token alloc overhead)└────┬────┬────┬────┬────┬────
DeepSeek’s paper claims V4-Flash uses only ~7% of V3.2’s KV cache and ~10% of V3.2’s inference FLOPs. The benchmarks back this up. The bandwidth-bound bottleneck you’d expect on this hardware just isn’t there.
This sets up the real question.
The wrong question, and how I asked it for three days
The wrong question is: “how do I get V4 to run faster?”
That implies there’s a speedup left on the table. There mostly isn’t, because V4’s authors already took most of them. The right question is “where is the actual decode time going, and which of those buckets is amenable to optimization?”
I asked the wrong question for most of the weekend.
Attempt 1: MTP via antirez’s ds4 engine
DeepSeek V4 ships with a trained MTP head (multi-token prediction). The idea: predict 1 token ahead, verify in the same forward pass as the main model. If accepted, you got 2 tokens for the price of one decode step.
Antirez wrote a dedicated C+CUDA inference engine, ds4 , with native MTP support. Build: make cuda-spark on GB10. Works, fast, polished.
I ran a 4×3 sweep across two MTP-tuning knobs:Parameter notes (ds4’s MTP API):
--mtp-draft N : maximum number of draft tokens the MTP head proposes per speculative
step. N=1 means "predict one token ahead." N=4 means "try predicting 4 tokens ahead via recursive draft."
--mtp-margin F : minimum confidence threshold (in log-prob space) for the draft to
"commit" to a proposal before verification. Higher = stricter = lower acceptance but fewer wasted verifications.

MTP doesn’t help. Best case (draft=1): -1% (noise). Anything > 1: -40–50%. Antirez’s own README acknowledges: “the MTP/speculative decoding path is still experimental: at most a slight speedup, not a meaningful generation-speed win.” My measurements confirm.
Why? The MTP head is itself a 256-expert MoE layer (~3.6 GB Q4). Each draft forward is roughly as expensive as one main-model step. With 100% acceptance, that nets zero. With realistic acceptance < 100% (at temperature > 0), it’s a net loss. This was the first hint that V4 isn’t bandwidth-bound — if it were, the cheap MTP draft would amortize over the saved main-model decode step. Instead the draft costs about the same as what it saves.
Attempt 2: implement MTP in llama.cpp from scratch
I wanted MTP in llama-server specifically — that's what opencode (my coding-agent harness) talks to via OpenAI-compatible API. Fringe210's CUDA fork doesn't have MTP. I delegated to a Claude Code subagent with a detailed spec.
The subagent produced ~400 LOC of patches across 14 files: new arch enum, 32 new tensor enums, new layer struct, new C API for sharing parent’s token_embd / output / vocab with the draft, new llm_build_deepseek4_mtp graph class. The build passed, the server loaded the draft, and at temperature=0 it reported 100% acceptance. By every box-checking metric, MTP was working.
The actual benchmark numbers were a different story:
Decode tokens/sec vs context size (our MTP impl)
15 ┤● (16K: 10.8)12 ┤9 ┤ ● (64K: 8.5)6 ┤3 ┤ ● (256K: 1.4)0 ┤ ● (1M: 0.24) ← catastrophic└────┬────┬────┬────┬────
16K 64K 256K 1M ds4 reference impl (same hardware, same model, same algorithm):
●─────●─────●─────●─────●─────● ~13.5 t/s flat across 32K to 1M
A 56× regression at 1M context versus the reference implementation. Same hardware, same model, same algorithm — different code.
Short version of what was wrong: the draft layer used a h_t = e_t shortcut (token embedding as a stand-in for parent hidden state) and disabled the compressed-attention pathway in the MTP layer. The result is effectively O(N²) attention scaling with the configured max context, not actual sequence length. Patching it hits a deeper issue with V4's recurrent state cache also scaling with n_ctx_seq — fixable, but ~8-15 hours of work I didn't commit to.
Lesson #1: code completeness ≠ performance correctness. The MTP code loads and generates valid tokens — but 56× slower than a known-good reference. You don’t catch this with unit tests; you catch it by benchmarking against a reference implementation on day 1.
(Bonus: also worth noting that upstream ggml-org/llama.cpp merged MTP support for Qwen — PR #22673 by am17an, May 16 2026, the day before this experiment. Their architecture uses separate llama_context for the draft but shares token embeddings and uses an after-ubatch hook to plumb the parent hidden state. On Qwen3.6-27B + RTX 3090: 1.85× decode speedup with 72-75% acceptance. So MTP can work — it just doesn't on V4 in its current form.)
TurboQuant: an excellent paper, a poor fit
After MTP didn’t pan out, I went looking for orthogonal speedups. TurboQuant (Zandieh et al., ICLR 2026) caught my eye: a Walsh-Hadamard-rotation + Lloyd-Max-codebook KV cache quantizer with 3.5 bits-per-channel lossless or 2.5 bpc with marginal degradation. 4.5× compression of f16 KV cache. Implementations exist for llama.cpp.
If decode is bandwidth-bound and the KV cache is sitting in HBM, a 4.5× KV compression should give ~4.5× decode speedup.
I delegated the port to another subagent. The subagent’s first action — and credit to it — was to read the V4-specific code and report back: TurboQuant doesn’t help on V4. Three reasons:
V4’s KV cache is already quantized. The fork has a custom
GGML_OP_DSV4_FP8_KV_QUANTIZE kernel that quantizes the compressed-cache outputs to FP8 (8-bit). TurboQuant on top of FP8 would be double-quantization, not 4.5× over f16.
V4 attention doesn’t go through ggml_flash_attn_ext . It uses explicit mul_mat → soft_max → mul_mat . The "fused dequant inside Flash Attention" optimization (which is where TurboQuant's wins live in the reference impl) has nowhere to dispatch.
The compressed KV at 128K context is only 860 MiB, not the multi-GB I assumed. It’s not the dominant memory traffic; the dense per-token matmuls and the indexer top-k path are.
The subagent refused to do the work and wrote a clean report.
This was the most productive 3 minutes of the weekend.
The actual bottleneck: Flash Attention auto-disabled
The same subagent’s investigation pointed me at the real issue. V4’s server log had been quietly emitting:
sched_reserve: layer 2 is assigned to device CUDA0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
sched_reserve: Flash Attention was auto, set to disabled
Flash Attention is disabled for 41 of V4’s 43 layers.
Why? V4 has head_dim=512 (large MLA latent rank). llama.cpp’s fattn-mma-* CUDA kernels require K_seqlen % 256 == 0 for head_dim=512 to enable Tensor Core packing. V4's compressed-attention path produces K_seqlen = 1 + n_comp_visible , which is essentially never divisible by 256 (e.g. decode at 128K: 1 + 32767 = 32769, not divisible).
So CUDA refuses the kernel, scheduler falls back to explicit mul_mat + softmax + mul_mat , and we lose the Flash Attention win on 95% of attention layers. A subagent did a Path A + Path B experiment:
Path A: enable FA for the 2 dense layers (compress_ratio=0) by suppressing the broken cases at the graph level. Server log now says “Flash Attention was auto, set to enabled”.
Path B: swap the FP8 quantization in dsv4.cu for Walsh-Hadamard + Lloyd-Max (still 8-bit storage, just a different math path).
Combined gain: +1 to +5% decode across context sizes. Modest because only 2 of 43 layers benefit.
The real win — +10–15% decode — requires writing a custom CUDA Flash Attention kernel for head_dim=512 without the 256-alignment constraint. I delegated this to another subagent. The result:
Outcome of the custom kernel (updated)
The subagent wrote a from-scratch CUDA kernel ( ggml/src/ggml-cuda/fattn-v4-head512.cu , ~300 LOC):
Standard online-softmax Flash Attention (Rabe-Staats / FA2 pattern)
Single-block per (head, sequence), 256 threads, 67 KB shared memory
Handles arbitrary K seqlen via per-tile clamping — solves the alignment problem fp32 accumulators, half I/O
Result on the full matrix:

Total weekend gain: +8–11% decode across context sizes. Quality preserved on factual, narrative, and code prompts.
But here’s the kicker — the subagent’s honest debrief: the kernel uses scalar fp32 fmadd, not Tensor Cores. The stock MMA kernel uses Blackwell’s Tensor Core instructions for ~30× higher arithmetic throughput per op. The pragmatic alternative, which I didn’t ask for: just relax the existing MMA dispatcher to accept
V4’s head_dim=512 + Q->ne[1]==1 case by adding 4 template instantiations.
Estimated 3–5 hours of work. Expected gain: +12–15% (using actual Tensor Cores instead of scalar math).
So there’s still more on the table. The custom kernel was 1–2 days of work that the subagent did honestly and well, but the cheaper path was sitting in the existing codebase the whole time. Lesson #6: read the existing dispatcher before writing a new kernel.
What I’d tell someone starting this experiment over
The goal of the weekend wasn’t to ship a 50%-faster V4 in production. It was to try things: get a feel for what GB10 can actually run, touch the new wave of optimization techniques, see which ones transfer and which don’t. By that bar, every “failed” attempt was a successful experiment — I now know things about V4’s internals that I couldn’t have learned without trying.
That said, a few patterns showed up worth writing down:
Don’t trust general-purpose optimization claims for highly-specialized models. V4 is sufficiently custom (MLA, CSA, HCA, MTP, hyper-connections, gated delta net) that off-the-shelf tricks miss. The “TurboQuant gives 4.5× compression” claim is true for f16 KV on standard transformers; it’s irrelevant for V4 because the KV path is already FP8-quantized and the attention doesn’t use ggml_flash_attn_ext . Read the model's source first, then optimize.
Reference implementations are gold. ds4 saved me hours of debugging by being a known-good baseline. The 56× MTP regression in my llama.cpp port was invisible from inside my own code — it only showed up when compared against ds4 running the same algorithm on the same hardware.
The unified-memory pitch of GB10 is real. 87 GB models that wouldn’t fit on any single discrete GPU run at usable speeds on this thing. Long context comes essentially for free thanks to V4’s hybrid attention. This is a genuinely useful machine.
The 273 GB/s bandwidth is not the bottleneck for V4 specifically. I went in expecting it to be the limiter and it isn’t. Indexer top-k and per-token MoE compute dominate. Don’t assume your slowest hardware metric is the binding constraint.
The new optimization techniques aren’t interchangeable. MTP, TurboQuant, EAGLE-3, Flash Attention variants — they each target a specific layer of the
inference stack. You can’t just pick the one with the biggest headline number;
you need to understand which bottleneck yours hits. For V4 on GB10, that turned out to be Flash Attention auto-disabled by a CUDA alignment constraint — not what I would have guessed going in.
Visualization 3 — cumulative weekend speedup
Visualization 4 — what failed and why

─────────────────────┼────────┼────────────────────────────────────────
MTP via ds4 | ~2h | Best case -1% (no speedup)
MTP in llama.cpp | ~5h | -27% to -98% (catastrophic, bug)
TurboQuant adapt | ~3min | Subagent refused: doesn't fit V4 ★
Path A (FA dense) | ~4h | +2.4% (only 2 of 43 layers benefit)
Path B (Lloyd-Max) | ~3h | +0.6% (storage already FP8)
Custom FA kernel | ~2h | +7.3% (the actual win, no TC use yet)
★ Most productive 3 minutes of the weekend.
Where this lands
For my coding-agent setup (V4 Flash served on :8005 via opencode -compatible API): 15–19 t/s decode at 128K context, no MTP, full quality
Long-context (256K+) works without performance degradation
Quality is genuinely good — IQ2_XXS at 87 GB stays close to FP8 native (160 GB) for chat/code tasks
+8–11% over baseline from the full weekend’s optimization stack (FA enable + Lloyd-Max KV + custom head_dim=512 kernel)
The remaining ~4–7% (to hit the +15% theoretical) is one dispatcher tweak away (Tensor Cores via MMA template instances)
Net of the weekend: V4 Flash on GB10 is demonstrably faster than the day I started, the optimizations are upstream-able, and there’s a clear (and cheap) next step. The final pragmatic recommendation is to skip the custom kernel and instead extend the MMA dispatcher with V4-shape-aware template instances. That’s the path that uses Blackwell’s Tensor Cores properly and is ~3–5 hours of focused work. Appendix: full data tables
[See “DATA TABLES” section below]
Appendix: source code
All four attempts are preserved as branches in the local working copy on gx10 : master — Fringe210 upstream, commit 44c7b01 (untouched)
weekend/03-path-a-b — after Attempts 1+2 (MTP scaffolding) and 3 (FA enable + Lloyd-Max). Production state mid-Sunday afternoon.
weekend/04-final — + Custom V4 head_dim=512 Flash Attention kernel .
Production state Sunday evening, what :8005 is running.
Each commit message documents the full chain of changes. The MTP-only states from Attempts 1 and 2 weren’t separately checkpointed because they were never production candidates — refer to the weekend/03-path-a-b commit message for the cumulative diff narrative.
Appendix: tools used llama.cpp (Fringe210 fork of antirez/llama.cpp-deepseek-v4-flash, sm_121a build)
antirez/ds4 (independent V4 inference engine, make cuda-spark )
antirez/deepseek-v4-gguf (IQ2_XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 quantization)
Claude Code with subagents for parallel CUDA work huggingface-cli + aria2c for model download opencode for harness integration
Appendix: useful links
DeepSeek V4 paper: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro antirez ds4 engine: https://github.com/antirez/ds4 antirez’s llama.cpp V4 fork: https://github.com/antirez/llama.cpp-deepseek-v4-flash
Fringe210 CUDA fork: https://github.com/Fringe210/llama.cpp-deepseek-v4-flash-cuda
TurboQuant paper (ICLR 2026): https://arxiv.org/abs/2504.19874 am17an’s MTP support PR for llama.cpp (Qwen-focused):
https://github.com/ggml-org/llama.cpp/pull/22673 davide.zenati’s MXFP4 single-GB10 proof-of-life:
📊 DATA TABLES (for reference / inclusion in blog)
Hardware
ASUS Ascent GX10 (ASUS-branded GB10 workstation, same Superchip as
NVIDIA’s own DGX Spark)
GB10 Grace Blackwell Superchip

10× Cortex-X925 + 10× Cortex-A725 ARM cores
Blackwell GPU, compute capability 12.1 (sm_121a), native MXFP4 tensor cores 128 GiB LPDDR5X unified memory @ 273 GB/s
CUDA 13.0, driver 580.142
Ubuntu 24.04
Model
DeepSeek V4 Flash (released 2026–04–24, MIT license)
284B total / 13B active params, 256 MoE experts top-K=6
Native 1M context, hybrid attention (CSA + HCA)
Built-in MTP head (1 extra trained layer)
Quantization (antirez/deepseek-v4-gguf)
Routed experts: IQ2_XXS (2-bit i-quant, imatrix-calibrated)
Attention LoRA Q/Q_b: Q8_0
Shared experts: Q8_0
Output projection: Q8_0
File size: 86.72 GB (vs 160 GB FP4-FP8 native)
MTP draft (separate): 3.6 GB
Baseline benchmarks (no MTP, llama.cpp Fringe210 fork)
Full prompt-size × gen matrix @ 16K context (baseline)


Full matrix @ 1M context (baseline)
Key takeaway: 1M decode ≈ 16K decode
The 1M cliff in prefill (3.44 vs 30.91 at small prompts) is real (fixed ~4.5s slot-prep overhead). But decode at small prompts is essentially the same: 14.82 (1M) vs 14.74 (16K). The KV cache pressure that destroys long-context decode in other models doesn’t exist in V4 thanks to hybrid CSA+HCA.
ds4 engine (antirez’s native C+CUDA, independent of llama.cpp)
Build: make cuda-spark on GB10.
Baseline at default ctx (32K): 14.99 t/s decode, 32.17 t/s prefill.
ds4 + MTP sweep (4×3, prompt = haiku about quantization, default ctx):



ds4 scaling test (MTP enabled, draft=1):
Our llama.cpp MTP port (vs ds4)
MTP code stats (our impl)
14 files modified, ~400 LOC of new patches
New arch enum, 32 new tensor enums, new layer struct
New llm_build_deepseek4_mtp graph class (~180 LOC)
New shared-vocab API ( llama_vocab::share_from() )
Works (loads, generates valid text, 100% draft acceptance at T=0)

Catastrophically slow (O(N²) cost over n_ctx_seq due to disabled compressed attention)
Path A + Path B experiment (Flash Attention + Lloyd-Max swap)
Path A: enable FA for 2 of 43 V4 layers (the 2 dense layers, compress_ratio=0). Identified via sched_reserve diagnostic. 41 compressed layers still fall back because CUDA's fattn-mma-f16 requires K->ne[1] % 256 == 0 for head_dim=512.
Path B: replace FP8 KV quantization in ggml/src/ggml-cuda/dsv4.cu with WHT + Lloyd-Max codebook (per TurboQuant paper, 16 centroids from Beta((d-1)/2, (d-1)/2) distribution).
Combined results:
Pattern: short prompts benefit most (FA helps the 2 dense layers), longer prompts barely move because compressed-path layers (95% of compute) don’t use FA. Custom V4 Flash Attention kernel (the final step)
Wrote ggml/src/ggml-cuda/fattn-v4-head512.cu — a from-scratch CUDA kernel handling head_dim=512 with arbitrary K seqlen:
Online softmax (Rabe-Staats / FA2 pattern), single-token decode
One block per (head, sequence), 256 threads
67 KB shared memory (Q tile + reused K/V tile + scratch)
fp32 accumulators, half I/O via float4 vectorized loads

Per-tile K seqlen clamping → handles unaligned K (the whole point)
Dispatcher hook in ggml/src/ggml-cuda/fattn.cu : only activates for head_dim=512 &&
Q->ne[1]==1 && f16 K/V && single KV head && stock dispatcher would reject . Other
models / shapes untouched.
Removed previous Path A’s local FA guard in src/llama-graph.cpp — the kernel now handles those cases properly.
Final benchmark (custom kernel on top of FA+TBQ):
Final cumulative gain over Saturday baseline: +8–11% decode for context-significant prompts (285+ tokens).
Why not the +15% stretch: the custom kernel uses scalar fp32 fmadd, no Tensor Cores. Blackwell’s Tensor Core instructions are ~30× faster per op. The cheaper alternative (the agent’s recommended next step, not done): add 4 MMA template instantiations under template-instances/ for (512, 512, N, 1) shapes and tweak the dispatcher. Estimated 3-5 hours, expected to land the missing 4-7%.
Memory breakdown (V4 + main, no MTP, ctx 128K)
CUDA0:
Model buffer: 81687 MiB (the 87 GB GGUF)
Compressed KV cache: 860 MiB (V4 hybrid attention)
Standard KV cache: 32 MiB (SWA)
Recurrent state: 20 MiB (Gated Delta Net)
Compute buffer: 697 MiB
─────────────────────────────────
Total used: ~83.3 GiB
Free: ~41 GiB (room for prompt cache, slots)
Memory breakdown @ 1M ctx
Only the compressed KV grows: 860 MiB → 6880 MiB (+6 GB). Everything else stays the same. The model fits 1M context with 30+ GiB free.
Quality samples (FA+TBQ binary, T=0)
Prompt: "The capital of France is"
Output: Paris. The capital of France is Paris. The capital of France is Paris... Prompt: "Once upon a time, in a far away land"
Output: , there was a little girl who lived in a scary house...
Prompt: "def fibonacci(n):"
Output: \n if n <= 1:\n return n\n else:\n return fibonacci(n-1) + fibonacci(n-2)\n...
All coherent, no NaN, quality matches FP8 baseline.
What’s in production
Server llama-server from Fringe210 fork (sm_121a build) with Path A + Path B + custom V4 FA kernelpatches
DeepSeek-V4-Flash-IQ2XXS as deepseek-v4-flash (no MTP, baseline)
128K context, OpenAI-compatible /v1/chat/completions + /completion endpoints Wired to opencode config as provider v4flash-gx10
Decode: 15–19 t/s @ small prompts, 11.4 t/s @ 8K prompts (+10% over Saturday) All Flash Attention paths report Flash Attention was auto, set to enabled Cumulative weekend speedup summary

Theoretical reachable (Tensor Core MMA via template instances): ~12.0–12.4 t/s, +17–20% vs Saturday baseline. Estimated 3–5 hours additional work — not done this weekend.
AI
284 billion parameters on your desk: running DeepSeek V4 Flash on the NVIDIA GB10 Superchip

