[Thymos2027]

Problem and result

The discussions around #4335 and #4347 did not include independent Windows/Blackwell measurements or a directly inspectable evidence bundle. On an RTX 5060 Ti, the two proposed hybrid VRAM policies also showed different prefill/generation tradeoffs that are not captured by a single throughput number.

This PR adds an independent report of the two proposals, tested separately, with original inputs, outputs, per-run timings, failed gates, and reproducible offline checks:

• [Engine] q36: a 4096-token prompt chunk in the hybrid mode when the card has room #4335: at a configured 32K context, chunk 4096 reduced the 16K-input fixed-history prefill+generation cost from 10.48 s to 8.90 s (faster in 5/5 pairs), while the 1K-input sum increased from 3.33 s to 3.40 s (slower in 5/5 pairs).
• [Engine] q36: give the packed Q8_0 decode copies back to the expert slots in the hybrid mode #4347: releasing 1.78 GiB increased resident GPU experts from 63 to 89 and improved the 16K generation component, but the prefill cost offset that gain in the 96-token fixed-history workload. An earlier free-generation identity failure is retained and is not reclassified as a valid speed comparison.

The engine, server, and defaults behave exactly as before; this PR adds benchmark evidence only.

Related discussion

• [Engine] q36: a 4096-token prompt chunk in the hybrid mode when the card has room #4335 and the Windows/Blackwell result comment
• [Engine] q36: give the packed Q8_0 decode copies back to the expert slots in the hybrid mode #4347 and the Windows/Blackwell result comment

No new feature, refactor, or default change is proposed.

Approach

Add one self-contained report directory under bench/results/2026-10-28-community-rtx-5060-ti-qwen36-hybrid/. It contains all 40 fixed-history runs/20 pairs, synthetic prompts and exact token IDs, 96-token histories, all task inputs, redacted engine logs, matching full-logit dump hashes, original driver/plan archives, and an offline verifier.

The measurements were made on 28–29 October 2026 with native Windows 11, RTX 5060 Ti 16 GB, i7-12700, 128 GB DDR4, driver 610.88, MSVC 19.42/CUDA 13.2/sm120, and Qwen3.6 UD-Q4_K_XL. The measured baseline was 5e8a4cca; the original proposal heads were bdf0bce2 and 7be0a3a1. The report branch is based on newer upstream 46c152a1, which is not presented as a newly measured engine.

Post-measurement portable replay utilities are separated from the original executed drivers. These are nonstandard controlled measurements, not API/free-generation throughput or broad quality equivalence.

Validation

Passed:

• Pinned native builds and 16 selected tests per build.
• Same-chunk and matched-placement full-logit gates.
• 24/24 and 48/48 small deterministic task checks.
• All 40 formal requests matched their arm's archived argmax IDs with zero expert-slot replacements.
• Before/after HtoD controls remained approximately 24.8 GB/s.
• The publication-time offline verifier recomputed all pairs/statistics against the logs and checked fixture/file hashes.
• New utility syntax, help, planning modes, adapter compile/link, and missing-file argument checks passed without loading a model.
• Staged Git bytes, local links, change scope, whitespace, private paths, and credentials were checked.

Failed:

• The initial [Engine] q36: give the packed Q8_0 decode copies back to the expert slots in the hybrid mode #4347 dynamic 256-token output-identity gate failed; the original protocol stopped and its outputs are retained.
• The first adapter link omitted the CUDA library path; the failed attempt is documented and the corrected link passed.
• Initial task scoring included the transport EOS marker; raw answers and original judgments are retained alongside the corrected scoring.

Not run:

• Portable GPU replay after publication packaging.
• Full CTest.
• Broad quality evaluation or new dynamic-cache/free-generation performance acceptance.
• API/concurrency, longer outputs, other contexts or VRAM sizes, batch/MTP, HIP/SYCL/Metal, or critical-path profiling.

Limits

The results cover one Windows/CUDA system, fixed expert residency, synthetic inputs, and a 96-token forced history. They do not establish broad quality equivalence, dynamic-cache behavior, standard decode throughput, or a universally faster default. The two proposals were not combined. No engine or default change is requested, and no combined-PR result is claimed.

[github-actions]

Thanks for the pull request. To keep the tracker readable, every item needs an area prefix in the title and the parts of the pull request template filled in. This one is missing:

• Fill in "Problem and result": what triggered the change, and how it behaves before and after.

Edit the pull request (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment /reopen.

Details: CONTRIBUTING.md

[github-actions]

Thanks, this follows the format now.

[Niko1221]

Thank you for measuring this and writing it up. We are building a Strata website with a page for community benchmarks, and results like these will be added there when it launches, so we are closing this pull request here; the numbers are not lost. If your numbers change with a new version, you are welcome to send them again.