Skip to content

feat(models): Hy4-flash with sparse attention, engram table streaming, MTP and vision - #294

Merged
asher merged 34 commits into
mainfrom
feat/hy4-flash
Nov 19, 2026
Merged

asher merged 34 commits into
mainfrom
feat/hy4-flash

Conversation

@asher

@asher asher commented Nov 19, 2026

Copy link
Copy Markdown
Owner

No description provided.

asher added 30 commits November 18, 2026 07:32
…, a lazy node, so decode builds the next step while the gpu runs
…tops copying each layer's cache per token and stays flat with depth
…s, reading the pool in place, with pool keys stored in fp16
…olling cache, so cache memory past the window grows with the sparse layers only
…in one kernel each and reuses its constants, 400 fewer graph nodes a token with the same bits
…tches spread over the GPU instead of one threadgroup on one core, 3.7% faster hy4-flash decode
…d starts each read when the step is built, so a 16k prefill idles the gpu 3% for the table instead of 18% and decode is 1% faster
… and radix select, 1.4x faster prefill at 128k and 4% faster at 16k
…eys with one dense matmul over the dequantized stack, 4% faster prefill at 16k
…4 threads on the fp32 gate, 0.6% faster decode
…ds and the pooled indexer scores, a request no longer fails on the per-layer head list
…rom the drive during its own forward, 4% faster prefill at 16k
…ed at prefill, glm-5.3-flash drafts from the whole prompt again
…kernels, the read-ahead row fits the table lint
…le format, and the released file's tensor names against the model tree
…ags, registry rows and the vendor manifest follow the mtp and vision wiring
…s that table as weights when sizing the mlx cache
@asher
asher merged commit 69b2506 into main Nov 19, 2026
11 checks passed
@asher
asher deleted the feat/hy4-flash branch November 19, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant