[AlpinDale]

Optimizes DeepSeek-V4.1-Flash prefill and batched decode on SM120 with bounded DSA indexing, fused kernels, native MoE, live-row Engram, TP-sharded DSpark, and opt-in paired PCIe collectives. Sparse MLA/MoE kernels adapt BRRR/vLLM-Moet. Keeps sources, registrations, preparation and configuration with subsystem owners. Fixes V4.1 tool/media ordering, MoE layout/bounds, and late-draft fault propagation.

Performance

4× RTX PRO 6000 Blackwell, TP4, 1,048,576-token context, MNBT4096, DSpark K7, 2,048 output tokens, thinking=false. Rates are aggregate tok/s; TTFT is p50/p95 in seconds.

Input tokens	C	Main prefill	PR prefill	Main decode	PR decode	Main TTFT	PR TTFT	Draft accept main/PR
2,674	1	6,742.3	7,164.6	167.2	432.0	0.397/0.400	0.372/0.376	50.86%/58.73%
2,674	8	7,062.0	7,718.8	428.3	1,083.3	1.904/3.209	1.697/2.811	50.88%/59.27%
8,185	1	6,946.8	7,626.4	111.8	362.6	1.170/1.174	1.068/1.069	31.64%/46.03%
8,185	8	7,245.9	7,996.2	294.4	866.9	5.811/9.609	5.166/8.441	31.97%/44.21%
131,071	1	6,659.1	7,787.0	99.9	383.1	19.670/19.709	16.822/16.827	23.93%/48.88%
131,071	8	6,774.2	7,945.1	238.5	882.0	90.562/164.726	75.560/134.550	29.32%/47.11%

PR abd41971d3: three measured batches per cell from one fresh server; main: six batches from two servers. Warmups excluded. Prefill uses cold one-token requests; decode uses warm prefixes and exact token IDs in the common-active window. TTFT uses separate cold 2,048-token requests. Acceptance is pooled over draft tokens.

PR: rebuilt SM120 extension, B12x W4A8 + paired PCIe, FP8 SWA/NVFP4 main KV, 90% GPU memory. Main nightly 03fbc02b7, isolated venv: stock Marlin + FlashInfer/NCCL, FP8 KV, 98% GPU memory; stock B12x failed startup. Deployment comparison, not a kernel-only A/B: math/cache modes and draft acceptance differ.

Launch

Use FlashInfer 0.7.2's bundled B12x. All four selected GPUs need CUDA peer reads and writes without crossing a NUMA system link. Paired all-reduce is experimental and rounds differently from BF16 NCCL.

export CUDA_VISIBLE_DEVICES=0,1,2,3
export OMP_NUM_THREADS=1 MKL_NUM_THREADS=1
export APHRODITE_WORKER_MULTIPROC_METHOD=spawn
export APHRODITE_ENGINE_READY_TIMEOUT_S=3600
export APHRODITE_USE_V2_MODEL_RUNNER=1
export APHRODITE_USE_RUST_FRONTEND=0
export APHRODITE_USE_DEEP_GEMM=1
export APHRODITE_DSV41_SM120_NATIVE=1
export APHRODITE_ALLREDUCE_USE_FLASHINFER=0
export APHRODITE_ALLREDUCE_USE_PCIE_PULL=1
export APHRODITE_B12X_MOE_FP4_FORCE_A16=0
export B12X_AUTOTUNE=1 B12X_DENSE_SPLITK_TURBO=1
export B12X_DYNAMIC_DETERMINISTIC_OUTPUT=1
export NCCL_ALGO='allreduce:Tree' NCCL_PROTO=Simple

taskset -c 0-31 aphrodite serve deepseek-ai/DeepSeek-V4.1-Flash-1017 \
  --served-model-name deepseek-v41 \
  --trust-remote-code \
  --tokenizer-mode deepseek_v41 \
  --tool-call-parser deepseek_v41 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v41 \
  --tensor-parallel-size 4 \
  --moe-backend b12x \
  --attention-backend B12X_DSV41 \
  --engram-config '{"cpu_offload":true}' \
  --kv-cache-dtype fp8 \
  --max-model-len 1048576 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 4096 \
  --max-cudagraph-capture-size 64 \
  --enable-prefix-caching \
  --attention-config '{"indexer_tp_shard_min_rows":2048}' \
  --gpu-memory-utilization 0.90 \
  --no-enable-flashinfer-autotune \
  --mm-encoder-tp-mode data \
  --hf-overrides '{"head_dtype":"float32"}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_tensor_parallel_size":4,"moe_backend":"b12x","attention_backend":"B12X_DSV41","draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T06:31:15.505041Z	5b48bdf	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f653ba4e15

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 346b243b27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: abd41971d3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[AlpinDale]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 5b48bdf012

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".