Skip to content

Optimize context parallel attention layouts and gradient accumulation - #3683

Draft
YangXu1990uiuc wants to merge 10 commits into
NVIDIA:mainfrom
YangXu1990uiuc:codex/optimize-context-parallel-attention
Draft

Optimize context parallel attention layouts and gradient accumulation#3683
YangXu1990uiuc wants to merge 10 commits into
NVIDIA:mainfrom
YangXu1990uiuc:codex/optimize-context-parallel-attention

Extend context parallel AllGather accumulator reuse to CP8

cd8fb41
Select commit
Loading
Failed to load commit list.
DCO / DCO succeeded Oct 17, 2026 in 0s

DCO

All commits are signed off!

View more details on DCO