[PyTorch] Fix DistributedWeight handling under saved-tensor hooks and in the fused grouped MLP - #3545

[PyTorch] Fix DistributedWeight handling under saved-tensor hooks and in the fused grouped MLP#3545

xrennvidia wants to merge 9 commits into

NVIDIA:mainNVIDIA/TransformerEngine:mainfrom

xrennvidia:xren/dist-weight-saved-tensor-hooksxrennvidia/TransformerEngine:xren/dist-weight-saved-tensor-hooksCopy head branch name to clipboard