From NVIDIA Megatron-LM for visibility #18
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ci: Add copy-pr-bot See merge request ADLR/megatron-lm!3829
Signed-off-by: oliver könig <[email protected]>
…model grads refactor Co-authored-by: Pranav Prashant Thombre <[email protected]> Co-authored-by: yaoyu-33 <[email protected]> Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: Hongbin Liu <[email protected]> Co-authored-by: Yu Yao <[email protected]>
M4 p2p communication, schedules and finalize model grads refactor See merge request ADLR/megatron-lm!3378
Co-authored-by: Tong Liu <[email protected]>
feat(moe): Add MoE router fusion See merge request ADLR/megatron-lm!3809
Apex.contrib.nccl_allocator migration See merge request ADLR/megatron-lm!3814
Signed-off-by: oliver könig <[email protected]>
Signed-off-by: oliver könig <[email protected]>
Signed-off-by: oliver könig <[email protected]>
…rnorm/moe_act/shared_experts
perf(MoE): Support recomputation for FP8 layernorm/moe_act/shared_experts See merge request ADLR/megatron-lm!3465
…rallel inference Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: Oliver Koenig <[email protected]> Co-authored-by: Youngeun Kwon <[email protected]> Co-authored-by: Helen Ngo <[email protected]> Co-authored-by: Shifang Xu <[email protected]> Co-authored-by: James Shen <[email protected]> Co-authored-by: Kunlun Li <[email protected]> Co-authored-by: Slawek Kierat <[email protected]> Co-authored-by: Zijie Yan <[email protected]> Co-authored-by: Li Tao <[email protected]> Co-authored-by: Mikolaj Blaz <[email protected]> Co-authored-by: Charlie Truong <[email protected]> Co-authored-by: Dong Hyuk Chang <[email protected]> Co-authored-by: Chenjie Luo <[email protected]>
ZMQ based communication of requests during parallel inference See merge request ADLR/megatron-lm!3757
… to support full CUDA graph capture
Add is_cg_capturable flag to CrossEntropyLoss to support full CUDA graph capture See merge request ADLR/megatron-lm!3815
…ndently installable Co-authored-by: jianbinc <[email protected]> Co-authored-by: Youngeun Kwon <[email protected]> Co-authored-by: Cory Ye <[email protected]> Co-authored-by: Boxiang Wang <[email protected]>
[FSDP] Decouple Custom FSDP to make it independently installable See merge request ADLR/megatron-lm!3443
…ation gives an error about missing eval_iters
This fixes the bug where not using full_validation gives an error about missing eval_iters See merge request ADLR/megatron-lm!3842
Fix cuda graph when VPP is used See merge request ADLR/megatron-lm!3824
Co-authored-by: oliver könig <[email protected]> Co-authored-by: Mcore Bot <[email protected]>
chore: Upgrade dependencies (2025-08-18) See merge request ADLR/megatron-lm!3834
…oss 2/5] Co-authored-by: Mcore Bot <[email protected]>
Co-authored-by: Wil Kong <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
… independetly parallel modules. Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: Pranav Prashant Thombre <[email protected]>
…ture changed and wasn't refactored
…uf-for-mxfp8-param-ag
Co-authored-by: Selvaraj Anandaraj <[email protected]>
Co-authored-by: Selvaraj Anandaraj <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
This reverts commit c173615.
…alization for Megatron FSDP
Author: Robin Zhang <[email protected]> Signed-off-by: oliver könig <[email protected]>
…TE fused MLP Co-authored-by: Mcore Bot <[email protected]>
…gradient existence assertion to fully_shard tests.
…case with decode-only graphs
… engine case with decode-only graphs" This reverts commit 4cf968c.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.