Search results

[BUG] aux_loss and z_loss is incorrect when use calculate_per_token_loss and cp

Status: Open.

#1652 In NVIDIA/Megatron-LM;

· Suparjie opened on Jun 26, 2025

[BUG] Missing roll operation for position_ids in MultiTokenPredictionBlock

Status: Open.

#1650 In NVIDIA/Megatron-LM;

· iansheng opened on Jun 26, 2025

[ENHANCEMENT] Increase lru_cache maxsize in YarnRotaryEmbedding to avoid cache misses with deep models

Status: Open.

#1649 In NVIDIA/Megatron-LM;

· HuangHunag-MT opened on Jun 25, 2025

[QUESTION]The current Megatron MTP implementation in the rotary embedding part may have problems.

Status: Open.

#1648 In NVIDIA/Megatron-LM;

· youth123 opened on Jun 24, 2025

[ENHANCEMENT] Supporting Peri-LN architecture (Gemma2 & 3, Olmo2 architecture)

Status: Open.

#1644 In NVIDIA/Megatron-LM;

· penpaperkeycode opened on Jun 23, 2025

[BUG] Megatron stable has a bug with router-dype + FSDP with Pipeline-parallelism leads to assertion error.

Status: Open.

#1643 In NVIDIA/Megatron-LM;

· Skylion007 opened on Jun 22, 2025

[BUG]:ImportError: cannot import name 'check_set_window_size' from 'transformer_engine.pytorch.attention'

Status: Open.

#1642 In NVIDIA/Megatron-LM;

· njw1123 opened on Jun 21, 2025

[QUESTION] --use-distributed-optimizer and --data-parallel-sharding-strategy

Status: Open.

#1641 In NVIDIA/Megatron-LM;

· bugm opened on Jun 20, 2025

[QUESTION] Dataset construction behavior for variable-length documents

Status: Open.

#1639 In NVIDIA/Megatron-LM;

· Abstractjkc opened on Jun 19, 2025

[QUESTION] Setting for testing Nemotron-H throughput

Status: Open.

#1638 In NVIDIA/Megatron-LM;

· iamtonymwt opened on Jun 18, 2025

[BUG] GPT2BPETokenizer (and possibly others) missing decode and offsets methods

Status: Open.

#1633 In NVIDIA/Megatron-LM;

· jzzhuang opened on Jun 13, 2025

[BUG] when using moe parallel folding feature and set etp > 1 && ep == 1, the grad sync is incorrect and the loss curve is bad

Status: Open.

#1628 In NVIDIA/Megatron-LM;

· Louis-J opened on Jun 13, 2025