Search results

MOE uses more memory than dense model and is slower

Status: Open.

#166 In databricks/megablocks;

· samuelwheeler opened on Mar 3, 2025

bump version for PyTorch 2.6

Status: Open.

#165 In databricks/megablocks;

· ad8e opened on Feb 26, 2025

[rank5]: TypeError: SortOp.forward() takes from 2 to 3 positional arguments but 5 were given When running moe script

Status: Open.

#164 In databricks/megablocks;

· rtmadduri opened on Feb 8, 2025

Grouped GEMM execution not possible with HW

Status: Open.

#163 In databricks/megablocks;

· cassanof opened on Jan 3, 2025

CUDA Error When Running Single GPU Experiment

Status: Open.

#161 In databricks/megablocks;

· kevin3567 opened on Nov 14, 2024

Is current megablocks compatible with distributed optimizer in Megatron-LM?

Status: Open.

#160 In databricks/megablocks;

· Spico197 opened on Nov 11, 2024

amp_C undefined symbol after installing Megablocks

Status: Open.

#157 In databricks/megablocks;

· RachitBansal opened on Oct 11, 2024

Do you support the bias of mlp？

Status: Open.

#156 In databricks/megablocks;

· maobenz opened on Oct 9, 2024

what devices are supported?

Status: Open.

#155 In databricks/megablocks;

· Guodanding opened on Oct 8, 2024

Does it work with torch.compile?

Status: Open.

#153 In databricks/megablocks;

· Muennighoff opened on Sep 30, 2024

[Do not Merge] Add CI/CD Milestones

Status: Open.

#132 In databricks/megablocks;

· eitanturok opened on Aug 1, 2024

Routing

Status: Open.

#118 In databricks/megablocks;

· alexliap opened on Jun 27, 2024