Parallelism config + TP + HSDP + BYODM (Bring Your Own Device Mesh)#3682

Merged

S1ro1 merged 76 commits intohuggingface:mainfrom

SalmanMohammadi:device_mesh_parallelism_config

Jul 30, 2025

Commits on Jun 24, 2025

Commits on Jul 4, 2025

Commits on Jul 7, 2025

remove import
S1ro1
committed

Commits on Jul 15, 2025

adding support for pre-configured device mesh
SalmanMohammadi
committed

Commits on Jul 16, 2025

Commits on Jul 17, 2025

tests
SalmanMohammadi
committed

Commits on Jul 20, 2025

FSDP dim names
SalmanMohammadi
committed

Commits on Jul 21, 2025

adding example
Salman Mohammadi
committed
WIP
Salman Mohammadi
committed
fixing HSDP
Salman Mohammadi
committed
Feat: add back old options
S1ro1
committed
working example
Salman Mohammadi
committed
debugging
Salman Mohammadi
committed
adding parallelism config to partialstate
Salman Mohammadi
committed
Feat: revert ddp changes
S1ro1
committed
Revert DDP
S1ro1
committed

Commits on Jul 22, 2025

Feat: (untested) update mesh dims and some minor tweaks
S1ro1
committed
adding dp_cp dims
Salman Mohammadi
committed
updating comments
Salman Mohammadi
committed
WIP
Salman Mohammadi
committed
wip 2
Salman Mohammadi
committed
reverting
Salman Mohammadi
committed
storing state in accelerator rather than acceleratorstate
Salman Mohammadi
committed
Fix: minor tweaks
S1ro1
committed
wip example update
Salman Mohammadi
committed
merging
Salman Mohammadi
committed
Fixes for non-fsdp2 case
S1ro1
committed
Feat: ensure ddp/tp only works
S1ro1
committed

Commits on Jul 23, 2025

updating example
Salman Mohammadi
committed
updating example
Salman Mohammadi
committed
updating examples, fixing state
Salman Mohammadi
committed
fixed state
Salman Mohammadi
committed
comments
Salman Mohammadi
committed
fixing partial state check
Salman Mohammadi
committed
linting
Salman Mohammadi
committed
comments
Salman Mohammadi
committed
removing fn
Salman Mohammadi
committed
merging
Salman Mohammadi
committed
WIP: fix tp
S1ro1
committed

Commits on Jul 24, 2025

comments
Salman Mohammadi
committed
removing return
Salman Mohammadi
committed
reverting upcast
Salman Mohammadi
committed

Commits on Jul 25, 2025

Commits on Jul 26, 2025

Commits on Jul 28, 2025

Feat: working examples
S1ro1
committed

Commits on Jul 30, 2025

Fix: remove implicit_replication and style
S1ro1
committed
Fix: remove optional
S1ro1
committed
add guard on parallelism_config.tp_enabled
winglian
committed
fix import
winglian
committed
fixing empty parallelism_config
Salman Mohammadi
committed
fix import path for test patch
winglian
committed
fixing patch
Salman Mohammadi
committed
merging
Salman Mohammadi
committed