Feat: context parallel v2.0 #6176

Job	Run time
run-tests (latest, test_checkpoint_step)	4m 11s
run-tests (latest, test_big_modeling)	2m 47s
run-tests (latest, test_prod)	1m 55s
run-tests (latest, test_deepspeed)	2m 43s
run-tests (latest, test_rest)	10m 34s
run-tests (latest, test_example_differences)	2m 47s
run-tests (minimum, test_rest)	11m 43s
run-tests (latest, test_checkpoint_epoch)	4m 2s
run-tests (minimum, test_example_differences)	3m 43s
run-tests (minimum, test_cli)	4m 48s
run-tests (minimum, test_fsdp)	3m 45s
run-tests (latest, test_core)	3m 52s
run-tests (minimum, test_big_modeling)	3m 44s
run-tests (latest, test_fsdp)	2m 44s
run-tests (latest, test_cli)	3m 13s
run-tests (minimum, test_core)	4m 50s
run-tests (minimum, test_checkpoint_epoch)	5m 53s
run-tests (minimum, test_deepspeed)	3m 52s
run-tests (minimum, test_checkpoint_step)	5m 8s
run-tests (minimum, test_prod)	2m 58s
	1h 29m 12s

Provide feedback