Add defence for DeepCompile w/o optimizer

[HollowMan6]

Similar to #7211

When the optimizer is not specified, the optimizer will be type DeepSpeedZeRoOffload instead of DeepSpeedZeroOptimizer_Stage3 (e.g. for ZeRO-3 pure inference), while DeepSpeedZeRoOffload doesn't have parameter_offload.

DeepSpeed/deepspeed/runtime/engine.py

Lines 1684 to 1707 in 56005d2

	if isinstance(optimizer, DummyOptim):
	log_dist("Creating ZeRO Offload", ranks=[0])
	zero_param_parallel_group = groups._get_zero_param_intra_parallel_group()
	if self.zero_hpz_partition_size() > 1 and zero_param_parallel_group is None:
	self._set_zero_group_parallelism()
	zero_param_parallel_group = groups._get_zero_param_intra_parallel_group()
	optimizer = DeepSpeedZeRoOffload(
	self.module,
	timers=timers,
	ds_config=self.config,
	overlap_comm=self.zero_overlap_comm(),
	prefetch_bucket_size=self.zero_prefetch_bucket_size(),
	max_reuse_distance=self.zero_max_reuse_distance(),
	max_live_parameters=self.zero_max_live_parameters(),
	param_persistence_threshold=self.zero_param_persistence_threshold(),
	model_persistence_threshold=self.zero_model_persistence_threshold(),
	offload_param_config=self.zero_offload_param(),
	mpu=self.mpu,
	zero_param_parallel_group=zero_param_parallel_group,
	zero_quantized_weights=self.zero_quantized_weights(),
	zero_quantized_nontrainable_weights=self.zero_quantized_nontrainable_weights(),
	zero_module_granularity_threshold=self.zero_module_granularity_threshold(),
	log_trace_cache_warnings=self.zero_log_trace_cache_warnings(),
	)

  File "deepspeed/runtime/engine.py", line 3919, in compile
    backend = init_z3(self, backend, compile_config, compile_kwargs, schedule)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "deepspeed/compile/init_z3.py", line 36, in init_z3
    optimizer.parameter_offload._remove_module_hooks()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'DeepSpeedZeRoOffload' object has no attribute 'parameter_offload'

[tohtana]

@HollowMan6 Great catch, we appreciate your contribution!

[loadams]

Thanks @HollowMan6 - we will prioritize merging this so we can push out a patch release for better DeepCompile support.

[HollowMan6]

Thanks for the quick review! After this PR, together with #7226, #7224 are merged, it should build (compile) fine for AMD machines as well. But I encountered 2 separate issues when I used DeepCompile together with OpenRLHF. Will open separate issues explaining the problems I got. Opened at #7229 and #7228