Unrecognized argument policy.path. How to load a pretrained model?

[milong26]

When I run this command:

python lerobot/scripts/control_robot.py --robot.type so100 --control.type record --control.fps 30 --control.single_task "Grasp a yellow tape and put it to yellow square." --control.repo_id a_cam_1/result --control.tags '["tutorial"]' --control.warmup_time_s 5 --control.episode_time_s 30 --control.reset_time_s 10 --control.m_episodes 1 --control.push_to_hub false --control.policy,path output/checkpoints/last/pretrained_model

I got:

usage: control_robot.py [-h] [--config_path str] [--robot str] [--robot.type {aloha,koch,koch_bimanual,moss,so101,so100,stretch,lekiwi}] [--robot.gripper_open_degree str]
                        [--robot.max_relative_target str] [--robot.ip str] [--robot.port str] [--robot.video_port str] [--robot.cameras str] [--robot.calibration_dir str]
                        [--robot.leader_arms str] [--robot.follower_arms str] [--robot.teleop_keys str] [--robot.mock str] [--control str]
                        [--control.type {calibrate,teleoperate,record,replay,remote_robot}] [--control.arms str] [--control.teleop_time_s str] [--control.single_task str]
                        [--policy str] [--control.policy.type {act,diffusion,pi0,tdmpc,vqbet,pi0fast}] [--control.policy.replace_final_stride_with_dilation str]
                        [--control.policy.pre_norm str] [--control.policy.dim_model str] [--control.policy.n_heads str] [--control.policy.dim_feedforward str]
                        [--control.policy.feedforward_activation str] [--control.policy.n_encoder_layers str] [--control.policy.n_decoder_layers str]
                        [--control.policy.use_vae str] [--control.policy.n_vae_encoder_layers str] [--control.policy.temporal_ensemble_coeff str]
                        [--control.policy.kl_weight str] [--control.policy.optimizer_lr_backbone str] [--control.policy.drop_n_last_frames str]
                        [--control.policy.use_separate_rgb_encoder_per_camera str] [--control.policy.down_dims str] [--control.policy.kernel_size str]
                        [--control.policy.n_groups str] [--control.policy.diffusion_step_embed_dim str] [--control.policy.use_film_scale_modulation str]
                        [--control.policy.noise_scheduler_type str] [--control.policy.num_train_timesteps str] [--control.policy.beta_schedule str]
                        [--control.policy.beta_start str] [--control.policy.beta_end str] [--control.policy.prediction_type str] [--control.policy.clip_sample str]
                        [--control.policy.clip_sample_range str] [--control.policy.num_inference_steps str] [--control.policy.do_mask_loss_for_padding str]
                        [--control.policy.scheduler_name str] [--control.policy.num_steps str] [--control.policy.attention_implementation str]
                        [--control.policy.train_expert_only str] [--control.policy.train_state_proj str] [--control.policy.n_action_repeats str] [--control.policy.horizon str]
                        [--control.policy.image_encoder_hidden_dim str] [--control.policy.state_encoder_hidden_dim str] [--control.policy.latent_dim str]
                        [--control.policy.q_ensemble_size str] [--control.policy.mlp_dim str] [--control.policy.discount str] [--control.policy.use_mpc str]
                        [--control.policy.cem_iterations str] [--control.policy.max_std str] [--control.policy.min_std str] [--control.policy.n_gaussian_samples str]
                        [--control.policy.n_pi_samples str] [--control.policy.uncertainty_regularizer_coeff str] [--control.policy.n_elites str]
                        [--control.policy.elite_weighting_temperature str] [--control.policy.gaussian_mean_momentum str] [--control.policy.max_random_shift_ratio str]
                        [--control.policy.reward_coeff str] [--control.policy.expectile_weight str] [--control.policy.value_coeff str] [--control.policy.consistency_coeff str]
                        [--control.policy.advantage_scaling str] [--control.policy.pi_coeff str] [--control.policy.temporal_decay_coeff str]
                        [--control.policy.target_model_momentum str] [--control.policy.n_action_pred_token str] [--control.policy.action_chunk_size str]
                        [--control.policy.vision_backbone str] [--control.policy.crop_shape str] [--control.policy.crop_is_random str]
                        [--control.policy.pretrained_backbone_weights str] [--control.policy.use_group_norm str] [--control.policy.spatial_softmax_num_keypoints str]
                        [--control.policy.n_vqvae_training_steps str] [--control.policy.vqvae_n_embed str] [--control.policy.vqvae_embedding_dim str]
                        [--control.policy.vqvae_enc_hidden_dim str] [--control.policy.gpt_block_size str] [--control.policy.gpt_input_dim str]
                        [--control.policy.gpt_output_dim str] [--control.policy.gpt_n_layer str] [--control.policy.gpt_n_head str] [--control.policy.gpt_hidden_dim str]
                        [--control.policy.dropout str] [--control.policy.mlp_hidden_dim str] [--control.policy.offset_loss_weight str]
                        [--control.policy.primary_code_loss_weight str] [--control.policy.secondary_code_loss_weight str] [--control.policy.bet_softmax_temperature str]
                        [--control.policy.sequentially_select str] [--control.policy.optimizer_vqvae_lr str] [--control.policy.optimizer_vqvae_weight_decay str]
                        [--control.policy.n_obs_steps str] [--control.policy.normalization_mapping str] [--control.policy.input_features str]
                        [--control.policy.output_features str] [--control.policy.device str] [--control.policy.use_amp str] [--control.policy.chunk_size str]
                        [--control.policy.n_action_steps str] [--control.policy.max_state_dim str] [--control.policy.max_action_dim str]
                        [--control.policy.resize_imgs_with_padding str] [--control.policy.interpolate_like_pi str] [--control.policy.empty_cameras str]
                        [--control.policy.adapt_to_pi_aloha str] [--control.policy.use_delta_joint_actions_aloha str] [--control.policy.tokenizer_max_length str]
                        [--control.policy.proj_width str] [--control.policy.max_decoding_steps str] [--control.policy.fast_skip_tokens str]
                        [--control.policy.max_input_seq_len str] [--control.policy.use_cache str] [--control.policy.freeze_vision_encoder str]
                        [--control.policy.freeze_lm_head str] [--control.policy.optimizer_lr str] [--control.policy.optimizer_betas str] [--control.policy.optimizer_eps str]
                        [--control.policy.optimizer_weight_decay str] [--control.policy.scheduler_warmup_steps str] [--control.policy.scheduler_decay_steps str]
                        [--control.policy.scheduler_decay_lr str] [--control.policy.checkpoint_path str] [--control.policy.padding_side str] [--control.policy.precision str]
                        [--control.policy.grad_clip_norm str] [--control.policy.relaxed_action_decoding str] [--control.warmup_time_s str] [--control.episode_time_s str]
                        [--control.reset_time_s str] [--control.num_episodes str] [--control.video str] [--control.push_to_hub str] [--control.private str] [--control.tags str]
                        [--control.num_image_writer_processes str] [--control.num_image_writer_threads_per_camera str] [--control.resume str] [--control.repo_id str]
                        [--control.episode str] [--control.root str] [--control.fps str] [--control.play_sounds str] [--control.log_interval str] [--control.display_data str]
                        [--control.viewer_ip str] [--control.viewer_port str]
control_robot.py: error: unrecognized arguments: --control.policy.path output/checkpoints/last/pretrained_model

lerobot version:42bf1e8b9df3d38b3898b24e46b5e0386910c466

[milong26]

I tried this:

--robot.type so100 --control.type record --control.fps 30 --control.single_task "Grasp a yellow tape and put it to yellow square." --control.repo_id a_cam_1/eval_result --control.tags '["tutorial"]' --control.warmup_time_s 5 --control.episode_time_s 30 --control.reset_time_s 10 --control.num_episodes 1 --control.push_to_hub false --control.policy.type act --control.policy.pretrained_path outputs/train/act_so100_test/checkpoints/last/pretrained_model

When I just use --control.policy.type act it's ok but no pretrained model path in make_policy() function.

    if cfg.pretrained_path:
        print("path=",cfg.pretrained_path)
        # Load a pretrained policy and override the config if needed (for example, if there are inference-time
        # hyperparameters that we want to vary).
        kwargs["pretrained_name_or_path"] = cfg.pretrained_path
        policy = policy_cls.from_pretrained(**kwargs)

But when I add --control.policy.pretrained_path outputs/train/act_so100_test/checkpoints/last/pretrained_model It raises error in lerobot/config/parser.py:

else:
                    cfg = draccus.parse(config_class=argtype, config_path=config_path, args=cli_args)

reporting this:
Exception has occurred: SystemExit
2
File "/xx/lerobot/lerobot/configs/parser.py", line 226, in wrapper_inner
cfg = draccus.parse(config_class=argtype, config_path=config_path, args=cli_args)
File "/xx/lerobot/lerobot/scripts/control_robot.py", line 437, in
control_robot()
SystemExit: 2

[milong26]

I can't register cfg,policy.pretrained_path=/outputs/train/act_so100_test/checkpoints/last/pretrained_model in control_robot.py How to do it?

[milong26]

1. In lerobot/configs/policies.py
   find __post_init__

    # add this line before def __post_init__
    pretrained_path: str = None

    def __post_init__(self):
        # delete this line
        # self.pretrained_path = None

2. command like:
   I'm using act policy so type=act.(I don't know why I can't use = to write arguments)
   repo_id name must start with eval
   pretrained_path: I just use copy path to get the path. In pretrained.py you should watch these lines if your program tried to get model from hugging face:

        if os.path.isdir(model_id):
            print("Loading weights from local directory")
            model_file = os.path.join(model_id, SAFETENSORS_SINGLE_FILE)
            policy = cls._load_as_safetensor(instance, model_file, config.device, strict)

--control.policy.type act \
--control.policy.pretrained_path /xx/lerobot/output/checkpoints/last/pretrained_model
--control.repo_id a_cam_1/eval_result8

after this I can evaluate and record (But my model works bad) I'm bad at code so if my work is right can anyone merge it to main and fix tutorials I really struggled at it :P

Also i don't know whether my logic of code is right, although it runs successfullt :P