HSMR Female

[DHPark98]

Thank you for your great work!
I have a simple question.

From what I can tell by checking the configuration file below, the checkpoint used in the demo appears to be the one trained with male data:

[data_inputs/released_models/HSMR-ViTH-r1d1/.hydra/config.yaml]

( ... )
  models:
    body_models:
      skel_mix_hsmr:
        _target_: lib.body_models.skel_wrapper.SKELWrapper
        model_path: ${_pm_.inputs}/body_models/skel
        gender: male
        joint_regressor_extra: ${_pm_.inputs}/body_models/SMPL_to_J19.pkl
        joint_regressor_custom: ${_pm_.inputs}/body_models/J_regressor_SKEL_mix_MALE.pkl
      skel_hsmr:
        _target_: lib.body_models.skel_wrapper.SKELWrapper
        model_path: ${_pm_.inputs}/body_models/skel
        gender: male
        joint_regressor_extra: ${_pm_.inputs}/body_models/SMPL_to_J19.pkl
      smpl_hsmr:
        _target_: lib.body_models.smpl_wrapper.SMPLWrapper
        model_path: ${_pm_.inputs}/body_models/smpl
        gender: male
        num_body_joints: 23
        joint_regressor_extra: ${_pm_.inputs}/body_models/SMPL_to_J19.pkl
( ... )

How can I run the demo with female inputs?
Could you also provide a checkpoint trained with female data?

[IsshikiHugh]

Hi @DHPark98,

Unfortunately, we don't have a model for female SKEL model.
Previous SMPL methods only predict neutral model parameters in general. But there are no neutral model for SKEL currently.
So we label all the data with male model and so that the trained model can only predict SKEL parameters for male model only.

In case of female inputs, we still predict male parameters, which indeed can cause some problems. If you really need a female model output, you can consider to finetune a female model. Unfortunately, currently we don't have such plan.

Thanks.