Pi0 Inference in agilex cobot magic(mobile aloha)

[kcyoung98]

Thank you so much for sharing the excellent codebase!

I’m currently working on a new robot, the Agilex Cobot Magic, which is an another version of the Mobile Aloha. I’ve written an inference code based on the Aloha_real code, but I’ve noticed a significantly lower success rate compared to the ACT model (the base model for Mobile Aloha) when performing simple pick tasks.

I’ve plotted the action rollout for both ACT and Pi0:

It seems that Pi0 behaves more hurry up than ACT, resulting in jerkier motions and a reduced success rate. Do you have any ideas on what might be causing this issue?

[Michael-Equi]

Hi @kcyoung98, based on the description you provided and the plots it is not immediately evident to me what the issue is. The fact that the action chunks look sped up does suggest it might be an issue with the main observation/action loop frequency or how you are executing the action chunk. A couple questions I have:

1. Are you using this to control the base as well? I only see 14 dimensions and not the full 16 for the platform.
2. Which norm stats asset are you using?
3. Did you make any changes to code outside of the environment loop and device code?

[tangger2000]

Thank you so much for sharing the excellent codebase!

I’m currently working on a new robot, the Agilex Cobot Magic, which is an another version of the Mobile Aloha. I’ve written an inference code based on the Aloha_real code, but I’ve noticed a significantly lower success rate compared to the ACT model (the base model for Mobile Aloha) when performing simple pick tasks.

I’ve plotted the action rollout for both ACT and Pi0:

It seems that Pi0 behaves more hurry up than ACT, resulting in jerkier motions and a reduced success rate. Do you have any ideas on what might be causing this issue?

Can I get your email? I meet the same issue.

[gjw0506]

Hi, I am also working on the Agilex Cobot Magic, can I get your version of inference code?

[tangger2000]

+1

[kcyoung98]

@Michael-Equi Thank you for replaying.
Are you using this to control the base as well? I only see 14 dimensions and not the full 16 for the platform.
I didn't control the base. Base is fixed.
Which norm stats asset are you using?
Originally, I used pizero base model but after realizing agilex mobile aloha is different from trossent aloha, I used my own state from agilex mobile aloha.
Did you make any changes to code outside of the environment loop and device code?
I change the code of aloha_policy so that disable # Flip the joints. state = _joint_flip_mask() * state. and change _gripper_to_angular so that only gripper is normalized between 0 and 0.07 which is agilex piper arm.
My problem is that joint position seems wrong and pizero makes over-shot than ACT model.