Questions about importing Motion (SMPL-X) and trajectories for humanoid

[zoeyliu1999]

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: master

Habitat-Sim: master

Habitat is under active development, and we advise users to restrict themselves to stable releases. Are you using the latest release versions of Habitat-Lab and Habitat-Sim? Your question may already be addressed in the latest versions. We may also not be able to help with problems in earlier versions because they sometimes lack the more verbose logging needed for debugging.

Master branch contains 'bleeding edge' code and should be used at your own risk.

Docs and Tutorials

Did you read the docs? https://aihabitat.org/docs/habitat-lab/

Did you check out the tutorials? https://aihabitat.org/tutorial/2020/

Perhaps your question is answered there. If not, carry on!

❓ Questions and Help

Hello, thank you for your excellent work! I am currently working with the social_nav task. Would it be possible to import motions and trajectories for a humanoid character so it can walk according to my specifications?

1. Regarding motion importing, I noticed that there are two HumanoidControllers in examples/tutorials/humanoids_tutorial.ipynb. Would it be advisable to utilize HumanoidSeqPoseController for the social_nav task?
2. Regarding trajectory importing, would it be compatible with the habitat simulator? For instance, only one joint is specified in the skeleton but others are generated. Please inform me if I've overlooked something.

[aclegg3]

@xavierpuigf could you advise here?

[xavierpuigf]

Hi @zoeyliu1999! It is definitely possible to import motion trajectories from other models. The only thing you need to do is to run a motion transformation script. You can see here how to get a motion in SMPL-X format (for instance motion coming from a Motion Diffusion Model) and convert it so that it is compatible with Habitat:

https://github.com/facebookresearch/habitat-lab/tree/main/habitat-lab/habitat/articulated_agents/humanoids#sequentialposecontroller

If you just want to play the motion as is, you can do that by using the SequentialPoseController (see this). You can also use the HumanoidRearrangeController, but that will require you to decompose the motion into a forward direction and a rotation, so that the motion can be used to guide the character anywhere.

Let me know if you have more questions!

[zoeyliu1999]

Hi @zoeyliu1999! It is definitely possible to import motion trajectories from other models. The only thing you need to do is to run a motion transformation script. You can see here how to get a motion in SMPL-X format (for instance motion coming from a Motion Diffusion Model) and convert it so that it is compatible with Habitat:

https://github.com/facebookresearch/habitat-lab/tree/main/habitat-lab/habitat/articulated_agents/humanoids#sequentialposecontroller

If you just want to play the motion as is, you can do that by using the SequentialPoseController (see this). You can also use the HumanoidRearrangeController, but that will require you to decompose the motion into a forward direction and a rotation, so that the motion can be used to guide the character anywhere.

Let me know if you have more questions!

@xavierpuigf @aclegg3 thank you for the support!

[zoeyliu1999]

@xavierpuigf @aclegg3 Hi, apologies for reopening this issue after some time. I am curious whether the robot can interact with the humanoid using imported motions within the scene. I understand that Habitat has a collision-avoidance mechanism used in tasks like Rearrangement and Social Navigation. Can I expect the same functionality with custom-defined motions? Specifically, I am considering importing some real-world motions to see how the robot will react.