Unseen Hand

[jmchumbo]

Hello,

I hope this message finds you well. I am interested in implementing GenDexGrasp for a robot different from the ones used in your project. Could you please provide me with a step-by-step procedure to follow? It seems that some modifications to the code are also required.

Thank you in advance for your assistance. I would like also to take this opportunity to congratulate you on the amazing project you have accomplished.

Best regards,
João Chumbinho

[Xiaoyao-Li]

Sorry for the late reply and thanks for your interest.

The first thing you should do is to provide your robot's URDF or other format descriptor in the ./data/urdf folder.

However, there are still some things you need to modify in the class HandModel at ./utils_model/HandModel.py, and class CMapAdam at ./utils_model/CMapAdam. In class HandModel, you can comment out all variables requiring the contact_{robot_name}..json file, which is only needed for the grasping dataset synthesis. In class CMapAdam, you should add a specific robotic hand center position and palm normal at line73.

I am happy to assist further if you have any other issues. Just let me know.

[jmchumbo]

Hello @Xiaoyao-Li,

I hope you're doing well. First of all, thanks once more. I have a few questions regarding the Class CmMapAdam in the GenDexGrasp project:

• When you mention that the palm normal should be added, could you please clarify which reference frame should be used for this addition?

• Additionally, when the palm normal is added, there is a scale factor involved (attached is an example with a scale factor of 1.2). Could you provide some guidance on determining the appropriate value for this scale factor?

Lastly, would you recommend incorporating the new unseen hand in Phase 1: Data Collection, using DFC? Or do you believe that the results will be similar regardless?

Thank you once again for your time and assistance.

Best regards,
João Chumbinho

[Xiaoyao-Li]

Hi @jmchumbo ,

• When adding the palm normal, please ensure that the robotic hand is set to he canonical pose. In the canonical pose, the global rotation should be set to identity, and all joint poses should be set to 0.
• Regarding the scale factor, it's recommended to set it between [1.0, 1.5] based on empirical observations. This factor influences the initial position of the robotic hand before optimization.
• If you're not concerned about inference time, you may consider using the DFC method to synthesize more diverse and stable grasping poses. Alternatively, if you prefer to run the inference pipeline of GenDexGrasp, the results should be similar due to its generalizability.

Hope this information proves helpful in addressing the issue.

Best regards,
Puhao.

[jmchumbo]

Hello once more @Xiaoyao-Li!

I greatly appreciate your kind assistance in addressing my previous inquiries. Now that I have successfully implemented the changes, I would like to proceed with conducting a stability test for the new robot. Could you please provide guidance on the necessary steps to perform this test?

Thank you once again for your support, and my apologies for the multitude of questions.

Best regards,
João Chumbinho

[jmchumbo]

Hello,

During the process of constructing the "grasp_test_force_robotname.py" file, where "robotname" represents the name of the new robot for which I wish to conduct a stability test, I studied similar files created for robots like barrett, shadowhand, etc. in order to discern the underlying pattern. Nevertheless, I encountered a few questions, and I would greatly appreciate your assistance in addressing them:

1. In the function "_set_init_pose(self)" found in files such as "grasp_test_force_barrett.py," you initialize the values of self.closure_q to 0.1 and 0.3. Could you please explain how these values are obtained, and under what circumstances should the matrix values be updated?

2. Regarding "_set_normal_force_pose(self)," I'm unsure about distinguishing between the values for "opt_q_global" and "opt_q_angle." Could you provide some clarification on this matter?

3. In the functions "q_transfer_o2s" and "get_q_as_opt" how are the values for "sim_joint_angle" and "opt_q_angle" chosen, respectively?

I apologize for the barrage of questions, but your insights would be invaluable in my progress.

Thank you for your time and assistance.

Best regards,
João Chumbinho

[Xiaoyao-Li]

Apologies for overlooking some previous issues due to an oversight. I hope you have already resolved them.
Regarding the two problems you raised, they were indeed caused by unclear variable naming and an incompletely cleaned repository. Sorry for the confusion.

1. As I commented out the self._set_init_pose() call here, I did not actually invoke this function. It was just used initially to verify the correctness of the simulator pose control.
2. q_global maintains the global translation (3D representation) and rotation (6D representation) values of the robot hand. q_angle maintains the rotation angles of the non-fixed joints in the robot hand.
3. Due to differences in the joint order definitions between HandModel.py and the Isaac Gym simulator, I maintain two types of variables named sim_* and opt_* for usage with the HandModel.py and Isaac Gym robot joint representations, respectively.

Please let me know if this helps explain and improve the original message.

[jmchumbo]

Hello, Thank you once again for all the support provided.

Regarding the third point in your response, in your opinion, what would be the best approach to differentiate the joint order definitions between HandModel.py and the Isaac Gym simulator?

Thank you once again for all the support.