6DoFHPE

Paper titled: Real-time 6DoF Full-Range Markerless Head Pose Estimation here

Results visualization

Gif: 6DoF-HPE (left side) and one by one (right side)

• Video: You can see a demo video here and here, which displays a comparison between our work with other works.

• Fig. Snapshots of 7 different views from one sequence in the CMU Panoptic val-set dataset.

Our results

• Trained and tested on CMU Panoptic datasets.

Method	Retrain?	Yaw	Pitch	Roll	MAE
WHENet	No	37.96	22.7	16.54	25.73
HopeNet	No	20.40	17.47	13.40	17.09
FSA-Net	No	17.52	16.31	13.02	15.62
Img2Pose	No	12.99	16.69	15.64	15.11
DirectMHP	Yes	5.86	8.25	7.25	7.12
DirectMHP	No	5.75	8.01	6.96	6.91
6DRepNet	Yes	5.20	7.22	6.00	6.14
6DoF-HPE (ours)	Yes	5.13	6.99	5.77	5.96
WHENet	No	29.87	19.88	14.66	21.47
DirectMHP	Yes	7.38	8.56	7.47	7.80
DirectMHP	No	7.32	8.54	7.35	7.74
6DRepNet	Yes	5.89	7.76	6.39	6.68
6DoF-HPE (ours)	Yes	5.83	7.63	6.35	6.60

• Trained on 300W-LP, and then test on AFLW2000 and BIWI.

Method	Retrain?	AFLW2000 Yaw	AFLW2000 Pitch	AFLW2000 Roll	AFLW2000 MAE	BIWI Yaw	BIWI Pitch	BIWI Roll	BIWI MAE
HopeNet	No	6.47	6.56	5.44	6.16	4.81	6.60	3.27	4.89
FSA-Net	No	4.50	6.08	4.64	5.07	4.27	4.96	2.76	4.00
WHENet	No	4.44	5.75	4.31	4.83	3.60	4.10	2.73	3.48
LSR	No	4.26	5.27	3.89	4.47	4.29	3.09	3.18	3.52
DirectMHP	No	2.99	5.35	3.77	4.04	3.57	5.47	4.02	4.35
6DRepNet	No	3.63	4.91	3.37	3.97	3.24	4.48	2.68	3.47
Img2Pose	No	3.43	5.03	3.28	3.91	4.57	3.55	3.24	3.79
DirectMHP	Yes	3.31	5.36	3.75	4.14	3.54	5.45	4.01	4.33
6DRepNet	Yes	3.50	4.81	3.47	3.93	3.79	4.53	2.89	3.74
6DoF-HPE (ours)	Yes	3.56	4.74	3.35	3.88	3.91	4.43	2.69	3.68

Datasets

• CMU Panoptic from here for the full range angles.

• 300W-LP, and AFLW2000 from here for the narrow range angles.

• BIWI from here for the narrow range angles.

Run demo codes

Download the pre-trained RepVGG model 'cmu.pth' for the full range angles or '300W_LP.pth' for narrow range angles from here and the pre-trained SSD model 'Head_detection_300x300.pb' from here and then save them in their certain directory (please see our code to know their paths).

To run demo_3DoF.py. You can run it directly.

demo_3DoF.py

But, to run demo_6DoF.py. You need to install ROS from here.

Maybe you need some time to learn how to install ROS and how it works.

After installing ROS. The roscore can be launched using the roscore executable:

roscore

Then run an RGB-D camera, for example, if it is realsense d435i.

roslaunch realsense2_camera rs_camera.launch filters:=pointcloud,colorizer align_depth:=true ordered_pc:=true

Then, run:

demo_6DoF.py

Training for rotational (yaw, pitch, and roll) components:

If you only need to change the pre-trained RepVGG model 'RepVGG-B1g4-train.pth' please see here and save it in the root directory.

python3 train.py

After training is done. Next step.

Deploy models for rotational (yaw, pitch, and roll) components:

For reparameterization, the trained models into inference models use the convert script.

python3 convert.py input-model.tar output-model.pth

After converting the training model into an inference model. Then, you can test your model.

Testing for rotational (yaw, pitch, and roll) components:

python3 test.py

Citing

@article{algabri2024real,
  title={Real-time 6DoF full-range markerless head pose estimation},
  author={Algabri, Redhwan and Shin, Hyunsoo and Lee, Sungon},
  journal={Expert Systems with Applications},
  volume={239},
  pages={122293},
  year={2024},
  publisher={Elsevier}
}