Home
Here you can find some additional information of things that are not mentioned in the front-page README.md.
Some example fitting results on the HELEN database, created with the fit-model example app:
 |
 |
 |
 |
 |
 |
To use the Basel Face Model 2009 in eos, you can convert it to the format that eos uses with the Python script in share/scripts/convert-bfm2009-to-eos.py. It will read the PublicMM1/01_MorphableModel.mat file from the BFM2009 distribution and save an eos .bin model.
If you want to perform any serious fitting, a model_contours.json and edge_topology.json file for the BFM is required as well. These two files are offered without any guarantees and are possibly untested:
-
model_contours_bfm.json.txt (remove the .txt at the end, GitHub doesn't allow to upload .json files). This file is fairly easy to create as you just need to mark a number of vertices along the left and right contour that correspond to ibug landmarks.
-
bfm_edge_topology.json. This file is actually taken from https://github.com/waps101/3DMM_edges and just converted to json.
According to the authors, the Basel Face Model 2017 has the same topology as the BFM2009, so the ibug_to_bfm.txt file in the repository and the model_contours_bfm.json file linked above should be valid. The bfm_edge_topology.json may needs to be recomputed, as the BFM2017 does not contain the mouth interior (and the face12 variant of the model has much fewer vertices anyway).
In the fit-model app, the edge fitting is just used to fit the occluding face contour to the ibug contour landmarks. However, the edge fitting is very general, and can be used to fit to edges from an edge detector (e.g. Canny edges). In fact this is how it is used in the original paper.
To achieve this in eos, the fitting::find_occluding_edge_correspondences() function just has to be called with a list of edges, instead of the landmarks. For example like so:
Mat edge_image;
double canny_thresh = 150.0;
cv::Canny(image, edge_image, canny_thresh*0.4, canny_thresh);
vector<cv::Point> image_edges;
cv::findNonZero(edge_image, image_edges);
vector<Eigen::Vector2f> image_edges_; // Need to convert these points to Eigen::Vector2f
std::for_each(begin(image_edges), end(image_edges), [&image_edges_](auto&& p) { image_edges_.push_back({ p.x, p.y }); });
auto mesh = morphable_model.get_mean();
auto edge_correspondences = fitting::find_occluding_edge_correspondences(mesh, edge_topology, rendering_params, image_edges_);
This will return a list of edge correspondences that can then be added to the landmark correspondences in the subsequent fitting.