How to train on train.slp and validate on val.slp?

[tctco]

Hi! I'm trying to reproduce some of the results in your paper SLEAP: A deep learning system for multi-animal pose tracking. The datasets provided in the paper split data into train.slp, val.slp, and test.slp. How should I train, validate, and test on the datasets respectively?

BTW, could you please share the training configurations that generate results provided in the supplementary materials so that I can reproduce the results on my own machine? I found some configs in this OSF|SLEAP repo but not all of them :(

Thanks!

[roomrys]

Hi @tctco,

The parameters are included in the model folder (on the OSF home: sleap-data/datasets/"experiment"/models/"model_used"). You can download this folder and use this same model for training/inference on you local computer. In the Training Pipeline on the SLEAP GUI, make sure the "Training/Inference Pipeline Type" matches the one used in the model (i.e. single-instance, top-down, or bottom-up). Then, in the subsequent Model(s) tab, there is a dropdown where you can located an already existing model to load - you will need to locate the training_config.json file which will be inside the models folder that you've downloaded from OSF.

Fig 1: Dropdown in Training Pipeline to select an already existing model.

Inside the training_config.json, the data to be used for training, validation, and testing is specified as a path to the train.pkg.slp, val.pkg.slp, and test.pkg.slp respectively. Thus, if you load the training_config.json (and have all the aforementioned files downloaded), everything should be taken care of for you. Note, you may want to follow the same directory structure as on the OSF repo as we use relative paths to find these datasets.

Let us know if this helps!

Thanks,
Liezl

[tctco]

Thanks!

[tctco]

I am currently using sleap to test on my own dataset. However, it seems that sometimes the prediction erroneously shifts from the ground truth using the top-down method. I wonder if it is a bug or if it is because the detected bounding box is not accurate enough.

[talmo]

Might this be related to the video compression? See (this FAQ item](https://sleap.ai/help.html#i-get-strange-results-where-the-poses-appear-to-be-correct-but-shifted-relative-to-the-image).

[tctco]

Seems not the case. This image is imported from a coco-style dataset. Part of the images in the dataset has similar issues while the others are okay.

[roomrys]

Hi @tctco,

Just wondering if you are on the latest version of SLEAP - v1.2.8? This looks similar to an issue we recently resolved in #596. Do the images have different dimensions from each other?

If this is not the case, can you open an issue here describing the steps taken to reproduce the bug? I am guessing you just ran inference, but any details would be appreciated. For troubleshooting purposes, would it be possible to share your dataset with us at [email protected]? The bounding box is drawn after the nodes are plotted so this is likely a visualization bug if it isn't related to the video compression.

Thanks,
Liezl

[tctco]

Updated to the latest version and the problem is resolved. Thanks!