tracking mixed up when instances too close

[ttunja]

If we focus on the two mice on the left (with blue marks) once they go over each other the tracking and marking gets mixed up. My question is: In the situation like this, where instances are obviously overlapping how to ensure that the right tracker ID remains assigned and the IDs dont get mixed up.

Any suggestions in the tracking configuration to be more strickt about this and take for example relative position in previous frames?

Second question is: Choice of the model and its configuration does not matter right? For this problem its only about configuring tracking correctly?

[talmo]

Hi @ttunja,

First off: great images! Looks like it'll be a terrific dataset.

Do you mind sharing which tracking parameters you've tried so far to get the results you're currently seeing?

Also, are you just tracking a single node on each animal?

Cheers,

Talmo

[ttunja]

Hi @talmo, thanks for the quick feedback.

here is my configuration:

And no, I hope I specified correctly that I am using 4 nodes not one for tracking. (because the 4 nodes for training were head, 2*x´body and tail)

[talmo]

That looks pretty good -- you could try using the "flow" method instead of simple, decreasing the frame window to ~5, and using "instance" for similarity.

Do you get a lot of merged skeletons when the animals are too close or do the poses mostly look ok?

[ttunja]

Thank you for the suggestions. So I played around a little bit and with the following config most of the conflicts when animals get too close get resolved. (I do have a subquestion: What do you suggest we set "Cull using IoU threshold?" the higher values were better for me)

Now with this conflicts get resolved but I get A LOT of empty frame predictions (as you can see in the frames underneath) now as a result of non of the tracks getting predicting for a long time, once it starts predicting back there are mixing ups.
How can one remedy and make sure that every frame MUST have always 4 predictions. Is this enforcable? In this case I suspect it has something to do with the modell too, not only the tracking. I posed a seperate question for this, because I think it will be interesting for the community in the future: #939

[roomrys]

Hi @ttunja,

If your instances overlap often and by a lot (i.e. positioned on top of each other), then a higher IOU threshold makes sense. The higher the IOU threshold, the less instances will disappear when they overlap.

While there is not way to ensure that SLEAP will predict a minimum number of instances, if you call sleap-track through the CLI, you can set the --peak-threshold parameter to a lower number (the default is 0.2). Predictions will be removed based on how confident SLEAP is that these predictions are correct. Using a lower --peak-threshold value will lower how confident SLEAP needs to be for the predictions to stay. Consequently, you will get more predictions, but they may not be as good as expected. Alternatively, you can train the model with more labeled frames which should also raise the number of predictions returned as confidence increases.

Hope this helps!

Thanks,
Liezl