Error in inference with topdown models on Apple Silicon

[phragmosis]

Using an M1 Mac on Mac OS Monterey 12.6.1

When tracking a single fish that often disappears behind obstacles within the video, the tracking crashes and an error notification appears stating that the terminal will contain more information. On emailing Talmo, his advice was:

"I think this is actually a problem specific to M1 Macs. We have to use a newer version of TensorFlow on those, and it looks like it introduces a bug with top-down models in rare situations where there are no animals detect for multiple frames in a row -- I think this happens in your case because the fish sometimes is occluded by the other objects."

Text of terminal output following error: https://www.dropbox.com/home/Work/Bristol/Projects/Motion%20illusions/Experiments/Tracking/Troubleshooting_7_11_22?preview=Terminal+output+following+error.rtf

[talmo]

We're still investigating this, but just to link things up -- this appears to be the same issue reported in #1151.

We'll update this thread with progress as we troubleshoot this.

[phragmosis]

Hi Talmo, is there any update on this by any chance? We're currently debating whether to buy a Windows desktop with a big graphics card to run the tracking.... but it would save a lot of money if the tracking could be run on our current M1 Mac!

[talmo]

Hi @phragmosis,

We're actively working on it! We're updating TensorFlow and Python as part of the 1.3.0 release, as well as providing pip and conda packages for Apple Silicon, so we're hoping this bug will just get resolved as part of the upgrades, but we'll test it and try for a fix if not.

Will let you know, but expected timescale for that is 1-3 weeks.

That said, not a bad idea to get a Linux or Windows box just in case? Usually the most expensive component is the GPU, but RTX 3060 is super cheap and should work great for SLEAP.

[phragmosis]

Hi Talmo, hope all is well. Are there any further updates (/projected timelines) on whether it's possible to use SLEAP on an M1/M2 Mac yet?

Cheers!
Rob

[roomrys]

Hi @phragmosis,

While we have updated dependency ranges and added both conda packages and unit testing for Macs in the last release 1.3.1, there are still some Mac-specific bugs (see this thread).

So, we've made some progress on the Apple Silicon-front, but the issue described here would require an update/bug-fix to a dependency... or migration to a library that replaces the aforementioned dependency.

Thanks,
Liezl

[talmo]

Apologies for the delay, but we finally do have an update that should solve this issue!!

The problem was that the Tensorflow backend for Apple Silicon works differently than other platform and would break when we there are 0 detected centroids in topdown models. This is handled gracefully in every other platform, so it required a sizeable refactor of how we have our models set up.

We're working on testing it out now across the other platforms to make sure we don't break more things than we fix, but if all goes well we should have a fix by the next release.

We'll post updates here as we have them.