Messed-up skeletons after running inference

[heiley8181]

Hello,

I recently encountered an issue after running inference. I have a model that performed well in the past but recently I am encountering an error message after running inference.

2024-09-18 13:23:27.063190: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -58 } dim { size: -59 } dim { size: -60 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA GeForce RTX 3060" frequency: 1777 num_cores: 28 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 2359296 shared_memory_size_per_multiprocessor: 102400 memory_size: 10064232448 bandwidth: 360048000 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -61 } dim { size: -62 } dim { size: 1 } } }
2024-09-18 13:23:27.064526: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_UINT8 } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_UINT8 shape { dim { size: 4 } dim { size: 1080 } dim { size: 1920 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -3 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -3 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 2112 num_cores: 24 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 2097152 l3_cache_size: 31457280 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -3 } dim { size: -74 } dim { size: -75 } dim { size: 1 } } }
2024-09-18 13:23:27.069717: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -159 } dim { size: -160 } dim { size: -161 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -8 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -8 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA GeForce RTX 3060" frequency: 1777 num_cores: 28 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 2359296 shared_memory_size_per_multiprocessor: 102400 memory_size: 10064232448 bandwidth: 360048000 } outputs { dtype: DT_FLOAT shape { dim { size: -8 } dim { size: -162 } dim { size: -163 } dim { size: 1 } } }
2024-09-18 13:23:27.665339: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8201

I have never seen this error message before. The inference successfully runs through the whole video (around 9000 frames each), but for every frame, the skeletons are messed up. I am attaching a screenshot of the messed up skeletons.

Could you help me fix this issue? Thank you so much in advance!

-Haeri

[github-actions[bot]]

Thank you for starting a new discussion!

We appreciate your input and will review it soon.

Warning

A friendly reminder that this is a public forum. Please be cautious when clicking links, downloading files, or running scripts posted by others.

• Always verify the credibility of links and code.
• Avoid running scripts or installing files from untrusted sources.
• If you're unsure, ask for clarification before proceeding.

Stay safe and happy SLEAPing!

Best regards,
The Team

[gitttt-1234]

Hi @heiley8181,

Could you please let me know how you installed SLEAP? So the issue is the keypoints are predicted right and only the node names are mixed-up? Could you please share how your skeleton looks like in the training_config.json?

Thanks,

Divya

[heiley8181]

Hi @gitttt-1234 ,

I recently updated sleap using pip, so I uninstalled sleap and reinstalled using mamba. I can share the training_config.json file for the centroid and centered instance. Could you possibly provide me with a google form to uploade the files?

Thanks,
Haeri

[gitttt-1234]

Thank you, I just wanted to confirm if you installed it from source or not.
Here's the link to upload your files.

Thanks,

Divya

[heiley8181]

Hi @gitttt-1234 ,

Thank you for the link. I have uploaded the training files for centroid and centered instance. Let me know if you need additional files for inspection.

[gitttt-1234]

Hi @heiley8181!

Thank you for sharing the data and apologies for the delay. We recently added the functionality to handle ser/deser of skeletons internally (Related PRs: #1961 , #1970 ), instead of relying on external dependencies. This is now available in our develop branch and could be used by installing sleap from source:

1. git clone https://github.com/talmolab/sleap && cd sleap
2. conda env create -f environment.yml -n sleap_dev
   Ref: conda from source

I modified the skeleton dict in your training configs. You can replace your current training_config.json with the ones here and then run inference on the test file.
Let us know if this works!

Thanks,

Divya