INVALID_ARGUMENT: Conv2DSlowBackpropInput: Size of out_backprop doesn't match computed Error During Training #2039
Comments
|
Hi, Could you send us your training package so we can troubleshoot? You can upload it here. Thanks! Elizabeth |
|
Dear Elizabeth,
Thank you for reaching out. I have uploaded our training package, let me
know if other files are needed.
Thank you again,
Thomas
Il giorno mar 10 dic 2024 alle ore 22:39 Elizabeth ***@***.***>
ha scritto:
… Hi,
Could you send us your training package so we can troubleshoot? You can
upload it here
<https://docs.google.com/forms/d/e/1FAIpQLSfWBwcuZH8lxGyFabMAODLjSUE4YrSCYbrrnYvXCpgBqZpcJA/viewform>
.
Thanks!
Elizabeth
—
Reply to this email directly, view it on GitHub
<#2039 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3C4KU7C6C44NZTC4K6MUFT2E5NQJAVCNFSM6AAAAABTIN7RQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZSHE2DKOJYGE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Hi @Thomasbush9, Apologies for the delay! We're looking into this issue, Could you please let me know how you installed SLEAP? (We also have an updated stable version: https://github.com/talmolab/sleap/tree/v1.3.4) It would be great if you could also share your Thanks, Divya Update: I tried installing the same SLEAP version as yours and tried training with your data. I couldn't replicate the error. Could you try updating SLEAP to 1.3.4? |
|
Hi Divya,
Sorry for the delay and thank you for the help!
We have installed SLEAP through pip install. I have attached to this email
the training_config.json file.
Thanks again for your help,
Thomas
Il giorno mar 17 dic 2024 alle ore 00:20 DivyaSesh ***@***.***>
ha scritto:
… Hi @Thomasbush9 <https://github.com/Thomasbush9>,
Apologies for the delay! We're looking into this issue, Could you please
let me know how you installed SLEAP? It would be great if you could also
share your training_config.json with us to debug better?
Thanks,
Divya
—
Reply to this email directly, view it on GitHub
<#2039 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3C4KU4ZKUJTFOBDXHQRTLT2F5N4TAVCNFSM6AAAAABTIN7RQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBXGEYDCMZVHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hi @Thomasbush9, Sorry I don't see the attachment in my email. Could you please upload it here? Could you also try to uodate SLEAP to v1.3.4 and check if it's working? Thanks, Divya |
Bug description
During the training of a model: single instance, unet (max stride:16, filters:16[rate=2], sigma: 1.5) we get this error at the end the training loop when the model has reached the plateau:
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) INVALID_ARGUMENT: Conv2DSlowBackpropInput: Size of out_backprop doesn't match computed: actual = 39, computed = 40 spatial_dim: 3 input: 80 filter: 3 output: 39 stride: 2 dilation: 1
[[node single_instance_inference_model/single_instance_inference_layer_1/model/stack0_dec0_s16_to_s8_trans_conv/conv2d_transpose
(defined at C:\Users\SNeurobiology\miniconda3\envs\sleap\lib\site-packages\keras\backend.py:5532)
]]
[[single_instance_inference_model/single_instance_inference_layer_1/PartitionedCall_1/range/_20]]
(1) INVALID_ARGUMENT: Conv2DSlowBackpropInput: Size of out_backprop doesn't match computed: actual = 39, computed = 40 spatial_dim: 3 input: 80 filter: 3 output: 39 stride: 2 dilation: 1
[[node single_instance_inference_model/single_instance_inference_layer_1/model/stack0_dec0_s16_to_s8_trans_conv/conv2d_transpose
(defined at C:\Users\SNeurobiology\miniconda3\envs\sleap\lib\site-packages\keras\backend.py:5532)
]]
0 successful operations.
Expected behaviour
We have trained the model before and we did not encountered such error. We have also run the model inference over all the video used for extracting the training labels and no error has been observed.
Actual behaviour
Your personal set up
Windows 10
Python 3.7.12, SLEAP 1.3.3
Environment packages
Logs
Screenshots
How to reproduce
The text was updated successfully, but these errors were encountered: