BiRealNet test issue

[FrancescoLoro]

Hi everyone and thanks for making public the fantastic larq library.
I am trying to reproduce the networks present in the larq zoo in order to customize them and make some comparisons.
I started from the BiRealNet and I was able to reproduce it exactly based both on your code on github and on the references of the summary present in: https://docs.larq.dev/zoo/api/literature/.
I have downloaded the pretrained weights using the functions on your code and correctly load them into the model.
To test the network I used a reduced version of ImageNet (about 50000 samples) applying preprocess I found here:

zoo/tests/models_test.py

Line 16 in 2e76138

def test_image(request):

to all images.
However the network always predicts the same class regardless of the sample presented to it, and I can't figure out why.
Could you please tell me if I'm doing it wrong somewhere?
Thanks a lot for you help!

[AdamHillier]

Hi @FrancescoLoro, sorry for the delayed response!

Could you perhaps share the exact evaluation script that you're using to test birealnet? It will make it easier to work out what might be going wrong.

[FrancescoLoro]

Sorry for the delay, I managed to solve it 👍

[AdamHillier]

Ah okay, great to hear :)