Did you try use alignment and uniformity as loss instead?

[jiqiujia]

In my dataset, I found in-batch sampled softmax loss donesn't work, and using alignment and uniformity as loss works well. I doubt if it related to the number of items.