Experimenting with very high weights

[Rounique]

The results showed that giving very high weights in the new objective function doesn't make considerable changes in the metrics reflecting that after some point the increase in weight does not increase women's representation. It was found that the reason behind this is that we do not have enough women experts and there lack of their presence leads to this. We figured this out by manually adding some women experts in the teams and saw that the loss function could affect again in the women presence.

Link to the results

The experiments were done by augmenting the females with different values(5,10,20) and setting the b function weight to different values (10, 100, 200, 500,1000) while we were considering the top-k for evaluation based on the number of female experts being augmented. The expected ratio for NDKL was set to 0.9 since we expect a high ratio of females in our predictions but the values didn't have a considerable change across these experiments. However the NDKL is best we have had so far for this ratio. It has a considerable improvement. The reason why should be the augmentation but the problem is the same approach (Augmenting) is not following the pattern we expect. It is likely that the severe bias is causing this issue.