Unsatisfactory results

[kukukiki765]

Hi, thank you for your work! But I seem to have some problems reproducing the model's performance. I followed the process you provided step by step, but I always couldn't achieve the optimal results given in the article despite trying several different sets of parameters. So I want to know what parameters can achieve the optimal results and how they differ from the default parameters
default:
-h --help Show this screen.
--bs= Batch size [default: 32]
--lr= Learning rate [default: 5e-4]
--ratio= Ratio of positive:negative, were negative is the controlled list (ratio=-1 => no ratio) [default: -1]
--itr= Number of iterations [default: 10]
--cuda= True/False - Whether to use cuda device or not [default: True]
--ft= Fine-tune the LM or not [default: False]
--wd= Adam optimizer Weight-decay [default: 0.01]
--hidden= hidden layers size [default: 150]
--dataset= wec/ecb - which dataset to generate for [default: wec]

[AlonEirew]

Hi @LinfanLiu01, I don't remember which parameters I've used, however did you try and reproduce using the model I've shared in huggingface model hub (link in the project readme)?

[kukukiki765]

Yes, I also use the weights you shared for the inference process and clustering, but strangely, the results obtained from this are even much lower than those given in the paper. I don't know where the problem comes from. Do I need to make adjustments in other aspects besides the parameters you set in the command

[AlonEirew]

Given these results, I assume you have a bug or an error in the code or in following the training procedure..

[kukukiki765]

It is very likely. I will re-examine my program and execution process