New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

预训练数据构造有误 #72

Open

hy-struggle wants to merge 1 commit into brightmart:master from hy-struggle:master

hy-struggle commented Jun 29, 2020

原来的代码中，在wwm阶段，MLM任务中标签会被构造为开头带'##'的汉字，而在下游任务中这种token是没有意义的。
这就导致wwm在下游任务中其实并没有任何作用，反而伤害了模型。


          create_p_data

049a646

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet