Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

目前的语义匹配榜使用的QBQTC,SimCLUE后续会有单独的榜单吗? #2

Open
catqaq opened this issue Oct 8, 2022 · 4 comments

Comments

@catqaq
Copy link

catqaq commented Oct 8, 2022

No description provided.

@brightmart
Copy link
Member

simclue可以考虑有自己单独的榜单。用处大不大啊?

@catqaq
Copy link
Author

catqaq commented Oct 13, 2022

您好,用处基本上也是搜索场景里面的召回排序之类的,评估指标如果纯看语义的话就是类似于英文STS-B上的相关性指标,如果是评价搜索召回效果的话可以是类似于QBQTC上的F1、ACC,当然了还可以有排序指标。

@jerry1993-tech
Copy link

`效果对比
SimCLUE有什么用? 可以使用SimCLUE进行对比学习或作为通用数据训练模型,使得语义理解或搜索等相关场景模型具有较好的基础,并最终促进业务场景效果提升。

召回场景:语义检索场景,使用train_pair.json训练后,再使用业务数据训练,相对于直接业务训练,效果提升5个点(84%--->89%)。

排序场景:同一个语义检索场景,在召回基础上,进一步使用train_rank.json训练后,再使用业务排序数据训练模型,效果进一步提升2个点(92%--->94%)`

您好 请问咱这个对比实验的项目是否已开源?求链接,谢谢!

@Satonio1
Copy link

期待能有SimCLUE的榜单

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants