Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug/Assistance] mind2web的unknown是怎么回事？ #129

Open
Tangent-90C opened this issue Mar 24, 2024 · 1 comment
Open

[Bug/Assistance] mind2web的unknown是怎么回事？ #129

Tangent-90C opened this issue Mar 24, 2024 · 1 comment
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@Tangent-90C
Copy link
Contributor

Describe the bug
我在使用gpt-3.5-turbo复现AgentBench中的mind2web(m2w)时，注意到有35%的结果为unknown，在runs.jsonl中，这35%的unknown结果没有任何的输出。
原以为是自己的问题，但注意到我复现出的分数与论文中的分数近乎一致（原论文20分，我的23分），所以这应该是AgentBench本身的问题，希望作者能修复这个unknown。

Screenshots or Terminal Copy&Paste
image
image
image

Desktop (please complete the following information):

  • OS: windows11 + WSL2(Ubuntu) + Docker
@Tangent-90C Tangent-90C added bug Something isn't working help wanted Extra attention is needed labels Mar 24, 2024
@zhc7
Copy link
Collaborator

zhc7 commented Mar 26, 2024

Hi, @Tangent-90C 这是正常的，这个涉及到Mind2Web环境本身的一些问题，可以暂时忽略

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Tangent-90C @zhc7