Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

当我在第0阶段之后,进入第一阶段生成数据之前,我得到的交互后的数据,我该运行哪个脚本? #27

Closed
K-THU opened this issue Dec 18, 2024 · 4 comments

Comments

@K-THU
Copy link

K-THU commented Dec 18, 2024

当我在第0阶段之后,进入第一阶段生成数据之前,我得到的交互后的数据,我该运行哪个脚本?是先python scripts/gen_task.py这个脚本还是你们后来发布的scripts/process_data.py这个脚本?

@lzy37ld
Copy link

lzy37ld commented Dec 18, 2024

hi,请问你是如何获得每个training example所对应的config file的呢?

Detail in : #24

@K-THU
Copy link
Author

K-THU commented Dec 18, 2024

hi,请问你是如何获得每个training example所对应的config file的呢?

Detail in : #24

自己改写的,在另外一个仓库VAB中有一个脚本generate_test_data.py
链接是https://github.com/THUDM/VisualAgentBench/tree/main/VAB-WebArena-Lite#-evaluating-in-webrl-setting-text-modal

@QZH-777
Copy link
Collaborator

QZH-777 commented Dec 26, 2024

当我在第0阶段之后,进入第一阶段生成数据之前,我得到的交互后的数据,我该运行哪个脚本?是先python scripts/gen_task.py这个脚本还是你们后来发布的scripts/process_data.py这个脚本?

在得到WebArena-Lite的交互数据后,需要先执行gen_task.py得到新任务,然后对新任务进行rollout,对rollout的结果执行process_data.py

@QZH-777 QZH-777 closed this as completed Dec 26, 2024
@K-THU
Copy link
Author

K-THU commented Dec 26, 2024

可否详细说明一下进入第0阶段后gen_task.py中第43行critic_lm和第44行指定critic_resume的参数吗,也就是用critic的模型对新任务指令进行打分和筛选,这里的critic模型是指定的orm吗?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants