Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] Support the -F flag for inspect eval #551

Open
tw-aisi opened this issue Sep 27, 2024 · 1 comment
Open

[feature request] Support the -F flag for inspect eval #551

tw-aisi opened this issue Sep 27, 2024 · 1 comment

Comments

@tw-aisi
Copy link
Contributor

tw-aisi commented Sep 27, 2024

Currently you can use the -F flag to filter tasks when running inspect list tasks (see https://inspect.ai-safety-institute.org.uk/eval-sets.html#attributes-and-filters).

It would be nice to also support this flag on the inspect eval subcommand, so instead of having to run this:

inspect list tasks security -F light=true | xargs inspect eval-set --log-dir logs-security-42

you can run this:

inspect eval-set --log-dir logs-security-42 -F light=true
@tw-aisi tw-aisi changed the title [feature requests] Support the -F flag for inspect eval [feature request] Support the -F flag for inspect eval Sep 27, 2024
@jjallaire-aisi
Copy link
Collaborator

Yes, that would be so much nicer! Will try to take care of this soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants