Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Just getting started with this framework, not sure what to do next.... #736

Open
jolson-ibm opened this issue Dec 9, 2024 · 0 comments
Open

Comments

@jolson-ibm
Copy link

Congrats on release v1.0.0 last week!

I ran through the modelbench README.md. I cloned the repo, set everything up, and executed the "Running Your First Benchmark" without issue. I then followed the step under "Using the Journal" and saw the results for prompt id airr_practice_1_0_41321, and the scores for the various models. I think I understand what the sample prompt is, and what it is trying to accomplish. Is there any further documentation on airr_practice_1_0_41321 ?

Now I am not sure what to do next. Are there other prompts or prompt suites I can test in a similar matter? I did checkout the ML Common website, and the white paper, but I am not sure what else I can execute besides the practice prompt.

Would be willing to contribute to the README.md for next steps if someone can guide me on what my options are to proceed with executing other prompts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant