Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync SDK & CLI for eval_tasks / scoring_functions / datasets #20

Merged
merged 2 commits into from
Nov 12, 2024

Conversation

yanxi0830
Copy link
Contributor

TL;DR

  • sync latest OpenAPI change
  • CLI for eval_tasks / scoring_functions / datasets list

Tests

llama stack run 

with following config

apis:
- inference
- telemetry
- datasetio
- eval
- scoring
providers:
  scoring:
  - provider_id: meta-reference-0
    provider_type: meta-reference
    config: {}
  datasetio:
  - provider_id: huggingface-0
    provider_type: remote::huggingface
    config: {}
  eval:
  - provider_id: meta-reference-0
    provider_type: meta-reference
    config: {}
  inference:
  - provider_id: remote::fireworks-0
    provider_type: remote::fireworks
    config:
      url: https://api.fireworks.ai/inference
      api_key: null
  telemetry:
  - provider_id: meta-reference-0
    provider_type: meta-reference
    config: {}
metadata_store: null
image

@yanxi0830 yanxi0830 marked this pull request as ready for review November 12, 2024 03:08
@yanxi0830 yanxi0830 changed the title Sync SDK & eval_tasks / scoring_functions / datasets Sync SDK & CLI for eval_tasks / scoring_functions / datasets Nov 12, 2024
@yanxi0830 yanxi0830 merged commit 85f2fb0 into main Nov 12, 2024
3 checks passed
@yanxi0830 yanxi0830 deleted the stainless_sync_NDrrRs branch November 12, 2024 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants