Code

Add Goodfire API Provider Support#1161

Code

Open

menhguin wants to merge 33 commits intoUKGovernmentBEIS:mainUKGovernmentBEIS/inspect_ai:mainfrom menhguin:mainmenhguin/inspect_ai:mainCopy head branch name to clipboard

+307-1

Commits

Commits on Jan 17, 2025

• GSM8K Test Script

  menhguin

  committed
  ee9a647

  Copy full SHA for ee9a647

• Merge branch 'main' of https://github.com/UKGovernmentBEIS/inspect_ai

  menhguin

  committed
  31f7d08

  Copy full SHA for 31f7d08

• Add Goodfire API provider and configuration support

  Show description for 8397a93
  menhguin

  committed
  8397a93

  Copy full SHA for 8397a93

• to fix tmr

  menhguin

  committed
  17bb100

  Copy full SHA for 17bb100

Commits on Jan 18, 2025

• Merge branch 'UKGovernmentBEIS:main' into main

  menhguin

  authored
  e2eb13a

  Copy full SHA for e2eb13a

• Merge branch 'UKGovernmentBEIS:main' into main

  menhguin

  authored
  63b1ad1

  Copy full SHA for 63b1ad1

• registry.py full revert

  menhguin

  committed
  8c0a4d8

  Copy full SHA for 8c0a4d8

• i reverted model.py and it worked???

  menhguin

  committed
  b466dea

  Copy full SHA for b466dea

• [why tf did that increase the score] Enhance Goodfire API provider with version verification and improved error handling

  Show description for a7cc7ad
  menhguin

  committed
  a7cc7ad

  Copy full SHA for a7cc7ad

Commits on Jan 20, 2025

• Merge branch 'UKGovernmentBEIS:main' into main

  menhguin

  authored
  7503d70

  Copy full SHA for 7503d70

• generate config reset

  menhguin

  committed
  89a9ec0

  Copy full SHA for 89a9ec0

• [latest working script with standardisation improvements] Refactor GoodfireAPI generate method for improved error handling and parameter management

  Show description for 1f9a9ca
  menhguin

  committed
  1f9a9ca

  Copy full SHA for 1f9a9ca

• [further standardisation changes]

  menhguin

  committed
  65da3b1

  Copy full SHA for 65da3b1

• Delete gsm8k_example.py

  menhguin

  authored
  254e0a8

  Copy full SHA for 254e0a8

• Update _model.py w space lol

  menhguin

  authored
  15eda0a

  Copy full SHA for 15eda0a

• Update registry.py w space lol

  menhguin

  authored
  65f507f

  Copy full SHA for 65f507f

• Update registry.py

  menhguin

  authored
  2af6a9a

  Copy full SHA for 2af6a9a

• Update _model.py

  menhguin

  authored
  89da81b

  Copy full SHA for 89da81b

• Merge branch 'UKGovernmentBEIS:main' into main

  menhguin

  authored
  b628884

  Copy full SHA for b628884

Commits on Jan 22, 2025

• Merge branch 'UKGovernmentBEIS:main' into main

  menhguin

  authored
  dfa4d0e

  Copy full SHA for dfa4d0e

• [changes not in goodfire.py] Update Goodfire integration: add goodfire package dependency and remove GoodfireConfig class from GenerateConfig. Enhance goodfire provider with version verification.

  menhguin

  committed
  45d5609

  Copy full SHA for 45d5609

• [model args implementation]

  menhguin

  committed
  06d8972

  Copy full SHA for 06d8972

• [UKGovernmentBEIS#7 remove exception logging]

  menhguin

  committed
  0f15aea

  Copy full SHA for 0f15aea

• last stable state

  menhguin

  committed
  68ddb3b

  Copy full SHA for 68ddb3b

• Fix type hint mismatch in GoodfireAPI: update variant assignment to use runtime-safe string. Add note and TODO for potential issue in Goodfire's repo.

  menhguin

  committed
  0796eac

  Copy full SHA for 0796eac

• [implement UKGovernmentBEIS#9] Enhance GoodfireAPI output structure: replace content-based output with choices array including ChatCompletionChoice. This change improves the handling of chat respon…

  Show description for 67cd044
  menhguin

  committed
  67cd044

  Copy full SHA for 67cd044

• Refactor GoodfireAPI: streamline model argument handling, enhance message conversion, and implement connection pooling methods. This update improves code organization by consolidating model argumen…

  Show description for 0cd262f
  menhguin

  committed
  0cd262f

  Copy full SHA for 0cd262f

• Enhance GoodfireAPI: Add error handling for API calls, implement timeout and retry logic, and improve rate limit detection. This update introduces a new method for handling API errors, ensuring gra…

  Show description for 3aa4d0a
  menhguin

  committed
  3aa4d0a

  Copy full SHA for 3aa4d0a

• revert all changes to _generate_config

  menhguin

  committed
  6d6adf0

  Copy full SHA for 6d6adf0

• Remove version verification from goodfire.py as it was duplicated from providers.py

  menhguin

  committed
  7597cc3

  Copy full SHA for 7597cc3

• Refactor GoodfireAPI: Update default temperature and top_p values to baseline settings, streamline model name validation, and remove redundant message collapsing methods. This enhances the API's co…

  Show description for 77b1f16
  menhguin

  committed
  77b1f16

  Copy full SHA for 77b1f16

Commits on Jan 23, 2025

• Merge branch 'main' into main

  jjallaire

  authored
  831f8de

  Copy full SHA for 831f8de

• Merge branch 'UKGovernmentBEIS:main' into main

  menhguin

  authored
  004b982

  Copy full SHA for 004b982