Code

Normal baselines#618

Code

Open

AkshitaB wants to merge 50 commits intomainallenai/OLMo:mainfrom normal-baselinesallenai/OLMo:normal-baselinesCopy head branch name to clipboard

+8,012-14

Commits

Commits on Jun 12, 2024

• 7b normal baseline scripts

  AkshitaB

  committed
  c5913aa

  Copy full SHA for c5913aa

• add new evals

  AkshitaB

  committed
  e2cd59b

  Copy full SHA for e2cd59b

• add 1b config

  AkshitaB

  committed
  3d02325

  Copy full SHA for 3d02325

• 1b scripts

  AkshitaB

  committed
  995247f

  Copy full SHA for 995247f

• turn off fused_loss

  AkshitaB

  committed
  b71dff9

  Copy full SHA for b71dff9

• fix name

  AkshitaB

  committed
  0de7234

  Copy full SHA for 0de7234

• make executable

  AkshitaB

  committed
  75ae73f

  Copy full SHA for 75ae73f

• temporarily don't run new evals

  AkshitaB

  committed
  ed51f61

  Copy full SHA for ed51f61

• switch to pete's torch2.3 image

  AkshitaB

  committed
  3293cbb

  Copy full SHA for 3293cbb

• no clipping warmup

  AkshitaB

  committed
  d77add5

  Copy full SHA for d77add5

• wait longer

  AkshitaB

  committed
  eff21ee

  Copy full SHA for eff21ee

• priority

  AkshitaB

  committed
  c1075ce

  Copy full SHA for c1075ce

• Load from checkpoint, and also more data loading workers

  dirkgr

  committed
  1c58105

  Copy full SHA for 1c58105

• Fewer nodes :-(

  dirkgr

  committed
  cd04bad

  Copy full SHA for cd04bad

• start from scratch again

  AkshitaB

  committed
  63d2a26

  Copy full SHA for 63d2a26

• restart 1b

  AkshitaB

  committed
  aa8ca13

  Copy full SHA for aa8ca13

• add weka configs

  AkshitaB

  committed
  4fb871a

  Copy full SHA for 4fb871a

• 1b with weka and shard_grad_op

  AkshitaB

  committed
  f89437d

  Copy full SHA for f89437d

• oops, actually use weka with 1b

  AkshitaB

  committed
  8083742

  Copy full SHA for 8083742

• 7b with weka and more nodes

  AkshitaB

  committed
  1e7a243

  Copy full SHA for 1e7a243

• weka credentials

  AkshitaB

  committed
  57cc378

  Copy full SHA for 57cc378

• use my creds for s3

  AkshitaB

  committed
  fe716ec

  Copy full SHA for fe716ec

• weka creds for 7b

  AkshitaB

  committed
  910d25b

  Copy full SHA for 910d25b

• reduce num workers

  AkshitaB

  committed
  8d273e2

  Copy full SHA for 8d273e2

• fewer nodes again

  AkshitaB

  committed
  ad14ba8

  Copy full SHA for ad14ba8

• go back to s3 with more nodes

  AkshitaB

  committed
  c784548

  Copy full SHA for c784548

• more workers

  AkshitaB

  committed
  fa8ffa0

  Copy full SHA for fa8ffa0

• back to 256

  AkshitaB

  committed
  1bbc892

  Copy full SHA for 1bbc892

Commits on Jun 13, 2024

• more nodes

  AkshitaB

  committed
  a7cbcd6

  Copy full SHA for a7cbcd6

• nope; 256 it is

  AkshitaB

  committed
  db1151e

  Copy full SHA for db1151e

• fix the lr schedule :(

  AkshitaB

  committed
  700cea9

  Copy full SHA for 700cea9

• fix

  AkshitaB

  committed
  faf5fb5

  Copy full SHA for faf5fb5

• baseline explicit configs

  AkshitaB

  committed
  d4e3716

  Copy full SHA for d4e3716

• use newer configs

  AkshitaB

  committed
  2c3e1e6

  Copy full SHA for 2c3e1e6

• add back the new evals

  AkshitaB

  committed
  fd3ff93

  Copy full SHA for fd3ff93

Commits on Jun 14, 2024

• weka fs

  AkshitaB

  committed
  96c338c

  Copy full SHA for 96c338c

• 7b with wekafs

  AkshitaB

  committed
  35b51ce

  Copy full SHA for 35b51ce

• Make olmo-core checkpointer more robust on weka

  

  epwalsh

  authored and

  AkshitaB

  committed
  02b42f7

  Copy full SHA for 02b42f7

Commits on Jun 18, 2024

• spike debugging scripts

  AkshitaB

  committed
  dbe1e15

  Copy full SHA for dbe1e15

• remove line

  AkshitaB

  committed
  7b47d8b

  Copy full SHA for 7b47d8b

• rerun but skip the problematic batch

  AkshitaB

  committed
  6853be8

  Copy full SHA for 6853be8

• more nodes

  AkshitaB

  committed
  9ab633b

  Copy full SHA for 9ab633b

• add nccl ib env var

  AkshitaB

  committed
  56e6c32

  Copy full SHA for 56e6c32

• reduce num workers?

  AkshitaB

  committed
  8c7a35a

  Copy full SHA for 8c7a35a

Commits on Jun 19, 2024

• 256 nodes

  AkshitaB

  committed
  28bb493

  Copy full SHA for 28bb493

Commits on Jun 21, 2024

• 128 nodes

  AkshitaB

  committed
  7ed76d4

  Copy full SHA for 7ed76d4

• back to 256

  AkshitaB

  committed
  283a775

  Copy full SHA for 283a775

Commits on Jun 22, 2024

• Fewer nodes

  dirkgr

  committed
  ca7f696

  Copy full SHA for ca7f696

• NCCL env var to try

  dirkgr

  committed
  914a2b5

  Copy full SHA for 914a2b5

Commits on Jul 17, 2024

• Merge branch 'main' into normal-baselines

  AkshitaB

  authored
  51ae8fa

  Copy full SHA for 51ae8fa