Code

Support latest TransformerEngine#98

Code

Merged

tocean merged 21 commits intomainAzure/MS-AMP:mainfrom yuxiang/teAzure/MS-AMP:yuxiang/teCopy head branch name to clipboardOct 18, 2023

+715-7

Commits

Commits on Oct 17, 2023

• Support latest TransformerEngine

  

  [email protected]

  authored and

  tocean

  committed
  28fa896

  Copy full SHA for 28fa896

• add UT

  tocean

  committed
  b25ff42

  Copy full SHA for b25ff42

• remove pytorch 1.13

  tocean

  committed
  4bb27fd

  Copy full SHA for 4bb27fd

• fix bug

  tocean

  committed
  22fd1ca

  Copy full SHA for 22fd1ca

• fix lint issue

  tocean

  committed
  79b3181

  Copy full SHA for 79b3181

• fix flash attention

  tocean

  committed
  3e7ed21

  Copy full SHA for 3e7ed21

• restore dockerfile

  tocean

  committed
  7688c09

  Copy full SHA for 7688c09

• try to fix the runner has received a shutdown signal

  tocean

  committed
  4cb76f1

  Copy full SHA for 4cb76f1

• change to use GPU machine for building image

  tocean

  committed
  245e4a1

  Copy full SHA for 245e4a1

• add environment variable

  tocean

  committed
  8fd5712

  Copy full SHA for 8fd5712

• install latedt te in dockerfile

  tocean

  committed
  f5b7720

  Copy full SHA for f5b7720

• fix bug

  tocean

  committed
  ed410ce

  Copy full SHA for ed410ce

• Add fp8_cast_transpose_fused and more UT

  tocean

  committed
  a6790f6

  Copy full SHA for a6790f6

• remove transformer layer forward/backward since it can only

  tocean

  committed
  cbc523f

  Copy full SHA for cbc523f

• fix bug

  tocean

  committed
  6dd1f90

  Copy full SHA for 6dd1f90

• use flash attention 2

  tocean

  committed
  a699bb6

  Copy full SHA for a699bb6

• use flash attention 2

  tocean

  committed
  902e21e

  Copy full SHA for 902e21e

• use ubuntu-2004

  tocean

  committed
  a8dd4e1

  Copy full SHA for a8dd4e1

• change to 1.0.9

  tocean

  committed
  6175523

  Copy full SHA for 6175523

• fix comment

  tocean

  committed
  29c6aab

  Copy full SHA for 29c6aab

• skip te unit test since fused attention only works when cuda>=12.1

  tocean

  committed
  caeb253

  Copy full SHA for caeb253