Merge M0 ops #54

jdh8 · 2024-08-12T11:37:12Z

There are still conflicts to resolve. I kept conflict markers for a clear history containing atomic commits. The following unchecked files still contain conflicts.

Files containing conflicts

Refactor tests

models

~~test_real_world.py~~

lowering/eltwise

test_pointwise_binary.py
~~test_pointwise_trinary.py~~
test_pointwise_unary.py

lowering/matmul

test_matrix_multiplication.py
test_only_add_matmul.py

lowering/misc

test_cse.py
~~test_fall_back.py~~
test_if.py

lowering/normalization

test_norm.py
- layer_norm

lowering/tensor_manipulation

test_datamove.py

This reverts commit fc09080.

Update tracer API See merge request tenstorrent/torch-ttnn!22

- ttnn.open(0) -> ttnn.open_device(device_id=0) - ttnn.close(d) -> ttnn.close_device(d)

Add conversion and unit test cases - add - eq - gt - logical_and - logical_or - logical_xor - lt - maximum - minimum - mul - ne - pow - sub - xlogy

- aten.abs - aten.acos - aten.acosh - aten.asin - aten.asinh - aten.atan - aten.atan2 # binary - aten.atanh - aten.clone - aten.cos - aten.cosh - aten.erf - aten.exp - aten.expm1 - aten.gelu - aten.hardtanh - aten.isinf - aten.isnan - aten.leaky_relu - aten.log - aten.log10 - aten.log1p - aten.log2 - aten.logical_not - aten.neg - aten.reciprocal - aten.relu - aten.rsqrt - aten.sigmoid - aten.sign - aten.sin - aten.sinh - aten.sqrt - aten.tan - aten.tanh

- addcdiv - addcmul - where

Also use fx.subgraph_rewriter - matmul - linear

- ttnn.add(and other ops) don't have __name__, so torch.compile will fail. We hard patch the op with the __name__ - Now ttnn need a to_layout before computation

jdh8 · 2024-08-22T19:48:34Z

All the lowering tests now either (x)pass or xfail for me. All the xfailed tests have their corresponding issues (#64, tenstorrent/tt-metal#11925, #66, tenstorrent/tt-metal#12853) so that we can track them independently. There might still be 0-1 sporadic RNG-based numerical failure (when comparing inference results), but not about the graph (the identity, number, place, etc. of ops).

============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.2.2, pluggy-1.5.0
rootdir: /home/ubuntu/jdh8/pytorch2.0_ttnn
plugins: dash-2.15.0, split-0.8.2, timeout-2.2.0, anyio-4.4.0, xdist-3.6.1
collected 147 items                                                            

tests/lowering/creation/test_arange.py xxx                               [  2%]
tests/lowering/creation/test_clone.py ..                                 [  3%]
tests/lowering/creation/test_full.py .                                   [  4%]
tests/lowering/creation/test_ones.py .                                   [  4%]
tests/lowering/creation/test_to_copy.py xx                               [  6%]
tests/lowering/creation/test_zeros_like.py .                             [  6%]
tests/lowering/eltwise/binary/test_add.py ...x                           [  9%]
tests/lowering/eltwise/binary/test_atan2.py xxx                          [ 11%]
tests/lowering/eltwise/binary/test_div.py ...x..                         [ 15%]
tests/lowering/eltwise/binary/test_eq.py Xxxx.                           [ 19%]
tests/lowering/eltwise/binary/test_gt.py xxxx                            [ 21%]
tests/lowering/eltwise/binary/test_logical_and.py xxxx                   [ 24%]
tests/lowering/eltwise/binary/test_logical_or.py xxxx                    [ 27%]
tests/lowering/eltwise/binary/test_logical_xor.py xxxx                   [ 29%]
tests/lowering/eltwise/binary/test_lt.py xxxxx                           [ 33%]
tests/lowering/eltwise/binary/test_maximum.py ..xx                       [ 36%]
tests/lowering/eltwise/binary/test_minimum.py .x.x                       [ 38%]
tests/lowering/eltwise/binary/test_mul.py ...x                           [ 41%]
tests/lowering/eltwise/binary/test_ne.py xxxx                            [ 44%]
tests/lowering/eltwise/binary/test_sub.py .x.x..xx..                     [ 51%]
tests/lowering/eltwise/binary/test_xlogy.py ..x                          [ 53%]
tests/lowering/eltwise/unary/test_abs.py .                               [ 53%]
tests/lowering/eltwise/unary/test_acos.py .                              [ 54%]
tests/lowering/eltwise/unary/test_acosh.py .                             [ 55%]
tests/lowering/eltwise/unary/test_asin.py .                              [ 55%]
tests/lowering/eltwise/unary/test_asinh.py .                             [ 56%]
tests/lowering/eltwise/unary/test_atan.py .                              [ 57%]
tests/lowering/eltwise/unary/test_atanh.py .                             [ 57%]
tests/lowering/eltwise/unary/test_clamp.py .                             [ 58%]
tests/lowering/eltwise/unary/test_cos.py .                               [ 59%]
tests/lowering/eltwise/unary/test_cosh.py .                              [ 59%]
tests/lowering/eltwise/unary/test_erf.py .                               [ 60%]
tests/lowering/eltwise/unary/test_exp.py .                               [ 61%]
tests/lowering/eltwise/unary/test_expm1.py .                             [ 61%]
tests/lowering/eltwise/unary/test_gelu.py .                              [ 62%]
tests/lowering/eltwise/unary/test_hardtanh.py .                          [ 63%]
tests/lowering/eltwise/unary/test_isinf.py .                             [ 63%]
tests/lowering/eltwise/unary/test_isnan.py .                             [ 64%]
tests/lowering/eltwise/unary/test_leaky_relu.py .                        [ 65%]
tests/lowering/eltwise/unary/test_log.py .                               [ 65%]
tests/lowering/eltwise/unary/test_log10.py .                             [ 66%]
tests/lowering/eltwise/unary/test_log1p.py .                             [ 67%]
tests/lowering/eltwise/unary/test_log2.py .                              [ 68%]
tests/lowering/eltwise/unary/test_logical_not.py .                       [ 68%]
tests/lowering/eltwise/unary/test_neg.py .                               [ 69%]
tests/lowering/eltwise/unary/test_pow_scalar.py .                        [ 70%]
tests/lowering/eltwise/unary/test_reciprocal.py .                        [ 70%]
tests/lowering/eltwise/unary/test_relu.py .                              [ 71%]
tests/lowering/eltwise/unary/test_rsqrt.py .                             [ 72%]
tests/lowering/eltwise/unary/test_sigmoid.py .                           [ 72%]
tests/lowering/eltwise/unary/test_sign.py .                              [ 73%]
tests/lowering/eltwise/unary/test_silu.py .                              [ 74%]
tests/lowering/eltwise/unary/test_sin.py .                               [ 74%]
tests/lowering/eltwise/unary/test_sinh.py .                              [ 75%]
tests/lowering/eltwise/unary/test_sqrt.py .                              [ 76%]
tests/lowering/eltwise/unary/test_tan.py .                               [ 76%]
tests/lowering/eltwise/unary/test_tanh.py .                              [ 77%]
tests/lowering/eltwise/unary/test_tril.py .                              [ 78%]
tests/lowering/embedding/test_embedding.py x.                            [ 79%]
tests/lowering/matmul/test_addmm.py .                                    [ 80%]
tests/lowering/matmul/test_baddbmm.py .                                  [ 80%]
tests/lowering/matmul/test_linear.py x                                   [ 81%]
tests/lowering/matmul/test_matmul.py .                                   [ 82%]
tests/lowering/matmul/test_only_add_matmul.py ....                       [ 85%]
tests/lowering/misc/test_if.py .                                         [ 85%]
tests/lowering/normalization/test_layer_norm.py ..                       [ 87%]
tests/lowering/normalization/test_softmax.py .                           [ 87%]
tests/lowering/pool/test_avg_pool_2d.py .                                [ 88%]
tests/lowering/reduction/test_mean.py .                                  [ 89%]
tests/lowering/tensor_manipulation/test_aten_t.py ...                    [ 91%]
tests/lowering/tensor_manipulation/test_concat.py x                      [ 91%]
tests/lowering/tensor_manipulation/test_expand.py xxxx                   [ 94%]
tests/lowering/tensor_manipulation/test_permute.py .                     [ 95%]
tests/lowering/tensor_manipulation/test_repeat.py x                      [ 95%]
tests/lowering/tensor_manipulation/test_reshape.py x                     [ 96%]
tests/lowering/tensor_manipulation/test_squeeze.py .                     [ 97%]
tests/lowering/tensor_manipulation/test_transpose.py .                   [ 97%]
tests/lowering/tensor_manipulation/test_unsqueeze.py ...                 [100%]

================== 89 passed, 57 xfailed, 1 xpassed in 55.11s ==================

jdh8 · 2024-08-22T21:13:12Z

Is there a way to skip approval for running GitHub workflow/CI? That would let us spot errors without human intervention and fix things faster, especially with this time difference.

README.md

ayerofieiev-tt · 2024-08-22T21:18:48Z

@jdh8 this is due to the limitation for first time contributors using forks.
I sent an invite, you can start direct contribution to the repo w/o forking, it makes things nicer.
I also expect that once this PR is in that you will be able to start processes yourself w/o approval.

jdh8 · 2024-08-22T21:29:14Z

torch_ttnn/passes/lowering/to_tt_pass.py

+            return self.call_function_prop_meta(ttnn.matmul, args, kwargs)
+
+        if target == torch.ops.aten.linear.default:
+            return self.call_function_prop_meta(ttnn.linear, args, kwargs)


(For #66)
The conversion for aten.linear is here, but it is somehow unused.

xlogy asserts the same size for inputs for now

swimdi and others added 30 commits February 1, 2024 17:14

change tracer type as a decorator, its API change, too

406fcd1

torch_ttnn.backend apply tracer config

f74ca37

mv json dump out of parse_fx_graph

bd8ade4

Wrap leaf func with function form instead of decorator

126ac69

Rename TorchTtnnOption to TenstorrentBackendOption

76ef450

Merge branch 'feature/package'

1b722e2

Revert "Use subgraph_rewriter"

65208fd

This reverts commit fc09080.

Extrace mock_ttnn to a standalone package

d61466a

Merge branch 'feature/wrap-terminal-funcs'

f07734e

Register ttnn backend

6988e36

Add setup.py and pyproject.toml

bac760b

Update README.md

ae1b5f1

merge master

4804fff

mv torch_ttnn/tracer.py tracer/tracer.py

1df9ce1

Merge branch 'update-tracer-API' into 'master'

2936d0d

Update tracer API See merge request tenstorrent/torch-ttnn!22

Add model_sow2_list1 in tools/stat_models

9fd5175

Fix counter bug

60417a5

fix try except

318c12e

detr_resnet50 retain_graph=True

bebb995

Update README.md for package building

0bcd3b2

Merge remote-tracking branch 'github/master'

f3c0e3a

Update test case for ttnn inferface change

ff619f9

- ttnn.open(0) -> ttnn.open_device(device_id=0) - ttnn.close(d) -> ttnn.close_device(d)

Convert 14 pointwise binary operations

5d8ac8e

Add conversion and unit test cases - add - eq - gt - logical_and - logical_or - logical_xor - lt - maximum - minimum - mul - ne - pow - sub - xlogy

Convert 3 pointwise trinary operations

1d4d22f

- addcdiv - addcmul - where

Convert 2 matmul operations

0f0fd06

Also use fx.subgraph_rewriter - matmul - linear

Simplify op conversion

3bfefee

Fix wrap for ttnn & update test

b41d9c2

- ttnn.add(and other ops) don't have __name__, so torch.compile will fail. We hard patch the op with the __name__ - Now ttnn need a to_layout before computation

Fix more ops unit test

138e67e

Simpify pass insertion for gen_graphviz

1c01dfd

jdh8 added 9 commits August 22, 2024 17:46

Reflect that we convert torch.hardtanh to ttnn.clip for now

fcd1002

Remove test for unsupported group norm

1db9305

Mark conversion failure with linear

701b5c0

Fix test_clone.py for the patch

6b740fd

Mark argument mismtach in arange (#65)

3897230

Link tenstorrent#66 to test_linear.py

6950a2e

Mark lowering issues with tensor manipulation (#67)

d3eeca5

Reciprocal needs an offset because it has a pole at 0

26ce0b2

More tolerance for matmul for its accumulated error

d242144

ayerofieiev-tt added this pull request to the merge queue Aug 22, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 22, 2024

kevinwuTT requested changes Aug 22, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

Symetrically mark xfail for (min|max)imum

a46af89

jdh8 commented Aug 22, 2024

View reviewed changes

jdh8 added 3 commits August 22, 2024 21:36

Merge human-made docs from README.md to docs/README.md.in

e7f0de9

Do not use braces for shell variables to avoid clashing with .format

cf46f46

Generate README.md with new metrics

28ac8b0

jdh8 force-pushed the main branch from 4ccc40f to 28ac8b0 Compare August 22, 2024 21:42

kevinwuTT approved these changes Aug 22, 2024

View reviewed changes

ayerofieiev-tt added this pull request to the merge queue Aug 22, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 22, 2024

Mark xfail for xlogy involving broadcasting

f9a9b2e

xlogy asserts the same size for inputs for now

jdh8 added this pull request to the merge queue Aug 22, 2024

Merged via the queue into tenstorrent:main with commit f87d7b3 Aug 22, 2024
1 check passed

jdh8 mentioned this pull request Sep 12, 2024

Failed to lower some tensor manipulation ops from aten to ttnn tenstorrent/tt-metal#12853

Open

jdh8 mentioned this pull request Oct 2, 2024

#13337: Optimize trigonometric functions tenstorrent/tt-metal#13339

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge M0 ops #54

Merge M0 ops #54

jdh8 commented Aug 12, 2024 •

edited

Loading

jdh8 commented Aug 22, 2024

jdh8 commented Aug 22, 2024

ayerofieiev-tt commented Aug 22, 2024

jdh8 Aug 22, 2024

Merge M0 ops #54

Merge M0 ops #54

Conversation

jdh8 commented Aug 12, 2024 • edited Loading

Files containing conflicts

Refactor tests

models

lowering/eltwise

lowering/matmul

lowering/misc

lowering/normalization

lowering/tensor_manipulation

jdh8 commented Aug 22, 2024

jdh8 commented Aug 22, 2024

ayerofieiev-tt commented Aug 22, 2024

jdh8 Aug 22, 2024

Choose a reason for hiding this comment

jdh8 commented Aug 12, 2024 •

edited

Loading