Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add models from Hugging Face/transformers from MLAgility - part 1 #615

Merged
merged 63 commits into from
Jul 26, 2023
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
b326b3c
popular_on_huggingface/bert-base-uncased.py
jcwchen Jun 21, 2023
510f609
add transformers models
jcwchen Jun 21, 2023
fede10e
remove gpt1 and gpt2 for now
jcwchen Jun 22, 2023
97de789
config
jcwchen Jun 22, 2023
3af24e7
get model name from build_dir
jcwchen Jul 11, 2023
543ba43
find_model_hash_name
jcwchen Jul 11, 2023
8e41e92
subprocess.PIPE
jcwchen Jul 11, 2023
5dda38a
new models
jcwchen Jul 12, 2023
22f6123
7 models
jcwchen Jul 12, 2023
7dbf731
only keep 4
jcwchen Jul 12, 2023
2ce4161
remove 4
jcwchen Jul 12, 2023
ee483be
remove albert-base-v2
jcwchen Jul 12, 2023
e1690be
del model and sess
jcwchen Jul 13, 2023
d466f26
check_path
jcwchen Jul 13, 2023
777f0a6
drop models in CI
jcwchen Jul 13, 2023
94cd553
add bert_generation
jcwchen Jul 13, 2023
c865e0f
--binary
jcwchen Jul 13, 2023
6426b71
Merge branch 'jcw/add-hgf' of https://github.com/jcwchen/models into …
jcwchen Jul 13, 2023
f29974b
disable bert_generation.py
jcwchen Jul 13, 2023
da253f3
no binary
jcwchen Jul 13, 2023
5d1fc5f
cancel in progress
jcwchen Jul 13, 2023
fcffd4a
binary
jcwchen Jul 13, 2023
8272854
minimal
jcwchen Jul 13, 2023
db26881
--mini
jcwchen Jul 14, 2023
ebfbc04
manually check
jcwchen Jul 14, 2023
9f528a9
only keep
jcwchen Jul 14, 2023
b1c9ae4
run_test_dir
jcwchen Jul 14, 2023
5c42644
coma
jcwchen Jul 14, 2023
9db1f1d
cache_converted_dir = "~/.cache"
jcwchen Jul 14, 2023
19a9851
delete and clean cache
jcwchen Jul 14, 2023
3fc4703
clean
jcwchen Jul 14, 2023
240c30c
clean all
jcwchen Jul 14, 2023
abe2b16
only clean
jcwchen Jul 14, 2023
f36b21b
--cache-dir", cache_converted_dir
jcwchen Jul 14, 2023
ceefb83
disable openai_clip-vit-large-patch14
jcwchen Jul 14, 2023
da4cfa9
disable
jcwchen Jul 14, 2023
e05446a
only keep 4
jcwchen Jul 14, 2023
dd2c86a
comma
jcwchen Jul 14, 2023
51e2e09
runs-on: macos-latest
jcwchen Jul 14, 2023
abefcb0
not using conda
jcwchen Jul 14, 2023
9dfe0ed
final_model_path
jcwchen Jul 15, 2023
63f2b00
git-lfst pull dir
jcwchen Jul 15, 2023
288d29a
git diff
jcwchen Jul 15, 2023
22ede0b
Merge branch 'new-models' into jcw/add-hgf
jcwchen Jul 18, 2023
7c34b5a
use onnx.load to compare
jcwchen Jul 20, 2023
35a3ca8
test_utils.pull_lfs_file(final_model_path)
jcwchen Jul 21, 2023
20817c2
only test changed models
jcwchen Jul 24, 2023
a1483a8
test_utils
jcwchen Jul 24, 2023
d126f3b
get_cpu_info
jcwchen Jul 24, 2023
4495108
ext names
jcwchen Jul 24, 2023
5acb493
test_utils.get_changed_models()
jcwchen Jul 24, 2023
0a2ae7c
compare 2
jcwchen Jul 24, 2023
1ddb303
fix init
jcwchen Jul 24, 2023
cd3e260
transformers==4.29.2
jcwchen Jul 24, 2023
8ad0a61
test
jcwchen Jul 24, 2023
251d682
initializer
jcwchen Jul 24, 2023
c9b2756
update bert-generation
jcwchen Jul 24, 2023
c5178b0
fixed numpy
jcwchen Jul 24, 2023
9dafea5
print(f"initializer {k}")
jcwchen Jul 24, 2023
3774ffa
update bert from mac
jcwchen Jul 24, 2023
1de728d
remove bert-generation
jcwchen Jul 24, 2023
450e2ad
mlagility_subdir_count number
jcwchen Jul 24, 2023
3cd5bba
remove unused onnx
jcwchen Jul 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ on:
schedule:
- cron: '31 11 * * 4'

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
analyze:
name: Analyze
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/linux_ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ on:
pull_request:
branches: [ main, new-models]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
# This workflow contains a single job called "build"
build:
Expand Down
17 changes: 10 additions & 7 deletions .github/workflows/mlagility_validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,23 @@ on:
pull_request:
branches: [ main, new-models]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-latest
runs-on: macos-latest
strategy:
matrix:
python-version: ['3.8']
python-version: ["3.8"]

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
name: Checkout repo
- uses: conda-incubator/setup-miniconda@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@61a6322f88396a6271a6ee3565807d608ecaddd1 # v4.7.0
with:
miniconda-version: "latest"
activate-environment: mla
python-version: ${{ matrix.python-version }}

- name: Install dependencies and mlagility
Expand All @@ -34,4 +37,4 @@ jobs:
run: |
# TODO: remove the following after mlagility has resovled version contradict issue
pip install -r models/mlagility/requirements.txt
python workflow_scripts/run_mlagility.py
python workflow_scripts/run_mlagility.py --drop
4 changes: 4 additions & 0 deletions .github/workflows/windows_ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ on:
pull_request:
branches: [ main, new-models]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
# This workflow contains a single job called "build"
build:
Expand Down
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/bert_generation-16.onnx
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/input_0.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/input_1.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_0.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_1.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_10.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_11.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_12.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_13.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_14.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_15.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_16.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_17.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_18.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_19.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_2.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_20.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_21.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_22.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_23.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_24.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_25.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_26.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_27.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_28.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_29.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_3.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_30.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_31.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_32.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_33.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_34.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_35.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_36.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_37.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_38.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_39.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_4.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_40.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_41.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_42.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_43.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_44.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_45.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_46.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_47.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_48.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_49.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_5.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_50.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_51.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_52.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_53.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_54.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_55.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_56.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_57.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_58.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_59.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_6.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_60.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_61.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_62.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_63.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_64.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_65.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_66.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_67.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_68.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_69.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_7.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_70.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_71.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_72.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_73.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_74.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_75.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_76.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_77.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_78.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_79.pb
Git LFS file not shown
3 changes: 3 additions & 0 deletions models/mlagility/bert_generation/test_data_set_0/output_8.pb
Git LFS file not shown
Loading