Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] 🚀🚀🚀 Transformers.js V3 🚀🚀🚀 #545

Open
wants to merge 447 commits into
base: main
Choose a base branch
from
Open

[WIP] 🚀🚀🚀 Transformers.js V3 🚀🚀🚀 #545

wants to merge 447 commits into from
+27,686 −9,579
This pull request is big! We’re only showing the most recent 250 commits.

Commits on Jun 10, 2024

  1. Formatting

    @xenova
    xenova committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    939f735 View commit details
    Browse the repository at this point in the history

  2. Optimize spectogram function

    @xenova
    xenova committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    edfa94f View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Add rfft/stft tensor ops via ORT 

    Will be much more useful when WebGPU support is ready for these ops (see microsoft/onnxruntime#20997)
    @xenova
    xenova committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    490a4b8 View commit details
    Browse the repository at this point in the history

  2. Optimize top_k sampling

    @xenova
    xenova committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    39bd0aa View commit details
    Browse the repository at this point in the history

  3. Allow priority queue to specify max heap size

    @xenova
    xenova committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    4d2c819 View commit details
    Browse the repository at this point in the history

  4. Create tiny random pipeline test (fill-mask)

    @xenova
    xenova committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    49ace06 View commit details
    Browse the repository at this point in the history

  5. Create text-classification pipeline test

    @xenova
    xenova committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    9ac56c8 View commit details
    Browse the repository at this point in the history

  6. Add problem_type = 'multi_label_classification' test

    @xenova
    xenova committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    3eaa484 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Support top_k w/ k < dims.at(-1)

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    f45cdd8 View commit details
    Browse the repository at this point in the history

  2. Create image-classification tiny pipeline test

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    a018842 View commit details
    Browse the repository at this point in the history

  3. Create audio-classification tiny pipeline test

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    4db5afd View commit details
    Browse the repository at this point in the history

  4. Add support for cohere models

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    fe931f1 View commit details
    Browse the repository at this point in the history

  5. Add CohereModel unit test

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    9ee75bd View commit details
    Browse the repository at this point in the history

  6. Add token-classification pipeline test

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    3ecfcc1 View commit details
    Browse the repository at this point in the history

  7. Add question-answering pipeline test

    @xenova
    xenova committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    222669f View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. (temp) Ignore beam search

    @xenova
    xenova committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e0a7a4d View commit details
    Browse the repository at this point in the history

  2. Implement ForcedEOSTokenLogitsProcessor

    @xenova
    xenova committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    2a3a71b View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Update pipelines

    @xenova
    xenova committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    6006fe7 View commit details
    Browse the repository at this point in the history

  2. Create webgpu depth estimation demo

    @xenova
    xenova committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    168588a View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Update package-lock.json

    @xenova
    xenova committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    fc53d15 View commit details
    Browse the repository at this point in the history

  2. Add support for MobileNet (v1/v2/v3/v4)

    @xenova
    xenova committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    db7c1f3 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Fix vision-encoder-decoder configs

    @xenova
    xenova committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    63cdcb2 View commit details
    Browse the repository at this point in the history

  2. Use switch statement for model type checks

    @xenova
    xenova committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    c32476c View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Re-enable text-to-audio pipeline

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    6a886f4 View commit details
    Browse the repository at this point in the history

  2. Support fallback for not-found errors

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    fd38113 View commit details
    Browse the repository at this point in the history

  3. Add florence2 configs

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d3c56f3 View commit details
    Browse the repository at this point in the history

  4. Implement Florence2ForConditionalGeneration

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    65ce093 View commit details
    Browse the repository at this point in the history

  5. Create Florence2Processor

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    666dfcc View commit details
    Browse the repository at this point in the history

  6. Add florence2 unit tests

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0a5bbd9 View commit details
    Browse the repository at this point in the history

  7. Throw error if neither decoder_input_ids nor `decoder_inputs_embeds… 

    …` are specified
    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4ca37d4 View commit details
    Browse the repository at this point in the history

  8. Improve merging input embeds

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    dab703d View commit details
    Browse the repository at this point in the history

  9. Add comments

    @xenova
    xenova committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    11da868 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. Map ngram bigint -> number

    @xenova
    xenova committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    1ab49a5 View commit details
    Browse the repository at this point in the history

  2. Add florence2 to list of supported models

    @xenova
    xenova committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    2649fd6 View commit details
    Browse the repository at this point in the history

  3. Minor conversion script improvements

    @xenova
    xenova committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    4d92c7f View commit details
    Browse the repository at this point in the history

  4. add onnxslim intergration (#811) 

    * add onnxslim intergration

* fix onnxslim version
    @inisis
    inisis committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    e8cfaf6 View commit details
    Browse the repository at this point in the history

  5. Update onnxslim version

    @xenova
    xenova committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    af37e87 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Remove debug log

    @xenova
    xenova committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    e4c4848 View commit details
    Browse the repository at this point in the history

  2. Implement post_process_generation for florence2 processor

    @xenova
    xenova committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    8cfc9ad View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Rename construct_prompts method

    @xenova
    xenova committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    92954af View commit details
    Browse the repository at this point in the history

  2. Create Florence-2 demo

    @xenova
    xenova committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    1f4ad16 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Remove unused line

    @xenova
    xenova committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    aa97d1e View commit details
    Browse the repository at this point in the history

  2. Add MobileCLIP to list of supported models

    @xenova
    xenova committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    f37eae0 View commit details
    Browse the repository at this point in the history

  3. Add support for RT-DETR models

    @xenova
    xenova committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    10e2431 View commit details
    Browse the repository at this point in the history

  4. Support per-model use_external_data_format

    @xenova
    xenova committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4cb26a1 View commit details
    Browse the repository at this point in the history

  5. Ensure options.use_external_data_format !== null

    @xenova
    xenova committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7ede237 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Fix musicgen progress

    @xenova
    xenova committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    29af02c View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. Fix custom whisper configs

    @xenova
    xenova committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    0b5469b View commit details
    Browse the repository at this point in the history

  2. Improve custom whisper ONNX config

    @xenova
    xenova committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    14a2990 View commit details
    Browse the repository at this point in the history

  3. Fix whisper word-level timestamps

    @xenova
    xenova committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    d010cc2 View commit details
    Browse the repository at this point in the history

  4. Optimize DTW (50x!)

    @xenova
    xenova committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    5ab1ea3 View commit details
    Browse the repository at this point in the history

  5. Early dereferencing for performance boosts

    @xenova
    xenova committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    0dba266 View commit details
    Browse the repository at this point in the history

  6. cleanup

    @xenova
    xenova committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    5e4e20f View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Move quantization logic to quantize.py

    @xenova
    xenova committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    dd6af93 View commit details
    Browse the repository at this point in the history

  2. update deps

    @xenova
    xenova committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    04af3d5 View commit details
    Browse the repository at this point in the history

  3. Fix q4 quantization

    @xenova
    xenova committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    9128651 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. save q4 quantization

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    83cbb21 View commit details
    Browse the repository at this point in the history

  2. Add decode ASR test

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    eb61344 View commit details
    Browse the repository at this point in the history

  3. Do not process last chunk unnecessarily 

    huggingface/transformers@1518508
    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    cec2400 View commit details
    Browse the repository at this point in the history

  4. fp16 disable_shape_infer if model is too large

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    c835b54 View commit details
    Browse the repository at this point in the history

  5. Use check_and_save_model for saving fp16 model

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    45cd8d4 View commit details
    Browse the repository at this point in the history

  6. Reorder functions

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    88f3e44 View commit details
    Browse the repository at this point in the history

  7. formatting

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    23440f0 View commit details
    Browse the repository at this point in the history

  8. Remove debug log

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    b411e9f View commit details
    Browse the repository at this point in the history

  9. Fix q8 quantization for models > 2GB

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    04a334a View commit details
    Browse the repository at this point in the history

  10. correct attribute

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    cd1ea69 View commit details
    Browse the repository at this point in the history

  11. Fix TextGenerationPipeline

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a167f6e View commit details
    Browse the repository at this point in the history

  12. Fix pauses in whisper word-level timestamps 

    Adapted from @MatteoFasulo's version.
    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ea73289 View commit details
    Browse the repository at this point in the history

  13. Formatting

    @xenova
    xenova committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    344af32 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. Sort added tokens by length to avoid early partial matches

    @xenova
    xenova committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    c305c38 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Add new tokenizer test

    @xenova
    xenova committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    d6f6fd4 View commit details
    Browse the repository at this point in the history

  2. Only finish with newline if running in Node.js

    @xenova
    xenova committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    1557b8d View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. Consider token timestamps when selecting longest common sequence

    @xenova
    xenova committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    9ac7ceb View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Create whisper word-level timestamps demo

    @xenova
    xenova committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    79ed46e View commit details
    Browse the repository at this point in the history

  2. cleanup

    @xenova
    xenova committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    8da6886 View commit details
    Browse the repository at this point in the history

  3. Fallback to WASM if WebGPU not supported

    @xenova
    xenova committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    d709bd0 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. Reload model for each quantization mode

    @xenova
    xenova committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    9ef3a6d View commit details
    Browse the repository at this point in the history

  2. Update converstion script requirements

    @xenova
    xenova committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    9787b75 View commit details
    Browse the repository at this point in the history

  3. Separate IO and Quantization args

    @xenova
    xenova committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    974f086 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. Use const where possible

    @xenova
    xenova committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    d042868 View commit details
    Browse the repository at this point in the history

  2. Add InterruptableStoppingCriteria

    @xenova
    xenova committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    1b4d242 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. @xenova/transformers -> @huggingface/transformers

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    31101c8 View commit details
    Browse the repository at this point in the history

  2. Override semver version

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    e84322b View commit details
    Browse the repository at this point in the history

  3. Add support for pyannote models

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    bd94334 View commit details
    Browse the repository at this point in the history

  4. Update README.md

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    3dbc633 View commit details
    Browse the repository at this point in the history

  5. Add listed support for pyannote

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    858e55d View commit details
    Browse the repository at this point in the history

  6. Add pyannote example code

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    8bf0349 View commit details
    Browse the repository at this point in the history

  7. Support specifying min_num_frames

    @xenova
    xenova committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c52618c View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2024

  1. Support simultaneous instantiation of multiple inference sessions 

    - Only the first call needs to be sequential due to the call to `initWasm()`
    @xenova
    xenova committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    96f19b0 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Support broadcasting encoder outputs over decoder inputs 

    More efficient than duplicating encoder inputs
    @xenova
    xenova committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    4ad43e2 View commit details
    Browse the repository at this point in the history

  2. Fix test

    @xenova
    xenova committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c6aeb4b View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. fix bundler config for latest ORT

    @fs-eire
    fs-eire committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    6d3ea4b View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Only check fp16 support for webgpu device 

    WASM/CPU works with fp16, but WebGPU is device-dependent
    @xenova
    xenova committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    38a3bf6 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. Remove default chat templates 

    huggingface/transformers#31733
    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    9df84c4 View commit details
    Browse the repository at this point in the history

  2. Add support for gemma2

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    fc3d860 View commit details
    Browse the repository at this point in the history

  3. Add gemma2 generation test

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    939920d View commit details
    Browse the repository at this point in the history

  4. Update gemma2 config mapping

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    5bb93a0 View commit details
    Browse the repository at this point in the history

  5. Prioritize high-performance adapter when possible 

    https://gpuweb.github.io/gpuweb/#dictdef-gpurequestadapteroptions
    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    72ec168 View commit details
    Browse the repository at this point in the history

  6. Set defaults for tools and documents in apply_chat_template

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    9068a53 View commit details
    Browse the repository at this point in the history

  7. bump @huggingface/jinja -> 0.3.0

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    824538b View commit details
    Browse the repository at this point in the history

  8. Add apply_chat_template default parameters unit test

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    836c0af View commit details
    Browse the repository at this point in the history

  9. Merge branch 'v3' into @huggingface/transformers

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    487d8b2 View commit details
    Browse the repository at this point in the history

  10. Add prettier

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    1f6e0e1 View commit details
    Browse the repository at this point in the history

  11. prettier format config files

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    55494d1 View commit details
    Browse the repository at this point in the history

  12. remove incorrect comment

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    5a68461 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'pr/864' into @huggingface/transformers

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    437cb34 View commit details
    Browse the repository at this point in the history

  14. Update onnxruntime-web version

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    5a6c926 View commit details
    Browse the repository at this point in the history

  15. Update webpack.config.js

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    b19251b View commit details
    Browse the repository at this point in the history

  16. Fix copy path

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    820c1e2 View commit details
    Browse the repository at this point in the history

  17. Run npm ci

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    b0dab91 View commit details
    Browse the repository at this point in the history

  18. Fix bundling

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    86b9b62 View commit details
    Browse the repository at this point in the history

  19. Do not set preferredOutputLocation if we are proxying 

    session option "preferredOutputLocation" is not supported for proxy.'
    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    222b94e View commit details
    Browse the repository at this point in the history

  20. Merge branch 'v3' into @huggingface/transformers

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    b326cc9 View commit details
    Browse the repository at this point in the history

  21. Update @webgpu/types

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    ca67092 View commit details
    Browse the repository at this point in the history

  22. Update SAM example

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    42076fd View commit details
    Browse the repository at this point in the history

  23. Use ??= operator where possible

    @xenova
    xenova committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    48d3142 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Fix commonjs usage

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3b1a4fd View commit details
    Browse the repository at this point in the history

  2. Mark onnxruntime-node and sharp as externals

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9a73b5e View commit details
    Browse the repository at this point in the history

  3. Move externals into config

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9951aa5 View commit details
    Browse the repository at this point in the history

  4. Downgrade to onnxruntime 1.18.0

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    c04d37e View commit details
    Browse the repository at this point in the history

  5. Finalize module/commonjs build

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    d32fe2b View commit details
    Browse the repository at this point in the history

  6. Separate web and node builds

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    1530d50 View commit details
    Browse the repository at this point in the history

  7. [version] Update to 3.0.0-alpha.1

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    b4df0e2 View commit details
    Browse the repository at this point in the history

  8. Default to CDN-hosted .wasm files

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ab59c51 View commit details
    Browse the repository at this point in the history

  9. [version] Update to 3.0.0-alpha.2

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    866b219 View commit details
    Browse the repository at this point in the history

  10. bump versions

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    4a3398d View commit details
    Browse the repository at this point in the history

  11. [version] Update to 3.0.0-alpha.3

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8891a14 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'improve-conversion-script' into v3

    @xenova
    xenova committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    a315933 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Consolidate conversion and quantization script

    @xenova
    xenova committed Aug 9, 2024
    1 Configuration menu
    Copy the full SHA
    12569b8 View commit details
    Browse the repository at this point in the history

  2. Downgrade onnxconverter-common

    @xenova
    xenova committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    83f5718 View commit details
    Browse the repository at this point in the history

  3. Link to types in exports

    @xenova
    xenova committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    6fa5fa6 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. Update list of supported tasks

    @xenova
    xenova committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    2f1b210 View commit details
    Browse the repository at this point in the history

  2. Fixed unit tests

    @xenova
    xenova committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    27bc55d View commit details
    Browse the repository at this point in the history

  3. Update imports

    @xenova
    xenova committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    23d1150 View commit details
    Browse the repository at this point in the history

  4. Bump versions to 3.0.0-alpha.4

    @xenova
    xenova committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    f9070dc View commit details
    Browse the repository at this point in the history

  5. [version] Update to 3.0.0-alpha.4

    @xenova
    xenova committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    c3494e1 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. Fix "Default condition should be last one"

    @xenova
    xenova committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    973fb0d View commit details
    Browse the repository at this point in the history

  2. Bump versions

    @xenova
    xenova committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    7376ecf View commit details
    Browse the repository at this point in the history

  3. [version] Update to 3.0.0-alpha.5

    @xenova
    xenova committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    0a04bc0 View commit details
    Browse the repository at this point in the history

  4. Update next.js client-side demo

    @xenova
    xenova committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    e4603cd View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. Initial WebNN Support

    @ibelem
    ibelem committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    ff1853c View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. Mark fs, path and url as external packages for node build

    @xenova
    xenova committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    15574bc View commit details
    Browse the repository at this point in the history

  2. Move content type map outside of FileResponse object

    @xenova
    xenova committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    7282862 View commit details
    Browse the repository at this point in the history

  3. Add GPU support for Node.js

    @xenova
    xenova committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    22f7ced View commit details
    Browse the repository at this point in the history

  4. Bump versions

    @xenova
    xenova committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    1e319a4 View commit details
    Browse the repository at this point in the history

  5. [version] Update to 3.0.0-alpha.6

    @xenova
    xenova committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    d278891 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Fix conflicts

    @ibelem
    ibelem committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    3fefa17 View commit details
    Browse the repository at this point in the history

  2. bump dependency versions

    @xenova
    xenova committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    fa6cc70 View commit details
    Browse the repository at this point in the history

  3. Add support for device auto-detection

    @xenova
    xenova committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    7fa5326 View commit details
    Browse the repository at this point in the history

  4. Fix default device selection

    @xenova
    xenova committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    4ec77c1 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'pr/ibelem/890-1' into v3

    @xenova
    xenova committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    5799e30 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2024

  1. Improve WebNN selection

    @xenova
    xenova committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    5b2cac2 View commit details
    Browse the repository at this point in the history

  2. Skip token callback if skip_prompt is set

    @xenova
    xenova committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    ad23c50 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. Bump versions

    @xenova
    xenova committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    5b84b62 View commit details
    Browse the repository at this point in the history

  2. [version] Update to 3.0.0-alpha.7

    @xenova
    xenova committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    bcf6a86 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. bump versions

    @xenova
    xenova committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    b97ed0d View commit details
    Browse the repository at this point in the history

  2. [version] Update to 3.0.0-alpha.8

    @xenova
    xenova committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    c5b7083 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. bump versions

    @xenova
    xenova committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    cbeefde View commit details
    Browse the repository at this point in the history

  2. [version] Update to 3.0.0-alpha.9

    @xenova
    xenova committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    59600f2 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. Add support for Sapiens

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b2e025a View commit details
    Browse the repository at this point in the history

  2. Update default ONNX env

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    8661d95 View commit details
    Browse the repository at this point in the history

  3. Fix types

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    57db34d View commit details
    Browse the repository at this point in the history

  4. Topologically sort fp16 nodes

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    1b7f978 View commit details
    Browse the repository at this point in the history

  5. Add marian unit test

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    45d1526 View commit details
    Browse the repository at this point in the history

  6. Re-order imports

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b903757 View commit details
    Browse the repository at this point in the history

  7. Fix NoBadWordsLogitsProcessor

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    633976f View commit details
    Browse the repository at this point in the history

  8. Update package.json

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    24d8787 View commit details
    Browse the repository at this point in the history

  9. [jest] Disable coverage

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    9412ec4 View commit details
    Browse the repository at this point in the history

  10. Bump versions

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    08e7388 View commit details
    Browse the repository at this point in the history

  11. [version] Update to 3.0.0-alpha.10

    @xenova
    xenova committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    d5a8f87 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Improve node/web interoperability

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7843ad0 View commit details
    Browse the repository at this point in the history

  2. Fix scripts/requirements.txt

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    bf093ae View commit details
    Browse the repository at this point in the history

  3. Bump versions

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    9a5ee42 View commit details
    Browse the repository at this point in the history

  4. [version] Update to 3.0.0-alpha.11

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    535cdfe View commit details
    Browse the repository at this point in the history

  5. Add support for JAIS models (#906)

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    4e1acf0 View commit details
    Browse the repository at this point in the history

  6. Add JAIS to README

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    488548d View commit details
    Browse the repository at this point in the history

  7. Fix node/web interop (again)

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    13aed41 View commit details
    Browse the repository at this point in the history

  8. Bump versions

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7655f81 View commit details
    Browse the repository at this point in the history

  9. [version] Update to 3.0.0-alpha.12

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    1c7e226 View commit details
    Browse the repository at this point in the history

  10. Set SapiensForNormalEstimation to encoder-only

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    ab6b28b View commit details
    Browse the repository at this point in the history

  11. Implement sub tensor operation

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    66c05d5 View commit details
    Browse the repository at this point in the history

  12. Bump versions

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    31e8b2a View commit details
    Browse the repository at this point in the history

  13. [version] Update to 3.0.0-alpha.13

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    bf3f7d5 View commit details
    Browse the repository at this point in the history

  14. Improve typing for wrap helper function

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    c025356 View commit details
    Browse the repository at this point in the history

  15. Update preferredOutputLocation type

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7ebdaf2 View commit details
    Browse the repository at this point in the history

  16. Make wrap type more generic

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    3b8ddcb View commit details
    Browse the repository at this point in the history

  17. Re-use segmentation_data

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    a385c6e View commit details
    Browse the repository at this point in the history

  18. Fix min type

    @xenova
    xenova committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    537e958 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. Add support for Hiera models

    @xenova
    xenova committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    bcb28b3 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Fix reused loop variable (closes #910)

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    d21c87c View commit details
    Browse the repository at this point in the history

  2. Add logits processor test file

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    1d281f6 View commit details
    Browse the repository at this point in the history

  3. Fix test imports

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    ba0427f View commit details
    Browse the repository at this point in the history

  4. Bump versions

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    3bc3e86 View commit details
    Browse the repository at this point in the history

  5. [version] Update to 3.0.0-alpha.14

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    0518960 View commit details
    Browse the repository at this point in the history

  6. Add another bad_words logits processor test (closes #913)

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    552cdea View commit details
    Browse the repository at this point in the history

  7. Add support for GroupViT

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    3422a8b View commit details
    Browse the repository at this point in the history

  8. Add zero-shot-image-classification unit test

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    3599902 View commit details
    Browse the repository at this point in the history

  9. Add maskformer model definitions

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    5892ee8 View commit details
    Browse the repository at this point in the history

  10. Support universal image segmentation in image-segmentation pipeline

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c4dac77 View commit details
    Browse the repository at this point in the history

  11. Add support for PVT models

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    f0c47be View commit details
    Browse the repository at this point in the history

  12. Add post_process_instance_segmentation function template

    @xenova
    xenova committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    d80d3a4 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. Add library_name option to convert.py

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    844099d View commit details
    Browse the repository at this point in the history

  2. Wrap onnxslim with try block

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    ba5d725 View commit details
    Browse the repository at this point in the history

  3. Use const where possible

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    b3691c8 View commit details
    Browse the repository at this point in the history

  4. Use const where possible (again)

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    dcf117f View commit details
    Browse the repository at this point in the history

  5. Create MaskFormerFeatureExtractor

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    9af026c View commit details
    Browse the repository at this point in the history

  6. Add support for MaskFormer

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    0f8200c View commit details
    Browse the repository at this point in the history

  7. Improve tool-use chat template detection

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    e278c5e View commit details
    Browse the repository at this point in the history

  8. Add object detection pipeline unit test

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    83fa58f View commit details
    Browse the repository at this point in the history

  9. Add support for ViTMSN and VitMAE

    @xenova
    xenova committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    86d6da4 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2024

  1. Bump ORT versions

    @xenova
    xenova committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    93b25fb View commit details
    Browse the repository at this point in the history

  2. Create get_chat_template helper function

    @xenova
    xenova committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    2f680ee View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Fix CI

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    2f9b2ed View commit details
    Browse the repository at this point in the history

  2. Run prettier on tests/**

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    deec350 View commit details
    Browse the repository at this point in the history

  3. move certain tests to utils subfolder

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    48fa226 View commit details
    Browse the repository at this point in the history

  4. Bump onnxruntime-web version

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    a10828f View commit details
    Browse the repository at this point in the history

  5. Bump onnxruntime==1.19.2 in scripts/requirements.txt

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    ba58ea2 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into v3

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    4f17e95 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into v3

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    c40a151 View commit details
    Browse the repository at this point in the history

  8. Sort this.added_tokens before creating regex (.toSorted is not av… 

    …ailable in Node.js < 20)
    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    30315b2 View commit details
    Browse the repository at this point in the history

  9. Rather make a copy of this.added_tokens

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    d7df575 View commit details
    Browse the repository at this point in the history

  10. Fix .tokenize with fuse_unk=true

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    a519379 View commit details
    Browse the repository at this point in the history

  11. Add blenderbot tokenizer tests

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    89ddccf View commit details
    Browse the repository at this point in the history

  12. Add t5 tokenizer tests

    @xenova
    xenova committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    36ad144 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Add falcon tokenizer tests

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    4765dd6 View commit details
    Browse the repository at this point in the history

  2. Run prettier

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    fd8b9a2 View commit details
    Browse the repository at this point in the history

  3. Add ESM tokenizer tests

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    710816e View commit details
    Browse the repository at this point in the history

  4. Run unit tests in parallel

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    0d3cd30 View commit details
    Browse the repository at this point in the history

  5. Fix fuse_unk for tokenizers with byte_fallback=true but no byte f… 

    …allback tokens
    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    cc258c2 View commit details
    Browse the repository at this point in the history

  6. Add llama tokenizer unit tests

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    4798755 View commit details
    Browse the repository at this point in the history

  7. Update emoji test string names

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    c6c3ae1 View commit details
    Browse the repository at this point in the history

  8. Move whisper-specific unit tests to subfolder

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    79a7409 View commit details
    Browse the repository at this point in the history

  9. Code formatting

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    1a38804 View commit details
    Browse the repository at this point in the history

  10. Bump versions

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    dabe6ae View commit details
    Browse the repository at this point in the history

  11. [version] Update to 3.0.0-alpha.15

    @xenova
    xenova committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    54f1f21 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. Add emoji tokenizer test cases for LlamaTokenizer

    @xenova
    xenova committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    a912d79 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Attempt to fix encoder-decoder memory leak

    @xenova
    xenova committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    969d10e View commit details
    Browse the repository at this point in the history

  2. Remove unused code

    @xenova
    xenova committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    072cbbc View commit details
    Browse the repository at this point in the history

  3. Fix BertNormalizer (strip Mn unicode characters)

    @xenova
    xenova committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    14b4bd4 View commit details
    Browse the repository at this point in the history

  4. Handle ZERO WIDTH JOINER (U+200D) characters

    @xenova
    xenova committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    6797771 View commit details
    Browse the repository at this point in the history

  5. Add more spm normalization characters

    @xenova
    xenova committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    f148afd View commit details
    Browse the repository at this point in the history

  6. Add emoji unit tests for bert/t5

    @xenova
    xenova committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    ca4b5b9 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. [WebNN] Add support for specifying free_dimension_overrides in config

    @xenova
    xenova committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    113c81e View commit details
    Browse the repository at this point in the history

  2. Log warning if webnn is selected by free_dimension_overrides is not… 

    … set
    @xenova
    xenova committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    9005acc View commit details
    Browse the repository at this point in the history

  3. Fix unigram for multi-byte tokens

    @xenova
    xenova committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    682c7d0 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2024

  1. Add gemma tokenizer tests

    @xenova
    xenova committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    4a31e54 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. Allow user to specify device and dtype in config.json

    @xenova
    xenova committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    7a16065 View commit details
    Browse the repository at this point in the history

  2. Update dependency versions

    @xenova
    xenova committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    4c1d21b View commit details
    Browse the repository at this point in the history

  3. Bump versions

    @xenova
    xenova committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    3c6a95a View commit details
    Browse the repository at this point in the history

  4. [version] Update to 3.0.0-alpha.16

    @xenova
    xenova committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    ac391d2 View commit details
    Browse the repository at this point in the history