Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[KV-Cache] Make k_scale, v_scale as attributes of self_attn using HFCache #148

Merged
merged 21 commits into from
Sep 25, 2024
Merged

[KV-Cache] Make k_scale, v_scale as attributes of self_attn using HFCache #148

merged 21 commits into from
Sep 25, 2024

Commits on Aug 28, 2024

  1. init

    @horheynm
    horheynm committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    bd63b68 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. init

    @horheynm
    horheynm committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    fbaadec View commit details
    Browse the repository at this point in the history

  2. delete unnces file

    @horheynm
    horheynm committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    01acd6c View commit details
    Browse the repository at this point in the history

  3. pass if no seed

    @horheynm
    horheynm committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    cffc1a0 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2024

  1. pre polish

    @horheynm
    horheynm committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    79a7194 View commit details
    Browse the repository at this point in the history

  2. clean up

    @horheynm
    horheynm committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    d228522 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into kv-cache

    @horheynm
    horheynm authored Aug 31, 2024
    Configuration menu
    Copy the full SHA
    40f12f0 View commit details
    Browse the repository at this point in the history

  4. post clean up, merge main

    @horheynm
    horheynm committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    e5ea57c View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2024

  1. tests and get rid of iter_named_leaf_modules, use iter_named_modules

    @horheynm
    horheynm committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    23fe917 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. comments

    @horheynm
    horheynm committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    eba35d9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into kv-cache

    @horheynm
    horheynm authored Sep 3, 2024
    Configuration menu
    Copy the full SHA
    511d04d View commit details
    Browse the repository at this point in the history

  3. pass tests

    @horheynm
    horheynm committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    b92b870 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'kv-cache' of github.com:neuralmagic/compressed-tensors … 

    …into kv-cache
    @horheynm
    horheynm committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    ab91291 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. mgoin comments

    @horheynm
    horheynm committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    86239c4 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Merge branch 'main' of github.com:neuralmagic/compressed-tensors into… 

    … kv-cache
    @horheynm
    horheynm committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    69ced96 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. frozen state for inference

    @horheynm
    horheynm committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    bea90f3 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. only compute scale, zp, do not keep quantized_key|value_states

    @horheynm
    horheynm committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    49ff395 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. do calibration if there is kv_cache_scheme

    @horheynm
    horheynm committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    b06cb5c View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. pass test

    @horheynm
    horheynm committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    89d0c58 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. Merge branch 'main' of github.com:neuralmagic/compressed-tensors into… 

    … kv-cache
    @horheynm
    horheynm committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    938243c View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. comments

    @horheynm
    horheynm committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    cf442f4 View commit details
    Browse the repository at this point in the history