Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add PagedAttention #13

Merged
merged 54 commits into from
Dec 3, 2023
Merged

Add PagedAttention #13

merged 54 commits into from
Dec 3, 2023

Commits on Nov 25, 2023

  1. Add new paged attention basic layout

    @EricLBuehler
    EricLBuehler committed Nov 25, 2023
    Configuration menu
    Copy the full SHA
    ec98e4f View commit details
    Browse the repository at this point in the history

  2. Update readme

    @EricLBuehler
    EricLBuehler committed Nov 25, 2023
    Configuration menu
    Copy the full SHA
    f6dc172 View commit details
    Browse the repository at this point in the history

  3. Add commend

    @EricLBuehler
    EricLBuehler committed Nov 25, 2023
    Configuration menu
    Copy the full SHA
    859338c View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2023

  1. Add calls to vllm paged attention impl via FFI

    @EricLBuehler
    EricLBuehler committed Nov 26, 2023
    Configuration menu
    Copy the full SHA
    680df93 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Debug state

    @EricLBuehler
    EricLBuehler committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    15a8194 View commit details
    Browse the repository at this point in the history

  2. Use cc .std, .opt_level, .include, still debug

    @EricLBuehler
    EricLBuehler committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    da1cd99 View commit details
    Browse the repository at this point in the history

  3. Use setup.py

    @EricLBuehler
    EricLBuehler committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    c0c9e0e View commit details
    Browse the repository at this point in the history

  4. Better install

    @EricLBuehler
    EricLBuehler committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    268a963 View commit details
    Browse the repository at this point in the history

  5. Add build.rs

    @EricLBuehler
    EricLBuehler committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    fec3e9e View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Add some installation instructions

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    4c21697 View commit details
    Browse the repository at this point in the history

  2. Update installation instructions

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    2e1575f View commit details
    Browse the repository at this point in the history

  3. Add pytorch installation method

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    e382d8e View commit details
    Browse the repository at this point in the history

  4. Update pytorch installation method

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    11cfa2d View commit details
    Browse the repository at this point in the history

  5. Update pytorch installation method

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    7e61a17 View commit details
    Browse the repository at this point in the history

  6. Update pytorch installation method

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    be6bf5f View commit details
    Browse the repository at this point in the history

  7. Update pytorch installation method

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    56a14c2 View commit details
    Browse the repository at this point in the history

  8. Update readme

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    1d4700a View commit details
    Browse the repository at this point in the history

  9. Add the Compiling PagedAttention CUDA kernels section

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    75cd454 View commit details
    Browse the repository at this point in the history

  10. Fix typo

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    8eeb250 View commit details
    Browse the repository at this point in the history

  11. Make directions explicit

    @EricLBuehler
    EricLBuehler committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    e453c05 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Only compile specific kernel

    @EricLBuehler
    EricLBuehler committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    1df0f81 View commit details
    Browse the repository at this point in the history

  2. Small update

    @EricLBuehler
    EricLBuehler committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    32a72c0 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Add basic framework of PagedAttention.forward

    @EricLBuehler
    EricLBuehler committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    0746cc5 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Add initial BlockDiagonalCausalMask

    @EricLBuehler
    EricLBuehler committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    3dfb563 View commit details
    Browse the repository at this point in the history

  2. temporary save

    @EricLBuehler
    EricLBuehler committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    c7cfca9 View commit details
    Browse the repository at this point in the history

  3. Implement attention bias

    @EricLBuehler
    EricLBuehler committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    97e91c5 View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2023

  1. Refactor, mostly implement PagedAttention

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    eba2ce8 View commit details
    Browse the repository at this point in the history

  2. Clippy fixes

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    ad0dfd6 View commit details
    Browse the repository at this point in the history

  3. Implement .copy_ manually

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    75105db View commit details
    Browse the repository at this point in the history

  4. Hide memory-efficient-attention behind a feature flag

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    d76f42e View commit details
    Browse the repository at this point in the history

  5. Describe feature flags

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    fbbe865 View commit details
    Browse the repository at this point in the history

  6. TODO: multi_query_kv_attention

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    9fe413f View commit details
    Browse the repository at this point in the history

  7. TODO: memory_efficient_attention_forward

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    3f57043 View commit details
    Browse the repository at this point in the history

  8. Update readme

    @EricLBuehler
    EricLBuehler committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    9ac36a5 View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2023

  1. Add scaled dot product attn

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    3a850bc View commit details
    Browse the repository at this point in the history

  2. Add flash_attn support

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    7e2e4e6 View commit details
    Browse the repository at this point in the history

  3. Remove all lossy 'as' conversions

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    f0d9832 View commit details
    Browse the repository at this point in the history

  4. Update readme

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    d946898 View commit details
    Browse the repository at this point in the history

  5. Fix typos

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    f40f9c9 View commit details
    Browse the repository at this point in the history

  6. Update readme

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    0d9f3b5 View commit details
    Browse the repository at this point in the history

  7. Update readme

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    85de7d5 View commit details
    Browse the repository at this point in the history

  8. Add minimal CUDA install script for CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    8eff99a View commit details
    Browse the repository at this point in the history

  9. Debugging CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    173ca49 View commit details
    Browse the repository at this point in the history

  10. Debugging CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    82ce3da View commit details
    Browse the repository at this point in the history

  11. Debugging CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    6d81221 View commit details
    Browse the repository at this point in the history

  12. Debugging CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    40a8589 View commit details
    Browse the repository at this point in the history

  13. Debugging CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    c8145b9 View commit details
    Browse the repository at this point in the history

  14. Add no-paged-attention for CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    9a26979 View commit details
    Browse the repository at this point in the history

  15. Add no-paged-attention for CI

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    a3e2d90 View commit details
    Browse the repository at this point in the history

  16. Update CI to not use CUDA

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    eb25e82 View commit details
    Browse the repository at this point in the history

  17. Update CI to not use CUDA

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    323da9f View commit details
    Browse the repository at this point in the history

  18. Split up attn_bias

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    cb9c583 View commit details
    Browse the repository at this point in the history

  19. Remove link from Contributing

    @EricLBuehler
    EricLBuehler committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    7e6607f View commit details
    Browse the repository at this point in the history

  20. Merge branch 'master' into paged_attention

    @EricLBuehler
    EricLBuehler authored Dec 3, 2023
    Configuration menu
    Copy the full SHA
    416bd19 View commit details
    Browse the repository at this point in the history