Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ggml/examples: add backend support for numerical optimization #949

Merged
merged 36 commits into from
Sep 20, 2024
Merged

ggml/examples: add backend support for numerical optimization #949

merged 36 commits into from
Sep 20, 2024

Commits on Sep 13, 2024

  1. CUDA eval works

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a3c341d View commit details
    Browse the repository at this point in the history

  2. stochastic gradient descent op

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    03c5b72 View commit details
    Browse the repository at this point in the history

  3. Adam except decay

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    1b2a4e5 View commit details
    Browse the repository at this point in the history

  4. CUDA CROSS_ENTROPY_LOSS_BACK

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    06bf41b View commit details
    Browse the repository at this point in the history

  5. CUDA mnist-fc training works

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    fd31a57 View commit details
    Browse the repository at this point in the history

  6. backend CLI arg

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    dacab7b View commit details
    Browse the repository at this point in the history

  7. refactor gguf load

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c7adfba View commit details
    Browse the repository at this point in the history

  8. remove sched from opt_step_adam

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7094b55 View commit details
    Browse the repository at this point in the history

  9. implement l1 regularization (weight decay)

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    2040338 View commit details
    Browse the repository at this point in the history

  10. extra call to add optimizer

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    5d687c0 View commit details
    Browse the repository at this point in the history

  11. initialize gradients with ggml_graph_reset

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    16cb38f View commit details
    Browse the repository at this point in the history

  12. gradient accumulation

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    3e93361 View commit details
    Browse the repository at this point in the history

  13. increment iter per eval instead of epoch

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    14d19f6 View commit details
    Browse the repository at this point in the history

  14. adjust backend interfaces

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7dd2c94 View commit details
    Browse the repository at this point in the history

  15. fix ggml_graph_reset without backend

    @JohannesGaessler
    JohannesGaessler committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7e12e32 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2024

  1. fix ggml graph export/import

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    c37fb9a View commit details
    Browse the repository at this point in the history

  2. fixup

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    46722f9 View commit details
    Browse the repository at this point in the history

  3. rename

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    544d86a View commit details
    Browse the repository at this point in the history

  4. revert ggml_opt changes

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    1c0a888 View commit details
    Browse the repository at this point in the history

  5. more general CUDA repeat_back

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    2472d51 View commit details
    Browse the repository at this point in the history

  6. update documentation, fix CNN

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    6e64c8c View commit details
    Browse the repository at this point in the history

  7. validation split

    @JohannesGaessler
    JohannesGaessler committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    bde1131 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2024

  1. add clarifying comment

    @JohannesGaessler
    JohannesGaessler committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    7b90357 View commit details
    Browse the repository at this point in the history

  2. optimize PyTorch training

    @JohannesGaessler
    JohannesGaessler committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    d813691 View commit details
    Browse the repository at this point in the history

  3. adjust buffer size, thread count

    @JohannesGaessler
    JohannesGaessler committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    cf0f60e View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. fix 0.0f validation split

    @JohannesGaessler
    JohannesGaessler committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    d07b6e3 View commit details
    Browse the repository at this point in the history

  2. Update examples/mnist/mnist-common.cpp 

    Co-authored-by: Georgi Gerganov <[email protected]>
    @JohannesGaessler @ggerganov
    JohannesGaessler and ggerganov authored Sep 16, 2024
    Configuration menu
    Copy the full SHA
    c1d13df View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. fix gradient accumulation

    @JohannesGaessler
    JohannesGaessler committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    478472b View commit details
    Browse the repository at this point in the history

  2. tensor flag for accumulators -> tensor hash set

    @JohannesGaessler
    JohannesGaessler committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    1d0e3ca View commit details
    Browse the repository at this point in the history

  3. Update include/ggml.h 

    Co-authored-by: slaren <[email protected]>
    @JohannesGaessler @slaren
    JohannesGaessler and slaren authored Sep 17, 2024
    Configuration menu
    Copy the full SHA
    db89f4e View commit details
    Browse the repository at this point in the history

  4. Update tests/test-backend-ops.cpp 

    Co-authored-by: slaren <[email protected]>
    @JohannesGaessler @slaren
    JohannesGaessler and slaren authored Sep 17, 2024
    Configuration menu
    Copy the full SHA
    02e1a37 View commit details
    Browse the repository at this point in the history

  5. Update tests/test-backend-ops.cpp 

    Co-authored-by: slaren <[email protected]>
    @JohannesGaessler @slaren
    JohannesGaessler and slaren authored Sep 17, 2024
    Configuration menu
    Copy the full SHA
    dbcd543 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. fix test prints

    @JohannesGaessler
    JohannesGaessler committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    594a143 View commit details
    Browse the repository at this point in the history

  2. Update src/ggml-backend.c 

    Co-authored-by: Georgi Gerganov <[email protected]>
    @JohannesGaessler @ggerganov
    JohannesGaessler and ggerganov authored Sep 18, 2024
    Configuration menu
    Copy the full SHA
    0642b69 View commit details
    Browse the repository at this point in the history

  3. better CUDA support for noncontiguous out_prod

    @JohannesGaessler
    JohannesGaessler committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    00b43cf View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. add comment

    @JohannesGaessler
    JohannesGaessler committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    461b648 View commit details
    Browse the repository at this point in the history