Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement batch inference #30

Merged
merged 20 commits into from
Jul 12, 2023
Merged

Implement batch inference #30

merged 20 commits into from
Jul 12, 2023

Commits on Jun 29, 2023

  1. WIP: Implement batch inference

    @monatis
    monatis committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    c37aa34 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2023

  1. WIP: use broadcastable mul_mat

    @monatis
    monatis committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    09dff96 View commit details
    Browse the repository at this point in the history

  2. Batched Conv2D is working

    @monatis
    monatis committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    bfab116 View commit details
    Browse the repository at this point in the history

  3. Batched Conv2D is working

    @monatis
    monatis committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    c80a020 View commit details
    Browse the repository at this point in the history

  4. Fix concat

    @monatis
    monatis committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    502a669 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2023

  1. Batched output normalization

    @monatis
    monatis committed Jul 2, 2023
    Configuration menu
    Copy the full SHA
    734f6db View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2023

  1. Full batch inference is working

    @monatis
    monatis committed Jul 3, 2023
    Configuration menu
    Copy the full SHA
    236b755 View commit details
    Browse the repository at this point in the history

  2. Full batch inference is working

    @monatis
    monatis committed Jul 3, 2023
    Configuration menu
    Copy the full SHA
    53250a5 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Sync ggml

    @monatis
    monatis committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    3348896 View commit details
    Browse the repository at this point in the history

  2. Sync ggml

    @monatis
    monatis committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    b8d0f43 View commit details
    Browse the repository at this point in the history

  3. Sync ggml

    @monatis
    monatis committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    b1c0289 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. add multithreaded batched image preprocessing

    @monatis
    monatis committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    ee6ceca View commit details
    Browse the repository at this point in the history

  2. add multithreaded batched image preprocessing

    @monatis
    monatis committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    ef3e0c9 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2023

  1. Merge remote-tracking branch 'origin/main' into batch-inference

    @monatis
    monatis committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    51d1f08 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2023

  1. Merge branch 'main' into batch-inference2

    @monatis
    monatis committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    9d0556d View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Update batch preprocess function signature

    @monatis
    monatis committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    f573ba5 View commit details
    Browse the repository at this point in the history

  2. Implement batch inference in benchmark util

    @monatis
    monatis committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    0907b40 View commit details
    Browse the repository at this point in the history

  3. sync ggml

    @monatis
    monatis committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    8c09113 View commit details
    Browse the repository at this point in the history

  4. set n_threads as const

    @monatis
    monatis committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    8ea50bf View commit details
    Browse the repository at this point in the history

  5. Sync ggml

    @monatis
    monatis committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    5f4fa2f View commit details
    Browse the repository at this point in the history