Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Text Generation][V2] NonKVCachePipeline #1417

Closed
wants to merge 7 commits into from
Closed

[Text Generation][V2] NonKVCachePipeline #1417

wants to merge 7 commits into from

Commits on Dec 18, 2023

  1. Pipelines Refactor - Initial Impl (#1287)

    @bfineran @dbogunowicz
    bfineran authored and dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    aa18bac View commit details
    Browse the repository at this point in the history

  2. [Pipeline Refactor] Additional functionality, engine operator, linear… 

    … router and image classification pipeline/operators/example (#1325)

* initial functionality and working example with image classification

* remove testing image

* update args

* initial functionality and working example with image classification

* remove testing image

* pr comments

* defines schemas for operators and test

* add image classification test, PR comments

* fix input/output handling in pipeline and operator base classes to be more generic; remove context

* add additional operator input message

* typo fix
    @dsikka @dbogunowicz
    dsikka authored and dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    4e5f2af View commit details
    Browse the repository at this point in the history

  3. [v2] EngineOperator updates to make continuous batching easier (#1371) 

    * [v2] EngineOperator updates to make continuous batching easier

* test fixes
    @bfineran @dbogunowicz
    bfineran authored and dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    59fb587 View commit details
    Browse the repository at this point in the history

  4. [Pipeline Refactor] Update routes, text generation initial functional… 

    …ity (#1348)

* initial functionality and working example with image classification

* remove testing image

* rebase fixes

* initial functionality and working example with image classification

* text gen

* updates func

* prompt inference, initial functionality

* remove image; update state docstring

* Fix typo

* add todo for split/join

* remove context, clean-up args, remove prefill_preprocess_operaator

* fix docstrings
    @dsikka @dbogunowicz
    dsikka authored and dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    d54ef26 View commit details
    Browse the repository at this point in the history

  5. [Pipeline Refactor] Additional Operators, Route update and completed … 

    …generation functionality (#1356)

* initial functionality and working example with image classification

* remove testing image

* rebase fixes

* initial functionality and working example with image classification

* text gen

* updates func

* prompt inference, initial functionality

* remove image; update state docstring

* Fix typo

* add todo for split/join

* remove context, clean-up args, remove prefill_preprocess_operaator

* fix docstrings

* initial functionality and working example with image classification

* updates func

* prompt inference, initial functionality

* finish generation operators and update routes

* further breakdown operators

* add operators

* fix can_operate condition

* update can_operate to not rely on the inference_state

* rebase + update

* fix condition

* fix capacity settting again

* typo fixes
    @dsikka @dbogunowicz
    dsikka authored and dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    5ee36d6 View commit details
    Browse the repository at this point in the history

  6. initial commit

    @dbogunowicz
    dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    7f3eb12 View commit details
    Browse the repository at this point in the history

  7. ready for reviews

    @dbogunowicz
    dbogunowicz committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    0901a01 View commit details
    Browse the repository at this point in the history