Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debug pipeline #1566

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Debug pipeline #1566

wants to merge 6 commits into from

Conversation

ProExpertProg
Copy link
Contributor

This is currently a hack but it would be great to get a version of this into production so that we can use debug_analysis on the pipeline and pass real inputs.

ProExpertProg and others added 6 commits January 26, 2024 23:36
… cases, handle kv_cache full during prefill (#1562)

* split prep_for_generation operator

* fix logits

* update non-kv cache pipeline and tests

* add tests to address edge cases

* add condition to check of kv_cache full during prompt inference, add test to cover this case, revert debugging changes

* fix typing

* remove commented code

* remove irrelevant condition

* address PR comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants