Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ Speculative decoding ] Support different tokenizers for draft and main models #1617

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

[ Speculative decoding ] Support different tokenizers for draft and main models #1617

wants to merge 11 commits into from
+197 −10

Conversation

iefode
Copy link
Contributor

@iefode iefode commented Jan 22, 2025

Ticket:

@iefode iefode marked this pull request as draft January 22, 2025 11:50
@github-actions github-actions bot added the category: speculative decoding Speculative decoding label Jan 22, 2025
@github-actions github-actions bot added the category: GHA CI based on Github actions label Jan 22, 2025
@iefode iefode marked this pull request as ready for review January 22, 2025 13:59
@iefode iefode requested a review from ilya-lavrenov January 22, 2025 13:59
@ilya-lavrenov
Copy link
Contributor

If we don't see perf gain, why do we need to merge it?

@github-actions github-actions bot added category: sampling Sampling / Decoding algorithms category: samples GenAI samples labels Jan 23, 2025
@github-actions github-actions bot removed the category: samples GenAI samples label Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ilya-lavrenov ilya-lavrenov Awaiting requested review from ilya-lavrenov

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
category: GHA CI based on Github actions category: sampling Sampling / Decoding algorithms category: speculative decoding Speculative decoding
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@iefode @ilya-lavrenov