You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, @Sakits! Thanks for sharing amazing work again.
i have a question regarding pass key retrieval task.
in passkey.py, fixed_length is multiplied with 4, (fixed_length = fixed_length * 4)
and i observe n_garbage_prefix incrases as 400, 800, 1200, ..., 34000, ...
Could you explain why fixed_length is multiplied by 4? Is there a specific reason or intention behind this?
(If multiplying with 4 is for making a fixed_length to a number of tokens, does setting fixed_length=10k equals to 40k tokens?)
Thank you!
The text was updated successfully, but these errors were encountered:
Hi, @Sakits! Thanks for sharing amazing work again.
i have a question regarding pass key retrieval task.
in
passkey.py
, fixed_length is multiplied with 4, (fixed_length = fixed_length * 4
)and i observe
n_garbage_prefix
incrases as 400, 800, 1200, ..., 34000, ...Could you explain why
fixed_length
is multiplied by 4? Is there a specific reason or intention behind this?(If multiplying with 4 is for making a
fixed_length
to anumber of tokens
, does setting fixed_length=10k equals to 40k tokens?)Thank you!
The text was updated successfully, but these errors were encountered: