You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this (very) particular case, we had two equivalent tokenizations of all the words: as we saw earlier, for example, "pug" could be tokenized ["p", "ug"] with the same score. Thus, removing the "pu" token from the vocabulary will give the exact same loss.
But as the following list from the link shows that "pun" needs "pu" and "n". If "pu" token is removed, the score for "pun" will change. So only if "pun" has the same score after "pu" is removed, the loss does not change.
https://huggingface.co/learn/nlp-course/chapter6/7?fw=pt says:
But as the following list from the link shows that "pun" needs "pu" and "n". If "pu" token is removed, the score for "pun" will change. So only if "pun" has the same score after "pu" is removed, the loss does not change.
The text was updated successfully, but these errors were encountered: