You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TL;DR: empty tokens are tricky so try and get rid of them as soon as possible.
PTB trees contain empty tokens for which educe provides good basic support.
However, we still have to come up with a satisfactory solution to handle empty tokens in a proper way according to the task at hand, e.g. when aligning tokens with EDUs.
My attempts at providing a simple solution with offsets have so far failed to provide a perfectly robust solution, so I won't explain much further.
The text was updated successfully, but these errors were encountered:
TL;DR: empty tokens are tricky so try and get rid of them as soon as possible.
PTB trees contain empty tokens for which educe provides good basic support.
However, we still have to come up with a satisfactory solution to handle empty tokens in a proper way according to the task at hand, e.g. when aligning tokens with EDUs.
My attempts at providing a simple solution with offsets have so far failed to provide a perfectly robust solution, so I won't explain much further.
The text was updated successfully, but these errors were encountered: