Code

WIP: Remove torchtext dependency#4038

Code

Closed

mhabedank wants to merge 4 commits intoludwig-ai:masterludwig-ai/ludwig:masterfrom mhabedank:remove_torchtextmhabedank/ludwig:remove_torchtextCopy head branch name to clipboard

+91-375

Commits

Commits on Oct 25, 2024

• Removed torchtext from NGramTokenizer

  

  m.habedank

  committed
  c27d123

Commits on Oct 26, 2024

• Refactored SentencePieceTokenizer

  Show description for 8e90e70
  

  m.habedank

  committed
  8e90e70

Commits on Oct 28, 2024

• removed torchtext

  

  m.habedank

  committed
  92a3ec0

Commits on Oct 29, 2024

• Rewrote tests

  

  m.habedank

  committed
  01f308e