Skip to content

Commit

Permalink
tests : fix test-tokenizer-0.sh
Browse files Browse the repository at this point in the history
  • Loading branch information
Georgi Gerganov committed May 28, 2024
1 parent 8b99e2a commit edc2943
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions tests/test-tokenizer-0.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ printf "Tokenizing using (cpp) llama.cpp ...\n"
cat /tmp/test-tokenizer-0-$name-py.log | grep "tokenized in"
cat /tmp/test-tokenizer-0-$name-cpp.log | grep "tokenized in"

set +e

diff $input.tok $input.tokcpp > /dev/null 2>&1

if [ $? -eq 0 ]; then
Expand Down

0 comments on commit edc2943

Please sign in to comment.