root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-12-28 12:24:35 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Update test-tokenizer-random.py

Browse Source
This commit is contained in:
Robert 2024-11-12 22:24:35 -08:00 committed by GitHub
parent db26ba5b5c
commit 82a4012c2a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 0 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
tests/test-tokenizer-random.py
View File

@ -25,10 +25,6 @@ from typing_extensions import Buffer
import cffi import cffi
from transformers import AutoTokenizer, PreTrainedTokenizer from transformers import AutoTokenizer, PreTrainedTokenizer
# #
####################################################################################################
#
# Classes:
logger = logging.getLogger("test-tokenizer-random") logger = logging.getLogger("test-tokenizer-random")