Commit Graph

2 Commits

Author SHA1 Message Date
Georgi Gerganov
7494c78428
llama : sync gguf-llama with llama (#2613)
* llama : sync gguf-llama with llama

* tests : fix build + warnings (test-tokenizer-1 still fails)

* tests : fix wstring_convert

* convert : fix layer names

* llama : sync gguf-llama.cpp

* convert : update HF converter to new tokenizer voodoo magics
2023-08-14 21:33:33 +03:00
goerch
ec1b100720
llama : tokenizer fixes (#2549)
* Merge tokenizer fixes into the gguf branch.

* Add test vocabularies
2023-08-14 19:30:28 +03:00