Commit Graph

4 Commits

Author SHA1 Message Date
Georgi Gerganov
2d6c2c757c
llama : remove C++ API + reorganize common source in /common dir 2023-08-18 16:22:48 +03:00
Georgi Gerganov
660ca9bbca
llama : re-order functions 2023-08-18 14:56:36 +03:00
Georgi Gerganov
7494c78428
llama : sync gguf-llama with llama (#2613)
* llama : sync gguf-llama with llama

* tests : fix build + warnings (test-tokenizer-1 still fails)

* tests : fix wstring_convert

* convert : fix layer names

* llama : sync gguf-llama.cpp

* convert : update HF converter to new tokenizer voodoo magics
2023-08-14 21:33:33 +03:00
goerch
ec1b100720
llama : tokenizer fixes (#2549)
* Merge tokenizer fixes into the gguf branch.

* Add test vocabularies
2023-08-14 19:30:28 +03:00