root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-11-11 21:39:52 +00:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
2,638 Commits 342 Branches 2,621 Tags 300 MiB
c4a3a4ff47
Commit Graph

2 Commits

Author SHA1 Message Date
Jared Van Bortel
32c8486e1f
wpm : portable unicode tolower (#6305) 
Also use C locale for ispunct/isspace, and split unicode-data.cpp from unicode.cpp.
 2024-03-26 17:46:21 -04:00
Georgi Gerganov
83796e62bc
llama : refactor unicode stuff (#5992) 
* llama : refactor unicode stuff

ggml-ci

* unicode : names

* make : fix c++ compiler

* unicode : names

* unicode : straighten tables

* zig : fix build

* unicode : put nfd normalization behind API

ggml-ci

* swift : fix build

* unicode : add BOM

* unicode : add <cstdint>

ggml-ci

* unicode : pass as cpts as const ref
 2024-03-11 17:47:47 +02:00