llama.cpp/examples/llama.android
Pedro Cuenca b97bc3966e
llama : support Llama 3 HF conversion (#6745)
* Support Llama 3 conversion

The tokenizer is BPE.

* style

* Accept suggestion

Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com>

* llama : add llama_token_is_eog()

ggml-ci

* llama : auto-detect more EOT tokens when missing in KV data

* convert : replacing EOS token is a hack

* llama : fix codegemma EOT token + add TODOs

* llama : fix model type string for 8B model

---------

Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2024-04-21 14:50:41 +03:00
..
app llama : support Llama 3 HF conversion (#6745) 2024-04-21 14:50:41 +03:00
gradle/wrapper android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
.gitignore android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
build.gradle.kts android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
gradle.properties android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
gradlew android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
README.md android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
settings.gradle.kts android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00