llama.cpp/examples/embedding
2024-07-02 12:18:10 -04:00
..
CMakeLists.txt build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809) 2024-06-13 00:41:52 +01:00
embedding.cpp embedding : more cli arguments (#7458) 2024-06-24 08:30:24 +03:00
README.md Removes multiple newlines at the end of files that is breaking the editorconfig step of CI. (#8258) 2024-07-02 12:18:10 -04:00

llama.cpp/example/embedding

This example demonstrates generate high-dimensional embedding vector of a given text with llama.cpp.

Quick Start

To get started right away, run the following command, making sure to use the correct path for the model you have:

Unix-based systems (Linux, macOS, etc.):

./llama-embedding -m ./path/to/model --log-disable -p "Hello World!" 2>/dev/null

Windows:

llama-embedding.exe -m ./path/to/model --log-disable -p "Hello World!" 2>$null

The above command will output space-separated float values.

extra parameters

--embd-normalize integer

integer description formula
-1 none
0 max absolute int16 \Large{{32760 * x_i} \over\max \lvert x_i\rvert}
1 taxicab \Large{x_i \over\sum \lvert x_i\rvert}
2 euclidean (default) \Large{x_i \over\sqrt{\sum x_i^2}}
>2 p-norm \Large{x_i \over\sqrt[p]{\sum \lvert x_i\rvert^p}}

--embd-output-format 'string'

'string' description
'' same as before (default)
'array' single embeddings [[x_1,...,x_n]]
multiple embeddings [[x_1,...,x_n],[x_1,...,x_n],...,[x_1,...,x_n]]
'json' openai style
'json+' add cosine similarity matrix

--embd-separator "string"

"string"
"\n" (default)
"<#embSep#>" for exemple
"<#sep#>" other exemple

examples

Unix-based systems (Linux, macOS, etc.):

./embedding -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --embd-separator '<#sep#>' --embd-normalize 2  --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null

Windows:

embedding.exe -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --embd-separator '<#sep#>' --embd-normalize 2  --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null