mirror of
https://github.com/ggerganov/llama.cpp.git
synced 2024-11-11 21:39:52 +00:00
b853d45601
* detect NUMA systems and pin work threads to nodes (linux) * disable mmap prefetch/readahead for NUMA systems * avoid sending finalize op to thread pool if it does nothing * silence robot * fix args * make --numa a param * recommendation that n_nodes evenly divide n_threads did not warrant such aggressive enforcement * lower synchronization overhead * statically allocate * move numa state to g_state * add description for --numa * ggml : minor style changes * ggml : minor style + try fix sanitizer build * llama : allow to initialize backend with NUMA support * llama : avoid ggml include in llama-util.h * ggml : style / formatting * ggml : fix handling of ops with n_threads > n_tasks > 1 * server : utilize numa parameter --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> |
||
|---|---|---|
| .. | ||
| baby-llama | ||
| benchmark | ||
| embedding | ||
| jeopardy | ||
| main | ||
| metal | ||
| perplexity | ||
| quantize | ||
| quantize-stats | ||
| save-load-state | ||
| server | ||
| simple | ||
| train-text-from-scratch | ||
| alpaca.sh | ||
| chat-13B.bat | ||
| chat-13B.sh | ||
| chat-persistent.sh | ||
| chat-vicuna.sh | ||
| chat.sh | ||
| CMakeLists.txt | ||
| common.cpp | ||
| common.h | ||
| gpt4all.sh | ||
| Miku.sh | ||
| reason-act.sh | ||