root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-11-14 06:49:54 +00:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity

Created Feature matrix (markdown)

Romain D 2024-03-05 12:36:14 +00:00
parent 26c482f18e
commit 27130d440e
1 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
Feature-matrix.md Normal file

@ -0,0 +1,11 @@
# llama.cpp feature matrix
| | **CPU (AVX2)** | **CPU (ARM NEON)** | **Metal** | **cuBLAS** | **rocBLAS** | **SYCL** | **CLBlast** | **Vulkan** | **Kompute** |
|:--------------------:|:--------------:|:------------------:|:---------:|:----------:|:----------------:|:--------:|:-----------:|:----------:|:-----------:|
| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🚫 |
| **I-quants** | ✅ (SLOW) | ✅ (SLOW) | ✅ (SLOW) | ✅ | ✅ | Partial¹ | 🚫 | 🚫 | 🚫 |
| **Multi-GPU** | N/A | N/A | N/A | ✅ | ❓ | 🚫 | ❓ | ✅ | ❓ |
| **K cache quants** | ✅ | ❓ | ❓ | ✅ | Only q8_0 (SLOW) | ❓ | ✅ | 🚫 | 🚫 |
| **MoE architecture** | ✅ | ❓ | ✅ | ✅ | ✅ | ❓ | Only -ngl 0 | 🚫 | 🚫 |
* ¹: IQ3_S and IQ1_S, see #5886