Compare commits

...

4 Commits

Author SHA1 Message Date
Yann Follet
795ac0975b
Merge 14f64dab74 into 5cd85b5e00 2024-12-22 10:34:38 +04:00
Yann Follet
14f64dab74
Merge branch 'ggerganov:master' into cuda-build-doc 2024-12-12 17:15:04 +08:00
Yann Follet
f70b5148e1 Merge branch 'cuda-build-doc' of https://github.com/YannFollet/llama.cpp into cuda-build-doc 2024-12-10 01:35:13 +00:00
Yann Follet
de1bb5a4ad Add build command for CUDA with path example 2024-12-10 01:32:10 +00:00

View File

@ -134,6 +134,12 @@ This provides GPU acceleration using an NVIDIA GPU. Make sure to have the CUDA t
cmake --build build --config Release
```
- Using `CMake` with path :
```bash
rm -rf build & /usr/local/bin/cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
/usr/local/bin/cmake --build build --config Release -j
```
The environment variable [`CUDA_VISIBLE_DEVICES`](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) can be used to specify which GPU(s) will be used.
The environment variable `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` can be used to enable unified memory in Linux. This allows swapping to system RAM instead of crashing when the GPU VRAM is exhausted. In Windows this setting is available in the NVIDIA control panel as `System Memory Fallback`.