diff --git a/docs/llama.cpp-models.md b/docs/llama.cpp-models.md index 153f70af..be368fd3 100644 --- a/docs/llama.cpp-models.md +++ b/docs/llama.cpp-models.md @@ -12,9 +12,9 @@ Follow the instructions in the llama.cpp README to generate the `ggml-model.bin` ## GPU offloading -Enabled with the `--n-gpu-layers` parameter. If you have enough VRAM, use a high number like `--n-gpu-layers 200000` to offload all layers to the GPU. +Enabled with the `--n-gpu-layers` parameter. If you have enough VRAM, use a high number like `--n-gpu-layers 200000` to offload all layers to the GPU. Otherwise, start with a low value like `--n-gpu-layers 10` and gradually increase it until you run out of memory. -Note that you need to manually install `llama-cpp-python` with GPU support. To do that: +To use this feature, you need to manually compile and install `llama-cpp-python` with GPU support. #### Linux