text-generation-webui/docs/What Works.md

## What Works

| Loader         | Loading 1 LoRA | Loading 2 or more LoRAs | Training LoRAs | Multimodal extension | Perplexity evaluation |
|----------------|----------------|-------------------------|----------------|----------------------|-----------------------|
| Transformers   |       ✅       |           ✅***            |       ✅*       |          ✅          |           ✅          |
| ExLlama_HF     |       ✅       |           ❌            |       ❌       |          ❌          |           ✅          |
| ExLlamav2_HF   |       ✅       |           ✅            |       ❌       |          ❌          |           ✅          |
| ExLlama        |       ✅       |           ❌            |       ❌       |          ❌          |           use ExLlama_HF      |
| ExLlamav2      |       ✅       |           ✅            |       ❌       |          ❌          |           use ExLlamav2_HF    |
| AutoGPTQ       |       ✅       |           ❌            |       ❌       |          ✅          |           ✅          |
| GPTQ-for-LLaMa |       ✅**       |           ✅***            |       ✅       |          ✅          |           ✅          |
| llama.cpp      |       ❌       |           ❌            |       ❌       |          ❌          |           use llamacpp_HF    |
| llamacpp_HF    |       ❌       |           ❌            |       ❌       |          ❌          |           ✅          |
| ctransformers  |       ❌       |           ❌            |       ❌       |          ❌          |           ❌          |
| AutoAWQ        |       ?        |           ❌            |       ?       |          ?          |           ✅          |

❌ = not implemented

✅ = implemented

\* Training LoRAs with GPTQ models also works with the Transformers loader. Make sure to check "auto-devices" and "disable_exllama" before loading the model.

\*\* Requires the monkey-patch. The instructions can be found [here](https://github.com/oobabooga/text-generation-webui/wiki/08-%E2%80%90-Additional-Tips#using-loras-with-gptq-for-llama).

\*\*\* Multi-LoRA in PEFT is tricky and the current implementation does not work reliably in all cases.
Add a proper documentation (#3885) 2023-10-21 18:15:54 -04:00			`## What Works`

			`\| Loader \| Loading 1 LoRA \| Loading 2 or more LoRAs \| Training LoRAs \| Multimodal extension \| Perplexity evaluation \|`
			`\|----------------\|----------------\|-------------------------\|----------------\|----------------------\|-----------------------\|`
Update docs/What Works.md 2023-11-19 10:57:16 -05:00			`\| Transformers \| ✅ \| ✅*** \| ✅* \| ✅ \| ✅ \|`
Add a proper documentation (#3885) 2023-10-21 18:15:54 -04:00			`\| ExLlama_HF \| ✅ \| ❌ \| ❌ \| ❌ \| ✅ \|`
			`\| ExLlamav2_HF \| ✅ \| ✅ \| ❌ \| ❌ \| ✅ \|`
			`\| ExLlama \| ✅ \| ❌ \| ❌ \| ❌ \| use ExLlama_HF \|`
			`\| ExLlamav2 \| ✅ \| ✅ \| ❌ \| ❌ \| use ExLlamav2_HF \|`
			`\| AutoGPTQ \| ✅ \| ❌ \| ❌ \| ✅ \| ✅ \|`
Update docs/What Works.md 2023-11-19 10:57:16 -05:00			`\| GPTQ-for-LLaMa \| ✅ \| ✅* \| ✅ \| ✅ \| ✅ \|`
Add a proper documentation (#3885) 2023-10-21 18:15:54 -04:00			`\| llama.cpp \| ❌ \| ❌ \| ❌ \| ❌ \| use llamacpp_HF \|`
			`\| llamacpp_HF \| ❌ \| ❌ \| ❌ \| ❌ \| ✅ \|`
			`\| ctransformers \| ❌ \| ❌ \| ❌ \| ❌ \| ❌ \|`
			`\| AutoAWQ \| ? \| ❌ \| ? \| ? \| ✅ \|`

			`❌ = not implemented`

			`✅ = implemented`

			`\* Training LoRAs with GPTQ models also works with the Transformers loader. Make sure to check "auto-devices" and "disable_exllama" before loading the model.`

			`\\ Requires the monkey-patch. The instructions can be found [here](https://github.com/oobabooga/text-generation-webui/wiki/08-%E2%80%90-Additional-Tips#using-loras-with-gptq-for-llama).`
Update docs/What Works.md 2023-11-19 10:57:16 -05:00
			`\\\* Multi-LoRA in PEFT is tricky and the current implementation does not work reliably in all cases.`