add readme

This commit is contained in:
James Ravenscroft 2023-04-09 17:56:27 +01:00
parent f76105eecb
commit f8713c089b

View File

@ -11,7 +11,7 @@ cd ggml
mkdir build mkdir build
cd build cd build
cmake .. cmake ..
make codegen make codegen codegen-quantize
``` ```
## Getting The Models ## Getting The Models
@ -24,4 +24,14 @@ Start by downloading either the [2B](https://huggingface.co/moyix/codegen-2B-mul
python convert-codegen-to-ggml.py ./codegen-6B-multi-gptj 0 python convert-codegen-to-ggml.py ./codegen-6B-multi-gptj 0
``` ```
## Build GGML ## Quantize the model
```bash
./bin/codegen-quantize ../../codegen-6B-multi-gptj/ggml-model-f32.bin ../../codegen-6B-multi-gptj/ggml-model-quant.bin 2
```
## Run the model
```bash
./bin/codegen -t 6 -m ../../codegen-6B-multi-gptj/ggml-model-quant.bin -p "def main("
```