From fb5ef235646f0f27d80af7f9c441d51ec31693cf Mon Sep 17 00:00:00 2001 From: James Ravenscroft Date: Mon, 10 Apr 2023 09:21:58 +0100 Subject: [PATCH] added acknowledgement of crow --- README.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 373f73f..a758db6 100644 --- a/README.md +++ b/README.md @@ -20,15 +20,21 @@ make codegen codegen-quantize ## Getting The Models +### Direct Access + +You can download the pre-converted, pre-quantized models from [Google Drive](https://drive.google.com/drive/folders/1wFy1Y0pqoK23ZeMWWCp8evxWOJQVdaGh?usp=sharing) + +### Convert The Models Yourself + Start by downloading either the [2B](https://huggingface.co/moyix/codegen-2B-multi-gptj) or [6B](https://huggingface.co/moyix/codegen-6B-multi-gptj) GPT-J versions of CodeGen. -## Convert The Model +#### Convert The Model ```bash python convert-codegen-to-ggml.py ./codegen-6B-multi-gptj 0 ``` -## Quantize the model +#### Quantize the model ```bash ./bin/codegen-quantize ../../codegen-6B-multi-gptj/ggml-model-f32.bin ../../codegen-6B-multi-gptj/ggml-model-quant.bin 2 @@ -47,4 +53,5 @@ python convert-codegen-to-ggml.py ./codegen-6B-multi-gptj 0 - It was completely inspired by [fauxpilot](https://github.com/fauxpilot/fauxpilot) which I did experiment with for a little while but wanted to try to make the models work without a GPU - The frontend of the project is powered by [Venthe's vscode-fauxpilot plugin](https://github.com/Venthe/vscode-fauxpilot) - The project uses the [Salesforce Codegen](https://github.com/salesforce/CodeGen) models. -- Thanks to [Moyix](https://huggingface.co/moyix) for his work on converting the Salesforce models to run in a GPT-J architecture. Not only does this [confer some speed benefits](https://gist.github.com/moyix/7896575befbe1b99162ccfec8d135566) but it also made it much easier for me to port the models to GGML using the [existing gpt-j example code](https://github.com/ggerganov/ggml/tree/master/examples/gpt-j) \ No newline at end of file +- Thanks to [Moyix](https://huggingface.co/moyix) for his work on converting the Salesforce models to run in a GPT-J architecture. Not only does this [confer some speed benefits](https://gist.github.com/moyix/7896575befbe1b99162ccfec8d135566) but it also made it much easier for me to port the models to GGML using the [existing gpt-j example code](https://github.com/ggerganov/ggml/tree/master/examples/gpt-j) +- The model server uses [CrowCPP](https://crowcpp.org/master/) to serve suggestions. \ No newline at end of file