AI/turbopilot
1
0
Fork 0
mirror of https://github.com/ravenscroftj/turbopilot.git synced 2024-06-25 22:02:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU
4 Commits 14 Branches 8 Tags 3.1 MiB
C++ 97.8%
Python 1.7%
CMake 0.5%
f8713c089b
Go to file
James Ravenscroft f8713c089b add readme
2023-04-09 17:56:27 +01:00
ggml@a25b0a84ab add ggml 2023-04-09 17:49:03 +01:00
.gitmodules add ggml 2023-04-09 17:49:03 +01:00
convert-codegen-to-ggml.py add conversion script 2023-04-09 17:49:42 +01:00
README.md add readme 2023-04-09 17:56:27 +01:00

README.md

TurboPilot

TurboPilot is a super fast fauxpilot clone which uses the library behind llama.cpp to run huge 6 Billion Parameter Salesforce Codegen models in 2GiB of RAM.

Getting Started

git clone https://github.com/ravenscroftj/turbopilot
git submodule init
cd ggml
mkdir build
cd build
cmake ..
make codegen codegen-quantize

Getting The Models

Start by downloading either the 2B or 6B GPT-J versions of CodeGen.

Convert The Model

python convert-codegen-to-ggml.py ./codegen-6B-multi-gptj 0

Quantize the model

./bin/codegen-quantize ../../codegen-6B-multi-gptj/ggml-model-f32.bin ../../codegen-6B-multi-gptj/ggml-model-quant.bin 2

Run the model

./bin/codegen -t 6 -m ../../codegen-6B-multi-gptj/ggml-model-quant.bin -p "def main("