mirror of
https://github.com/nomic-ai/gpt4all.git
synced 2024-10-01 01:06:10 -04:00
116 lines
3.5 KiB
Markdown
116 lines
3.5 KiB
Markdown
|
# C# GPT4All
|
|||
|
|
|
|||
|
|
This package contains a set of C# bindings around the `llmodel` C-API.
|
|||
|
|
|
|||
|
|
## Documentation
|
|||
|
|
TBD
|
|||
|
|
|
|||
|
|
## Installation
|
|||
|
|
TBD NuGet
|
|||
|
|
|
|||
|
|
## Project Structure
|
|||
|
|
```
|
|||
|
|
gpt4all-bindings/
|
|||
|
|
└── csharp
|
|||
|
|
├── Gpt4All // .NET Bindigs
|
|||
|
|
├── Gpt4All.Samples // Sample project
|
|||
|
|
├── build_win-msvc.ps1 // Native build scripts
|
|||
|
|
├── build_win-mingw.ps1
|
|||
|
|
├── build_linux.sh
|
|||
|
|
└── runtimes // [POST-BUILD] Platform-specific native libraries
|
|||
|
|
├── win-x64
|
|||
|
|
├── ...
|
|||
|
|
└── linux-x64
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Local Build Instructions
|
|||
|
|
> **Note**
|
|||
|
|
> Tested On:
|
|||
|
|
> - Windows 11 22H + VS2022 (CE) x64
|
|||
|
|
> - Linux Ubuntu x64
|
|||
|
|
> - Linux Ubuntu (WSL2) x64
|
|||
|
|
|
|||
|
|
1. Setup the repository
|
|||
|
|
2. Build the native libraries for the platform of choice (see below)
|
|||
|
|
3. Build the C# Bindings (NET6+ SDK is required)
|
|||
|
|
```
|
|||
|
|
git clone --recurse-submodules https://github.com/nomic-ai/gpt4all
|
|||
|
|
cd gpt4all/gpt4all-bindings/csharp
|
|||
|
|
```
|
|||
|
|
### Linux
|
|||
|
|
1. Setup build environment and install NET6+ SDK with the appropriate procedure for your distribution
|
|||
|
|
```
|
|||
|
|
sudo apt-get update
|
|||
|
|
sudo apt-get install -y cmake build-essential
|
|||
|
|
chmod +x ./build_linux.sh
|
|||
|
|
```
|
|||
|
|
2. `./build_linux.sh`
|
|||
|
|
3. The native libraries should be present at `.\native\linux-x64`
|
|||
|
|
|
|||
|
|
### Windows - MinGW64
|
|||
|
|
#### Additional requirements
|
|||
|
|
- [MinGW64](https://www.mingw-w64.org/)
|
|||
|
|
- CMAKE
|
|||
|
|
1. Setup
|
|||
|
|
```
|
|||
|
|
choco install mingw
|
|||
|
|
$env:Path += ";C:\ProgramData\chocolatey\lib\mingw\tools\install\mingw64\bin"
|
|||
|
|
choco install -y cmake --installargs 'ADD_CMAKE_TO_PATH=System'
|
|||
|
|
```
|
|||
|
|
2. Run the `./build_win-mingw.ps1` build script
|
|||
|
|
3. The native libraries should be present at `.\native\win-x64`
|
|||
|
|
|
|||
|
|
### Windows - MSVC
|
|||
|
|
#### Additional requirements
|
|||
|
|
- Visual Studio 2022
|
|||
|
|
1. Open a terminal using the `x64 Native Tools Command Prompt for VS 2022` (`vcvars64.bat`)
|
|||
|
|
2. Run the `./build_win-msvc.ps1` build script
|
|||
|
|
3. `libllmodel.dll` and `libllama.dll` should be present at `.\native\win-x64`
|
|||
|
|
|
|||
|
|
> **Warning**
|
|||
|
|
> If the build fails with: '**error C7555: use of designated initializers requires at least '/std:c++20'**'
|
|||
|
|
>
|
|||
|
|
> Modify `cd gpt4all/gpt4all-backends/CMakeLists.txt` adding `CXX_STANDARD_20` to `llmodel` properties.
|
|||
|
|
> ```cmake
|
|||
|
|
> set_target_properties(llmodel PROPERTIES
|
|||
|
|
> VERSION ${PROJECT_VERSION}
|
|||
|
|
> CXX_STANDARD 20 # <---- ADD THIS -----------------------
|
|||
|
|
> SOVERSION ${PROJECT_VERSION_MAJOR})
|
|||
|
|
> ```
|
|||
|
|
## C# Bindings Build Instructions
|
|||
|
|
Build the `Gpt4All` (or `Gpt4All.Samples`) projects from within VisualStudio.
|
|||
|
|
### Try the bindings
|
|||
|
|
```csharp
|
|||
|
|
using Gpt4All;
|
|||
|
|
|
|||
|
|
// load the model
|
|||
|
|
var modelFactory = new ModelFactory();
|
|||
|
|
|
|||
|
|
using var model = modelFactory.LoadModel("./path/to/ggml-gpt4all-j-v1.3-groovy.bin");
|
|||
|
|
|
|||
|
|
var input = "Name 3 Colors";
|
|||
|
|
|
|||
|
|
// request a prediction
|
|||
|
|
var result = await model.GetStreamingPredictionAsync(
|
|||
|
|
input,
|
|||
|
|
PredictRequestOptions.Defaults);
|
|||
|
|
|
|||
|
|
// asynchronously print the tokens as soon as they are produces by the model
|
|||
|
|
await foreach(var token in result.GetPredictionStreamingAsync())
|
|||
|
|
{
|
|||
|
|
Console.Write(token);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
Output:
|
|||
|
|
```
|
|||
|
|
gptj_model_load: loading model from 'ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
|
|||
|
|
gptj_model_load: n_vocab = 50400
|
|||
|
|
[...TRUNCATED...]
|
|||
|
|
gptj_model_load: ggml ctx size = 5401.45 MB
|
|||
|
|
gptj_model_load: kv self size = 896.00 MB
|
|||
|
|
gptj_model_load: ................................... done
|
|||
|
|
gptj_model_load: model size = 3609.38 MB / num tensors = 285
|
|||
|
|
|
|||
|
|
Black, Blue and White
|
|||
|
|
```
|