gpt4all/gpt4all-bindings/python/docs/gpt4all_cli.md

GPT4All CLI

The GPT4All command-line interface (CLI) is a Python script which is built on top of the Python bindings (repository) and the typer package. The source code, README, and local build instructions can be found here.

Installation

The Short Version

The CLI is a Python script called app.py. If you're already familiar with Python best practices, the short version is to download app.py into a folder of your choice, install the two required dependencies with some variant of:

pip install gpt4all typer

Then run it with a variant of:

python app.py repl

In case you're wondering, REPL is an acronym for read-eval-print loop.

Recommendations & The Long Version

Especially if you have several applications/libraries which depend on Python, to avoid descending into dependency hell at some point, you should:

• Consider to always install into some kind of virtual environment.
• On a Unix-like system, don't use sudo for anything other than packages provided by the system package manager, i.e. never with pip.

There are several ways and tools available to do this, so below are descriptions on how to install with a virtual environment (recommended) or a user installation on all three main platforms.

Different platforms can have slightly different ways to start the Python interpreter itself.

Note: Typer has an optional dependency for more fanciful output. If you want that, replace typer with typer[all] in the pip-install instructions below.

Virtual Environment Installation

You can name your virtual environment folder for the CLI whatever you like. In the following, gpt4all-cli is used throughout.

macOS

There are at least three ways to have a Python installation on macOS, and possibly not all of them provide a full installation of Python and its tools. When in doubt, try the following:

python3 -m venv --help
python3 -m pip --help

Both should print the help for the venv and pip commands, respectively. If they don't, consult the documentation of your Python installation on how to enable them, or download a separate Python variant, for example try an unified installer package from python.org.

Once ready, do:

python3 -m venv gpt4all-cli
. gpt4all-cli/bin/activate
python3 -m pip install gpt4all typer

Windows

Download the official installer from python.org if Python isn't already present on your system.

A Windows installation should already provide all the components for a virtual environment. Run:

py -3 -m venv gpt4all-cli
gpt4all-cli\Scripts\activate
py -m pip install gpt4all typer

Linux

On Linux, a Python installation is often split into several packages and not all are necessarily installed by default. For example, on Debian/Ubuntu and derived distros, you will want to ensure their presence with the following:

sudo apt-get install python3-venv python3-pip

The next steps are similar to the other platforms:

python3 -m venv gpt4all-cli
. gpt4all-cli/bin/activate
python3 -m pip install gpt4all typer

On other distros, the situation might be different. Especially the package names can vary a lot. You'll have to look it up in the documentation, software directory, or package search.

User Installation

macOS

There are at least three ways to have a Python installation on macOS, and possibly not all of them provide a full installation of Python and its tools. When in doubt, try the following:

python3 -m pip --help

That should print the help for the pip command. If it doesn't, consult the documentation of your Python installation on how to enable them, or download a separate Python variant, for example try an unified installer package from python.org.

Once ready, do:

python3 -m pip install --user --upgrade gpt4all typer

Windows

Download the official installer from python.org if Python isn't already present on your system. It includes all the necessary components. Run:

py -3 -m pip install --user --upgrade gpt4all typer

Linux

On Linux, a Python installation is often split into several packages and not all are necessarily installed by default. For example, on Debian/Ubuntu and derived distros, you will want to ensure their presence with the following:

sudo apt-get install python3-pip

The next steps are similar to the other platforms:

python3 -m pip install --user --upgrade gpt4all typer

On other distros, the situation might be different. Especially the package names can vary a lot. You'll have to look it up in the documentation, software directory, or package search.

Running the CLI

The CLI is a self-contained script called app.py. As such, you can download and save it anywhere you like, as long as the Python interpreter has access to the mentioned dependencies.

Note: different platforms can have slightly different ways to start Python. Whereas below the interpreter command is written as python you typically want to type instead:

• On Unix-like systems: python3
• On Windows: py -3

The simplest way to start the CLI is:

python app.py repl

This automatically selects the groovy model and downloads it into the .cache/gpt4all/ folder of your home directory, if not already present.

If you want to use a different model, you can do so with the -m/--model parameter. If only a model file name is provided, it will again check in .cache/gpt4all/ and might start downloading. If instead given a path to an existing model, the command could for example look like this:

python app.py repl --model /home/user/my-gpt4all-models/GPT4All-13B-snoozy.ggmlv3.q4_0.bin

When you're done and want to end a session, simply type /exit.

To get help and information on all the available commands and options on the command-line, run:

python app.py --help

And while inside the running REPL, write /help.

Note that if you've installed the required packages into a virtual environment, you don't need to activate that every time you want to run the CLI. Instead, you can just start it with the Python interpreter in the folder gpt4all-cli/bin/ (Unix-like) or gpt4all-cli/Script/ (Windows).

That also makes it easy to set an alias e.g. in Bash or PowerShell:

• Bash: alias gpt4all="'/full/path/to/gpt4all-cli/bin/python' '/full/path/to/app.py' repl"
• PowerShell:

  Function GPT4All-Venv-CLI {"C:\full\path\to\gpt4all-cli\Scripts\python.exe" "C:\full\path\to\app.py" repl}
  Set-Alias -Name gpt4all -Value GPT4All-Venv-CLI
  

Don't forget to save these in the start-up file of your shell.

Finally, if on Windows you see a box instead of an arrow ⇢ as the prompt character, you should change the console font to one which offers better Unicode support.