AI/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-10-01 01:26:03 -04:00
Code Issues Packages Projects Releases Wiki Activity

Add mention to alternative docker repository (#2145)

Browse Source
This commit is contained in:
Atinoda 2023-05-24 00:35:53 +01:00 committed by GitHub
parent 9714072692
commit 4155aaa96a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 50 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -118,9 +118,8 @@ cp docker/.env.example .env
docker compose up --build docker compose up --build
``` ```
You need to have docker compose v2.17 or higher installed in your system. To see how to install docker compose itself, see the guide in [here](https://github.com/oobabooga/text-generation-webui/blob/main/docs/Docker.md). * You need to have docker compose v2.17 or higher installed. See [this guide](https://github.com/oobabooga/text-generation-webui/blob/main/docs/Docker.md) for instructions.
* For additional docker files, check out [this repository](https://github.com/Atinoda/text-generation-webui/blob/docker-wrapper/docs/Docker.md#dedicated-docker-repository).
Contributed by [@loeken](https://github.com/loeken) in [#633](https://github.com/oobabooga/text-generation-webui/pull/633)
### Updating the requirements ### Updating the requirements

74
docs/Docker.md
View File

@ -7,7 +7,24 @@ In order to create the image as described in the main README, you must have dock
Docker Compose version v2.17.2 Docker Compose version v2.17.2
``` ```
# Intructions by [@loeken](https://github.com/loeken) Make sure to also create the necessary symbolic links:
```
cd text-generation-webui
ln -s docker/{Dockerfile,docker-compose.yml,.dockerignore} .
cp docker/.env.example .env
# Edit .env and set TORCH_CUDA_ARCH_LIST based on your GPU model
docker compose up --build
```
# Table of contents
* [Docker Compose installation instructions](#docker-compose-installation-instructions)
* [Repository with additional Docker files](#dedicated-docker-repository)
# Docker Compose installation instructions
By [@loeken](https://github.com/loeken).
- [Ubuntu 22.04](#ubuntu-2204) - [Ubuntu 22.04](#ubuntu-2204)
- [0. youtube video](#0-youtube-video) - [0. youtube video](#0-youtube-video)
@ -35,21 +52,21 @@ Docker Compose version v2.17.2
- [7. startup](#7-startup) - [7. startup](#7-startup)
- [notes](#notes) - [notes](#notes)
# Ubuntu 22.04 ## Ubuntu 22.04
## 0. youtube video ### 0. youtube video
A video walking you through the setup can be found here: A video walking you through the setup can be found here:
[![oobabooga text-generation-webui setup in docker on ubuntu 22.04](https://img.youtube.com/vi/ELkKWYh8qOk/0.jpg)](https://www.youtube.com/watch?v=ELkKWYh8qOk) [![oobabooga text-generation-webui setup in docker on ubuntu 22.04](https://img.youtube.com/vi/ELkKWYh8qOk/0.jpg)](https://www.youtube.com/watch?v=ELkKWYh8qOk)
## 1. update the drivers ### 1. update the drivers
in the the “software updater” update drivers to the last version of the prop driver. in the the “software updater” update drivers to the last version of the prop driver.
## 2. reboot ### 2. reboot
to switch using to new driver to switch using to new driver
## 3. install docker ### 3. install docker
```bash ```bash
sudo apt update sudo apt update
sudo apt-get install curl sudo apt-get install curl
@ -65,7 +82,7 @@ sudo usermod -aG docker $USER
newgrp docker newgrp docker
``` ```
## 4. docker & container toolkit ### 4. docker & container toolkit
```bash ```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu22.04/amd64 /" | \ echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu22.04/amd64 /" | \
@ -75,13 +92,13 @@ sudo apt install nvidia-docker2 nvidia-container-runtime -y
sudo systemctl restart docker sudo systemctl restart docker
``` ```
## 5. clone the repo ### 5. clone the repo
``` ```
git clone https://github.com/oobabooga/text-generation-webui git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui cd text-generation-webui
``` ```
## 6. prepare models ### 6. prepare models
download and place the models inside the models folder. tested with: download and place the models inside the models folder. tested with:
4bit 4bit
@ -91,30 +108,30 @@ https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941
8bit: 8bit:
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789
## 7. prepare .env file ### 7. prepare .env file
edit .env values to your needs. edit .env values to your needs.
```bash ```bash
cp .env.example .env cp .env.example .env
nano .env nano .env
``` ```
## 8. startup docker container ### 8. startup docker container
```bash ```bash
docker compose up --build docker compose up --build
``` ```
# Manjaro ## Manjaro
manjaro/arch is similar to ubuntu just the dependency installation is more convenient manjaro/arch is similar to ubuntu just the dependency installation is more convenient
## update the drivers ### update the drivers
```bash ```bash
sudo mhwd -a pci nonfree 0300 sudo mhwd -a pci nonfree 0300
``` ```
## reboot ### reboot
```bash ```bash
reboot reboot
``` ```
## docker & container toolkit ### docker & container toolkit
```bash ```bash
yay -S docker docker-compose buildkit gcc nvidia-docker yay -S docker docker-compose buildkit gcc nvidia-docker
sudo usermod -aG docker $USER sudo usermod -aG docker $USER
@ -122,32 +139,32 @@ newgrp docker
sudo systemctl restart docker # required by nvidia-container-runtime sudo systemctl restart docker # required by nvidia-container-runtime
``` ```
## continue with ubuntu task ### continue with ubuntu task
continue at [5. clone the repo](#5-clone-the-repo) continue at [5. clone the repo](#5-clone-the-repo)
# Windows ## Windows
## 0. youtube video ### 0. youtube video
A video walking you through the setup can be found here: A video walking you through the setup can be found here:
[![oobabooga text-generation-webui setup in docker on windows 11](https://img.youtube.com/vi/ejH4w5b5kFQ/0.jpg)](https://www.youtube.com/watch?v=ejH4w5b5kFQ) [![oobabooga text-generation-webui setup in docker on windows 11](https://img.youtube.com/vi/ejH4w5b5kFQ/0.jpg)](https://www.youtube.com/watch?v=ejH4w5b5kFQ)
## 1. choco package manager ### 1. choco package manager
install package manager (https://chocolatey.org/ ) install package manager (https://chocolatey.org/ )
``` ```
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1')) Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
``` ```
## 2. install drivers/dependencies ### 2. install drivers/dependencies
``` ```
choco install nvidia-display-driver cuda git docker-desktop choco install nvidia-display-driver cuda git docker-desktop
``` ```
## 3. install wsl ### 3. install wsl
wsl --install wsl --install
## 4. reboot ### 4. reboot
after reboot enter username/password in wsl after reboot enter username/password in wsl
## 5. git clone && startup ### 5. git clone && startup
clone the repo and edit .env values to your needs. clone the repo and edit .env values to your needs.
``` ```
cd Desktop cd Desktop
@ -157,19 +174,19 @@ COPY .env.example .env
notepad .env notepad .env
``` ```
## 6. prepare models ### 6. prepare models
download and place the models inside the models folder. tested with: download and place the models inside the models folder. tested with:
4bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105 4bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105
8bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789 8bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789
## 7. startup ### 7. startup
``` ```
docker compose up docker compose up
``` ```
# notes ## notes
on older ubuntus you can manually install the docker compose plugin like this: on older ubuntus you can manually install the docker compose plugin like this:
``` ```
@ -179,3 +196,8 @@ curl -SL https://github.com/docker/compose/releases/download/v2.17.2/docker-comp
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
export PATH="$HOME/.docker/cli-plugins:$PATH" export PATH="$HOME/.docker/cli-plugins:$PATH"
``` ```
# Dedicated docker repository
An external repository maintains a docker wrapper for this project as well as several pre-configured 'one-click' `docker compose` variants (e.g., updated branches of GPTQ). It can be found at: [Atinoda/text-generation-webui-docker](https://github.com/Atinoda/text-generation-webui-docker).