text-generation-webui/docs/09 - Docker.md

209 lines
6.9 KiB
Markdown
Raw Permalink Normal View History

2023-04-18 17:50:33 +02:00
Docker Compose is a way of installing and launching the web UI in an isolated Ubuntu image using only a few commands.
2023-11-18 04:52:30 +01:00
## Installing Docker Compose
In order to create the image as described in the main README, you must have Docker Compose installed (2.17 or higher is recommended):
2023-04-18 17:50:33 +02:00
```
~$ docker compose version
2023-11-18 04:52:30 +01:00
Docker Compose version v2.21.0
2023-04-18 17:50:33 +02:00
```
2023-11-18 04:52:30 +01:00
The installation instructions for various Linux distributions can be found here:
https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository
## Launching the image
Use these commands to launch the image:
```
cd text-generation-webui
2024-05-20 01:09:37 +02:00
ln -s docker/{nvidia/Dockerfile,nvidia/docker-compose.yml,.dockerignore} .
cp docker/.env.example .env
# Edit .env and set TORCH_CUDA_ARCH_LIST based on your GPU model
docker compose up --build
```
2023-11-18 04:52:30 +01:00
## More detailed installation instructions
* [Docker Compose installation instructions](#docker-compose-installation-instructions)
* [Repository with additional Docker files](#dedicated-docker-repository)
By [@loeken](https://github.com/loeken).
2023-04-18 17:50:33 +02:00
2023-04-18 17:47:43 +02:00
- [Ubuntu 22.04](#ubuntu-2204)
- [0. youtube video](#0-youtube-video)
- [1. update the drivers](#1-update-the-drivers)
- [2. reboot](#2-reboot)
- [3. install docker](#3-install-docker)
- [4. docker \& container toolkit](#4-docker--container-toolkit)
- [5. clone the repo](#5-clone-the-repo)
- [6. prepare models](#6-prepare-models)
- [7. prepare .env file](#7-prepare-env-file)
- [8. startup docker container](#8-startup-docker-container)
- [Manjaro](#manjaro)
- [update the drivers](#update-the-drivers)
- [reboot](#reboot)
- [docker \& container toolkit](#docker--container-toolkit)
- [continue with ubuntu task](#continue-with-ubuntu-task)
- [Windows](#windows)
- [0. youtube video](#0-youtube-video-1)
- [1. choco package manager](#1-choco-package-manager)
- [2. install drivers/dependencies](#2-install-driversdependencies)
- [3. install wsl](#3-install-wsl)
- [4. reboot](#4-reboot)
- [5. git clone \&\& startup](#5-git-clone--startup)
- [6. prepare models](#6-prepare-models-1)
- [7. startup](#7-startup)
- [notes](#notes)
2023-10-22 00:15:54 +02:00
### Ubuntu 22.04
2023-04-18 17:47:43 +02:00
2023-10-22 00:15:54 +02:00
#### 0. youtube video
2023-04-18 17:47:43 +02:00
A video walking you through the setup can be found here:
[![oobabooga text-generation-webui setup in docker on ubuntu 22.04](https://img.youtube.com/vi/ELkKWYh8qOk/0.jpg)](https://www.youtube.com/watch?v=ELkKWYh8qOk)
2023-10-22 00:15:54 +02:00
#### 1. update the drivers
2023-04-18 17:47:43 +02:00
in the the “software updater” update drivers to the last version of the prop driver.
2023-10-22 00:15:54 +02:00
#### 2. reboot
2023-04-18 17:47:43 +02:00
to switch using to new driver
2023-10-22 00:15:54 +02:00
#### 3. install docker
2023-04-18 17:47:43 +02:00
```bash
sudo apt update
sudo apt-get install curl
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose -y
sudo usermod -aG docker $USER
newgrp docker
```
2023-10-22 00:15:54 +02:00
#### 4. docker & container toolkit
2023-04-18 17:47:43 +02:00
```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu22.04/amd64 /" | \
sudo tee /etc/apt/sources.list.d/nvidia.list > /dev/null
sudo apt update
sudo apt install nvidia-docker2 nvidia-container-runtime -y
sudo systemctl restart docker
```
2023-10-22 00:15:54 +02:00
#### 5. clone the repo
2023-04-18 17:47:43 +02:00
```
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
```
2023-10-22 00:15:54 +02:00
#### 6. prepare models
2023-04-18 17:47:43 +02:00
download and place the models inside the models folder. tested with:
4bit
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105
8bit:
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789
2023-10-22 00:15:54 +02:00
#### 7. prepare .env file
2023-04-18 17:47:43 +02:00
edit .env values to your needs.
```bash
cp .env.example .env
nano .env
```
2023-10-22 00:15:54 +02:00
#### 8. startup docker container
2023-04-18 17:47:43 +02:00
```bash
2023-05-02 05:51:10 +02:00
docker compose up --build
2023-04-18 17:47:43 +02:00
```
2023-10-22 00:15:54 +02:00
### Manjaro
2023-04-18 17:47:43 +02:00
manjaro/arch is similar to ubuntu just the dependency installation is more convenient
2023-10-22 00:15:54 +02:00
#### update the drivers
2023-04-18 17:47:43 +02:00
```bash
sudo mhwd -a pci nonfree 0300
```
2023-10-22 00:15:54 +02:00
#### reboot
2023-04-18 17:47:43 +02:00
```bash
reboot
```
2023-10-22 00:15:54 +02:00
#### docker & container toolkit
2023-04-18 17:47:43 +02:00
```bash
yay -S docker docker-compose buildkit gcc nvidia-docker
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl restart docker # required by nvidia-container-runtime
```
2023-10-22 00:15:54 +02:00
#### continue with ubuntu task
2023-04-18 17:47:43 +02:00
continue at [5. clone the repo](#5-clone-the-repo)
2023-10-22 00:15:54 +02:00
### Windows
#### 0. youtube video
2023-04-18 17:47:43 +02:00
A video walking you through the setup can be found here:
[![oobabooga text-generation-webui setup in docker on windows 11](https://img.youtube.com/vi/ejH4w5b5kFQ/0.jpg)](https://www.youtube.com/watch?v=ejH4w5b5kFQ)
2023-10-22 00:15:54 +02:00
#### 1. choco package manager
2023-04-18 17:47:43 +02:00
install package manager (https://chocolatey.org/ )
```
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
```
2023-10-22 00:15:54 +02:00
#### 2. install drivers/dependencies
2023-04-18 17:47:43 +02:00
```
choco install nvidia-display-driver cuda git docker-desktop
```
2023-10-22 00:15:54 +02:00
#### 3. install wsl
2023-04-18 17:47:43 +02:00
wsl --install
2023-10-22 00:15:54 +02:00
#### 4. reboot
2023-04-18 17:47:43 +02:00
after reboot enter username/password in wsl
2023-10-22 00:15:54 +02:00
#### 5. git clone && startup
2023-04-18 17:47:43 +02:00
clone the repo and edit .env values to your needs.
```
cd Desktop
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
COPY .env.example .env
notepad .env
```
2023-10-22 00:15:54 +02:00
#### 6. prepare models
2023-04-18 17:47:43 +02:00
download and place the models inside the models folder. tested with:
4bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105
8bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789
2023-10-22 00:15:54 +02:00
#### 7. startup
2023-04-18 17:47:43 +02:00
```
2023-05-02 05:51:10 +02:00
docker compose up
2023-04-18 17:47:43 +02:00
```
2023-10-22 00:15:54 +02:00
### notes
2023-04-18 17:47:43 +02:00
on older ubuntus you can manually install the docker compose plugin like this:
```
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.17.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
export PATH="$HOME/.docker/cli-plugins:$PATH"
```
2023-10-22 00:15:54 +02:00
## Dedicated docker repository
An external repository maintains a docker wrapper for this project as well as several pre-configured 'one-click' `docker compose` variants (e.g., updated branches of GPTQ). It can be found at: [Atinoda/text-generation-webui-docker](https://github.com/Atinoda/text-generation-webui-docker).