diff --git a/README.md b/README.md index 40ae94d5..fe6a33ca 100644 --- a/README.md +++ b/README.md @@ -10,27 +10,30 @@ Its goal is to become the [AUTOMATIC1111/stable-diffusion-webui](https://github. ## Features -* 3 interface modes: default (two columns), notebook, and chat. -* Multiple model backends: [Transformers](https://github.com/huggingface/transformers), [llama.cpp](https://github.com/ggerganov/llama.cpp) (through [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)), [ExLlamaV2](https://github.com/turboderp/exllamav2), [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ), [AutoAWQ](https://github.com/casper-hansen/AutoAWQ), [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM). -* Dropdown menu for quickly switching between different models. -* Large number of extensions (built-in and user-contributed), including Coqui TTS for realistic voice outputs, Whisper STT for voice inputs, translation, [multimodal pipelines](https://github.com/oobabooga/text-generation-webui/tree/main/extensions/multimodal), vector databases, Stable Diffusion integration, and a lot more. See [the wiki](https://github.com/oobabooga/text-generation-webui/wiki/07-%E2%80%90-Extensions) and [the extensions directory](https://github.com/oobabooga/text-generation-webui-extensions) for details. -* [Chat with custom characters](https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab#character). -* Precise chat templates for instruction-following models, including Llama-2-chat, Alpaca, Vicuna, Mistral. -* LoRA: train new LoRAs with your own data, load/unload LoRAs on the fly for generation. -* Transformers library integration: load models in 4-bit or 8-bit precision through bitsandbytes, use llama.cpp with transformers samplers (`llamacpp_HF` loader), CPU inference in 32-bit precision using PyTorch. +* Multiple backends for text generation in a single UI and API, including [Transformers](https://github.com/huggingface/transformers), [llama.cpp](https://github.com/ggerganov/llama.cpp) (through [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)), [ExLlamaV2](https://github.com/turboderp/exllamav2), [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ), [AutoAWQ](https://github.com/casper-hansen/AutoAWQ), and [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM). * OpenAI-compatible API server with Chat and Completions endpoints -- see the [examples](https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API#examples). +* Automatic prompt formatting for each model using the Jinja2 template in its metadata, ensuring high-quality outputs without manual intervention. +* Three chat modes: `instruct`, `chat-instruct`, and `chat`, allowing for both task-based interactions and casual conversations with characters. +* High-quality outputs in chat scenarios with the `chat-instruct` mode, which uses the model's template automatically. +* Easy switching between conversations and starting new ones through the "Past chats" menu in the main interface tab. +* Flexible text generation through autocompletion in the Default/Notebook tabs without being limited to chat turns. Send formatted chat conversations from the Chat tab to these tabs. +* Multiple sampling parameters and options for sophisticated text generation control. +* Quick downloading and loading of new models through the interface without restarting, using the "Model" tab. +* Simple LoRA fine-tuning tool to customize models with your data. +* Self-contained dependencies in the `installer_files` folder, avoiding interference with the system's Python environment. Precompiled Python wheels for the backends are in the `requirements.txt` and are transparently compiled using GitHub Actions. +* Extensions support, including numerous built-in and user-contributed extensions. See [the wiki](https://github.com/oobabooga/text-generation-webui/wiki/07-%E2%80%90-Extensions) and [the extensions directory](https://github.com/oobabooga/text-generation-webui-extensions) for details. ## How to install 1) Clone or [download](https://github.com/oobabooga/text-generation-webui/archive/refs/heads/main.zip) the repository. 2) Run the `start_linux.sh`, `start_windows.bat`, `start_macos.sh`, or `start_wsl.bat` script depending on your OS. 3) Select your GPU vendor when asked. -4) Once the installation ends, browse to `http://localhost:7860/?__theme=dark`. +4) Once the installation ends, browse to `http://localhost:7860`. 5) Have fun! -To restart the web UI in the future, just run the `start_` script again. This script creates an `installer_files` folder where it sets up the project's requirements. In case you need to reinstall the requirements, you can simply delete that folder and start the web UI again. +To restart the web UI in the future, just run the `start_` script again. This script creates an `installer_files` folder where it sets up the project's requirements. If you need to reinstall the requirements, you can simply delete that folder and start the web UI again. -The script accepts command-line flags. Alternatively, you can edit the `CMD_FLAGS.txt` file with a text editor and add your flags there. +The script accepts command-line flags. Alternatively, you can edit the `CMD_FLAGS.txt` file with a text editor and add your flags there, such as `--api` in case you need to use the API. To get updates in the future, run `update_wizard_linux.sh`, `update_wizard_windows.bat`, `update_wizard_macos.sh`, or `update_wizard_wsl.bat`. @@ -401,7 +404,7 @@ https://colab.research.google.com/github/oobabooga/text-generation-webui/blob/ma ## Community -* Subreddit: https://www.reddit.com/r/oobabooga/ +* Subreddit: https://www.reddit.com/r/Oobabooga/ * Discord: https://discord.gg/jwZCF2dPQN ## Acknowledgment diff --git a/requirements.txt b/requirements.txt index a4b864b1..08b7d56d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* aqlm[gpu,cpu]==1.1.6; platform_system == "Linux" auto-gptq==0.7.1 bitsandbytes==0.43.* @@ -14,7 +14,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_amd.txt b/requirements_amd.txt index db765afe..52e36510 100644 --- a/requirements_amd.txt +++ b/requirements_amd.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_amd_noavx2.txt b/requirements_amd_noavx2.txt index 3751af0e..18a81d04 100644 --- a/requirements_amd_noavx2.txt +++ b/requirements_amd_noavx2.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_apple_intel.txt b/requirements_apple_intel.txt index c38fad1c..af02904b 100644 --- a/requirements_apple_intel.txt +++ b/requirements_apple_intel.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_apple_silicon.txt b/requirements_apple_silicon.txt index 0f2f9656..8cdd8519 100644 --- a/requirements_apple_silicon.txt +++ b/requirements_apple_silicon.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_cpu_only.txt b/requirements_cpu_only.txt index 9a46e366..807c182a 100644 --- a/requirements_cpu_only.txt +++ b/requirements_cpu_only.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_cpu_only_noavx2.txt b/requirements_cpu_only_noavx2.txt index e78ce3e0..e2a89936 100644 --- a/requirements_cpu_only_noavx2.txt +++ b/requirements_cpu_only_noavx2.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_noavx2.txt b/requirements_noavx2.txt index 5cb0d19a..d22eb72c 100644 --- a/requirements_noavx2.txt +++ b/requirements_noavx2.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* aqlm[gpu,cpu]==1.1.6; platform_system == "Linux" auto-gptq==0.7.1 bitsandbytes==0.43.* @@ -14,7 +14,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml diff --git a/requirements_nowheels.txt b/requirements_nowheels.txt index a95098a5..ffb45fe3 100644 --- a/requirements_nowheels.txt +++ b/requirements_nowheels.txt @@ -1,4 +1,4 @@ -accelerate==0.32.* +accelerate==0.33.* colorama datasets einops @@ -11,7 +11,7 @@ numba==0.59.* numpy==1.26.* optimum==1.17.* pandas -peft==0.8.* +peft==0.12.* Pillow>=9.5.0 psutil pyyaml