Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-09-20 18:45:09 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Update RWKV-model.md

Browse source
This commit is contained in:
oobabooga 2023-05-23 02:10:08 -03:00 committed by GitHub
parent cd3618d7fb
commit b0845ae4e8
WARNING! Although there is a key with this ID in the database it does not verify this commit! This commit is SUSPICIOUS.
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/RWKV-model.md
View file

@ -46,7 +46,7 @@ No additional steps are required. Just launch it as you would with any other mod
python server.py --listen --no-stream --model RWKV-4-Pile-169M-20220807-8023.pth
```
### Setting a custom strategy
#### Setting a custom strategy
It is possible to have very fine control over the offloading and precision for the model with the `--rwkv-strategy` flag. Possible values include:
@ -59,6 +59,6 @@ It is possible to have very fine control over the offloading and precision for t
See the README for the PyPl package for more details: https://pypi.org/project/rwkv/
### Compiling the CUDA kernel
#### Compiling the CUDA kernel
You can compile the CUDA kernel for the model with `--rwkv-cuda-on`. This should improve the performance a lot but I haven't been able to get it to work yet.
You can compile the CUDA kernel for the model with `--rwkv-cuda-on`. This should improve the performance a lot but I haven't been able to get it to work yet.