Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-09-20 10:35:10 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Use the CPU if no GPU is detected

Browse source
This commit is contained in:
oobabooga 2023-02-11 23:17:06 -03:00
parent 337290777b
commit 5d3f15b915
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
server.py
View file

@ -120,6 +120,9 @@ def load_model(model_name):
else:
command = "AutoModelForCausalLM.from_pretrained"
params = ["low_cpu_mem_usage=True"]
if not args.cpu and not torch.cuda.is_available():
print("Warning: no GPU has been detected.\nFalling back to CPU mode.\n")
args.cpu = True
if args.cpu:
params.append("low_cpu_mem_usage=True")