Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-09-20 10:35:10 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Stop generation immediately when using "Maximum tokens/second" (#3952)

Browse source 
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
This commit is contained in:
BadisG 2023-09-18 19:27:06 +02:00 committed by GitHub
parent b7c55665c1
commit 893a72a1c5
WARNING! Although there is a key with this ID in the database it does not verify this commit! This commit is SUSPICIOUS.
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
modules/text_generation.py
View file

@ -96,7 +96,7 @@ def _generate_reply(question, state, stopping_strings=None, is_chat=False, escap
last_update = cur_time
yield reply
if stop_found:
if stop_found or (state['max_tokens_second'] > 0 and shared.stop_everything):
break
if not is_chat: