AI/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-10-01 01:26:03 -04:00
Code Issues Packages Projects Releases Wiki Activity

Set use_cache=True by default for all models

Browse Source
This commit is contained in:
oobabooga 2023-08-30 13:26:27 -07:00
parent 5190e153ed
commit 47e490c7b4
1 changed files with 1 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
modules/text_generation.py
View File

@ -265,9 +265,7 @@ def generate_reply_HF(question, original_question, seed, state, stopping_strings
if state['ban_eos_token']:
generate_params['suppress_tokens'] = [shared.tokenizer.eos_token_id]
if shared.args.no_cache:
generate_params.update({'use_cache': False})
generate_params.update({'use_cache': not shared.args.no_cache})
if shared.args.deepspeed:
generate_params.update({'synced_gpus': True})