AI/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-10-01 01:26:03 -04:00
Code Issues Packages Projects Releases Wiki Activity

Make long_replies ban the eos token as well

Browse Source
This commit is contained in:
oobabooga 2023-08-01 18:47:49 -07:00
parent 9ae0eab989
commit 6c521ce967
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
extensions/long_replies/script.py
View File

@ -28,7 +28,7 @@ class MyLogits(LogitsProcessor):
def __call__(self, input_ids, scores):
if input_ids.shape[-1] - initial_size < params["min_length"]:
scores[...,self.newline_id] = -1000
# scores[...,shared.tokenizer.eos_token_id] = -1000
scores[...,shared.tokenizer.eos_token_id] = -1000
# probs = torch.softmax(scores, dim=-1, dtype=torch.float)
# probs[0] /= probs[0].sum()