mirror of
https://github.com/oobabooga/text-generation-webui.git
synced 2024-10-01 01:26:03 -04:00
72 lines
2.5 KiB
Markdown
72 lines
2.5 KiB
Markdown
# Generation Parameters
|
|
|
|
For a technical description of the parameters, the [transformers documentation](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig) is a good reference.
|
|
|
|
The best presets, according to the [Preset Arena](https://github.com/oobabooga/oobabooga.github.io/blob/main/arena/results.md) experiment, are:
|
|
|
|
**Instruction following:**
|
|
|
|
1) Divine Intellect
|
|
2) Big O
|
|
3) simple-1
|
|
4) Space Alien
|
|
5) StarChat
|
|
6) Titanic
|
|
7) tfs-with-top-a
|
|
8) Asterism
|
|
9) Contrastive Search
|
|
|
|
**Chat:**
|
|
|
|
1) Midnight Enigma
|
|
2) Yara
|
|
3) Shortwave
|
|
|
|
### Temperature
|
|
|
|
Primary factor to control randomness of outputs. 0 = deterministic (only the most likely token is used). Higher value = more randomness.
|
|
|
|
### top_p
|
|
|
|
If not set to 1, select tokens with probabilities adding up to less than this number. Higher value = higher range of possible random results.
|
|
|
|
### top_k
|
|
|
|
Similar to top_p, but select instead only the top_k most likely tokens. Higher value = higher range of possible random results.
|
|
|
|
### typical_p
|
|
|
|
If not set to 1, select only tokens that are at least this much more likely to appear than random tokens, given the prior text.
|
|
|
|
### epsilon_cutoff
|
|
|
|
In units of 1e-4; a reasonable value is 3. This sets a probability floor below which tokens are excluded from being sampled. Should be used with top_p, top_k, and eta_cutoff set to 0.
|
|
|
|
### eta_cutoff
|
|
|
|
In units of 1e-4; a reasonable value is 3. Should be used with top_p, top_k, and epsilon_cutoff set to 0.
|
|
|
|
### repetition_penalty
|
|
|
|
Exponential penalty factor for repeating prior tokens. 1 means no penalty, higher value = less repetition, lower value = more repetition.
|
|
|
|
### repetition_penalty_range
|
|
|
|
The number of most recent tokens to consider for repetition penalty. 0 makes all tokens be used.
|
|
|
|
### encoder_repetition_penalty
|
|
|
|
Also known as the "Hallucinations filter". Used to penalize tokens that are *not* in the prior text. Higher value = more likely to stay in context, lower value = more likely to diverge.
|
|
|
|
### no_repeat_ngram_size
|
|
|
|
If not set to 0, specifies the length of token sets that are completely blocked from repeating at all. Higher values = blocks larger phrases, lower values = blocks words or letters from repeating. Only 0 or high values are a good idea in most cases.
|
|
|
|
### min_length
|
|
|
|
Minimum generation length in tokens.
|
|
|
|
### penalty_alpha
|
|
|
|
Contrastive Search is enabled by setting this to greater than zero and unchecking "do_sample". It should be used with a low value of top_k, for instance, top_k = 4.
|