Commit Graph

1124 Commits

Author SHA1 Message Date
oobabooga
365335e1ae Minor fix 2023-09-26 06:47:19 -07:00
oobabooga
1ca54faaf0 Improve --multi-user mode 2023-09-26 06:42:33 -07:00
oobabooga
019371c0b6 Lint 2023-09-25 20:31:11 -07:00
oobabooga
814520fed1 Extension install improvements 2023-09-25 20:27:06 -07:00
oobabooga
7f1460af29 Change a warning 2023-09-25 20:22:27 -07:00
oobabooga
862b45b1c7 Extension install improvements 2023-09-25 19:48:30 -07:00
oobabooga
c8952cce55 Move documentation from UI to docs/ 2023-09-25 12:28:28 -07:00
oobabooga
d0d221df49 Add --use_fast option (closes #3741) 2023-09-25 12:19:43 -07:00
oobabooga
b973b91d73 Automatically filter by loader (closes #4072) 2023-09-25 10:28:35 -07:00
oobabooga
63de9eb24f Clean up the transformers loader 2023-09-24 20:26:26 -07:00
oobabooga
36c38d7561 Add disable_exllama to Transformers loader (for GPTQ LoRA training) 2023-09-24 20:03:11 -07:00
oobabooga
55a685d999 Minor fixes 2023-09-24 14:15:10 -07:00
oobabooga
08cf150c0c
Add a grammar editor to the UI (#4061) 2023-09-24 18:05:24 -03:00
oobabooga
eb0b7c1053 Fix a minor UI bug 2023-09-24 07:17:33 -07:00
oobabooga
3edac43426 Remove print statement 2023-09-24 07:13:00 -07:00
oobabooga
b227e65d86 Add grammar to llama.cpp loader (closes #4019) 2023-09-24 07:10:45 -07:00
oobabooga
2e7b6b0014
Create alternative requirements.txt with AMD and Metal wheels (#4052) 2023-09-24 09:58:29 -03:00
oobabooga
7a3ca2c68f Better detect EXL2 models 2023-09-23 13:05:55 -07:00
oobabooga
b1467bd064
Move one-click-installers into the repository (#4028 from oobabooga/one-click) 2023-09-22 17:43:07 -03:00
oobabooga
c075969875 Add instructions 2023-09-22 13:10:03 -07:00
oobabooga
8ab3eca9ec Add a warning for outdated installations 2023-09-22 09:35:19 -07:00
oobabooga
95976a9d4f Fix a bug while deleting characters 2023-09-22 06:02:34 -07:00
oobabooga
d5330406fa Add a rename menu for chat histories 2023-09-21 19:16:51 -07:00
oobabooga
00ab450c13
Multiple histories for each character (#4022) 2023-09-21 17:19:32 -03:00
oobabooga
029da9563f Avoid redundant function call in llamacpp_hf 2023-09-19 14:14:40 -07:00
oobabooga
869f47fff9 Lint 2023-09-19 13:51:57 -07:00
oobabooga
13ac55fa18 Reorder some functions 2023-09-19 13:51:57 -07:00
oobabooga
03dc69edc5 ExLlama_HF (v1 and v2) prefix matching 2023-09-19 13:12:19 -07:00
oobabooga
5075087461 Fix command-line arguments being ignored 2023-09-19 13:11:46 -07:00
oobabooga
ff5d3d2d09 Add missing import 2023-09-18 16:26:54 -07:00
oobabooga
605ec3c9f2 Add a warning about ExLlamaV2 without flash-attn 2023-09-18 12:26:35 -07:00
oobabooga
f0ef971edb Remove obsolete warning 2023-09-18 12:25:10 -07:00
oobabooga
745807dc03 Faster llamacpp_HF prefix matching 2023-09-18 11:02:45 -07:00
BadisG
893a72a1c5
Stop generation immediately when using "Maximum tokens/second" (#3952)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-09-18 14:27:06 -03:00
Cebtenzzre
8466cf229a
llama.cpp: fix ban_eos_token (#3987) 2023-09-18 12:15:02 -03:00
oobabooga
0ede2965d5 Remove an error message 2023-09-17 18:46:08 -07:00
missionfloyd
cc8eda298a
Move hover menu shortcuts to right side (#3951) 2023-09-17 22:33:00 -03:00
oobabooga
280cca9f66 Merge remote-tracking branch 'refs/remotes/origin/main' 2023-09-17 18:01:27 -07:00
oobabooga
b062d50c45 Remove exllama import that causes problems 2023-09-17 18:00:32 -07:00
James Braza
fee38e0601
Simplified ExLlama cloning instructions and failure message (#3972) 2023-09-17 19:26:05 -03:00
Lu Guanghua
9858acee7b
Fix unexpected extensions load after gradio restart (#3965) 2023-09-17 17:35:43 -03:00
oobabooga
d9b0f2c9c3 Fix llama.cpp double decoding 2023-09-17 13:07:48 -07:00
oobabooga
d71465708c llamacpp_HF prefix matching 2023-09-17 11:51:01 -07:00
oobabooga
37e2980e05 Recommend mul_mat_q for llama.cpp 2023-09-17 08:27:11 -07:00
oobabooga
a069f3904c Undo part of ad8ac545a5 2023-09-17 08:12:23 -07:00
oobabooga
ad8ac545a5 Tokenization improvements 2023-09-17 07:02:00 -07:00
saltacc
cd08eb0753
token probs for non HF loaders (#3957) 2023-09-17 10:42:32 -03:00
kalomaze
7c9664ed35
Allow full model URL to be used for download (#3919)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-09-16 10:06:13 -03:00
saltacc
ed6b6411fb
Fix exllama tokenizers (#3954)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-09-16 09:42:38 -03:00
missionfloyd
2ad6ca8874
Add back chat buttons with --chat-buttons (#3947) 2023-09-16 00:39:37 -03:00
oobabooga
ef04138bc0 Improve the UI tokenizer 2023-09-15 19:30:44 -07:00
oobabooga
c3e4c9fdc2 Add a simple tokenizer to the UI 2023-09-15 19:09:03 -07:00
saltacc
f01b9aa71f
Add customizable ban tokens (#3899) 2023-09-15 18:27:27 -03:00
oobabooga
5b117590ad Add some scrollbars to Parameters tab 2023-09-15 09:17:37 -07:00
Johan
fdcee0c215
Allow custom tokenizer for llamacpp_HF loader (#3941) 2023-09-15 12:38:38 -03:00
oobabooga
fd7257c7f8 Prevent code blocks from flickering while streaming 2023-09-15 07:46:26 -07:00
oobabooga
a3ecf3bb65 Add cai-chat-square chat style 2023-09-14 16:15:08 -07:00
oobabooga
3d1c0f173d User config precedence over GGUF metadata 2023-09-14 12:15:52 -07:00
oobabooga
94dc64f870 Add a border 2023-09-14 07:20:36 -07:00
oobabooga
70aafa34dc Fix blockquote markdown rendering 2023-09-14 05:57:04 -07:00
oobabooga
644a9b8765 Change the chat generate button 2023-09-14 05:16:44 -07:00
oobabooga
ecc90f9f62 Continue on Alt + Enter 2023-09-14 03:59:12 -07:00
oobabooga
1ce3c93600 Allow "Your name" field to be saved 2023-09-14 03:44:35 -07:00
oobabooga
27dbcc59f5
Make the chat input expand upwards (#3920) 2023-09-14 07:06:42 -03:00
oobabooga
6b6af74e14 Keyboard shortcuts without conflicts (hopefully) 2023-09-14 02:33:52 -07:00
oobabooga
fc11d1eff0 Add chat keyboard shortcuts 2023-09-13 19:22:40 -07:00
oobabooga
9f199c7a4c Use Noto Sans font
Copied from 6c8bd06308/public/webfonts/NotoSans
2023-09-13 13:48:05 -07:00
oobabooga
8ce94b735c Show progress on impersonate 2023-09-13 11:22:53 -07:00
oobabooga
7cd437e05c Properly close the hover menu on mobile 2023-09-13 11:10:46 -07:00
oobabooga
1b47b5c676 Change the Generate/Stop buttons 2023-09-13 09:25:26 -07:00
oobabooga
8ea28cbfe0 Reorder chat buttons 2023-09-13 08:49:11 -07:00
oobabooga
5e3d2f7d44
Reorganize chat buttons (#3892) 2023-09-13 02:36:12 -03:00
Panchovix
34dc7306b8
Fix NTK (alpha) and RoPE scaling for exllamav2 and exllamav2_HF (#3897) 2023-09-13 02:35:09 -03:00
oobabooga
b7adf290fc Fix ExLlama-v2 path issue 2023-09-12 17:42:22 -07:00
oobabooga
b190676893 Merge remote-tracking branch 'refs/remotes/origin/main' 2023-09-12 15:06:33 -07:00
oobabooga
2f935547c8 Minor changes 2023-09-12 15:05:21 -07:00
oobabooga
18e6b275f3 Add alpha_value/compress_pos_emb to ExLlama-v2 2023-09-12 15:02:47 -07:00
Gennadij
460c40d8ab
Read more GGUF metadata (scale_linear and freq_base) (#3877) 2023-09-12 17:02:42 -03:00
oobabooga
16e1696071 Minor qol change 2023-09-12 10:44:26 -07:00
oobabooga
c2a309f56e
Add ExLlamaV2 and ExLlamav2_HF loaders (#3881) 2023-09-12 14:33:07 -03:00
oobabooga
df123a20fc Prevent extra keys from being saved to settings.yaml 2023-09-11 20:13:10 -07:00
oobabooga
dae428a967 Revamp cai-chat theme, make it default 2023-09-11 19:30:40 -07:00
oobabooga
78811dd89a Fix GGUF metadata reading for falcon 2023-09-11 15:49:50 -07:00
oobabooga
9331ab4798
Read GGUF metadata (#3873) 2023-09-11 18:49:30 -03:00
oobabooga
df52dab67b Lint 2023-09-11 07:57:38 -07:00
oobabooga
ed86878f02 Remove GGML support 2023-09-11 07:44:00 -07:00
John Smith
cc7b7ba153
fix lora training with alpaca_lora_4bit (#3853) 2023-09-11 01:22:20 -03:00
Forkoz
15e9b8c915
Exllama new rope settings (#3852) 2023-09-11 01:14:36 -03:00
oobabooga
4affa08821 Do not impose instruct mode while loading models 2023-09-02 11:31:33 -07:00
oobabooga
47e490c7b4 Set use_cache=True by default for all models 2023-08-30 13:26:27 -07:00
missionfloyd
787219267c
Allow downloading single file from UI (#3737) 2023-08-29 23:32:36 -03:00
oobabooga
cec8db52e5
Add max_tokens_second param (#3533) 2023-08-29 17:44:31 -03:00
oobabooga
2b58a89f6a Clear instruction template before loading new one 2023-08-29 13:11:32 -07:00
oobabooga
36864cb3e8 Use Alpaca as the default instruction template 2023-08-29 13:06:25 -07:00
oobabooga
9a202f7fb2 Prevent <ul> lists from flickering during streaming 2023-08-28 20:45:07 -07:00
oobabooga
439dd0faab Fix stopping strings in the chat API 2023-08-28 19:40:11 -07:00
oobabooga
c75f98a6d6 Autoscroll Notebook/Default textareas during streaming 2023-08-28 18:22:03 -07:00
oobabooga
558e918fd6 Add a typing dots (...) animation to chat tab 2023-08-28 13:50:36 -07:00
oobabooga
57e9ded00c
Make it possible to scroll during streaming (#3721) 2023-08-28 16:03:20 -03:00
Cebtenzzre
2f5d769a8d
accept floating-point alpha value on the command line (#3712) 2023-08-27 18:54:43 -03:00
oobabooga
b2296dcda0 Ctrl+S to show/hide chat controls 2023-08-27 13:14:33 -07:00
Ravindra Marella
e4c3e1bdd2
Fix ctransformers model unload (#3711)
Add missing comma in model types list

Fixes marella/ctransformers#111
2023-08-27 10:53:48 -03:00
oobabooga
0c9e818bb8 Update truncation length based on max_seq_len/n_ctx 2023-08-26 23:10:45 -07:00
oobabooga
3361728da1 Change some comments 2023-08-26 22:24:44 -07:00
oobabooga
8aeae3b3f4 Fix llamacpp_HF loading 2023-08-26 22:15:06 -07:00
oobabooga
7f5370a272 Minor fixes/cosmetics 2023-08-26 22:11:07 -07:00
jllllll
4d61a7d9da
Account for deprecated GGML parameters 2023-08-26 14:07:46 -05:00
jllllll
4a999e3bcd
Use separate llama-cpp-python packages for GGML support 2023-08-26 10:40:08 -05:00
oobabooga
83640d6f43 Replace ggml occurences with gguf 2023-08-26 01:06:59 -07:00
jllllll
db42b365c9
Fix ctransformers threads auto-detection (#3688) 2023-08-25 14:37:02 -03:00
cal066
960980247f
ctransformers: gguf support (#3685) 2023-08-25 11:33:04 -03:00
oobabooga
21058c37f7 Add missing file 2023-08-25 07:10:26 -07:00
oobabooga
f4f04c8c32 Fix a typo 2023-08-25 07:08:38 -07:00
oobabooga
5c7d8bfdfd Detect CodeLlama settings 2023-08-25 07:06:57 -07:00
oobabooga
52ab2a6b9e Add rope_freq_base parameter for CodeLlama 2023-08-25 06:55:15 -07:00
oobabooga
feecd8190f Unescape inline code blocks 2023-08-24 21:01:09 -07:00
oobabooga
3320accfdc
Add CFG to llamacpp_HF (second attempt) (#3678) 2023-08-24 20:32:21 -03:00
oobabooga
d6934bc7bc
Implement CFG for ExLlama_HF (#3666) 2023-08-24 16:27:36 -03:00
oobabooga
87442c6d18 Fix Notebook Logits tab 2023-08-22 21:00:12 -07:00
oobabooga
c0b119c3a3 Improve logit viewer format 2023-08-22 20:35:12 -07:00
oobabooga
8545052c9d Add the option to use samplers in the logit viewer 2023-08-22 20:18:16 -07:00
oobabooga
25e5eaa6a6 Remove outdated training warning 2023-08-22 13:16:44 -07:00
oobabooga
335c49cc7e Bump peft and transformers 2023-08-22 13:14:59 -07:00
cal066
e042bf8624
ctransformers: add mlock and no-mmap options (#3649) 2023-08-22 16:51:34 -03:00
oobabooga
6cca8b8028 Only update notebook token counter on input
For performance during streaming
2023-08-21 05:39:55 -07:00
oobabooga
2cb07065ec Fix an escaping bug 2023-08-20 21:50:42 -07:00
oobabooga
a74dd9003f Fix HTML escaping for perplexity_colors extension 2023-08-20 21:40:22 -07:00
oobabooga
57036abc76 Add "send to default/notebook" buttons to chat tab 2023-08-20 19:54:59 -07:00
oobabooga
429cacd715 Add a token counter similar to automatic1111
It can now be found in the Default and Notebook tabs
2023-08-20 19:37:33 -07:00
oobabooga
120fb86c6a
Add a simple logit viewer (#3636) 2023-08-20 20:49:21 -03:00
oobabooga
ef17da70af Fix ExLlama truncation 2023-08-20 08:53:26 -07:00
oobabooga
ee964bcce9 Update a comment about RoPE scaling 2023-08-20 07:01:43 -07:00
missionfloyd
1cae784761
Unescape last message (#3623) 2023-08-19 09:29:08 -03:00
Cebtenzzre
942ad6067d
llama.cpp: make Stop button work with streaming disabled (#3620) 2023-08-19 00:17:27 -03:00
oobabooga
f6724a1a01 Return the visible history with "Copy last reply" 2023-08-18 13:04:45 -07:00
oobabooga
b96fd22a81
Refactor the training tab (#3619) 2023-08-18 16:58:38 -03:00
oobabooga
c4733000d7 Return the visible history with "Remove last" 2023-08-18 09:25:51 -07:00
oobabooga
7cba000421
Bump llama-cpp-python, +tensor_split by @shouyiwang, +mul_mat_q (#3610) 2023-08-18 12:03:34 -03:00
oobabooga
bdb6eb5734 Restyle the chat input box + several CSS improvements
- Remove extra spacing below the last chat message
- Change the background color of code blocks in dark mode
- Remove border radius from selected header bar elements
- Make the chat scrollbar more discrete
2023-08-17 11:10:38 -07:00
oobabooga
cebe07f29c Unescape HTML inside code blocks 2023-08-16 21:08:26 -07:00
oobabooga
a4e903e932 Escape HTML in chat messages 2023-08-16 09:25:52 -07:00
oobabooga
73d9befb65 Make "Show controls" customizable through settings.yaml 2023-08-16 07:04:18 -07:00
oobabooga
2a29208224
Add a "Show controls" button to chat UI (#3590) 2023-08-16 02:39:58 -03:00
cal066
991bb57e43
ctransformers: Fix up model_type name consistency (#3567) 2023-08-14 15:17:24 -03:00
oobabooga
ccfc02a28d
Add the --disable_exllama option for AutoGPTQ (#3545 from clefever/disable-exllama) 2023-08-14 15:15:55 -03:00
oobabooga
7e57b35b5e Clean up old code 2023-08-14 10:10:39 -07:00
oobabooga
4d067e9b52 Add back a variable to keep old extensions working 2023-08-14 09:39:06 -07:00
oobabooga
d8a82d34ed Improve a warning 2023-08-14 08:46:05 -07:00
oobabooga
3e0a9f9cdb Refresh the character dropdown when saving/deleting a character 2023-08-14 08:23:41 -07:00
oobabooga
890b4abdad Fix session saving 2023-08-14 07:55:52 -07:00
oobabooga
619cb4e78b
Add "save defaults to settings.yaml" button (#3574) 2023-08-14 11:46:07 -03:00
oobabooga
a95e6f02cb Add a placeholder for custom stopping strings 2023-08-13 21:17:20 -07:00
oobabooga
ff9b5861c8 Fix impersonate when some text is present (closes #3564) 2023-08-13 21:10:47 -07:00
oobabooga
cc7e6ef645 Fix a CSS conflict 2023-08-13 19:24:09 -07:00
Eve
66c04c304d
Various ctransformers fixes (#3556)
---------

Co-authored-by: cal066 <cal066@users.noreply.github.com>
2023-08-13 23:09:03 -03:00
oobabooga
4a05aa92cb Add "send to" buttons for instruction templates
- Remove instruction templates from prompt dropdowns (default/notebook)
- Add 3 buttons to Parameters > Instruction template as a replacement
- Increase the number of lines of 'negative prompt' field to 3, and add a scrollbar
- When uploading a character, switch to the Character tab
- When uploading chat history, switch to the Chat tab
2023-08-13 18:35:45 -07:00
oobabooga
f6db2c78d1 Fix ctransformers seed 2023-08-13 05:48:53 -07:00
oobabooga
a1a9ec895d
Unify the 3 interface modes (#3554) 2023-08-13 01:12:15 -03:00
cal066
bf70c19603
ctransformers: move thread and seed parameters (#3543) 2023-08-13 00:04:03 -03:00
Chris Lefever
0230fa4e9c Add the --disable_exllama option for AutoGPTQ 2023-08-12 02:26:58 -04:00
oobabooga
0e05818266 Style changes 2023-08-11 16:35:57 -07:00
oobabooga
2f918ccf7c Remove unused parameter 2023-08-11 11:15:22 -07:00
oobabooga
28c8df337b Add repetition_penalty_range to ctransformers 2023-08-11 11:04:19 -07:00
cal066
7a4fcee069
Add ctransformers support (#3313)
---------

Co-authored-by: cal066 <cal066@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Co-authored-by: randoentity <137087500+randoentity@users.noreply.github.com>
2023-08-11 14:41:33 -03:00
oobabooga
8dbaa20ca8 Don't replace last reply with an empty message 2023-08-10 13:14:48 -07:00
oobabooga
0789554f65 Allow --lora to use an absolute path 2023-08-10 10:03:12 -07:00
oobabooga
3929971b66 Don't show oobabooga_llama-tokenizer in the model dropdown 2023-08-10 10:02:48 -07:00
oobabooga
c7f52bbdc1 Revert "Remove GPTQ-for-LLaMa monkey patch support"
This reverts commit e3d3565b2a.
2023-08-10 08:39:41 -07:00
jllllll
d6765bebc4
Update installation documentation 2023-08-10 00:53:48 -05:00
jllllll
d7ee4c2386
Remove unused import 2023-08-10 00:10:14 -05:00
jllllll
e3d3565b2a
Remove GPTQ-for-LLaMa monkey patch support
AutoGPTQ will be the preferred GPTQ LoRa loader in the future.
2023-08-09 23:59:04 -05:00
jllllll
bee73cedbd
Streamline GPTQ-for-LLaMa support 2023-08-09 23:42:34 -05:00
oobabooga
6c6a52aaad Change the filenames for caches and histories 2023-08-09 07:47:19 -07:00
oobabooga
d8fb506aff Add RoPE scaling support for transformers (including dynamic NTK)
https://github.com/huggingface/transformers/pull/24653
2023-08-08 21:25:48 -07:00
Friedemann Lipphardt
901b028d55
Add option for named cloudflare tunnels (#3364) 2023-08-08 22:20:27 -03:00
oobabooga
bf08b16b32 Fix disappearing profile picture bug 2023-08-08 14:09:01 -07:00
Gennadij
0e78f3b4d4
Fixed a typo in "rms_norm_eps", incorrectly set as n_gqa (#3494) 2023-08-08 00:31:11 -03:00
oobabooga
37fb719452
Increase the Context/Greeting boxes sizes 2023-08-08 00:09:00 -03:00
oobabooga
584dd33424
Fix missing example_dialogue when uploading characters 2023-08-07 23:44:59 -03:00
oobabooga
412f6ff9d3 Change alpha_value maximum and step 2023-08-07 06:08:51 -07:00
oobabooga
a373c96d59 Fix a bug in modules/shared.py 2023-08-06 20:36:35 -07:00
oobabooga
3d48933f27 Remove ancient deprecation warnings 2023-08-06 18:58:59 -07:00
oobabooga
c237ce607e Move characters/instruction-following to instruction-templates 2023-08-06 17:50:32 -07:00
oobabooga
65aa11890f
Refactor everything (#3481) 2023-08-06 21:49:27 -03:00
oobabooga
d4b851bdc8 Credit turboderp 2023-08-06 13:43:15 -07:00
oobabooga
0af10ab49b
Add Classifier Free Guidance (CFG) for Transformers/ExLlama (#3325) 2023-08-06 17:22:48 -03:00
missionfloyd
5134878344
Fix chat message order (#3461) 2023-08-05 13:53:54 -03:00
jllllll
44f31731af
Create logs dir if missing when saving history (#3462) 2023-08-05 13:47:16 -03:00
Forkoz
9dcb37e8d4
Fix: Mirostat fails on models split across multiple GPUs 2023-08-05 13:45:47 -03:00
oobabooga
8df3cdfd51
Add SSL certificate support (#3453) 2023-08-04 13:57:31 -03:00
missionfloyd
2336b75d92
Remove unnecessary chat.js (#3445) 2023-08-04 01:58:37 -03:00
oobabooga
4b3384e353 Handle unfinished lists during markdown streaming 2023-08-03 17:15:18 -07:00
Pete
f4005164f4
Fix llama.cpp truncation (#3400)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-08-03 20:01:15 -03:00
oobabooga
87dab03dc0
Add the --cpu option for llama.cpp to prevent CUDA from being used (#3432) 2023-08-03 11:00:36 -03:00
oobabooga
3e70bce576 Properly format exceptions in the UI 2023-08-03 06:57:21 -07:00
oobabooga
32c564509e Fix loading session in chat mode 2023-08-02 21:13:16 -07:00
oobabooga
0e8f9354b5 Add direct download for session/chat history JSONs 2023-08-02 19:43:39 -07:00
oobabooga
32a2bbee4a Implement auto_max_new_tokens for ExLlama 2023-08-02 11:03:56 -07:00
oobabooga
e931844fe2
Add auto_max_new_tokens parameter (#3419) 2023-08-02 14:52:20 -03:00
Pete
6afc1a193b
Add a scrollbar to notebook/default, improve chat scrollbar style (#3403)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-08-02 12:02:36 -03:00