Commit Graph

7 Commits

Author SHA1 Message Date
FartyPants (FP HAM)
6ab477f375
training: Added ChatML-format.json format example (#5899) 2024-07-20 19:05:09 -03:00
jeffbiocode
3168644152
Training: Update llama2-chat-format.json (#5593) 2024-03-03 12:42:14 -03:00
jeffbiocode
45c4cd01c5
Add llama 2 chat format for lora training (#5553) 2024-02-25 02:36:36 -03:00
kizinfo
5d513eea22
Add ability to load all text files from a subdirectory for training (#1997)
* Update utils.py

returns individual txt files and subdirectories to getdatasets to allow for training from a directory of text files

* Update training.py

minor tweak to training on raw datasets to detect if a directory is selected, and if so, to load in all the txt files in that directory for training

* Update put-trainer-datasets-here.txt

document

* Minor change

* Use pathlib, sort by natural keys

* Space

---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-07-12 11:44:30 -03:00
Fernando Tarin Morales
1f540fa4f8
Added the format to be able to finetune Vicuna1.1 models (#3037) 2023-07-07 02:22:39 -03:00
oobabooga
88ad86249d Remove unnecessary file 2023-03-28 13:19:52 -03:00
Alex "mcmonkey" Goodwin
566898a79a initial lora training tab 2023-03-25 12:08:26 -07:00