AI/gpt4all
1
0
Fork 0
mirror of https://github.com/nomic-ai/gpt4all.git synced 2024-10-01 01:06:10 -04:00
Code Issues Packages Projects Releases Wiki Activity
92bc92d232
gpt4all/.codespellrc

5 lines
81 B
Plaintext
Raw Normal View History

Add rudimentary codespell config
2023-05-15 16:44:10 -04:00
[codespell]
New tokenizer implementation for MPT and GPT-J Improves output quality by making these tokenizers more closely match the behavior of the huggingface `tokenizers` based BPE tokenizers these models were trained with. Featuring: * Fixed unicode handling (via ICU) * Fixed BPE token merge handling * Complete added vocabulary handling
2023-05-21 08:18:42 -04:00
skip = .git,*.pdf,*.svg,*_tokenizer_config.h
Add rudimentary codespell config
2023-05-15 16:44:10 -04:00
#
# ignore-words-list =
Reference in New Issue Copy Permalink