mirror of
https://github.com/nomic-ai/gpt4all.git
synced 2024-10-01 01:06:10 -04:00
69102a2859
* small edits and placeholder gif Signed-off-by: Max Cembalest <max@nomic.ai> * jul2 docs updates Signed-off-by: Max Cembalest <max@nomic.ai> * added video Signed-off-by: mcembalest <70534565+mcembalest@users.noreply.github.com> Signed-off-by: Max Cembalest <max@nomic.ai> * quantization nits Signed-off-by: Max Cembalest <max@nomic.ai> --------- Signed-off-by: Max Cembalest <max@nomic.ai> Signed-off-by: mcembalest <70534565+mcembalest@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| configs | ||
| figs | ||
| build_map.py | ||
| clean.py | ||
| create_hostname.sh | ||
| data.py | ||
| env.yaml | ||
| eval_figures.py | ||
| eval_self_instruct.py | ||
| generate.py | ||
| GPT-J_MAP.md | ||
| inference.py | ||
| launcher.sh | ||
| old-README.md | ||
| read.py | ||
| README.md | ||
| requirements.txt | ||
| train.py | ||
| TRAINING_LOG.md | ||
Training GPT4All-J
Technical Reports
📗 Technical Report 3: GPT4All Snoozy and Groovy
📗 Technical Report 2: GPT4All-J
GPT4All-J Training Data
- We are releasing the curated training data for anyone to replicate GPT4All-J here: GPT4All-J Training Data
We have released updated versions of our GPT4All-J model and training data.
v1.0: The original model trained on the v1.0 datasetv1.1-breezy: Trained on a filtered dataset where we removed all instances of AI language modelv1.2-jazzy: Trained on a filtered dataset where we also removed instances like I'm sorry, I can't answer... and AI language model
The models and data versions can be specified by passing a revision argument.
For example, to load the v1.2-jazzy model and dataset, run:
from datasets import load_dataset
from transformers import AutoModelForCausalLM
dataset = load_dataset("nomic-ai/gpt4all-j-prompt-generations", revision="v1.2-jazzy")
model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-j", revision="v1.2-jazzy")
GPT4All-J Training Instructions
accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16 --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config_gptj.json train.py --config configs/train/finetune_gptj.yaml