Commit Graph

154 Commits

Author SHA1 Message Date
Zach Nussbaum
cd6a054a6c chore: remove not needed 2023-04-11 12:39:07 +00:00
Zach Nussbaum
9056a46b55 chore: submodule ff 2023-04-10 02:16:05 +00:00
Zach Nussbaum
bbbf007ed9 Merge branch 'gptj' of github.com:nomic-ai/gpt4all into gptj 2023-04-10 02:15:47 +00:00
Zach Nussbaum
9dfd8e1a7c fix: num training steps for lr decay 2023-04-10 02:15:31 +00:00
Zach
311c818934 feat: evals on new gptj models 2023-04-10 02:14:20 +00:00
Zach
195f8a7d4e fix: topic model for embeddings 2023-04-09 15:12:49 +00:00
Zach Nussbaum
7807a80bbb fix: bs try one more time? 2023-04-08 21:47:07 +00:00
Zach Nussbaum
2f0eba211d fix: smaller bs for 40gb 2023-04-08 21:36:20 +00:00
Zach Nussbaum
7f95ab3a06 fix: config for lora gptj 2023-04-08 21:17:12 +00:00
Zach Nussbaum
9efdf56e38 fix: saving name 2023-04-08 20:56:13 +00:00
Zach Nussbaum
633df8edb4 Merge remote-tracking branch 'origin/mosaic' into gptj 2023-04-08 20:47:01 +00:00
Zach Nussbaum
31195270cb fix: eos/pad token + wd 2023-04-08 20:38:10 +00:00
Zach Nussbaum
c82ee7d882 fix: add wd + min lr to config 2023-04-08 20:37:51 +00:00
Zach Nussbaum
be3f528810 fix: tokenization error 2023-04-08 20:33:51 +00:00
Zach Nussbaum
b66f127ade fix: config + ignore pkl 2023-04-08 20:33:02 +00:00
Zach Nussbaum
0606ab46b9 feat: build map script 2023-04-08 19:30:53 +00:00
Zach
1c6d2d9622 fix: embeddings instead of logits!!! 2023-04-08 17:05:40 +00:00
zanussbaum
147c2fd7eb feat: lora gptj 2023-04-07 17:53:07 -04:00
zanussbaum
2b001e8932 fix: batch size 2023-04-07 17:41:45 -04:00
zanussbaum
7cfda6a21f feat: update for mosaic 2023-04-07 16:54:29 -04:00
Zach Nussbaum
4b51e6ef37 fix: pyarrow filter 2023-04-07 19:04:19 +00:00
Zach Nussbaum
7a9f6d1cdc fix: inference save shards 2023-04-07 16:23:34 +00:00
Zach
0bd6acb4dd fix: drop uneven batch size 2023-04-07 12:09:31 +00:00
Zach
985da51fbc fix: concat 2023-04-07 04:33:34 +00:00
Zach
1b14b1f723 fix: data for inference 2023-04-07 01:45:07 +00:00
Zach
fb9ff9c40d feat: inference for embedding plots 2023-04-07 01:40:39 +00:00
Zach
809680d621 fix: grad accum loss calc 2023-04-06 12:11:10 +00:00
Zach
7751f39432 fix: data processing 2023-04-06 03:03:34 +00:00
Zach
5baead45be fix: configs 2023-04-05 20:42:35 +00:00
Zach
a57adb0344 fix: try except push 2023-04-05 20:42:22 +00:00
Zach Nussbaum
399a65e779 feat: multinode setup 2023-04-05 02:53:04 +00:00
Zach Nussbaum
0a3834d086 fix: gptj multinode 2023-04-05 02:52:44 +00:00
Zach Nussbaum
fde7d9506f fix: ignore env 2023-04-05 02:52:21 +00:00
Zach Nussbaum
97d4499d79 fix: only on first process, not once on every node 2023-04-05 02:36:22 +00:00
Zach Nussbaum
d0402288bd fix: eval func 2023-04-04 23:25:37 +00:00
Zach
65ec606f21 fix: prompt len for larger 2023-04-04 22:01:55 +00:00
Zach Nussbaum
df2d5f7e46 feat: gpt-j config 2023-04-04 20:58:08 +00:00
Zach Nussbaum
3efc19ebc5 feat: adamw, fix training, log gradients 2023-04-04 20:57:42 +00:00
Zach Nussbaum
5c5f41ba36 fix: clean up data, pad at end 2023-04-04 20:53:23 +00:00
Zach Nussbaum
2e2e9f4339 fix: clean where prompt is randomly 1 char 2023-04-04 20:47:21 +00:00
Zach Nussbaum
2e3e35c7a2 chore: gitignore ckpts 2023-04-04 20:46:57 +00:00
Andriy Mulyar
252676ff05
Merge pull request #3 from nomic-ai/train
log wandb multi-epoch
2023-03-29 13:50:26 -04:00
Andriy Mulyar
aa4dd0eaef
Qualified number of epochs for LoRa weights 2023-03-29 12:26:47 -04:00
Andriy Mulyar
b10890f8f9
Merge pull request #42 from tiendung/main
Change git clone url to https://github.com/nomic-ai/gpt4all.git to avoid `Permission denied`
2023-03-29 10:50:44 -04:00
Andriy Mulyar
c197eee060
Added Torrent Magnet Link 2023-03-29 10:47:19 -04:00
Andriy Mulyar
8e5511896c
Merge pull request #36 from Hello1024/patch-1
Update README.md to add torrent link to data
2023-03-29 10:38:26 -04:00
Andriy Mulyar
753c6baf96
Merge branch 'main' into patch-1 2023-03-29 10:38:17 -04:00
Andriy Mulyar
220df0924f
Merge pull request #31 from EliasVincent/chat-windows-binary
Add chat binary for Windows
2023-03-29 10:35:38 -04:00
Andriy Mulyar
f5d46e8182
Merge branch 'main' into chat-windows-binary 2023-03-29 10:35:31 -04:00
Andriy Mulyar
e2c59517f2
Merge pull request #26 from dsernst/mac-intel-bin
Add binary for Intel Macs
2023-03-29 10:34:52 -04:00