164 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Zach Nussbaum	4903c4ca9f	fix: train gpt-j command	2023-04-13 17:59:19 +00:00
Zach Nussbaum	a4e8616c76	fix: update config	2023-04-13 17:58:59 +00:00
Zach Nussbaum	362dcee7e3	fix: map links	2023-04-13 17:52:04 +00:00
Zach Nussbaum	1a2703b1e9	fix: gpt-j data link	2023-04-13 17:41:07 +00:00
Andriy Mulyar	1b44dfbefd	Update README.md	2023-04-13 12:57:12 -04:00
Andriy Mulyar	049fd2fd50	Update README.md	2023-04-13 12:56:08 -04:00
Zach Nussbaum	a3485c4b32	Merge: main into gptj	2023-04-13 15:16:31 +00:00
Zach Nussbaum	8a94a8c068	fix: multi-turn data breaks	2023-04-12 03:51:29 +00:00
Zach Nussbaum	15f7c5b68f	chore: peft	2023-04-12 03:50:54 +00:00
Zach Nussbaum	e550e4ed34	feat: commits for eval + generation	2023-04-11 19:14:29 +00:00
Zach Nussbaum	cd6a054a6c	chore: remove not needed	2023-04-11 12:39:07 +00:00
Zach Nussbaum	9056a46b55	chore: submodule ff	2023-04-10 02:16:05 +00:00
Zach Nussbaum	bbbf007ed9	Merge branch 'gptj' of github.com:nomic-ai/gpt4all into gptj	2023-04-10 02:15:47 +00:00
Zach Nussbaum	9dfd8e1a7c	fix: num training steps for lr decay	2023-04-10 02:15:31 +00:00
Zach	311c818934	feat: evals on new gptj models	2023-04-10 02:14:20 +00:00
Zach	195f8a7d4e	fix: topic model for embeddings	2023-04-09 15:12:49 +00:00
Zach Nussbaum	7807a80bbb	fix: bs try one more time?	2023-04-08 21:47:07 +00:00
Zach Nussbaum	2f0eba211d	fix: smaller bs for 40gb	2023-04-08 21:36:20 +00:00
Zach Nussbaum	7f95ab3a06	fix: config for lora gptj	2023-04-08 21:17:12 +00:00
Zach Nussbaum	9efdf56e38	fix: saving name	2023-04-08 20:56:13 +00:00
Zach Nussbaum	633df8edb4	Merge remote-tracking branch 'origin/mosaic' into gptj	2023-04-08 20:47:01 +00:00
Zach Nussbaum	31195270cb	fix: eos/pad token + wd	2023-04-08 20:38:10 +00:00
Zach Nussbaum	c82ee7d882	fix: add wd + min lr to config	2023-04-08 20:37:51 +00:00
Zach Nussbaum	be3f528810	fix: tokenization error	2023-04-08 20:33:51 +00:00
Zach Nussbaum	b66f127ade	fix: config + ignore pkl	2023-04-08 20:33:02 +00:00
Zach Nussbaum	0606ab46b9	feat: build map script	2023-04-08 19:30:53 +00:00
Zach	1c6d2d9622	fix: embeddings instead of logits!!!	2023-04-08 17:05:40 +00:00
zanussbaum	147c2fd7eb	feat: lora gptj	2023-04-07 17:53:07 -04:00
zanussbaum	2b001e8932	fix: batch size	2023-04-07 17:41:45 -04:00
zanussbaum	7cfda6a21f	feat: update for mosaic	2023-04-07 16:54:29 -04:00
Zach Nussbaum	4b51e6ef37	fix: pyarrow filter	2023-04-07 19:04:19 +00:00
Zach Nussbaum	7a9f6d1cdc	fix: inference save shards	2023-04-07 16:23:34 +00:00
Zach	0bd6acb4dd	fix: drop uneven batch size	2023-04-07 12:09:31 +00:00
Zach	985da51fbc	fix: concat	2023-04-07 04:33:34 +00:00
Zach	1b14b1f723	fix: data for inference	2023-04-07 01:45:07 +00:00
Zach	fb9ff9c40d	feat: inference for embedding plots	2023-04-07 01:40:39 +00:00
Zach	809680d621	fix: grad accum loss calc	2023-04-06 12:11:10 +00:00
Zach	7751f39432	fix: data processing	2023-04-06 03:03:34 +00:00
Zach	5baead45be	fix: configs	2023-04-05 20:42:35 +00:00
Zach	a57adb0344	fix: try except push	2023-04-05 20:42:22 +00:00
Zach Nussbaum	399a65e779	feat: multinode setup	2023-04-05 02:53:04 +00:00
Zach Nussbaum	0a3834d086	fix: gptj multinode	2023-04-05 02:52:44 +00:00
Zach Nussbaum	fde7d9506f	fix: ignore env	2023-04-05 02:52:21 +00:00
Zach Nussbaum	97d4499d79	fix: only on first process, not once on every node	2023-04-05 02:36:22 +00:00
Zach Nussbaum	d0402288bd	fix: eval func	2023-04-04 23:25:37 +00:00
Zach	65ec606f21	fix: prompt len for larger	2023-04-04 22:01:55 +00:00
Zach Nussbaum	df2d5f7e46	feat: gpt-j config	2023-04-04 20:58:08 +00:00
Zach Nussbaum	3efc19ebc5	feat: adamw, fix training, log gradients	2023-04-04 20:57:42 +00:00
Zach Nussbaum	5c5f41ba36	fix: clean up data, pad at end	2023-04-04 20:53:23 +00:00
Zach Nussbaum	2e2e9f4339	fix: clean where prompt is randomly 1 char	2023-04-04 20:47:21 +00:00