13 Commits

Author	SHA1	Message	Date
Zach Nussbaum	9dfd8e1a7c	fix: num training steps for lr decay	2023-04-10 02:15:31 +00:00
Zach Nussbaum	9efdf56e38	fix: saving name	2023-04-08 20:56:13 +00:00
Zach Nussbaum	31195270cb	fix: eos/pad token + wd	2023-04-08 20:38:10 +00:00
Zach	809680d621	fix: grad accum loss calc	2023-04-06 12:11:10 +00:00
Zach	a57adb0344	fix: try except push	2023-04-05 20:42:22 +00:00
Zach Nussbaum	97d4499d79	fix: only on first process, not once on every node	2023-04-05 02:36:22 +00:00
Zach Nussbaum	d0402288bd	fix: eval func	2023-04-04 23:25:37 +00:00
Zach Nussbaum	3efc19ebc5	feat: adamw, fix training, log gradients	2023-04-04 20:57:42 +00:00
Zach Nussbaum	812b807003	fix: log for multiple epochs	2023-03-28 18:47:58 +00:00
Zach Nussbaum	b24ca03dc4	fix: append first_epoch	2023-03-27 17:30:24 +00:00
Zach Nussbaum	24765a1965	fix: add epoch train	2023-03-27 16:32:35 +00:00
Zach Nussbaum	2daecd6066	feat: update embeddings	2023-03-26 17:45:21 +00:00
Zach Nussbaum	723a50bdf1	feat: train and clean data	2023-03-25 16:17:48 +00:00