5 Commits

Author	SHA1	Message	Date
Aaron Miller	7a5f6e4726	limit prompt batch size to 128	2023-06-30 21:07:21 -03:00
Aaron Miller	88616fde7f	llmodel: change tokenToString to not use string_view (#968) ... fixes a definite use-after-free and likely avoids some other potential ones - std::string will convert to a std::string_view automatically but as soon as the std::string in question goes out of scope it is already freed and the string_view is pointing at freed memory - this is *mostly* fine if its returning a reference to the tokenizer's internal vocab table but it's, imo, too easy to return a reference to a dynamically constructed string with this as replit is doing (and unfortunately needs to do to convert the internal whitespace replacement symbol back to a space)	2023-06-13 07:14:02 -04:00
Adam Treat	301d2fdbea	Fix up for newer models on reset context. This fixes the model from totally failing after a reset context.	2023-06-04 19:31:20 -04:00
AT	bbe195ee02	Backend prompt dedup (#822) ... * Deduplicated prompt() function code	2023-06-04 08:59:24 -04:00
Adam Treat	70e3b7e907	Try and fix build on mac.	2023-06-02 10:47:12 -04:00