27 Commits

Author	SHA1	Message	Date
James Ravenscroft	215a69b5af	update clblast code in gpt-j model	2023-08-26 16:16:01 +01:00
James Ravenscroft	0b408510f4	add gpu offload for gpt-j models (codegen)	2023-08-26 16:11:16 +01:00
James Ravenscroft	604183380d	tidy up prints in stablecoder and starcoder	2023-08-26 16:04:41 +01:00
James Ravenscroft	4a47251822	update for gpu build	2023-08-26 15:13:08 +01:00
James Ravenscroft	b2b4a1480f	increase scratch on starcoder	2023-08-26 15:12:41 +01:00
James Ravenscroft	5f7155a314	add gpu offload for gptneox	2023-08-26 15:12:41 +01:00
James Ravenscroft	dc81abbc52	Merge branch 'main' into feature/model-lock	2023-08-24 14:58:44 +01:00
James Ravenscroft	f69a8f65d4	fix build?	2023-08-24 13:49:14 +01:00
James Ravenscroft	e8beac34e7	more attempts to build with boost threads	2023-08-24 13:17:51 +01:00
James Ravenscroft	cceee41f79	add boost threads	2023-08-24 13:03:12 +01:00
James Ravenscroft	2d617b458e	expose batch size flag to cli	2023-08-24 11:40:19 +01:00
James Ravenscroft	0c1fc1a04e	implement locking of model per request to prevent crashing when multiple requests reeived	2023-08-24 11:30:44 +01:00
James Ravenscroft	6a72e32dab	add debug output to starcoder	2023-08-23 15:53:38 +00:00
James Ravenscroft	bad53ad190	add debug logs to codegen	2023-08-23 14:57:31 +00:00
James Ravenscroft	cd9baacdd7	add debug logs to stablecode	2023-08-23 14:56:36 +00:00
James Ravenscroft	88c5ede768	fix cast problem due to incorrect float notation	2023-08-23 08:12:53 +00:00
James Ravenscroft	757cc64a59	add temp and top p as cli args	2023-08-23 06:27:37 +01:00
James Ravenscroft	f7f1991e2c	add huggingface request handler and refactor old req handler	2023-08-10 09:26:54 +01:00
James Ravenscroft	84869ff0f3	added support for gptneox models	2023-08-10 08:39:14 +01:00
James Ravenscroft	430733c7b8	sort out args	2023-08-05 08:29:18 +01:00
James Ravenscroft	c48cd786ed	replace crow submodule with header-only impl	2023-08-04 22:48:11 +01:00
James Ravenscroft	7dee51782d	tidy up arm args	2023-08-04 22:18:30 +01:00
James Ravenscroft	00ec442860	try to move arm options to top level	2023-08-04 07:14:38 +01:00
James Ravenscroft	3f3bff73b6	add processor specific flags for arm	2023-08-04 07:05:20 +01:00
James Ravenscroft	dfa4b5e74f	add starcoder/wizardcoder/santacoder support	2023-07-29 15:28:07 +01:00
James Ravenscroft	fd3a127aaa	add server component	2023-07-23 17:50:53 +01:00
James Ravenscroft	887d348188	add new code	2023-07-23 17:06:44 +01:00