Commit Graph

252 Commits

Author SHA1 Message Date
James Ravenscroft
b79ab46b50 add gpu offload for gptneox 2023-08-26 15:14:02 +01:00
James Ravenscroft
4a47251822 update for gpu build 2023-08-26 15:13:08 +01:00
James Ravenscroft
b2b4a1480f increase scratch on starcoder 2023-08-26 15:12:41 +01:00
James Ravenscroft
5f7155a314 add gpu offload for gptneox 2023-08-26 15:12:41 +01:00
James Ravenscroft
5b561f7b7e
Merge pull request #61 from c01o/patch-1
Fix download link on MODELS.md
2023-08-26 12:37:21 +01:00
c01o
e85492d8ba
Fix download link on MODELS.md 2023-08-26 19:16:15 +09:00
James Ravenscroft
f840ea0b73
Merge pull request #60 from nvtienanh/update-dockerfile-default
Change from alpine to ubuntu in Dockerfile.default
2023-08-26 09:43:24 +01:00
Anh Nguyen
3e37c4bb7c Update cmake command 2023-08-26 14:27:27 +07:00
Anh Nguyen
20b1460bd8 Using GGML_STATIC 2023-08-26 14:21:18 +07:00
Anh Nguyen
2308b9ae21 Change from alpine to ubuntu in dockerfile.default 2023-08-26 13:21:04 +07:00
James Ravenscroft
c4e57e0aab
Merge pull request #58 from ravenscroftj/feature/model-lock
WIP: implement locking of model per request
2023-08-25 06:57:05 +01:00
James Ravenscroft
dc81abbc52 Merge branch 'main' into feature/model-lock 2023-08-24 14:58:44 +01:00
James Ravenscroft
ae2d505a2f use std mutex instead of boost mutex 2023-08-24 14:55:23 +01:00
James Ravenscroft
143155dac3 boost 2023-08-24 14:28:01 +01:00
James Ravenscroft
596c835939 boost 2023-08-24 14:26:40 +01:00
James Ravenscroft
6b0a25cb71
Merge pull request #59 from ravenscroftj/feature/batch-flag
expose batch size flag to cli
2023-08-24 14:13:40 +01:00
James Ravenscroft
227501188c try to set boost librarydir 2023-08-24 13:57:22 +01:00
James Ravenscroft
f69a8f65d4 fix build? 2023-08-24 13:49:14 +01:00
James Ravenscroft
22f2993db4 try using stage lib dir for boost root 2023-08-24 13:28:58 +01:00
James Ravenscroft
e8beac34e7 more attempts to build with boost threads 2023-08-24 13:17:51 +01:00
James Ravenscroft
ccf425f019 update deps for boost 2023-08-24 13:05:10 +01:00
James Ravenscroft
cceee41f79 add boost threads 2023-08-24 13:03:12 +01:00
James Ravenscroft
113544400a try adding build boost dirs explicitely 2023-08-24 12:04:45 +01:00
James Ravenscroft
f0627cd567 add boost libraries to cmake 2023-08-24 11:50:21 +01:00
James Ravenscroft
2d617b458e expose batch size flag to cli 2023-08-24 11:40:19 +01:00
James Ravenscroft
0c1fc1a04e implement locking of model per request to prevent crashing when multiple requests reeived 2023-08-24 11:30:44 +01:00
James Ravenscroft
e0adf0519b
Merge pull request #57 from aperullo/model-docs-fix
Fix incorrect instructions in model docs
2023-08-24 10:18:36 +01:00
aperullo
ef1402a1a8 Fix errant command in model docs 2023-08-23 17:59:43 -04:00
James Ravenscroft
11f385066a
Merge pull request #54 from ravenscroftj/feature/debug-timing-logs
Implemented debug log level and added timings to model outputs
2023-08-23 17:17:20 +01:00
James Ravenscroft
6a72e32dab add debug output to starcoder 2023-08-23 15:53:38 +00:00
James Ravenscroft
bad53ad190 add debug logs to codegen 2023-08-23 14:57:31 +00:00
James Ravenscroft
cd9baacdd7 add debug logs to stablecode 2023-08-23 14:56:36 +00:00
James Ravenscroft
6d90e5d870
Merge pull request #53 from ravenscroftj/feature/expose-temp-top_p
fix cast problem due to incorrect float notation
2023-08-23 10:42:10 +01:00
James Ravenscroft
88c5ede768 fix cast problem due to incorrect float notation 2023-08-23 08:12:53 +00:00
James Ravenscroft
1e0d2d3434
Merge pull request #52 from ravenscroftj/feature/expose-temp-top_p
add temp and top p as cli args
2023-08-23 09:03:28 +01:00
James Ravenscroft
757cc64a59 add temp and top p as cli args 2023-08-23 06:27:37 +01:00
James Ravenscroft
8be7171573
Merge pull request #48 from ravenscroftj/fix/docker-static-builds
add check for GGML_STATIC and compile accordingly
2023-08-13 09:51:45 +01:00
James Ravenscroft
2f0e2dee94 add check for GGML_STATIC and compile accordingly 2023-08-13 08:34:07 +01:00
James Ravenscroft
c0cde10046
Merge pull request #44 from ravenscroftj/feature/hf-code
Feature/hf code
2023-08-10 10:08:20 +01:00
James Ravenscroft
12a3fe9367 Merge remote-tracking branch 'origin/main' into feature/hf-code 2023-08-10 09:35:22 +01:00
James Ravenscroft
de68ab022c
Merge pull request #43 from ravenscroftj/feature/gptneox
Feature/gptneox
2023-08-10 09:34:54 +01:00
James Ravenscroft
f7f1991e2c add huggingface request handler and refactor old req handler 2023-08-10 09:26:54 +01:00
James Ravenscroft
6ee2d3dc66 update readme 2023-08-10 08:57:19 +01:00
James Ravenscroft
18faa3e5f6 update readme with stablecode refs 2023-08-10 08:54:16 +01:00
James Ravenscroft
84869ff0f3 added support for gptneox models 2023-08-10 08:39:14 +01:00
James Ravenscroft
d618fb1aec update link to 2b multi codegen 2023-08-05 12:10:21 +01:00
James Ravenscroft
66270d3a9b
Merge pull request #39 from ravenscroftj/fix/build
add .exe
2023-08-05 10:09:59 +01:00
James Ravenscroft
c7b4fcddba add .exe 2023-08-05 10:09:05 +01:00
James Ravenscroft
14050e15ba
Merge pull request #38 from ravenscroftj/fix/build
Fix/build
2023-08-05 10:02:59 +01:00
James Ravenscroft
815ae3a088 fix windows release 2023-08-05 10:02:06 +01:00