Add better docs and threading support to bert.

gpt4all-bindings/python/docs/gpt4all_python.md

@@ -1,8 +1,7 @@
-# GPT4All Python API
+# GPT4All Python Generation API
 The `GPT4All` python package provides bindings to our C/C++ model backend libraries.
 The source code and local build instructions can be found [here](https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/python).
 
-
 ## Quickstart
 
 ```bash
@@ -109,22 +108,5 @@ with model.chat_session():
     print(model.current_chat_session)
 ```
 
-### Generating embeddings
-GPT4All includes a super simple means of generating embeddings for your text documents. The embedding model will automatically be downloaded if not installed.
-
-=== "Embed4All Example"
-    ``` py
-    from gpt4all import GPT4All, Embed4All
-    text = 'The quick brown fox jumps over the lazy dog'
-    embedder = Embed4All()
-    output = embedder.embed(text)
-    print(output)
-    ```
-=== "Output"
-    ```
-    [0.034696947783231735, -0.07192722707986832, 0.06923297047615051, ...]
-    ```
-
 ### API documentation
 ::: gpt4all.gpt4all.GPT4All
-::: gpt4all.gpt4all.Embed4All

gpt4all-bindings/python/docs/gpt4all_python_embedding.md

@@ -0,0 +1,35 @@
+# GPT4All Python Embedding API
+GPT4All includes a super simple means of generating embeddings for your text documents.
+
+## Quickstart
+
+```bash
+pip install gpt4all
+```
+
+### Generating embeddings
+The embedding model will automatically be downloaded if not installed.
+
+=== "Embed4All Example"
+    ``` py
+    from gpt4all import GPT4All, Embed4All
+    text = 'The quick brown fox jumps over the lazy dog'
+    embedder = Embed4All()
+    output = embedder.embed(text)
+    print(output)
+    ```
+=== "Output"
+    ```
+    [0.034696947783231735, -0.07192722707986832, 0.06923297047615051, ...]
+    ```
+### Speed of embedding generation
+The following table lists the generation speed for text documents of N tokens captured on an Intel i913900HX CPU with DDR5 5600 running with 8 threads under stable load.
+
+| Tokens          | 2^7  | 2^9  | 2^11 | 2^13 | 2^14 |
+| --------------- | ---- | ---- | ---- | ---- | ---- |
+| Wall time (s)   | .02  | .08  | .24  | .96  | 1.9  |
+| Tokens / Second | 6508 | 6431 | 8622 | 8509 | 8369 |
+
+
+### API documentation
+::: gpt4all.gpt4all.Embed4All

gpt4all-bindings/python/gpt4all/gpt4all.py

@@ -20,12 +20,16 @@ class Embed4All:
     Python class that handles embeddings for GPT4All.
     """
     def __init__(
-        self
+        self,
+        n_threads: Optional[int] = None,
     ):
         """
         Constructor
+
+        Args:
+            n_threads: number of CPU threads used by GPT4All. Default is None, then the number of threads are determined automatically.
         """
-        self.gpt4all = GPT4All(model_name='ggml-all-MiniLM-L6-v2-f16.bin')
+        self.gpt4all = GPT4All(model_name='ggml-all-MiniLM-L6-v2-f16.bin', n_threads=n_threads)
 
     def embed(
         self,
@@ -65,7 +69,7 @@ class GPT4All:
             model_type: Model architecture. This argument currently does not have any functionality and is just used as
                 descriptive identifier for user. Default is None.
             allow_download: Allow API to download models from gpt4all.io. Default is True.
-            n_threads: number of CPU threads used by GPT4All. Default is None, than the number of threads are determined automatically.
+            n_threads: number of CPU threads used by GPT4All. Default is None, then the number of threads are determined automatically.
         """
         self.model_type = model_type
         self.model = pyllmodel.LLModel()

gpt4all-bindings/python/gpt4all/tests/test_embed_timings.py

@@ -0,0 +1,18 @@
+import sys
+from io import StringIO
+
+from gpt4all import GPT4All, Embed4All
+import time
+
+def time_embedding(i, embedder):
+    text = 'foo bar ' * i
+    start_time = time.time()
+    output = embedder.embed(text)
+    end_time = time.time()
+    elapsed_time = end_time - start_time
+    print(f"Time report: {2 * i / elapsed_time} tokens/second with {2 * i} tokens taking {elapsed_time} seconds")
+
+if __name__ == "__main__":
+    embedder = Embed4All(n_threads=8)
+    for i in [2**n for n in range(6, 14)]:
+        time_embedding(i, embedder)

gpt4all-bindings/python/mkdocs.yml

@@ -10,7 +10,9 @@ use_directory_urls: false
 nav:
     - 'index.md'
     - 'Bindings':
-      - 'GPT4All in Python': 'gpt4all_python.md'
+      - 'GPT4All in Python':
+        - 'Generation': 'gpt4all_python.md'
+        - 'Embedding': 'gpt4all_python_embedding.md'
       - 'GPT4All Chat Client': 'gpt4all_chat.md'
       - 'gpt4all_cli.md'
 #    - 'Tutorials':

gpt4all-bindings/python/setup.py

@@ -61,7 +61,7 @@ copy_prebuilt_C_lib(SRC_CLIB_DIRECtORY,
 
 setup(
     name=package_name,
-    version="1.0.5",
+    version="1.0.6",
     description="Python bindings for GPT4All",
     author="Richard Guo",
     author_email="richard@nomic.ai",