gpt4all/README.md

GPT4All

Privacy-oriented software for chatting with large language models that run on your own computer.

Official Website • Documentation • Discord

Official Download Links: Windows — macOS — Ubuntu

NEW: Subscribe to our mailing list for updates and news!

GPT4All is made possible by our compute partner Paperspace.

Run on an M2 MacBook Pro (not sped up!)

About GPT4All

GPT4All is an ecosystem to run powerful and customized large language models that work locally on consumer grade CPUs and NVIDIA and AMD GPUs. Note that your CPU needs to support AVX instructions.

Learn more in the documentation.

A GPT4All model is a 3GB - 8GB file that you can download and plug into the GPT4All software. Nomic AI supports and maintains this software ecosystem to enforce quality and security alongside spearheading the effort to allow any person or enterprise to easily deploy their own on-edge large language models.

What's New

• October 19th, 2023: GGUF Support Launches with Support for:
  • Mistral 7b base model, an updated model gallery on gpt4all.io, several new local code models including Rift Coder v1.5
  • Nomic Vulkan support for Q4_0 and Q4_1 quantizations in GGUF.
  • Offline build support for running old versions of the GPT4All Local LLM Chat Client.
• September 18th, 2023: Nomic Vulkan launches supporting local LLM inference on NVIDIA and AMD GPUs.
• July 2023: Stable support for LocalDocs, a feature that allows you to privately and locally chat with your data.
• June 28th, 2023: Docker-based API server launches allowing inference of local LLMs from an OpenAI-compatible HTTP endpoint.

Building From Source

• Follow the instructions here to build the GPT4All Chat UI from source.

Bindings

• 🐍 Official Python Bindings
• 💻 Typescript Bindings

Integrations

• 🦜🔗 Langchain
• 🗃️ Weaviate Vector Database - module docs

Contributing

GPT4All welcomes contributions, involvement, and discussion from the open source community! Please see CONTRIBUTING.md and follow the issues, bug reports, and PR markdown templates.

Check project discord, with project owners, or through existing issues/PRs to avoid duplicate work. Please make sure to tag all of the above with relevant project identifiers or your contribution could potentially get lost. Example tags: backend, bindings, python-bindings, documentation, etc.

GPT4All 2024 Roadmap

To contribute to the development of any of the below roadmap items, make or find the corresponding issue and cross-reference the in-progress task.

Each item should have an issue link below.

• Chat UI Language Localization (localize UI into the native languages of users)

  • Chinese
  • German
  • French
  • Portuguese
  • Your native language here.

• UI Redesign: an internal effort at Nomic to improve the UI/UX of gpt4all for all users.

  • Design new user interface and gather community feedback
  • Implement the new user interface and experience.

• Installer and Update Improvements

  • Seamless native installation and update process on OSX
  • Seamless native installation and update process on Windows
  • Seamless native installation and update process on Linux

• Model discoverability improvements:

  • Support huggingface model discoverability
  • Support Nomic hosted model discoverability

• LocalDocs (towards a local perplexity)

  • Multilingual LocalDocs Support
    • Create a multilingual experience
    • Incorporate a multilingual embedding model
    • Specify a preferred multilingual LLM for localdocs
  • Improved RAG techniques
    • Query augmentation and re-writing
    • Improved chunking and text extraction from arbitrary modalities
      • Custom PDF extractor past the QT default (charts, tables, text)
    • Faster indexing and local exact search with v1.5 hamming embeddings and reranking (skip ANN index construction!)
  • Support queries like 'summarize X document'
  • Multimodal LocalDocs support with Nomic Embed
  • Nomic Dataset Integration with real-time LocalDocs
    • Include an option to allow the export of private LocalDocs collections to Nomic Atlas for debugging data/chat quality
    • Allow optional sharing of LocalDocs collections between users.
    • Allow the import of a LocalDocs collection from an Atlas Datasets
      • Chat with live version of Wikipedia, Chat with Pubmed, chat with the latest snapshot of world news.

• First class Multilingual LLM Support

  • Recommend and set a default LLM for German
  • Recommend and set a default LLM for English
  • Recommend and set a default LLM for Chinese
  • Recommend and set a default LLM for Spanish

• Server Mode improvements

  • Improved UI and new requested features:
    • Fix outstanding bugs and feature requests around networking configurations.
    • Support Nomic Embed inferencing
    • First class documentation
    • Improving developer use and quality of server mode (e.g. support larger batches)

Technical Reports

📗 Technical Report 3: GPT4All Snoozy and Groovy

📗 Technical Report 2: GPT4All-J

📗 Technical Report 1: GPT4All

Citation

If you utilize this repository, models or data in a downstream project, please consider citing it with:

@misc{gpt4all,
  author = {Yuvanesh Anand and Zach Nussbaum and Brandon Duderstadt and Benjamin Schmidt and Andriy Mulyar},
  title = {GPT4All: Training an Assistant-style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/nomic-ai/gpt4all}},
}