veilid-dot-com/pages/framework/rpc.md
2023-08-15 15:25:21 -04:00

3.7 KiB
Raw Blame History

title description weight layout
RPC Protocol An overview of the RPC protocol used in Veilid 4 subpage

Strong, appropriate, cryptography choices are essential to the functioning of Veilid.

Veilid provides applications guarantees about how data is handled on the wire and at rest.

Cryptosystems were chosen that work well together and provide a balance of speed and cryptographic hardness.

RPC Summary

  • Schema Language is Cap'n Proto

    Capn Proto is designed for deserialization speed and schema evolution. Flexible and well supported in Rust.
  • RPC is fully in-schema and documented

    Both Question/Answer and Statement RPC modes are supported. All schema fields are documented.
  • RPC fully supports Private Routing

    All private routing structures are expressed in the RPC schema itself, no magic encrypted blobs.
  • Schema Evolution is built-in

    Fields can be added and removed with full backward and forward compatibility. New features wont break older Veilid nodes.
  • RPC Schema is cryptography-independent

    As cryptosystems change, the language spoken by Veilid nodes remains the same.

Distributed Hash Table

Distributed Hash Tables are a way of storing data in records that have keys that are close to nodes in the network.

It may look complicated, but all the DHT algorithms out there are just search algorithms. Finding data that is stored on some node somewhere out there.

We built a better DHT by making both search and data locality more relevant. Veilid synchronizes popular data when nodes come and go from the network.

a tree diagram for the search ability

Locating a node by its ID. Here the node with the prefix 0011 finds the node with the prefix 1110 by successively learning of and querying closer and closer nodes. The line segment on top represents the space of 160-bit IDs, and shows how the lookups coverge to the target node. Below we illustrate RPC messages made by 1110. The first RPC is to node 101, already known to 1110. Subsequent RPCs are to nodes return by the previous RPC.

DHT Schema

Veilid DHT is built using GetValue and SetValue RPC operations. Nodes can opt out of DHT storage if they do not want to participate.

Veilid DHT records have schemas that define subkeys that are individually addressable and can have multiple writers.

DHT record subkeys have sequence numbers and are eventually consistent across multiple writes and background synchronizations.

a diagram showing key-value pairs

Veilid Default DHT Schema - DFLT

To zoom in on the details, view the image directly.

a diagram showing key-value pairs, but with more fields

Veild Simple DHT Schema - SMPL

To zoom in on the details, view the image directly.

The DHT gives you full control over your data

Our DHT is not based on a blockchain or a coin

Popular data becomes more available automatically