mirror of
https://git.anonymousland.org/anonymousland/synapse.git
synced 2025-08-05 10:04:16 -04:00
when we're talking about backfilling data in federation, call it backfilling - not pagination.
This commit is contained in:
parent
f729f13735
commit
59dfbaba3b
9 changed files with 54 additions and 54 deletions
|
@ -24,7 +24,7 @@ Where the bottom (the transport layer) is what talks to the internet via HTTP, a
|
|||
* duplicate pdu_id's - i.e., it makes sure we ignore them.
|
||||
* responding to requests for a given pdu_id
|
||||
* responding to requests for all metadata for a given context (i.e. room)
|
||||
* handling incoming pagination requests
|
||||
* handling incoming backfill requests
|
||||
|
||||
So it has to parse incoming messages to discover which are metadata and which aren't, and has to correctly clobber existing metadata where appropriate.
|
||||
|
||||
|
|
|
@ -155,9 +155,9 @@ To fetch all the state of a given context:
|
|||
PDUs that encode the state.
|
||||
|
||||
|
||||
To paginate events on a given context:
|
||||
To backfill events on a given context:
|
||||
|
||||
GET .../paginate/:context/
|
||||
GET .../backfill/:context/
|
||||
Query args: v, limit
|
||||
|
||||
Response: JSON encoding of a single Transaction containing multiple PDUs
|
||||
|
|
|
@ -1,11 +1,11 @@
|
|||
Versioning is, like, hard for paginating backwards because of the number of Home Servers involved.
|
||||
Versioning is, like, hard for backfilling backwards because of the number of Home Servers involved.
|
||||
|
||||
The way we solve this is by doing versioning as an acyclic directed graph of PDUs. For pagination purposes, this is done on a per context basis.
|
||||
The way we solve this is by doing versioning as an acyclic directed graph of PDUs. For backfilling purposes, this is done on a per context basis.
|
||||
When we send a PDU we include all PDUs that have been received for that context that hasn't been subsequently listed in a later PDU. The trivial case is a simple list of PDUs, e.g. A <- B <- C. However, if two servers send out a PDU at the same to, both B and C would point at A - a later PDU would then list both B and C.
|
||||
|
||||
Problems with opaque version strings:
|
||||
- How do you do clustering without mandating that a cluster can only have one transaction in flight to a given remote home server at a time.
|
||||
If you have multiple transactions sent at once, then you might drop one transaction, receive anotherwith a version that is later than the dropped transaction and which point ARGH WE LOST A TRANSACTION.
|
||||
- How do you do pagination? A version string defines a point in a stream w.r.t. a single home server, not a point in the context.
|
||||
If you have multiple transactions sent at once, then you might drop one transaction, receive another with a version that is later than the dropped transaction and which point ARGH WE LOST A TRANSACTION.
|
||||
- How do you do backfilling? A version string defines a point in a stream w.r.t. a single home server, not a point in the context.
|
||||
|
||||
We only need to store the ends of the directed graph, we DO NOT need to do the whole one table of nodes and one of edges.
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue