Erik Johnston
4fb44fb5b9
Expose DataStore._get_events as get_events_as_list
...
This is in preparation for reaction work which requires it.
2019-05-14 13:37:44 +01:00
Amber Brown
7efd1d87c2
Run black on the rest of the storage module ( #4996 )
2019-04-03 10:07:29 +01:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place ( #4969 )
...
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Richard van der Hoff
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room ( #4535 )
2019-01-31 15:34:17 +00:00
Amber Brown
3f189c902e
Fix flake8 ( #4519 )
2019-01-30 10:53:17 +00:00
Erik Johnston
c21b7cbc09
Update synapse/storage/events_worker.py
2019-01-29 21:53:48 +00:00
Erik Johnston
4db252c073
Check redaction state when event is pulled out of the database
2019-01-29 21:48:36 +00:00
Erik Johnston
6d23ec2111
Fix typo
2019-01-29 21:45:53 +00:00
Erik Johnston
b5d510ad64
Remove unused arg
2019-01-29 21:45:28 +00:00
Richard van der Hoff
82165eeb05
Update synapse/storage/events_worker.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-01-29 21:14:39 +00:00
Erik Johnston
7709d2bd16
Implement rechecking of redactions
2019-01-29 11:56:20 +00:00
Erik Johnston
678a92cb56
Replace missed usages of FrozenEvent
2019-01-25 10:32:30 +00:00
Erik Johnston
be6a7e47fa
Revert "Require event format version to parse or create events"
2019-01-25 10:23:51 +00:00
Erik Johnston
e8c9f15397
Replace missed usages of FrozenEvent
2019-01-24 11:14:07 +00:00
Erik Johnston
be1065af59
isort
2019-01-23 11:48:16 +00:00
Erik Johnston
c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 ( #3725 )
2018-08-31 00:19:58 +10:00
Erik Johnston
96a9a29645
Pull in necessary stores in federation_reader
2018-08-06 15:23:57 +01:00
Erik Johnston
62ace05c45
Move necessary storage functions to worker classes
2018-08-06 15:23:38 +01:00
Richard van der Hoff
704c3e6239
Merge branch 'master' into develop
2018-08-02 15:43:30 +01:00
Richard van der Hoff
14fa9d4d92
Avoid extra db lookups
...
Since we're about to look up the events themselves anyway, we can skip the
extra db queries here.
2018-08-02 13:55:51 +01:00
Richard van der Hoff
667fba68f3
Run things as background processes
...
This fixes #3518 , and ensures that we get useful logs and metrics for lots of
things that happen in the background.
(There are certainly more things that happen in the background; these are just
the common ones I've found running a single-process synapse locally).
2018-07-18 20:55:05 +01:00
Richard van der Hoff
c3c29aa196
Attempt to include db threads in cpu usage stats ( #3496 )
...
Let's try to include time spent in the DB threads in the per-request/block cpu
usage metrics.
2018-07-10 16:12:36 +01:00
Richard van der Hoff
e31e5dee38
Add CPU metrics for _fetch_event_list
...
add a Measure block on _fetch_event_list, in the hope that we can better
measure CPU usage here.
2018-07-09 18:15:54 +01:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
Richard van der Hoff
70e6501913
Merge pull request #3419 from matrix-org/rav/events_per_request
...
Log number of events fetched from DB
2018-06-22 11:17:56 +01:00
Richard van der Hoff
0495fe0035
Indirect evt_count updates via method call
...
so that we can stub it for the sentinel and not have a billion failing UTs
2018-06-22 10:42:28 +01:00
Amber Brown
77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
Richard van der Hoff
b088aafcae
Log number of events fetched from DB
...
When we finish processing a request, log the number of events we fetched from
the database to handle it.
[I'm trying to figure out which requests are responsible for large amounts of
event cache churn. It may turn out to be more helpful to add counts to the
prometheus per-request/block metrics, but that is an extension to this code
anyway.]
2018-06-21 06:15:03 +01:00
Adrian Tschira
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
Richard van der Hoff
2a13af23bc
Use run_in_background in preference to preserve_fn
...
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Erik Johnston
415aeefd89
Format docstring
2018-04-12 12:07:09 +01:00
Erik Johnston
23a7f9d7f4
Doc we raise on unknown event
2018-04-12 11:20:51 +01:00
Erik Johnston
4dae4a97ed
Track last processed event received_ts
2018-04-11 14:27:09 +01:00
Erik Johnston
a8ce159be4
Replace some ujson with simplejson to make it work
2018-03-16 00:27:09 +00:00
Erik Johnston
eba93b05bf
Split EventsWorkerStore into separate file
2018-02-23 11:01:21 +00:00
