Commit Graph

3083 Commits

Author SHA1 Message Date
Erik Johnston
a2f6d31a63 Refactor get_user_ids_changed to pull less from DB
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.

This PR changes this to only check for changes for users the client is
actually interested in.

Fixes #5553
2019-06-26 12:03:44 +01:00
Richard van der Hoff
e59a8cd2e5
Merge pull request #5499 from matrix-org/rav/cleanup_metrics
Cleanups and sanity-checking in cpu and db metrics
2019-06-24 17:12:54 +01:00
Richard van der Hoff
1793de6c6d black 2019-06-24 11:16:13 +01:00
Richard van der Hoff
5097aee740 Merge branch 'develop' into rav/cleanup_metrics 2019-06-24 10:00:13 +01:00
Richard van der Hoff
c753c098dd
Merge pull request #5498 from matrix-org/rav/fix_clock_reversal
Use monotonic clock where possible for metrics
2019-06-24 09:55:12 +01:00
Richard van der Hoff
0e8b35f7b0
Fix "Unexpected entry in 'full_schemas'" log warning (#5509)
There is a README.txt which always sets off this warning, which is a bit
alarming when you first start synapse. I don't think we need to warn about
this.
2019-06-21 15:11:57 +01:00
Erik Johnston
f8bd30af2a Black 2019-06-20 13:10:06 +01:00
Erik Johnston
45f28a9d2f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/histogram_extremities 2019-06-20 11:59:14 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff
f682af052d Simplify PerformanceCounters.update interface
we already have the duration for the update, so may as well use it rather than
passing extra params around and recalculating it.
2019-06-19 21:18:38 +01:00
Richard van der Hoff
68128d5626 Remove unused _get_event_counters
This has been redundant since cdb3757942.
2019-06-19 21:14:25 +01:00
Richard van der Hoff
15bf32e062 Use monotonic clock where possible for metrics
Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.

Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.
2019-06-19 21:09:43 +01:00
Erik Johnston
e0be8d7016
Merge pull request #5480 from matrix-org/erikj/extremities_dummy_events
Add experimental option to reduce extremities.
2019-06-19 13:19:18 +01:00
Erik Johnston
65787b0f7c Add descriptions and remove redundant set(..) 2019-06-19 11:49:39 +01:00
Brendan Abolivier
092e36457a
Fix typo in deactivation background job 2019-06-19 11:09:41 +01:00
Erik Johnston
b42f90470f Add experimental option to reduce extremities.
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.

THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
Erik Johnston
499d4a32cd Add metrics for len of new extremities persisted.
Of new events being persisted add metrics for total size of forward
extremities and number of unchanged, "stale" extremities.
2019-06-17 17:02:48 +01:00
Amber Brown
eba7caf09f
Remove Postgres 9.4 support (#5448) 2019-06-18 00:59:00 +10:00
Erik Johnston
9ca4ae7131
Merge pull request #5461 from matrix-org/erikj/histograms_are_cumalitive
Prometheus histograms are cumalative
2019-06-14 18:21:42 +01:00
Brendan Abolivier
e0b77b004d
Fix background job for deactivated flag 2019-06-14 16:00:45 +01:00
Brendan Abolivier
6d56a694f4 Don't send renewal emails to deactivated users 2019-06-14 15:05:56 +01:00
Erik Johnston
3ed595e327 Prometheus histograms are cumalative 2019-06-14 14:07:32 +01:00
Brendan Abolivier
d0530382ee
Track deactivated accounts in the database (#5378) 2019-06-14 13:18:24 +01:00
Amber Brown
6312d6cc7c
Expose statistics on extrems to prometheus (#5384) 2019-06-13 22:40:52 +10:00
Erik Johnston
5bc9484537 Merge branch 'release-v1.0.0' of github.com:matrix-org/synapse into develop 2019-06-11 10:37:43 +01:00
Neil Johnson
94dac0f3e5
add monthly active users to phonehome stats (#5252)
* add monthly active users to phonehome stats
2019-06-10 23:33:59 +01:00
Erik Johnston
49e01e5710 Fix defaults on checking threepids 2019-06-10 23:09:31 +01:00
Erik Johnston
c413540fb9 Fix bug sending federation transactions with lots of EDUs
If we try and send a transaction with lots of EDUs and we run out of
space, we call get_new_device_msgs_for_remote with a limit of 0, which
then failed.
2019-06-10 16:22:45 +01:00
Erik Johnston
43badd2cd4 Fix key verification when key stored with null valid_until_ms
Some keys are stored in the synapse database with a null valid_until_ms
which caused an exception to be thrown when using that key. We fix this
by treating nulls as zeroes, i.e. they keys will match verification
requests with a minimum_valid_until_ms of zero (i.e. don't validate ts)
but will not match requests with a non-zero minimum_valid_until_ms.

Fixes #5391.
2019-06-10 15:37:10 +01:00
Andrew Morgan
2d1d7b7e6f Prevent multiple device list updates from breaking a batch send (#5156)
fixes #5153
2019-06-06 23:54:00 +01:00
Erik Johnston
1b8cb64393
Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1
Make a full SQL schema
2019-06-06 18:35:38 +01:00
Amber H. Brown
ed872db8df fix maybe 2019-06-07 02:53:47 +10:00
Andrew Morgan
3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377)
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Amber H. Brown
837e32ef55 just user dir? 2019-06-07 01:49:25 +10:00
Amber H. Brown
3b6645d3bf remove background updates that arent needed 2019-06-07 01:20:58 +10:00
Erik Johnston
89d3d7b2c0
Merge pull request #5221 from matrix-org/erikj/fix_worker_sytest
Fix get_max_topological_token to never return None
2019-06-06 13:54:46 +01:00
Erik Johnston
7f08a3523a Better words 2019-06-06 11:09:38 +01:00
Amber Brown
6362e3af14 add more comments 2019-06-06 04:20:35 +10:00
Erik Johnston
75538813fc Fix background updates to handle redactions/rejections (#5352)
* Fix background updates to handle redactions/rejections

In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
2019-06-06 00:45:46 +10:00
Erik Johnston
0a56966f7d Fix 2019-06-03 17:42:52 +01:00
Erik Johnston
fa4b54aca5 Ignore room state with null bytes in for room stats 2019-06-03 17:06:54 +01:00
Amber Brown
2198b7ce2a add stuff in bg updates 2019-06-04 01:06:00 +10:00
Amber Brown
4e75c5e02a WHY IS THIS CALLED A SLIGHTLY DIFFERENT THING 2019-06-03 22:42:12 +10:00
Amber Brown
be452fc9ac more fix 2019-06-03 22:24:23 +10:00
Amber Brown
7f81b967ca fix schemas 2019-06-03 22:23:40 +10:00
Amber Brown
dc72b90cd6 full schema 2019-06-03 22:03:28 +10:00
Amber Brown
37057d5d60 prepare 2019-06-03 22:02:47 +10:00
Erik Johnston
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
Brendan Abolivier
4d794dae21 Move delta from +10% to -10% 2019-05-31 11:09:39 +01:00
Erik Johnston
31d44ec4bd
Merge pull request #5294 from matrix-org/erikj/speed_up_room_stats
Speed up room stats background update
2019-05-31 10:48:51 +01:00