Patrick Cloke
f460da6031
Consistently use db_to_json to convert from database values to JSON objects. ( #7849 )
2020-07-16 11:32:19 -04:00
Richard van der Hoff
e6027562e2
remove builtins.buffer code from storage code
...
this is no longer needed on python 3
2020-05-15 19:37:41 +01:00
Erik Johnston
d7983b63a6
Support any process writing to cache invalidation stream. ( #7436 )
2020-05-07 13:51:08 +01:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Erik Johnston
5d7a6ad223
Allow streaming cache invalidate all to workers. ( #6749 )
2020-01-22 10:37:00 +00:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory ( #6648 )
...
* Add a background update to clear tombstoned rooms from the directory
* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Erik Johnston
2284eb3a53
Add database config class ( #6513 )
...
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
Erik Johnston
d537be1ebd
Pass Database into the data store
2019-12-06 15:49:44 +00:00
Erik Johnston
9a4fb457cf
Change DataStores to accept 'database' param.
2019-12-06 13:30:06 +00:00
Erik Johnston
8863624f78
Comments
2019-12-05 10:46:37 +00:00
Erik Johnston
756d4942f5
Move DB pool and helper functions into dedicated Database class
2019-12-05 10:46:37 +00:00
Erik Johnston
ee86abb2d6
Remove underscore from SQLBaseStore functions
2019-12-04 16:23:43 +00:00
Erik Johnston
c2f525a525
Don't call SQLBaseStore methods from outside stores
2019-12-04 16:23:43 +00:00
Erik Johnston
9186c105a0
Revert "Move get_user_count_txn out of base store"
...
This reverts commit 00f0d67566
2019-12-04 15:46:19 +00:00
Erik Johnston
00f0d67566
Move get_user_count_txn out of base store
2019-12-04 15:21:14 +00:00
Erik Johnston
1056d6885a
Move cache invalidation to main data store
2019-12-04 15:21:14 +00:00
Erik Johnston
6b2867096b
Move event fetch vars to EventWorkStore
2019-12-04 15:21:14 +00:00
Erik Johnston
ddd48b6851
Move account validity bg updates to registration store
2019-12-04 15:21:14 +00:00
Richard van der Hoff
c01d543584
Make sure that we close cursors before returning from a query ( #6408 )
...
There are lots of words in the comment as to why this is a good idea.
Fixes #6403 .
2019-11-25 21:03:17 +00:00
Andrew Morgan
3916e1b97a
Clean up newline quote marks around the codebase ( #6362 )
2019-11-21 12:00:14 +00:00
Andrew Morgan
745a48625d
Fix guest -> real account upgrade with account validity enabled ( #6359 )
2019-11-14 12:02:05 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston
b54b1e759a
Fix SQLite take 2
2019-10-10 16:19:40 +01:00
Erik Johnston
3bc687508f
Remove add_in_list_sql_clause
2019-10-10 15:35:46 +01:00
Erik Johnston
9d06fb9cb1
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_any
2019-10-10 14:38:21 +01:00
Erik Johnston
b4fbf71187
Add helper funcs to use postgres ANY
...
This means that we can write queries with `col = ANY(?)`, which helps
postgres.
2019-10-10 13:15:24 +01:00
Erik Johnston
132279a46f
Patch inlinecallbacks for log contexts
2019-09-27 15:11:14 +01:00
Richard van der Hoff
49ef8ec399
Fix a cache-invalidation bug for worker-based deployments ( #5920 )
...
Some of the caches on worker processes were not being correctly invalidated
when a room's state was changed in a way that did not affect the membership
list of the room.
We need to make sure we send out cache invalidations even when no memberships
are changing.
2019-08-28 10:18:16 +01:00
Amber Brown
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
Erik Johnston
bd2e1a2aa8
LoggingTransaction accepts None for callback lists.
...
Its a bit disingenuousto give LoggingTransaction lists to append
callbacks to if we're not going to run the callbacks.
2019-07-19 13:36:04 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
Richard van der Hoff
e59a8cd2e5
Merge pull request #5499 from matrix-org/rav/cleanup_metrics
...
Cleanups and sanity-checking in cpu and db metrics
2019-06-24 17:12:54 +01:00
Richard van der Hoff
1793de6c6d
black
2019-06-24 11:16:13 +01:00
Richard van der Hoff
5097aee740
Merge branch 'develop' into rav/cleanup_metrics
2019-06-24 10:00:13 +01:00
Richard van der Hoff
c753c098dd
Merge pull request #5498 from matrix-org/rav/fix_clock_reversal
...
Use monotonic clock where possible for metrics
2019-06-24 09:55:12 +01:00
Amber Brown
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
Richard van der Hoff
f682af052d
Simplify PerformanceCounters.update interface
...
we already have the duration for the update, so may as well use it rather than
passing extra params around and recalculating it.
2019-06-19 21:18:38 +01:00
Richard van der Hoff
68128d5626
Remove unused _get_event_counters
...
This has been redundant since cdb3757942
2019-06-19 21:14:25 +01:00
Richard van der Hoff
15bf32e062
Use monotonic clock where possible for metrics
...
Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.
Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.
2019-06-19 21:09:43 +01:00
Brendan Abolivier
6d56a694f4
Don't send renewal emails to deactivated users
2019-06-14 15:05:56 +01:00
Andrew Morgan
3719680ee4
Add ability to perform password reset via email without trusting the identity server ( #5377 )
...
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.
This PR is a culmination of 3 smaller PRs which have each been separately reviewed:
* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Erik Johnston
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
...
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
Brendan Abolivier
4d794dae21
Move delta from +10% to -10%
2019-05-31 11:09:39 +01:00
Brendan Abolivier
847b9dcd1c
Make max_delta equal to period * 10%
2019-05-31 09:54:46 +01:00
Erik Johnston
7e8e683754
Log actual number of entries deleted
2019-05-29 15:11:28 +01:00
Brendan Abolivier
52839886d6
Allow configuring a range for the account validity startup job
...
When enabling the account validity feature, Synapse will look at startup for registered account without an expiration date, and will set one equals to 'now + validity_period' for them. On large servers, it can mean that a large number of users will have the same expiration date, which means that they will all be sent a renewal email at the same time, which isn't ideal.
In order to mitigate this, this PR allows server admins to define a 'max_delta' so that the expiration date is a random value in the [now + validity_period ; now + validity_period + max_delta] range. This allows renewal emails to be progressively sent over a configured period instead of being sent all in one big batch.
2019-05-28 16:52:45 +01:00
Brendan Abolivier
5ceee46c6b
Do the select and insert in a single transaction
2019-05-21 13:38:51 +01:00
Brendan Abolivier
ad5b4074e1
Add startup background job for account validity
...
If account validity is enabled in the server's configuration, this job will run at startup as a background job and will stick an expiration date to any registered account missing one.
2019-05-17 19:37:31 +01:00
Amber Brown
a33a5abc4c
Clean up the database pagination code ( #5007 )
...
* rewrite & simplify
* changelog
* cleanup potential sql injection
2019-04-05 00:21:16 +11:00
Amber Brown
7efd1d87c2
Run black on the rest of the storage module ( #4996 )
2019-04-03 10:07:29 +01:00
