Commit Graph

407 Commits

Author SHA1 Message Date
Patrick Cloke
637282bb50
Add additional type hints to the storage module. () 2020-12-30 08:09:53 -05:00
Richard van der Hoff
97647b33c2
Replace DeferredCache with LruCache where possible ()
Most of these uses don't need a full-blown DeferredCache; LruCache is lighter and more appropriate.
2020-10-19 12:20:29 +01:00
Patrick Cloke
eebf52be06
Be stricter about JSON that is accepted by Synapse () 2020-08-19 07:26:03 -04:00
Patrick Cloke
fbe930dad2
Convert the roommember database to async/await. () 2020-08-12 12:14:34 -04:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense () 2020-08-05 21:38:57 +01:00
Patrick Cloke
f460da6031
Consistently use db_to_json to convert from database values to JSON objects. () 2020-07-16 11:32:19 -04:00
Richard van der Hoff
e6027562e2 remove builtins.buffer code from storage code
this is no longer needed on python 3
2020-05-15 19:37:41 +01:00
Erik Johnston
d7983b63a6
Support any process writing to cache invalidation stream. () 2020-05-07 13:51:08 +01:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ()
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Erik Johnston
5d7a6ad223
Allow streaming cache invalidate all to workers. () 2020-01-22 10:37:00 +00:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory ()
* Add a background update to clear tombstoned rooms from the directory

* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Erik Johnston
2284eb3a53
Add database config class ()
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
Erik Johnston
d537be1ebd Pass Database into the data store 2019-12-06 15:49:44 +00:00
Erik Johnston
9a4fb457cf Change DataStores to accept 'database' param. 2019-12-06 13:30:06 +00:00
Erik Johnston
8863624f78 Comments 2019-12-05 10:46:37 +00:00
Erik Johnston
756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Erik Johnston
ee86abb2d6 Remove underscore from SQLBaseStore functions 2019-12-04 16:23:43 +00:00
Erik Johnston
c2f525a525 Don't call SQLBaseStore methods from outside stores 2019-12-04 16:23:43 +00:00
Erik Johnston
9186c105a0 Revert "Move get_user_count_txn out of base store"
This reverts commit 00f0d67566.

Its going to get removed soon, so lets not make merge conflicts.
2019-12-04 15:46:19 +00:00
Erik Johnston
00f0d67566 Move get_user_count_txn out of base store 2019-12-04 15:21:14 +00:00
Erik Johnston
1056d6885a Move cache invalidation to main data store 2019-12-04 15:21:14 +00:00
Erik Johnston
6b2867096b Move event fetch vars to EventWorkStore 2019-12-04 15:21:14 +00:00
Erik Johnston
ddd48b6851 Move account validity bg updates to registration store 2019-12-04 15:21:14 +00:00
Richard van der Hoff
c01d543584
Make sure that we close cursors before returning from a query ()
There are lots of words in the comment as to why this is a good idea.

Fixes .
2019-11-25 21:03:17 +00:00
Andrew Morgan
3916e1b97a
Clean up newline quote marks around the codebase () 2019-11-21 12:00:14 +00:00
Andrew Morgan
745a48625d
Fix guest -> real account upgrade with account validity enabled () 2019-11-14 12:02:05 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ()
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston
b54b1e759a Fix SQLite take 2 2019-10-10 16:19:40 +01:00
Erik Johnston
3bc687508f Remove add_in_list_sql_clause 2019-10-10 15:35:46 +01:00
Erik Johnston
9d06fb9cb1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_any 2019-10-10 14:38:21 +01:00
Erik Johnston
b4fbf71187 Add helper funcs to use postgres ANY
This means that we can write queries with `col = ANY(?)`, which helps
postgres.
2019-10-10 13:15:24 +01:00
Erik Johnston
132279a46f Patch inlinecallbacks for log contexts 2019-09-27 15:11:14 +01:00
Richard van der Hoff
49ef8ec399
Fix a cache-invalidation bug for worker-based deployments ()
Some of the caches on worker processes were not being correctly invalidated
when a room's state was changed in a way that did not affect the membership
list of the room.

We need to make sure we send out cache invalidations even when no memberships
are changing.
2019-08-28 10:18:16 +01:00
Amber Brown
4806651744
Replace returnValue with return () 2019-07-23 23:00:55 +10:00
Erik Johnston
bd2e1a2aa8 LoggingTransaction accepts None for callback lists.
Its a bit disingenuousto give LoggingTransaction lists to append
callbacks to if we're not going to run the callbacks.
2019-07-19 13:36:04 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ () 2019-07-04 00:07:04 +10:00
Richard van der Hoff
e59a8cd2e5
Merge pull request from matrix-org/rav/cleanup_metrics
Cleanups and sanity-checking in cpu and db metrics
2019-06-24 17:12:54 +01:00
Richard van der Hoff
1793de6c6d black 2019-06-24 11:16:13 +01:00
Richard van der Hoff
5097aee740 Merge branch 'develop' into rav/cleanup_metrics 2019-06-24 10:00:13 +01:00
Richard van der Hoff
c753c098dd
Merge pull request from matrix-org/rav/fix_clock_reversal
Use monotonic clock where possible for metrics
2019-06-24 09:55:12 +01:00
Amber Brown
32e7c9e7f2
Run Black. () 2019-06-20 19:32:02 +10:00
Richard van der Hoff
f682af052d Simplify PerformanceCounters.update interface
we already have the duration for the update, so may as well use it rather than
passing extra params around and recalculating it.
2019-06-19 21:18:38 +01:00
Richard van der Hoff
68128d5626 Remove unused _get_event_counters
This has been redundant since cdb3757942.
2019-06-19 21:14:25 +01:00
Richard van der Hoff
15bf32e062 Use monotonic clock where possible for metrics
Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.

Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.
2019-06-19 21:09:43 +01:00
Brendan Abolivier
6d56a694f4 Don't send renewal emails to deactivated users 2019-06-14 15:05:56 +01:00
Andrew Morgan
3719680ee4
Add ability to perform password reset via email without trusting the identity server ()
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* 
* 
* 
2019-06-06 17:34:07 +01:00
Erik Johnston
58cce39f3a
Merge pull request from matrix-org/babolivier/account_validity_job_delta
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
Brendan Abolivier
4d794dae21 Move delta from +10% to -10% 2019-05-31 11:09:39 +01:00
Brendan Abolivier
847b9dcd1c Make max_delta equal to period * 10% 2019-05-31 09:54:46 +01:00
Erik Johnston
7e8e683754 Log actual number of entries deleted 2019-05-29 15:11:28 +01:00