Richard van der Hoff
1e19ce00bf
Add 'failure_ts' column to 'destinations' table ( #6016 )
...
Track the time that a server started failing at, for general analysis purposes.
2019-09-17 11:41:54 +01:00
Richard van der Hoff
7902bf1e1d
Clean up some code in the retry logic ( #6017 )
...
* remove some unused code
* make things which were constants into constants for efficiency and clarity
2019-09-11 15:14:56 +01:00
Amber Brown
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
Erik Johnston
2410335507
Use upsert when updating destination retry interval
2019-07-19 13:40:24 +01:00
Richard van der Hoff
a6a776f3d8
remove dead transaction persist code ( #5622 )
...
this hasn't done anything for years
2019-07-05 12:59:42 +01:00
Amber Brown
7efd1d87c2
Run black on the rest of the storage module ( #4996 )
2019-04-03 10:07:29 +01:00
Amber Brown
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) ( #4068 )
2018-10-20 11:16:55 +11:00
Erik Johnston
7258d081a5
Fix bug when invalidating destination retry timings
2018-10-02 15:47:57 +01:00
Erik Johnston
fdd1a62e8d
Add a five minute cache to get_destination_retry_timings
...
Hopefully helps with #3931
2018-09-21 14:56:12 +01:00
Erik Johnston
bbab6ebfd9
Fix up changelog and remove spurious comment
2018-09-19 14:45:14 +01:00
Erik Johnston
392a54128c
pep8
2018-09-19 14:37:49 +01:00
Erik Johnston
b9158ac2bf
Remove get_destination_retry_timings cache
...
Currently we rely on the master to invalidate this cache promptly.
However, after having moved most federation endpoints off of master this
no longer happens, causing outbound fedeariont to get blackholed.
Fixes #3798
2018-09-19 14:22:57 +01:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 ( #3725 )
2018-08-31 00:19:58 +10:00
Richard van der Hoff
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Richard van der Hoff
371da42ae4
Wrap a number of things that run in the background
...
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
2018-07-25 09:41:12 +01:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
Adrian Tschira
4b9d0cde97
add remaining memoryview changes
2018-05-29 17:42:43 +02:00
Erik Johnston
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
Richard van der Hoff
6cfee09be9
Make __init__ consitstent across Store heirarchy
...
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
Erik Johnston
54fed21c04
Fix tests and flake8
2016-11-22 18:18:31 +00:00
Erik Johnston
90565d015e
Invalidate retry cache in both directions
2016-11-22 17:45:44 +00:00
Erik Johnston
760469c812
Continue to clean up received_transactions
2016-11-01 11:42:08 +00:00
Erik Johnston
f09d2b692f
Removed unused stuff
2016-10-31 17:10:56 +00:00
Erik Johnston
4c3eb14d68
Increase batching of sent transaction inserts
...
This should further reduce the number of individual inserts,
transactions and updates that are required for keeping sent_transactions
up to date.
2016-10-31 16:07:45 +00:00
Erik Johnston
cce957e254
Bump max_entries on get_destination_retry_timings
2016-09-02 14:08:33 +01:00
Erik Johnston
ef0cc648cf
Clean up old sent transactions
2016-08-31 11:12:02 +01:00
Erik Johnston
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
Mark Haines
87ffd21b29
Fix a couple of bugs in the transaction and keyring code
2016-07-26 19:19:08 +01:00
Erik Johnston
d13459636f
Pull prev txn from in memory
2016-05-06 11:30:55 +01:00
Erik Johnston
1d275dba69
Don't needlessly enter transaction
2016-05-06 11:25:58 +01:00
Erik Johnston
56b5e83e36
Reduce database inserts when sending transactions
2016-05-06 11:20:18 +01:00
Mark Haines
54172924c8
Load the current id in the IdGenerator constructor
...
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.
It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Mark Haines
c0a279e808
Delete the table objects from TransactionStore
2016-01-13 11:15:20 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Erik Johnston
39de87869c
Fix bug where assumed dict was namedtuple
2015-11-12 14:47:48 +00:00
Erik Johnston
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
Mark Haines
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
Erik Johnston
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
Erik Johnston
0cf7e480b4
And use buffer(...) there as well
2015-05-05 18:20:01 +01:00
Erik Johnston
977338a7af
Use buffer(...) when inserting into bytea column
2015-05-05 18:12:53 +01:00
Erik Johnston
2732be83d9
Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible.
2015-04-27 13:22:30 +01:00
Erik Johnston
25d2914fba
Turn off persistance of sent_transactions.response_json
2015-04-15 14:09:35 +01:00
Erik Johnston
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
Erik Johnston
304111afd0
Don't use AUTOINCREMENT, use an in memory version
2015-04-07 12:05:36 +01:00
Erik Johnston
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
Erik Johnston
1aa44939fc
Fix bugs in transactions storage
2015-03-24 16:20:05 +00:00
Erik Johnston
9115421ace
Use _simple_upsert
2015-03-24 16:17:39 +00:00
Erik Johnston
d5272b1d2c
Use 'update or insert' rather than on 'conflict replace'
2015-03-23 14:02:34 +00:00
Erik Johnston
278149f533
Sanitize TransactionStore
2015-03-23 13:43:21 +00:00
