Erik Johnston
b9158ac2bf
Remove get_destination_retry_timings cache
...
Currently we rely on the master to invalidate this cache promptly.
However, after having moved most federation endpoints off of master this
no longer happens, causing outbound fedeariont to get blackholed.
Fixes #3798
2018-09-19 14:22:57 +01:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 ( #3725 )
2018-08-31 00:19:58 +10:00
Richard van der Hoff
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Richard van der Hoff
371da42ae4
Wrap a number of things that run in the background
...
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
2018-07-25 09:41:12 +01:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
Adrian Tschira
4b9d0cde97
add remaining memoryview changes
2018-05-29 17:42:43 +02:00
Erik Johnston
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
Richard van der Hoff
6cfee09be9
Make __init__ consitstent across Store heirarchy
...
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
Erik Johnston
54fed21c04
Fix tests and flake8
2016-11-22 18:18:31 +00:00
Erik Johnston
90565d015e
Invalidate retry cache in both directions
2016-11-22 17:45:44 +00:00
Erik Johnston
760469c812
Continue to clean up received_transactions
2016-11-01 11:42:08 +00:00
Erik Johnston
f09d2b692f
Removed unused stuff
2016-10-31 17:10:56 +00:00
Erik Johnston
4c3eb14d68
Increase batching of sent transaction inserts
...
This should further reduce the number of individual inserts,
transactions and updates that are required for keeping sent_transactions
up to date.
2016-10-31 16:07:45 +00:00
Erik Johnston
cce957e254
Bump max_entries on get_destination_retry_timings
2016-09-02 14:08:33 +01:00
Erik Johnston
ef0cc648cf
Clean up old sent transactions
2016-08-31 11:12:02 +01:00
Erik Johnston
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
Mark Haines
87ffd21b29
Fix a couple of bugs in the transaction and keyring code
2016-07-26 19:19:08 +01:00
Erik Johnston
d13459636f
Pull prev txn from in memory
2016-05-06 11:30:55 +01:00
Erik Johnston
1d275dba69
Don't needlessly enter transaction
2016-05-06 11:25:58 +01:00
Erik Johnston
56b5e83e36
Reduce database inserts when sending transactions
2016-05-06 11:20:18 +01:00
Mark Haines
54172924c8
Load the current id in the IdGenerator constructor
...
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.
It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Mark Haines
c0a279e808
Delete the table objects from TransactionStore
2016-01-13 11:15:20 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Erik Johnston
39de87869c
Fix bug where assumed dict was namedtuple
2015-11-12 14:47:48 +00:00
Erik Johnston
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
Mark Haines
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
Erik Johnston
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
Erik Johnston
0cf7e480b4
And use buffer(...) there as well
2015-05-05 18:20:01 +01:00
Erik Johnston
977338a7af
Use buffer(...) when inserting into bytea column
2015-05-05 18:12:53 +01:00
Erik Johnston
2732be83d9
Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible.
2015-04-27 13:22:30 +01:00
Erik Johnston
25d2914fba
Turn off persistance of sent_transactions.response_json
2015-04-15 14:09:35 +01:00
Erik Johnston
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
Erik Johnston
304111afd0
Don't use AUTOINCREMENT, use an in memory version
2015-04-07 12:05:36 +01:00
Erik Johnston
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
Erik Johnston
1aa44939fc
Fix bugs in transactions storage
2015-03-24 16:20:05 +00:00
Erik Johnston
9115421ace
Use _simple_upsert
2015-03-24 16:17:39 +00:00
Erik Johnston
d5272b1d2c
Use 'update or insert' rather than on 'conflict replace'
2015-03-23 14:02:34 +00:00
Erik Johnston
278149f533
Sanitize TransactionStore
2015-03-23 13:43:21 +00:00
Erik Johnston
f6583796fe
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-03-20 16:31:48 +00:00
Erik Johnston
fce0114005
Start removing Tables
2015-03-20 15:05:44 +00:00
Erik Johnston
d288d273e1
Generate transaction id in code
2015-03-20 10:57:44 +00:00
Erik Johnston
d4f50f3ae5
decode_result takes an iterable
2015-03-20 10:57:26 +00:00
Erik Johnston
d7a0496f3e
Convert storage layer to be mysql compatible
2015-03-19 15:59:48 +00:00
Mark Haines
74048bdd41
Remove unused import
2015-02-23 18:17:43 +00:00
Paul "LeoNerd" Evans
044d813ef7
Use the @cached decorator to implement the destination_retry_timings cache
2015-02-23 16:04:40 +00:00
Erik Johnston
f6da237c35
Add index on transaction_id to sent_transcations
2015-01-06 15:40:38 +00:00
Erik Johnston
9bd07bed23
Actually time that function
2015-01-06 15:28:56 +00:00
Erik Johnston
03a501456c
Time how long calls to _get_destination_retry_timings take
2015-01-06 15:22:28 +00:00
Mark Haines
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
