Erik Johnston
|
9a05795619
|
Retry transaction, not SQL query
|
2015-04-08 13:11:28 +01:00 |
|
Erik Johnston
|
24d8134ac1
|
Fix maria engine to correctly recognize deadlocks
|
2015-04-08 13:10:54 +01:00 |
|
Erik Johnston
|
4fe95094d1
|
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
|
2015-04-07 18:05:39 +01:00 |
|
Erik Johnston
|
49d6aa1394
|
Retry on deadlock
|
2015-04-07 15:28:37 +01:00 |
|
Erik Johnston
|
0bfa78b39b
|
PEP8
|
2015-04-07 12:16:05 +01:00 |
|
Erik Johnston
|
6bc9edd8b2
|
Fix prepare_sqlite3_database's convert_param_style
|
2015-04-07 12:13:58 +01:00 |
|
Erik Johnston
|
05a35d62b6
|
Bump database version
|
2015-04-07 12:10:15 +01:00 |
|
Erik Johnston
|
8574bf62dc
|
Add index to presence table
|
2015-04-07 12:09:36 +01:00 |
|
Erik Johnston
|
0af5f5efaf
|
Don't use multiple UNIQUE constraints; it will cause deadlocks
|
2015-04-07 12:08:35 +01:00 |
|
Erik Johnston
|
c8d3f6486d
|
Implement or_ignore flag on inserts
|
2015-04-07 12:06:22 +01:00 |
|
Erik Johnston
|
304111afd0
|
Don't use AUTOINCREMENT, use an in memory version
|
2015-04-07 12:05:36 +01:00 |
|
Erik Johnston
|
779f7b0f44
|
Fix unicode support
|
2015-04-02 10:06:22 +01:00 |
|
Erik Johnston
|
9236136f3a
|
Make work in both Maria and SQLite. Fix tests
|
2015-04-01 14:12:33 +01:00 |
|
Kegan Dougal
|
5e88a09a42
|
Add same user_id char checks as registration.
|
2015-03-31 14:00:25 +01:00 |
|
Kegan Dougal
|
cf1fa59f4b
|
Use a sender localpart instead of a user ID.
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
|
2015-03-31 13:48:03 +01:00 |
|
Kegan Dougal
|
c217504949
|
Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.
|
2015-03-31 12:07:56 +01:00 |
|
Kegan Dougal
|
b59aa74556
|
Fix tests and missing returns on deferreds.
|
2015-03-31 11:35:45 +01:00 |
|
Kegan Dougal
|
d33ae65efc
|
Remove more reg/unreg methods. Read config not database for cache.
|
2015-03-31 11:00:00 +01:00 |
|
Paul "LeoNerd" Evans
|
3e420aebd8
|
Revert "Add another @cached wrapper, this time on get_presence_state()"
This reverts commit ff1fa0fbf8.
|
2015-03-27 16:16:58 +00:00 |
|
Paul "LeoNerd" Evans
|
ff1fa0fbf8
|
Add another @cached wrapper, this time on get_presence_state()
|
2015-03-27 15:57:16 +00:00 |
|
David Baker
|
a32e876ef4
|
Delete pushers when changing password
|
2015-03-26 13:40:16 +00:00 |
|
Paul "LeoNerd" Evans
|
a198894bf7
|
Appease pep8
|
2015-03-26 11:53:58 +00:00 |
|
Kegan Dougal
|
32206dde3f
|
Fixes from PR comments
|
2015-03-26 10:11:52 +00:00 |
|
Kegan Dougal
|
4edcbcee3b
|
Merge branch 'develop' into application-services-txn-reliability
Conflicts:
synapse/storage/__init__.py
|
2015-03-26 10:07:59 +00:00 |
|
Paul "LeoNerd" Evans
|
953e40f9dc
|
Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one
|
2015-03-25 19:12:16 +00:00 |
|
David Baker
|
c1a256cc4c
|
Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API.
|
2015-03-25 19:06:22 +00:00 |
|
Paul "LeoNerd" Evans
|
f173d40a32
|
Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key
|
2015-03-25 19:06:05 +00:00 |
|
Paul "LeoNerd" Evans
|
1b988b051b
|
Store the rejected reason in (Frozen)Event structs
|
2015-03-25 19:06:05 +00:00 |
|
Paul "LeoNerd" Evans
|
033a517feb
|
Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file
|
2015-03-25 19:06:05 +00:00 |
|
Paul "LeoNerd" Evans
|
9ba6487b3f
|
Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict()
|
2015-03-25 19:05:34 +00:00 |
|
Erik Johnston
|
0e8f5095c7
|
Fix unicode database support
|
2015-03-25 17:15:20 +00:00 |
|
David Baker
|
c7023f2155
|
1) Pushers are now associated with an access token
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
|
2015-03-24 17:24:15 +00:00 |
|
Erik Johnston
|
0ba393924a
|
Escape non printing ascii character
|
2015-03-24 16:31:52 +00:00 |
|
Erik Johnston
|
f488293d96
|
Don't reinsert into event_edges
|
2015-03-24 16:20:26 +00:00 |
|
Erik Johnston
|
1aa44939fc
|
Fix bugs in transactions storage
|
2015-03-24 16:20:05 +00:00 |
|
Erik Johnston
|
5a447098dd
|
Don't use room hosts table
|
2015-03-24 16:19:24 +00:00 |
|
Erik Johnston
|
9e98f1022a
|
Don't order by rowid
|
2015-03-24 16:19:01 +00:00 |
|
Erik Johnston
|
9115421ace
|
Use _simple_upsert
|
2015-03-24 16:17:39 +00:00 |
|
David Baker
|
d19e79ecc9
|
Make deleting other access tokens when you change your password actually work
|
2015-03-24 15:33:48 +00:00 |
|
Erik Johnston
|
6e7131f02f
|
Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable.
|
2015-03-23 15:38:56 +00:00 |
|
Erik Johnston
|
9a7f496298
|
Sanitize RoomMemberStore
|
2015-03-23 15:29:04 +00:00 |
|
David Baker
|
d98660a60d
|
Implement password changing (finally) along with a start on making client/server auth more general.
|
2015-03-23 14:20:28 +00:00 |
|
Erik Johnston
|
d5272b1d2c
|
Use 'update or insert' rather than on 'conflict replace'
|
2015-03-23 14:02:34 +00:00 |
|
Erik Johnston
|
278149f533
|
Sanitize TransactionStore
|
2015-03-23 13:43:21 +00:00 |
|
Paul "LeoNerd" Evans
|
72d8406409
|
Put a cache on get_aliases_for_room
|
2015-03-20 19:21:13 +00:00 |
|
Paul "LeoNerd" Evans
|
a63b4f7101
|
Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale
|
2015-03-20 18:25:49 +00:00 |
|
Paul "LeoNerd" Evans
|
0f86312c4c
|
Pull out the cache logic from the @cached wrapper into its own class we can reuse
|
2015-03-20 18:25:42 +00:00 |
|
Paul "LeoNerd" Evans
|
b1022ed8b5
|
func(*EXPR) is valid Python syntax, really...
|
2015-03-20 17:47:45 +00:00 |
|
Erik Johnston
|
f6583796fe
|
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
|
2015-03-20 16:31:48 +00:00 |
|
Erik Johnston
|
80cd08c190
|
PEP8
|
2015-03-20 16:03:25 +00:00 |
|
Erik Johnston
|
9517f4da4d
|
Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement
|
2015-03-20 16:02:47 +00:00 |
|
Erik Johnston
|
dc0c989ef4
|
Give sensible names for '_simple_...' transactions
|
2015-03-20 15:59:18 +00:00 |
|
Paul "LeoNerd" Evans
|
ceb61daa70
|
Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream
|
2015-03-20 15:44:06 +00:00 |
|
Erik Johnston
|
fce0114005
|
Start removing Tables
|
2015-03-20 15:05:44 +00:00 |
|
Erik Johnston
|
7e282a53a5
|
Tidy up _simple_... methods
|
2015-03-20 15:05:10 +00:00 |
|
Paul "LeoNerd" Evans
|
91cb46191d
|
Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though
|
2015-03-20 14:59:45 +00:00 |
|
Erik Johnston
|
87db64b839
|
Rearrange storage modules
|
2015-03-20 14:11:38 +00:00 |
|
Erik Johnston
|
cb8162d3d1
|
Rearrange storage modules
|
2015-03-20 13:52:56 +00:00 |
|
Erik Johnston
|
d288d273e1
|
Generate transaction id in code
|
2015-03-20 10:57:44 +00:00 |
|
Erik Johnston
|
d4f50f3ae5
|
decode_result takes an iterable
|
2015-03-20 10:57:26 +00:00 |
|
Erik Johnston
|
0d0610870d
|
Fix up schemas some more
|
2015-03-20 10:55:31 +00:00 |
|
Erik Johnston
|
d7a0496f3e
|
Convert storage layer to be mysql compatible
|
2015-03-19 15:59:48 +00:00 |
|
Erik Johnston
|
58ed393235
|
Remove redundant key
|
2015-03-19 15:12:05 +00:00 |
|
Erik Johnston
|
fae059cc18
|
Fix up schemas to work with mariadb
|
2015-03-19 13:42:39 +00:00 |
|
Erik Johnston
|
bb24609158
|
Clean out event_forward_extremities table when the server rejoins the room
|
2015-03-18 11:19:47 +00:00 |
|
Erik Johnston
|
89036579ed
|
Update schema to work with mariadb
|
2015-03-18 11:18:49 +00:00 |
|
Paul "LeoNerd" Evans
|
93978c5e2b
|
@cached() annotate get_user_by_token() - achieves a minor DB performance improvement
|
2015-03-17 17:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
1489521ee5
|
Be polite and ensure we use @functools.wraps() when creating a function decorator
|
2015-03-17 17:19:22 +00:00 |
|
Paul "LeoNerd" Evans
|
be170b1426
|
Add a metric for the scheduling latency of SQL queries
|
2015-03-16 17:21:59 +00:00 |
|
Erik Johnston
|
cd2539ab2a
|
Merge pull request #110 from matrix-org/fix_ban
Fix ban
|
2015-03-16 15:36:52 +00:00 |
|
Kegan Dougal
|
f0d6f724a2
|
Set the service ID as soon as it is known.
|
2015-03-16 15:24:32 +00:00 |
|
Erik Johnston
|
f1d2b94e0b
|
Copy dict of context.current_state before changing it.
|
2015-03-16 15:13:05 +00:00 |
|
Kegan Dougal
|
835e01fc70
|
Minor PR comment tweaks.
|
2015-03-16 10:16:59 +00:00 |
|
Kegan Dougal
|
f9232c7917
|
Merge branch 'develop' into application-services-txn-reliability
Conflicts:
synapse/storage/appservice.py
|
2015-03-16 10:09:15 +00:00 |
|
Paul "LeoNerd" Evans
|
e75fa8bbbf
|
Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
63cb7ece62
|
Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
f1fbe3e09f
|
Rename TimerMetric to DistributionMetric; as it could count more than just time
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
642f725fd7
|
Pretend the 'getEvent' cache is just another cache in the set of all the others for metric
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
b0cf867319
|
Use _ instead of . as a metric namespacing separator, for Prometheus
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
f9478e475b
|
Rename Metrics' "keys" to "labels"
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
fa319a5786
|
Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely
|
2015-03-12 16:24:51 +00:00 |
|
Paul "LeoNerd" Evans
|
25187ab674
|
Collect per-SQL-verb timer stats on query execution time
|
2015-03-12 16:24:50 +00:00 |
|
Paul "LeoNerd" Evans
|
e1a7e3564f
|
Delete a couple of TODO markers of monitoring stats now done
|
2015-03-12 16:24:50 +00:00 |
|
Paul "LeoNerd" Evans
|
8664599af7
|
Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache
|
2015-03-12 16:24:50 +00:00 |
|
Paul "LeoNerd" Evans
|
b0cdf097f4
|
Sprinkle some CacheCounterMetrics around the synapse.storage layer
|
2015-03-12 16:24:50 +00:00 |
|
Paul "LeoNerd" Evans
|
a7a28f85ae
|
Appease pep8
|
2015-03-11 17:32:43 +00:00 |
|
Paul "LeoNerd" Evans
|
59a5f012cc
|
Also give _execute() a description
|
2015-03-11 17:19:17 +00:00 |
|
Paul "LeoNerd" Evans
|
099e4b88d8
|
Add a description to storage layer's _execute_and_decode()
|
2015-03-11 17:08:57 +00:00 |
|
David Baker
|
04f8478aaa
|
Add the master push rule for the break-my-push button. Allow server default rules to be disabled by default.
|
2015-03-10 17:26:25 +00:00 |
|
David Baker
|
8916acbc13
|
These aren't defined for redacted events so don't crash
|
2015-03-10 11:21:37 +00:00 |
|
Kegan Dougal
|
db1fbc6c6f
|
Fix remaining scheduler bugs. Add more informative logging.
|
2015-03-10 10:04:20 +00:00 |
|
Kegan Dougal
|
7e0bba555c
|
Remove unused import
|
2015-03-09 17:48:37 +00:00 |
|
Kegan Dougal
|
04c9751f24
|
Bug fixes whilst putting it all together
|
2015-03-09 17:45:41 +00:00 |
|
Kegan Dougal
|
b98cd03193
|
Use event IDs instead of dumping event content in the txns table.
|
2015-03-09 17:25:20 +00:00 |
|
Kegan Dougal
|
21fd84dcb8
|
Use seconds; start gluing in the AS scheduler into the AS handler.
|
2015-03-09 17:01:19 +00:00 |
|
Kegan Dougal
|
0a60bbf4fa
|
Finish appservice txn storage impl and tests.
|
2015-03-09 15:53:03 +00:00 |
|
Kegan Dougal
|
1ead1caa18
|
Implement create_appservice_txn with tests.
|
2015-03-09 13:54:20 +00:00 |
|
Kegan Dougal
|
1c2dcf762a
|
Partially implement txn store methods with tests.
|
2015-03-09 13:10:31 +00:00 |
|
Kegan Dougal
|
406d32f8b5
|
Start implementing ApplicationServiceTransactionStore
|
2015-03-06 17:35:14 +00:00 |
|
Kegan Dougal
|
34ce2ca62f
|
Merge branch 'develop' into application-services-txn-reliability
|
2015-03-06 17:28:49 +00:00 |
|
Kegan Dougal
|
4a6afa6abf
|
Assign the AS ID from the database; replace old placeholder txn id.
|
2015-03-06 17:27:55 +00:00 |
|
Kegan Dougal
|
01c099d9ef
|
Add appservice txns sql schema
|
2015-03-06 17:16:47 +00:00 |
|
Kegan Dougal
|
2602ddc379
|
Apply clarity and docstrings
|
2015-03-06 16:16:14 +00:00 |
|
Kegan Dougal
|
0354659f9d
|
Finish synapse.appservice.scheduler implementation.
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
|
2015-03-06 16:09:05 +00:00 |
|
Kegan Dougal
|
f260cb72cd
|
Flesh out more stub functions.
|
2015-03-06 15:12:24 +00:00 |
|
Kegan Dougal
|
141ec04d19
|
Add stub ApplicationServiceTransactionStore. Bootstrap Recoverers. Fill in stub Transaction functions.
|
2015-03-06 14:53:35 +00:00 |
|
David Baker
|
cf66ddc1b4
|
Schema change as delta in v14
|
2015-03-06 14:11:49 +00:00 |
|
David Baker
|
ed72fc3a50
|
Merge branch 'develop' into pushrules2
Conflicts:
synapse/storage/schema/pusher.sql
|
2015-03-04 15:24:21 +00:00 |
|
Erik Johnston
|
d56c01fff4
|
Note that we don't specify execution order
|
2015-03-04 15:10:05 +00:00 |
|
Erik Johnston
|
17d319a20d
|
s/schema_deltas/applied_schema_deltas/
|
2015-03-04 15:06:22 +00:00 |
|
David Baker
|
92b3dc3219
|
Merge branch 'develop' into pushrules2
|
2015-03-04 14:56:41 +00:00 |
|
Erik Johnston
|
5681264faa
|
s/%r/%s/
|
2015-03-04 14:21:53 +00:00 |
|
Erik Johnston
|
f701197227
|
Add example directory structures in doc
|
2015-03-04 14:20:14 +00:00 |
|
David Baker
|
2a45f3d448
|
Use if not results rather than len, as per feedback.
|
2015-03-04 14:17:59 +00:00 |
|
Erik Johnston
|
16dd87d848
|
Don't assume db conn is a Context Manager.
Twisted adbapi wrapped connections aren't context managers.
|
2015-03-04 14:03:41 +00:00 |
|
Erik Johnston
|
5eefd1f618
|
Add unique constraint on schema_version.lock schema. Use conflict clause in sql.
|
2015-03-04 13:52:18 +00:00 |
|
Erik Johnston
|
b4c38738f4
|
Change to use logger in db upgrade script
|
2015-03-04 13:43:35 +00:00 |
|
Erik Johnston
|
640e53935d
|
Use context manager with db conn to correctly commit and rollback
|
2015-03-04 13:43:17 +00:00 |
|
Erik Johnston
|
8c8354e85a
|
Actually add full_schemas dir
|
2015-03-04 13:34:38 +00:00 |
|
Erik Johnston
|
c3530c3fb3
|
More docs. Rename 'schema/current' to 'schema/full_schemas'
|
2015-03-04 13:34:11 +00:00 |
|
Erik Johnston
|
811355ccd0
|
Add some docs and remove unused variables
|
2015-03-04 13:11:01 +00:00 |
|
Erik Johnston
|
82b34e813d
|
SYN-67: Finish up implementing new database schema management
|
2015-03-04 12:04:19 +00:00 |
|
Erik Johnston
|
8d33adfbbb
|
SYN-67: Begin changing the way we handle schema versioning
|
2015-03-02 18:23:55 +00:00 |
|
David Baker
|
6fab7bd2c1
|
s/user_name/user/ as per mjark's comment
|
2015-03-02 18:17:19 +00:00 |
|
Erik Johnston
|
9f03553f48
|
Add missing comma
|
2015-03-02 16:38:40 +00:00 |
|
Erik Johnston
|
b41dc68773
|
We purposefully don't have a version 14 delta script.
|
2015-03-02 16:36:19 +00:00 |
|
David Baker
|
20436cdf75
|
Blank lines
|
2015-03-02 15:58:12 +00:00 |
|
Kegsay
|
8ad024ea80
|
Merge pull request #93 from matrix-org/application-services-exclusive
Application services exclusive flag support
|
2015-03-02 14:56:32 +00:00 |
|
Erik Johnston
|
0a036944bd
|
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
|
2015-03-02 13:53:30 +00:00 |
|
Kegan Dougal
|
c3c01641d2
|
Run deltas and bump user_version in upgrade script
|
2015-03-02 13:38:57 +00:00 |
|
Kegan Dougal
|
cb97ea3ec2
|
PEP8
|
2015-03-02 11:23:46 +00:00 |
|
Kegan Dougal
|
377ae369c1
|
Wrap all of get_app_service_rooms in a txn.
|
2015-03-02 11:20:51 +00:00 |
|
Kegan Dougal
|
b216b36892
|
JOIN state_events rather than parsing unrecognized_keys to pull out member state_keys
|
2015-03-02 10:41:35 +00:00 |
|
Kegan Dougal
|
3d73383d18
|
Modify _simple_select_list to allow an empty WHERE clause. Use it for get_all_rooms and get_all_users.
|
2015-03-02 10:16:24 +00:00 |
|
Kegan Dougal
|
ebc4830666
|
PR tweaks: set earlier on and use 'as json' for compat
|
2015-03-02 09:53:00 +00:00 |
|
Kegan Dougal
|
16b90764ad
|
Convert expected format for AS regex to include exclusivity.
Previously you just specified the regex as a string, now it expects a JSON
object with a 'regex' key and an 'exclusive' boolean, as per spec.
|
2015-02-27 10:44:32 +00:00 |
|
Kegan Dougal
|
806a6c886a
|
PEP8
|
2015-02-27 09:48:57 +00:00 |
|
David Baker
|
1959088156
|
Add API for getting/setting enabled-ness of push rules.
|
2015-02-26 18:07:44 +00:00 |
|
Kegan Dougal
|
f0995436e7
|
Check for membership invite events correctly.
|
2015-02-26 17:21:17 +00:00 |
|
Kegan Dougal
|
dcec7175dc
|
Finish impl to get new events for AS. ASes should now be able to poll /events
|
2015-02-26 16:23:01 +00:00 |
|
Kegan Dougal
|
92478e96d6
|
Finish impl to extract all room IDs an AS may be interested in when polling the event stream.
|
2015-02-26 14:35:28 +00:00 |
|
David Baker
|
944003021b
|
whitespace
|
2015-02-26 13:43:05 +00:00 |
|
David Baker
|
94fa334b01
|
Add enable/disable overlay for push rules (REST API not yet hooked up)
|
2015-02-25 19:17:07 +00:00 |
|
Kegan Dougal
|
29267cf9d7
|
PEP8 and pyflakes
|
2015-02-25 17:42:28 +00:00 |
|
Kegan Dougal
|
978ce87c86
|
Comment unused variables.
|
2015-02-25 17:37:48 +00:00 |
|
Kegan Dougal
|
2c79c4dc7f
|
Fix alias query.
|
2015-02-25 17:37:14 +00:00 |
|
Kegan Dougal
|
2b8ca84296
|
Add support for extracting matching room_ids and room_aliases for a given AS.
|
2015-02-25 17:15:25 +00:00 |
|
Kegan Dougal
|
2d20466f9a
|
Add stub functions and work out execution flow to implement AS event stream polling.
|
2015-02-25 15:00:59 +00:00 |
|
Paul "LeoNerd" Evans
|
9640510de2
|
Use OrderedDict for @cached backing store, so we can evict the oldest key unbiased
|
2015-02-23 18:41:58 +00:00 |
|
Paul "LeoNerd" Evans
|
f53fcbce97
|
Use cache.pop() instead of a separate membership test + del []
|
2015-02-23 18:30:45 +00:00 |
|