Erik Johnston
8bf285e082
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-09 13:13:26 +01:00
Mark Haines
1280a47fc6
Add comment
2015-04-09 11:42:21 +01:00
Mark Haines
23d285ad57
Unset the timer in the timeout callback so that we don't try to cancel it if it has been called
2015-04-09 11:41:50 +01:00
Erik Johnston
8ad0f4912e
Stream ordering and out of order insertions.
...
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Mark Haines
6f9dea7483
SYN-339: Cancel the notifier timeout when the notifier fires
2015-04-09 11:07:20 +01:00
Erik Johnston
279a547a8b
Use generic db exceptions rather than sqlite3 specific ones
2015-04-08 16:53:48 +01:00
Mark Haines
83f5125d52
Merge pull request #114 from matrix-org/improve_get_event_cache
...
Improve get event cache
2015-04-08 16:50:27 +01:00
Erik Johnston
a43b40449b
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-08 16:46:56 +01:00
Erik Johnston
9cef051ce2
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-04-08 16:43:17 +01:00
Erik Johnston
45131b2bca
Bump version
2015-04-08 16:35:12 +01:00
Erik Johnston
ccda401dbf
SYN-338: Fix typo that caused the cache to throw an exception in some instances
2015-04-08 16:34:23 +01:00
Erik Johnston
3887350e47
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-08 14:11:07 +01:00
Erik Johnston
19234cc6c3
typo
2015-04-08 14:10:06 +01:00
Erik Johnston
e8f1521605
Don't yield on notifying all listeners
2015-04-08 14:08:30 +01:00
Mark Haines
605941ee26
Merge pull request #117 from matrix-org/notifier-leak
...
Fix a notifier leak
2015-04-08 14:02:37 +01:00
Erik Johnston
5bc41fe9f8
Move comment into docstring
2015-04-08 14:01:22 +01:00
Erik Johnston
638be5a6b9
Factor out loops into '_discard_if_notified'
2015-04-08 13:58:32 +01:00
Erik Johnston
830d07db82
Also perform paranoia checks in 'on_new_user_event'
2015-04-08 13:40:20 +01:00
Erik Johnston
65f5e4e3e4
Add paranoia checks to make sure that we evict stale NotificationListeners when we are about to process them
2015-04-08 13:33:38 +01:00
Erik Johnston
07d4041709
Fix bug where we didn't inform the NotificataionListeners about new rooms they have been subscribed to. This meant that the listeners didn't clean themselves up fully from all the dicts
2015-04-08 13:33:38 +01:00
Erik Johnston
c1b34af441
Move database timer logging to seperate logger
2015-04-08 13:31:19 +01:00
Erik Johnston
9a05795619
Retry transaction, not SQL query
2015-04-08 13:11:28 +01:00
Erik Johnston
24d8134ac1
Fix maria engine to correctly recognize deadlocks
2015-04-08 13:10:54 +01:00
Erik Johnston
d5e7e6b9b6
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:17:22 +01:00
Erik Johnston
0775c62469
Fix --enable-registration flag to work if you don't give a value
2015-04-07 18:16:23 +01:00
Erik Johnston
38928c6609
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:08:38 +01:00
Erik Johnston
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
Kegan Dougal
ae8ff92e05
Fix a bug which causes a send event level of 0 to not be honoured.
...
Caused by a bad if check, which incorrectly executes for both 0 and None,
when None was the original intent.
2015-04-07 15:48:20 +01:00
Erik Johnston
49d6aa1394
Retry on deadlock
2015-04-07 15:28:37 +01:00
Erik Johnston
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
Erik Johnston
6bc9edd8b2
Fix prepare_sqlite3_database's convert_param_style
2015-04-07 12:13:58 +01:00
Erik Johnston
05a35d62b6
Bump database version
2015-04-07 12:10:15 +01:00
Erik Johnston
8574bf62dc
Add index to presence table
2015-04-07 12:09:36 +01:00
Erik Johnston
0af5f5efaf
Don't use multiple UNIQUE constraints; it will cause deadlocks
2015-04-07 12:08:35 +01:00
Erik Johnston
c8d3f6486d
Implement or_ignore flag on inserts
2015-04-07 12:06:22 +01:00
Erik Johnston
304111afd0
Don't use AUTOINCREMENT, use an in memory version
2015-04-07 12:05:36 +01:00
Erik Johnston
d0e444a648
Explicitly name the __main__ module logger
2015-04-07 12:04:02 +01:00
David Baker
4eb6d66b45
Add app service auth back in to v2 register
2015-04-02 17:51:19 +01:00
David Baker
6b59650753
Throw sensible errors on not-json when allowing empty body
2015-04-02 17:45:16 +01:00
David Baker
41cd778d66
pep8
2015-04-02 17:06:17 +01:00
David Baker
70a84f17f3
Add shared secret auth into register v2 and switch the script over.
2015-04-02 17:01:29 +01:00
Erik Johnston
779f7b0f44
Fix unicode support
2015-04-02 10:06:22 +01:00
Paul "LeoNerd" Evans
ef1e019840
Appease pep8
2015-04-01 19:17:38 +01:00
Paul "LeoNerd" Evans
5583e29513
Report process open filehandles in metrics
2015-04-01 19:15:23 +01:00
David Baker
c5bf0343e8
Explain how I justified to myself making JsonResource not always send JSON.
2015-04-01 15:13:14 +01:00
Erik Johnston
e24c32e6f3
Fix SQLite support
2015-04-01 15:09:51 +01:00
David Baker
e9c908ebc0
Completely replace fallback auth for C/S V2:
...
* Now only the auth part goes to fallback, not the whole operation
* Auth fallback is a normal API endpoint, not a static page
* Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
Erik Johnston
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
Kegan Dougal
813e54bd5b
Fix more AS sender ID thinkos.
...
Specifically, the ASes own user ID wasn't being treated as 'exclusive' so
a human could nab it. Also, the HS would needlessly send user queries to the
AS for its own user ID.
2015-04-01 14:05:24 +01:00
Kegan Dougal
9fa8bda099
Merge branch 'develop' into application-services-registration-script
2015-04-01 10:19:17 +01:00
David Baker
f129ee1e18
Make docs a bit more true
2015-03-31 18:25:10 +01:00
Kegan Dougal
09cbff174a
Fix thinko whereby events *for the AS specifically* were not passed on.
...
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
2015-03-31 16:44:45 +01:00
David Baker
d18e7779ca
Grammar and deduplication
2015-03-31 14:40:02 +01:00
Kegan Dougal
5e88a09a42
Add same user_id char checks as registration.
2015-03-31 14:00:25 +01:00
Kegan Dougal
cf1fa59f4b
Use a sender localpart instead of a user ID.
...
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal
3470cb36a8
Pyflakes
2015-03-31 13:03:31 +01:00
Kegan Dougal
c217504949
Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.
2015-03-31 12:07:56 +01:00
Kegan Dougal
b59aa74556
Fix tests and missing returns on deferreds.
2015-03-31 11:35:45 +01:00
Kegan Dougal
d33ae65efc
Remove more reg/unreg methods. Read config not database for cache.
2015-03-31 11:00:00 +01:00
David Baker
9f642a93ec
pep8
2015-03-31 09:50:44 +01:00
Kegan Dougal
e7887e37a8
Remove appservice REST servlets
2015-03-31 09:32:40 +01:00
Kegan Dougal
af853a4cdb
Add AppServiceConfig
2015-03-31 09:22:31 +01:00
David Baker
59bf16eddc
New registration for C/S API v2. Only ReCAPTCHA working currently.
2015-03-30 18:13:10 +01:00
Matthew Hodgson
8366fde82f
turn --disable-registration into --enable-registration, given the default is for registration to be disabled by default now. this is backwards incompatible by removing the old --disable-registration arg, but makes for a much more intuitive arg
2015-03-30 12:01:09 -04:00
Paul "LeoNerd" Evans
3e420aebd8
Revert "Add another @cached wrapper, this time on get_presence_state()"
...
This reverts commit ff1fa0fbf8
2015-03-27 16:16:58 +00:00
Paul "LeoNerd" Evans
ff1fa0fbf8
Add another @cached wrapper, this time on get_presence_state()
2015-03-27 15:57:16 +00:00
David Baker
6f4f7e4e22
pep8
2015-03-26 14:12:06 +00:00
David Baker
a32e876ef4
Delete pushers when changing password
2015-03-26 13:40:16 +00:00
Paul "LeoNerd" Evans
a198894bf7
Appease pep8
2015-03-26 11:53:58 +00:00
Kegan Dougal
32206dde3f
Fixes from PR comments
2015-03-26 10:11:52 +00:00
Kegan Dougal
4edcbcee3b
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans
953e40f9dc
Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one
2015-03-25 19:12:16 +00:00
David Baker
df4c12c762
pep8 blank lines
2015-03-25 19:08:17 +00:00
David Baker
c1a256cc4c
Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API.
2015-03-25 19:06:22 +00:00
Paul "LeoNerd" Evans
f173d40a32
Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key
2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
1b988b051b
Store the rejected reason in (Frozen)Event structs
2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
033a517feb
Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file
2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
9ba6487b3f
Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict()
2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans
d6b3ea75d4
Implement the 'key in dict' test for LruCache()
2015-03-25 19:04:59 +00:00
Erik Johnston
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
David Baker
9aa0224cdf
unused import
2015-03-24 17:25:59 +00:00
David Baker
c7023f2155
1) Pushers are now associated with an access token
...
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston
0ba393924a
Escape non printing ascii character
2015-03-24 16:31:52 +00:00
Erik Johnston
f488293d96
Don't reinsert into event_edges
2015-03-24 16:20:26 +00:00
Erik Johnston
1aa44939fc
Fix bugs in transactions storage
2015-03-24 16:20:05 +00:00
Erik Johnston
5a447098dd
Don't use room hosts table
2015-03-24 16:19:24 +00:00
Erik Johnston
9e98f1022a
Don't order by rowid
2015-03-24 16:19:01 +00:00
Erik Johnston
9115421ace
Use _simple_upsert
2015-03-24 16:17:39 +00:00
David Baker
d19e79ecc9
Make deleting other access tokens when you change your password actually work
2015-03-24 15:33:48 +00:00
Paul "LeoNerd" Evans
ed008e85a8
Reduce activity timer granularity to avoid too many quick updates (SYN-247)
2015-03-23 17:25:50 +00:00
Erik Johnston
6e7131f02f
Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable.
2015-03-23 15:38:56 +00:00
Erik Johnston
9a7f496298
Sanitize RoomMemberStore
2015-03-23 15:29:04 +00:00
David Baker
78adccfaf4
pep8 / pyflakes
2015-03-23 14:23:51 +00:00
David Baker
d98660a60d
Implement password changing (finally) along with a start on making client/server auth more general.
2015-03-23 14:20:28 +00:00
Erik Johnston
d5272b1d2c
Use 'update or insert' rather than on 'conflict replace'
2015-03-23 14:02:34 +00:00
Erik Johnston
278149f533
Sanitize TransactionStore
2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans
72d8406409
Put a cache on get_aliases_for_room
2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans
a63b4f7101
Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale
2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans
0f86312c4c
Pull out the cache logic from the @cached wrapper into its own class we can reuse
2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans
b1022ed8b5
func(*EXPR) is valid Python syntax, really...
2015-03-20 17:47:45 +00:00
Erik Johnston
f6583796fe
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-03-20 16:31:48 +00:00
Erik Johnston
80cd08c190
PEP8
2015-03-20 16:03:25 +00:00
Erik Johnston
9517f4da4d
Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement
2015-03-20 16:02:47 +00:00
Erik Johnston
dc0c989ef4
Give sensible names for '_simple_...' transactions
2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans
ceb61daa70
Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream
2015-03-20 15:44:06 +00:00
Erik Johnston
fce0114005
Start removing Tables
2015-03-20 15:05:44 +00:00
Erik Johnston
7e282a53a5
Tidy up _simple_... methods
2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans
91cb46191d
Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though
2015-03-20 14:59:45 +00:00
Erik Johnston
87db64b839
Rearrange storage modules
2015-03-20 14:11:38 +00:00
Erik Johnston
cb8162d3d1
Rearrange storage modules
2015-03-20 13:52:56 +00:00
Erik Johnston
d288d273e1
Generate transaction id in code
2015-03-20 10:57:44 +00:00
Erik Johnston
d4f50f3ae5
decode_result takes an iterable
2015-03-20 10:57:26 +00:00
Erik Johnston
455579ca90
Make database selection configurable
2015-03-20 10:55:55 +00:00
Erik Johnston
0d0610870d
Fix up schemas some more
2015-03-20 10:55:31 +00:00
Erik Johnston
56f2d31676
Bump version
2015-03-19 17:48:33 +00:00
Erik Johnston
c178e4e6ca
Add missing servlet to list
2015-03-19 17:48:21 +00:00
Erik Johnston
d7a0496f3e
Convert storage layer to be mysql compatible
2015-03-19 15:59:48 +00:00
Erik Johnston
58ed393235
Remove redundant key
2015-03-19 15:12:05 +00:00
Erik Johnston
fae059cc18
Fix up schemas to work with mariadb
2015-03-19 13:42:39 +00:00
Erik Johnston
0dcb145c7e
Bump version
2015-03-19 11:26:03 +00:00
Erik Johnston
6cc046302f
Bump version
2015-03-18 11:41:00 +00:00
Erik Johnston
ed4d44d833
Merge pull request #109 from matrix-org/default_registration
...
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
Erik Johnston
f88db7ac0b
Factor out user id validation checks
2015-03-18 11:34:18 +00:00
Erik Johnston
57976f646f
Do more validation of incoming request
2015-03-18 11:30:04 +00:00
Erik Johnston
bb24609158
Clean out event_forward_extremities table when the server rejoins the room
2015-03-18 11:19:47 +00:00
Erik Johnston
89036579ed
Update schema to work with mariadb
2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans
93978c5e2b
@cached() annotate get_user_by_token() - achieves a minor DB performance improvement
2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans
1489521ee5
Be polite and ensure we use @functools.wraps() when creating a function decorator
2015-03-17 17:19:22 +00:00
David Baker
6d33f97703
pep8
2015-03-17 11:53:55 +00:00
David Baker
7564dac8cb
Wire up the webclient option
...
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
2015-03-17 12:45:37 +01:00
Paul "LeoNerd" Evans
3f7a31d366
Add a DistributionMetric to HTTP request/response processing time in the server
2015-03-16 18:31:29 +00:00
Paul "LeoNerd" Evans
be170b1426
Add a metric for the scheduling latency of SQL queries
2015-03-16 17:21:59 +00:00
Erik Johnston
cd2539ab2a
Merge pull request #110 from matrix-org/fix_ban
...
Fix ban
2015-03-16 15:36:52 +00:00
Kegan Dougal
f0d6f724a2
Set the service ID as soon as it is known.
2015-03-16 15:24:32 +00:00
Erik Johnston
f1d2b94e0b
Copy dict of context.current_state before changing it.
2015-03-16 15:13:05 +00:00
Erik Johnston
857810d2dd
Revert incorrect changes to where we send events
2015-03-16 15:12:47 +00:00
Kegan Dougal
d04fa1f712
Implement ServiceQueuer with tests.
2015-03-16 14:03:16 +00:00
Erik Johnston
c2c9471cba
Don't block waiting on waking up all the listeners when sending an event.
2015-03-16 13:16:37 +00:00
Kegan Dougal
6279285b2a
Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer.
2015-03-16 13:15:40 +00:00
Erik Johnston
8bad40701b
Comment.
2015-03-16 13:13:07 +00:00
Erik Johnston
250e143084
Use 403 instead of 400
2015-03-16 13:11:42 +00:00
Erik Johnston
b2e6ee5b43
Remove concept of context.auth_events, instead use context.current_state
2015-03-16 13:06:23 +00:00
Kegan Dougal
c9c444f562
Wrap polling/retry blocks in try/excepts to avoid sending to other ASes breaking permanently should an error occur.
2015-03-16 10:38:02 +00:00
Kegan Dougal
835e01fc70
Minor PR comment tweaks.
2015-03-16 10:16:59 +00:00
Kegan Dougal
f9232c7917
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Erik Johnston
758d114cbc
Send all membership events to the remote homeserver
2015-03-16 00:27:59 +00:00
Erik Johnston
ea8590cf66
Make context.auth_events grap auth events from current state. Otherwise auth is wrong.
2015-03-16 00:18:08 +00:00
Erik Johnston
ab8229479b
Respect ban membership
2015-03-16 00:17:25 +00:00
Matthew Hodgson
c8032aec17
actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0'
2015-03-14 00:12:20 +00:00
Paul Evans
e731d30d90
Merge pull request #108 from matrix-org/metrics
...
Metrics
2015-03-13 17:31:10 +00:00
Erik Johnston
98a3825614
Allow enabling of registration with --disable-registration false
2015-03-13 16:49:18 +00:00
Erik Johnston
9266cb0a22
PEP8
2015-03-13 15:26:00 +00:00
Erik Johnston
dea236e4fa
Add missing commas
2015-03-13 15:24:03 +00:00
Erik Johnston
69135f59aa
Implement registering with shared secret.
2015-03-13 15:23:37 +00:00
Erik Johnston
58367a9da2
Disable registration by default
2015-03-13 12:59:45 +00:00
Erik Johnston
58247c8b4b
Also bump dependency link version
2015-03-13 11:39:57 +00:00
Matthew Hodgson
f55bd3f94b
bump dep on syweb 0.6.5
2015-03-12 18:56:53 +00:00
Paul "LeoNerd" Evans
e90002ca1d
Merge remote-tracking branch 'origin/develop' into metrics
2015-03-12 16:55:25 +00:00
David Baker
bbb010a30f
More sacrifices to the pep8 gods.
2015-03-12 16:53:12 +00:00
Paul "LeoNerd" Evans
05a056a409
Appease pyflakes
2015-03-12 16:45:05 +00:00
Paul "LeoNerd" Evans
0eb7e6b9a8
Delete unused import of NOT_READY_YET
2015-03-12 16:39:52 +00:00
Paul "LeoNerd" Evans
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b98b4c135d
Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
a2cdd11d4a
Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening()
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e0214a263b
Build MetricsResource as a specific HomeServer dependency
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e75fa8bbbf
Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
c782e893ec
Neater metrics from TransactionQueue
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
89ac1fa8ba
Add a counter to track total number of events served by the notifier
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
2e4f0b2bd7
Replace the @metrics.counted annotations in federation with specifically-written counters and distributions
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
c1cdd7954d
Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
63cb7ece62
Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
493e3fa0ca
Don't forbid '_' in metric basenames any more, to allow things like foo_time
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f1fbe3e09f
Rename TimerMetric to DistributionMetric; as it could count more than just time
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
642f725fd7
Pretend the 'getEvent' cache is just another cache in the set of all the others for metric
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
cbc0406be8
Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
4d661ec0f3
Remember to emit final linefeed from /metrics page, or Prometheus gets upset
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
0e847540c3
Prometheus needs "escaped" label values
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
22b37b75db
Kill unused CounterMetric.fetch() method
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
0b96bb793e
Have all @metrics.counted use a single metric name vectored on the method name, rather than a brand new scalar counter per counted method
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b3a0179d64
Bugfix to rendering output of vectored TimerMetrics
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
399689dcc7
Provide some process resource usage metrics
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
fa319a5786
Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
6d146e15df
Put some gauge metrics on the number of notifier listeners, and notified-on objects (users, rooms, appservices)
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
25187ab674
Collect per-SQL-verb timer stats on query execution time
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
f52acf3b12
Neater register_* methods on overall Metrics container
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
a99d6edc05
Neater implementation of metric render methods by pulling out 'render' as a base method that calls self.render_item
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
72625f2f4d
Initial hack at a TimerMetric; for storing counts + duration accumulators
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e1a7e3564f
Delete a couple of TODO markers of monitoring stats now done
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
094803cf82
Put vector gauges on transaction queue pending PDU and EDU dicts
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e9c4b0d178
Ensure that /_synapse/metrics response is UTF-8 encoded
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
23ab0c68c2
Implement vector CallbackMetrics
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
849300bc73
Neater introspection methods on BaseMetric so that subclasses don't need to touch self.keys directly
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
8664599af7
Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e02cc249da
Ensure that exceptions while rendering individual metrics don't stop others from being rendered anyway - especially useful for CallbackMetric
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
59c448f074
Add a scalar gauge metric on the size of the presence user cachemap
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
d8caa5454d
Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
b0cdf097f4
Sprinkle some CacheCounterMetrics around the synapse.storage layer
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
ce8b5769f7
Create the concept of a cachecounter metric; generating two counters specific to caches
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
7d72e44eb9
Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
c53ec53d80
Pull out all uses of the underlying HTTP user agent .request() method into a single wrapper function, to make adding metrics easier
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
9470412316
Initial attempt at sprinkling some @metrics.counted decorations around the federation code
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
a594087f06
Have the MetricsResource actually render metric counters
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
74bc42cfdd
An initial implementation of a 'metrics' instance, similar to a 'logger' for keeping counter stats on method calls
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
120b689284
Delete pointless (and unreachable) __init__ method from FederationClient
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e7420a3bef
Initial tiny attempt at (vectorable) counter metrics
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e07fc62833
A trivial 'hello world'-style resource on /_synapse/metrics, with optional commandline flag
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
5b6e11d560
Commandline option to enable metrics system
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
211c14c391
No need to explicitly pass 'web_client' in to create_resource_tree as it can be found via config
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
ad5701f50f
Expose 'config' as a real HomeServer dependency key
2015-03-12 16:24:50 +00:00
David Baker
c92fdf88a3
Log the matching push rule.
2015-03-11 22:17:31 +00:00
Paul "LeoNerd" Evans
a7a28f85ae
Appease pep8
2015-03-11 17:32:43 +00:00
Paul "LeoNerd" Evans
59a5f012cc
Also give _execute() a description
2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans
099e4b88d8
Add a description to storage layer's _execute_and_decode()
2015-03-11 17:08:57 +00:00
David Baker
cdb2e045ee
Again, underscore, not hyphen
2015-03-11 14:22:35 +00:00
David Baker
465354ffde
'false' is not False
2015-03-11 11:24:50 +00:00
David Baker
83b1e7fb3c
PEP8 blank lines
2015-03-11 10:01:17 +00:00
David Baker
04f8478aaa
Add the master push rule for the break-my-push button. Allow server default rules to be disabled by default.
2015-03-10 17:26:25 +00:00
David Baker
8916acbc13
These aren't defined for redacted events so don't crash
2015-03-10 11:21:37 +00:00
Erik Johnston
abaf47bbb6
Fix bug in logging.
2015-03-10 10:28:29 +00:00
Erik Johnston
045afd6b61
in_thread takes no arguments
2015-03-10 10:19:03 +00:00
Erik Johnston
98b867f7b7
Fix bug in logging.
2015-03-10 10:16:09 +00:00
Kegan Dougal
db1fbc6c6f
Fix remaining scheduler bugs. Add more informative logging.
2015-03-10 10:04:20 +00:00
Erik Johnston
e84fe3599b
Merge pull request #105 from matrix-org/erikj-perf
...
Add a Twisted Service to synapse.app.homeserver
2015-03-10 10:02:26 +00:00
Erik Johnston
c37eceeb9e
Split out the 'run' from 'setup'
2015-03-10 09:58:33 +00:00
Erik Johnston
b8a6692657
Add documentation. When starting via twistd respect soft_file_limit config option.
2015-03-10 09:39:42 +00:00
Kegan Dougal
7e0bba555c
Remove unused import
2015-03-09 17:48:37 +00:00
Kegan Dougal
04c9751f24
Bug fixes whilst putting it all together
2015-03-09 17:45:41 +00:00
Erik Johnston
019422ebba
Merge pull request #101 from matrix-org/neaten-federation-servlets
...
Neaten federation servlets
2015-03-09 17:39:06 +00:00
Kegan Dougal
b98cd03193
Use event IDs instead of dumping event content in the txns table.
2015-03-09 17:25:20 +00:00
Kegan Dougal
21fd84dcb8
Use seconds; start gluing in the AS scheduler into the AS handler.
2015-03-09 17:01:19 +00:00
Erik Johnston
8e28db5cc9
Change room handlers get_rooms_for_user to get_joined_rooms_for_user. This uses the a storage api that is cached.
2015-03-09 16:43:09 +00:00
Kegan Dougal
0a60bbf4fa
Finish appservice txn storage impl and tests.
2015-03-09 15:53:03 +00:00
Kegan Dougal
1ead1caa18
Implement create_appservice_txn with tests.
2015-03-09 13:54:20 +00:00
Erik Johnston
f31e65ca8b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf
2015-03-09 13:29:41 +00:00
Kegan Dougal
1c2dcf762a
Partially implement txn store methods with tests.
2015-03-09 13:10:31 +00:00
David Baker
1df3ccf7ee
D'oh: underscore, not hyphen
2015-03-09 12:39:56 +00:00
David Baker
118c883429
Call notifications should be override else they'll get clobbered by sender/room rules.
2015-03-06 19:41:36 +00:00
Kegan Dougal
406d32f8b5
Start implementing ApplicationServiceTransactionStore
2015-03-06 17:35:14 +00:00
Kegan Dougal
34ce2ca62f
Merge branch 'develop' into application-services-txn-reliability
2015-03-06 17:28:49 +00:00
Kegan Dougal
4a6afa6abf
Assign the AS ID from the database; replace old placeholder txn id.
2015-03-06 17:27:55 +00:00
Kegan Dougal
01c099d9ef
Add appservice txns sql schema
2015-03-06 17:16:47 +00:00
Kegan Dougal
64345b7559
Upper bound the backoff.
2015-03-06 16:41:19 +00:00
Erik Johnston
5d43eaed61
Merge branch 'develop' into release-v0.8.0
2015-03-06 16:25:19 +00:00
Erik Johnston
9ccccd4874
When setting display name more graciously handle failures to update room state.
2015-03-06 16:24:05 +00:00
Kegan Dougal
2602ddc379
Apply clarity and docstrings
2015-03-06 16:16:14 +00:00
Kegan Dougal
0354659f9d
Finish synapse.appservice.scheduler implementation.
...
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
2015-03-06 16:09:05 +00:00
David Baker
be9dafcd37
Dial down logging for failed pushers
2015-03-06 15:32:38 +00:00
Kegan Dougal
7d3491c741
Add some loggers
2015-03-06 15:17:50 +00:00
David Baker
96eda876a4
Specify when we don't want to highlight
2015-03-06 15:12:37 +00:00
Kegan Dougal
f260cb72cd
Flesh out more stub functions.
2015-03-06 15:12:24 +00:00
David Baker
e7d7152c3c
Remove the fallback rule - we probably don't want to be notifying for everything even if we don't know what it is.
2015-03-06 15:03:34 +00:00
Kegan Dougal
141ec04d19
Add stub ApplicationServiceTransactionStore. Bootstrap Recoverers. Fill in stub Transaction functions.
2015-03-06 14:53:35 +00:00
David Baker
5ecc768970
Add attribute so push gateways can tell if a member event is about the user in question
2015-03-06 14:41:50 +00:00
Kegan Dougal
0fbfe1b08a
Add more tests; fix bugs.
2015-03-06 14:36:52 +00:00
Erik Johnston
369449827d
Bump version
2015-03-06 14:24:53 +00:00
Erik Johnston
c54773473f
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-03-06 14:23:41 +00:00
Erik Johnston
b102a87348
Merge pull request #96 from matrix-org/pushrules2
...
Evolution of push rules
2015-03-06 14:20:04 +00:00
David Baker
cf66ddc1b4
Schema change as delta in v14
2015-03-06 14:11:49 +00:00
David Baker
c06b45129c
Add more server default rules so we have default rules for whether you get notifs for invites / random member events
2015-03-06 11:50:51 +00:00
Kegan Dougal
192e228a98
Start adding some tests
2015-03-06 11:50:27 +00:00
Erik Johnston
e49d6b1568
Unused import
2015-03-06 11:37:24 +00:00
David Baker
657a0d2568
Comment typo
2015-03-06 11:34:30 +00:00
Erik Johnston
3ce8540484
Don't look for an TLS private key if we have set --no-tls
2015-03-06 11:34:06 +00:00
David Baker
1487bba226
Suppress notices should trump content/room/sender rules.
2015-03-06 10:27:32 +00:00
David Baker
83d31144eb
Add the highlight tweak where messages should be highlighted a different colour in appropriate clients.
2015-03-06 10:26:08 +00:00
Kegan Dougal
d516d68b29
Rejig structure given the appservice_handler already filters the correct ASes to use.
2015-03-06 10:25:50 +00:00
Erik Johnston
130df8fb01
Add some randomness to the user specified timeout on event streams to mitigate against thundering herds problems
2015-03-06 10:25:36 +00:00
Paul "LeoNerd" Evans
d79d91a4a7
Appease pep8
2015-03-05 20:55:40 +00:00
Paul "LeoNerd" Evans
5eab2549ab
Append a $ on PATH at registration time, meaning each PATH attribute doesn't need it
2015-03-05 20:36:05 +00:00
Paul "LeoNerd" Evans
7644cb79b2
Slightly neater(?) arrangement of authentication wrapper for HTTP servlet methods
2015-03-05 20:33:16 +00:00
Paul "LeoNerd" Evans
ba8ac996f9
Remove the dead 'rate_limit_origin' method from TransportLayerServer
2015-03-05 19:43:17 +00:00
Paul "LeoNerd" Evans
a901ed16b5
Move federation API responding code out of weird mix of lambdas into Servlet-style methods on instances
2015-03-05 19:10:57 +00:00
Kegan Dougal
0c838f9f5e
Minor tweaks
2015-03-05 17:45:52 +00:00
Kegan Dougal
773cb3b688
Add stub architecture for txn reliability.
2015-03-05 17:35:07 +00:00
Erik Johnston
5b5c7a28d6
Log error message when we fail to fetch remote server keys
2015-03-05 17:09:13 +00:00
Erik Johnston
12bcf3d179
Merge pull request #100 from matrix-org/missing_pdu_compat
...
Handle if get_missing_pdu returns 400 or not all events.
2015-03-05 16:42:15 +00:00
Erik Johnston
9708f49abf
Docs
2015-03-05 16:35:16 +00:00
Erik Johnston
96fee64421
Remove unecessary check
2015-03-05 16:31:47 +00:00
Erik Johnston
39aa968a76
Respect min_depth argument
2015-03-05 16:31:32 +00:00
Erik Johnston
6dfd8c73fc
Docs.
2015-03-05 16:31:13 +00:00
Kegan Dougal
e319071191
Add stub scheduler module for txn reliability
2015-03-05 16:30:33 +00:00
Paul "LeoNerd" Evans
9d9d39536b
Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through
2015-03-05 16:24:13 +00:00
Erik Johnston
ae702d161a
Handle if get_missing_pdu returns 400 or not all events.
2015-03-05 16:08:02 +00:00
Kegan Dougal
be09c23ff0
Add txn_id kwarg to push methods
2015-03-05 15:40:07 +00:00
Paul "LeoNerd" Evans
dc4b774f1e
Rename rooms_to_listeners to room_to_listeners, for consistency with user_ and appservice_*
2015-03-05 14:30:20 +00:00
Paul "LeoNerd" Evans
027fd1242c
Give LruCache a __len__, so that len(cache) works
2015-03-04 17:32:28 +00:00
David Baker
590b544f67
Add default rule to suppress notices.
2015-03-04 15:29:02 +00:00
David Baker
ed72fc3a50
Merge branch 'develop' into pushrules2
...
Conflicts:
synapse/storage/schema/pusher.sql
2015-03-04 15:24:21 +00:00
Erik Johnston
d56c01fff4
Note that we don't specify execution order
2015-03-04 15:10:05 +00:00
Erik Johnston
17d319a20d
s/schema_deltas/applied_schema_deltas/
2015-03-04 15:06:22 +00:00
David Baker
92b3dc3219
Merge branch 'develop' into pushrules2
2015-03-04 14:56:41 +00:00
Erik Johnston
5681264faa
s/%r/%s/
2015-03-04 14:21:53 +00:00
Erik Johnston
f701197227
Add example directory structures in doc
2015-03-04 14:20:14 +00:00
David Baker
2a45f3d448
Use if not results rather than len, as per feedback.
2015-03-04 14:17:59 +00:00
Erik Johnston
16dd87d848
Don't assume db conn is a Context Manager.
...
Twisted adbapi wrapped connections aren't context managers.
2015-03-04 14:03:41 +00:00
Erik Johnston
5eefd1f618
Add unique constraint on schema_version.lock schema. Use conflict clause in sql.
2015-03-04 13:52:18 +00:00
Erik Johnston
b4c38738f4
Change to use logger in db upgrade script
2015-03-04 13:43:35 +00:00
Erik Johnston
640e53935d
Use context manager with db conn to correctly commit and rollback
2015-03-04 13:43:17 +00:00
Erik Johnston
8c8354e85a
Actually add full_schemas dir
2015-03-04 13:34:38 +00:00
Erik Johnston
c3530c3fb3
More docs. Rename 'schema/current' to 'schema/full_schemas'
2015-03-04 13:34:11 +00:00
Erik Johnston
811355ccd0
Add some docs and remove unused variables
2015-03-04 13:11:01 +00:00
Erik Johnston
82b34e813d
SYN-67: Finish up implementing new database schema management
2015-03-04 12:04:19 +00:00
Erik Johnston
527e0c43a5
Bump version
2015-03-03 14:49:34 +00:00
Erik Johnston
ede89ae3b4
Also bump version of downloaded syweb
2015-03-03 14:49:19 +00:00
Erik Johnston
da877aad15
Bump syweb dependency
2015-03-03 13:31:50 +00:00
Erik Johnston
8d33adfbbb
SYN-67: Begin changing the way we handle schema versioning
2015-03-02 18:23:55 +00:00
David Baker
6fab7bd2c1
s/user_name/user/ as per mjark's comment
2015-03-02 18:17:19 +00:00
David Baker
09f9e8493c
Oops, missed a replacement.
2015-03-02 17:37:22 +00:00
Erik Johnston
9f03553f48
Add missing comma
2015-03-02 16:38:40 +00:00
Erik Johnston
b41dc68773
We purposefully don't have a version 14 delta script.
2015-03-02 16:36:19 +00:00
David Baker
20436cdf75
Blank lines
2015-03-02 15:58:12 +00:00
Kegan Dougal
2de5b14fe0
Fix bug which prevented the HS pushing events to the AS due to FrozenEvents
2015-03-02 15:36:37 +00:00
Erik Johnston
8486910b64
Bump webclient version
2015-03-02 14:57:37 +00:00
Kegsay
8ad024ea80
Merge pull request #93 from matrix-org/application-services-exclusive
...
Application services exclusive flag support
2015-03-02 14:56:32 +00:00
Erik Johnston
0a036944bd
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
2015-03-02 13:53:30 +00:00
Erik Johnston
3fce185c77
Merge pull request #83 from matrix-org/nofile_limit_config
...
Add config option to set the soft fd limit on start
2015-03-02 13:52:16 +00:00
Erik Johnston
4195e55ccc
Merge branch 'develop' of github.com:matrix-org/synapse into federation_rate_limit
2015-03-02 13:39:22 +00:00
Kegan Dougal
c3c01641d2
Run deltas and bump user_version in upgrade script
2015-03-02 13:38:57 +00:00
Erik Johnston
210d3c5d72
Merge pull request #95 from matrix-org/serialize_transaction_processing
...
Process transactions serially.
2015-03-02 13:33:05 +00:00
Erik Johnston
3077cb2915
Use contextlib.contextmanager instead of a custom class
2015-03-02 13:32:44 +00:00
David Baker
769f8b58e8
Rename the room-with-two-people rule to be more compatible if we have actual one to one rooms.
2015-03-02 13:28:24 +00:00
Kegsay
33f93d389e
Merge pull request #92 from matrix-org/application-services-event-stream
...
Application services event stream support
2015-03-02 12:02:48 +00:00
Erik Johnston
29481690c5
If we're yielding don't add errback
2015-03-02 11:50:43 +00:00
Erik Johnston
23d9bd1d74
Process transactions serially.
...
Since the events received in a transaction are ordered, later events
might depend on earlier events and so we shouldn't blindly process them
in parellel.
2015-03-02 11:39:57 +00:00
Erik Johnston
9d9b230501
Make the federation server ratelimiting configurable.
2015-03-02 11:33:45 +00:00
Kegan Dougal
cb97ea3ec2
PEP8
2015-03-02 11:23:46 +00:00
Kegan Dougal
377ae369c1
Wrap all of get_app_service_rooms in a txn.
2015-03-02 11:20:51 +00:00
Kegan Dougal
b216b36892
JOIN state_events rather than parsing unrecognized_keys to pull out member state_keys
2015-03-02 10:41:35 +00:00
Kegan Dougal
3d73383d18
Modify _simple_select_list to allow an empty WHERE clause. Use it for get_all_rooms and get_all_users.
2015-03-02 10:16:24 +00:00
Kegan Dougal
ebc4830666
PR tweaks: set earlier on and use 'as json' for compat
2015-03-02 09:53:00 +00:00
David Baker
2a6dedd7cc
It's set_tweak now, not set_sound
2015-02-27 18:38:56 +00:00
Erik Johnston
0554d07082
Move federation rate limiting out of transport layer
2015-02-27 15:41:52 +00:00
Erik Johnston
9dc9118e55
Document FederationRateLimiter
2015-02-27 15:16:47 +00:00
Kegan Dougal
58ff066064
Implement exclusive namespace checks.
2015-02-27 13:51:41 +00:00
Kegan Dougal
40c9896705
Add functions to return whether an AS has exclusively claimed a matching namespace.
2015-02-27 11:03:56 +00:00
Kegan Dougal
16b90764ad
Convert expected format for AS regex to include exclusivity.
...
Previously you just specified the regex as a string, now it expects a JSON
object with a 'regex' key and an 'exclusive' boolean, as per spec.
2015-02-27 10:44:32 +00:00
Kegan Dougal
806a6c886a
PEP8
2015-02-27 09:48:57 +00:00
Kegan Dougal
1cc77145d4
Notify appservices of invites mid-poll.
...
This requires the notifier to have knowledge of appservice listeners so it can
do the regex checks on incoming invites to see if the state_key matches. It
isn't enough to just rely on the room listeners and store.get_app_service_rooms
as the room will initially not exist or won't be on the ASes radar due to
having none of its users in the room.
2015-02-27 09:39:12 +00:00
David Baker
cfac3b7873
SYN-267 Add a fallback rule as an explicit server default rule and make the default dont-notify so you effectively have a "notify for everything else" switch you can turn on and off.
2015-02-26 18:58:14 +00:00
David Baker
1959088156
Add API for getting/setting enabled-ness of push rules.
2015-02-26 18:07:44 +00:00
Kegan Dougal
f0995436e7
Check for membership invite events correctly.
2015-02-26 17:21:17 +00:00
Kegan Dougal
dcec7175dc
Finish impl to get new events for AS. ASes should now be able to poll /events
2015-02-26 16:23:01 +00:00
Erik Johnston
93d90765c4
Initial implementation of federation server rate limiting
2015-02-26 16:15:26 +00:00
Erik Johnston
59362454dd
Must update pending_transactions map before yield'ing
2015-02-26 15:47:35 +00:00
Kegan Dougal
92478e96d6
Finish impl to extract all room IDs an AS may be interested in when polling the event stream.
2015-02-26 14:35:28 +00:00
David Baker
944003021b
whitespace
2015-02-26 13:43:05 +00:00
David Baker
94fa334b01
Add enable/disable overlay for push rules (REST API not yet hooked up)
2015-02-25 19:17:07 +00:00
