Commit Graph

5099 Commits

Author SHA1 Message Date
Erik Johnston
ba07d4a70e Add None checks to backfill 2016-08-24 10:31:05 +01:00
Kegan Dougal
4b31426a02 Pass through user-supplied content in /join/$room_id
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.

If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
Erik Johnston
122c7a43c9 Merge pull request #1038 from matrix-org/erikj/receved_txn_purge
Delete old received_transactions rows
2016-08-23 11:02:40 +01:00
Erik Johnston
14047126d8 Bump changelog and version 2016-08-22 18:26:07 +01:00
Erik Johnston
d143f211c8 Merge pull request #1028 from matrix-org/dbkr/notifications_api
Add the Notifications API
2016-08-22 18:23:24 +01:00
Erik Johnston
aad8a1a825 Delete old received_transactions 2016-08-22 16:29:46 +01:00
Mark Haines
689f4cb914 Update comment 2016-08-22 16:17:31 +01:00
Mark Haines
c8f9b45bc2 Add usage stats to prometheus monitoring 2016-08-22 15:34:38 +01:00
Erik Johnston
e65bc7d315 Merge pull request #1031 from matrix-org/erikj/measure_notifier
Add more Measure blocks
2016-08-22 12:13:07 +01:00
Erik Johnston
33f3624ff7 Add exception logging. Fix typo 2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07 Allow request handlers to override metric name 2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253 Use top level measure 2016-08-22 10:18:12 +01:00
Matthew Hodgson
6e80c03d45 Merge branch 'develop' into dbkr/notifications_api 2016-08-20 00:16:18 +01:00
Erik Johnston
27e0178da9 Add a top level measure 2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218 Remove redundant measure 2016-08-19 18:40:31 +01:00
Erik Johnston
8731197e54 Only abort Measure on Exceptions 2016-08-19 18:23:45 +01:00
Erik Johnston
afbf6b33fc defer.returnValue must not be called within Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
37adde32dc Move defer.returnValue out of Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
04fc8bbcb0 Update keyring Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
39b900b316 Measure http.server render 2016-08-19 18:23:44 +01:00
Erik Johnston
47dd8f02a1 Measure _get_event_from_row 2016-08-19 18:23:44 +01:00
Erik Johnston
2426c2f21a Measure keyrings 2016-08-19 18:23:44 +01:00
Erik Johnston
39242090e3 Add measure blocks to notifier 2016-08-19 18:23:44 +01:00
Erik Johnston
e6784daf07 Merge pull request #1030 from matrix-org/erikj/cache_contexts
Add concept of cache contexts
2016-08-19 16:29:58 +01:00
Erik Johnston
45fd2c8942 Ensure invalidation list does not grow unboundedly 2016-08-19 16:09:16 +01:00
Erik Johnston
c0d7d9d642 Rename to on_invalidate 2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909 Make cache_context an explicit option 2016-08-19 15:02:38 +01:00
Erik Johnston
f164fd9220 Move _bulk_get_push_rules_for_room to storage layer 2016-08-19 14:29:20 +01:00
Erik Johnston
ba214a5e32 Remove lru option 2016-08-19 14:17:11 +01:00
Erik Johnston
4161ff2fc4 Add concept of cache contexts 2016-08-19 14:17:07 +01:00
Erik Johnston
b770435389 Make get_new_events_for_appservice use indices 2016-08-19 10:28:42 +01:00
Paul Evans
5674ea3e6c Merge pull request #1026 from matrix-org/paul/thirdpartylookup
3rd party entity lookup
2016-08-18 20:52:50 +01:00
David Baker
1e4217c90c Explicit join 2016-08-18 17:53:44 +01:00
David Baker
0acdd0f1ea Use tuple comparison
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans
65201631a4 Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater 2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans
697872cf08 More warnings about invalid results from AS 3PE query 2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans
b515f844ee Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration 2016-08-18 17:19:55 +01:00
David Baker
602c84cd9c Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans
2a91799fcc Minor syntax neatenings 2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans
fcf1dec809 Appease pep8 2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans
105ff162d4 Authenticate 3PE lookup requests 2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans
06964c4a0a Copypasta the 3PU support code to also do 3PL 2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans
f3afd6ef1a Remove TODO note about request fields being strings - they're always strings 2016-08-18 15:53:01 +01:00
Erik Johnston
bcbd74dc5b Remove log lines 2016-08-18 15:52:10 +01:00
Paul "LeoNerd" Evans
d7b42afc74 Log a warning if an AS yields an invalid 3PU lookup result 2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans
80f4740c8f Scattergather the call out to ASes; validate received results 2016-08-18 15:40:41 +01:00
Erik Johnston
07229bbdae Add appservice worker 2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans
434bbf2cb5 Filter 3PU lookups by only ASes that declare knowledge of that protocol 2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans
d5bf7a4a99 Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup 2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans
718ffcf8bb Since empty lookups now return 200/empty list not 404, we can safely log failures as exceptions 2016-08-18 14:18:37 +01:00
Paul "LeoNerd" Evans
3856582741 Ensure that 3PU lookup request fields actually get passed in 2016-08-18 14:06:02 +01:00
Paul "LeoNerd" Evans
f0c73a1e7a Extend individual list results into the main return list, don't append 2016-08-18 13:53:54 +01:00
Paul "LeoNerd" Evans
b3511adb92 Don't catch the return-value-as-exception that @defer.inlineCallbacks will use 2016-08-18 13:45:18 +01:00
Erik Johnston
9da84a9a1e Make AppserviceHandler stream events from database
This is for two reasons:

1. Suppresses duplicates correctly, as the notifier doesn't do any
   duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
   stream.
2016-08-18 11:54:41 +01:00
Mark Haines
403ecd8a2c Missed a s/federation reader/media repository/ in a log message 2016-08-18 10:26:15 +01:00
Mark Haines
47fbff7a05 Merge pull request #1024 from matrix-org/markjh/media_repository
Add a media repository worker
2016-08-18 10:12:17 +01:00
Mark Haines
396624864a Add a media repository worker 2016-08-18 09:38:42 +01:00
Erik Johnston
e73dcb66da Merge pull request #1022 from matrix-org/erikj/as_notify_perf
Make notify_interested_services faster
2016-08-18 09:35:19 +01:00
Paul "LeoNerd" Evans
3ec10dffd6 Actually make 3PU lookup calls out to ASes 2016-08-18 00:39:09 +01:00
Erik Johnston
732cf72b86 Fix push_display_name_rename schema update 2016-08-17 18:12:21 +01:00
Erik Johnston
abcb9aee5b Make push Measure finer grained 2016-08-17 18:00:18 +01:00
Erik Johnston
2a0d8f8219 Merge pull request #1021 from matrix-org/erikj/mediasecurity_policy
Set `Content-Security-Policy` on media repo
2016-08-17 17:22:43 +01:00
Erik Johnston
320dfe523c Make notify_interested_services faster 2016-08-17 17:20:50 +01:00
Erik Johnston
0af9e1a637 Set Content-Security-Policy on media repo
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans
fa87c981e1 Thread 3PU lookup through as far as the AS API object; which currently noöps it 2016-08-17 16:17:28 +01:00
David Baker
0d7cef0943 Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override
Move display name rule
2016-08-17 16:15:45 +01:00
Erik Johnston
f90b3d83a3 Add None check to _iterate_over_text 2016-08-17 15:17:17 +01:00
Matrix
f743471380 Change name of metric 2016-08-17 15:05:50 +01:00
Erik Johnston
b9e888858c Move Measure block inside loop 2016-08-17 14:52:26 +01:00
Erik Johnston
973d67a033 Merge pull request #1019 from matrix-org/erikj/appservice_clean
Clean up _ServiceQueuer
2016-08-17 14:37:21 +01:00
Paul "LeoNerd" Evans
e3e3fbc23a Initial empty implementation that just registers an API endpoint handler 2016-08-17 12:46:49 +01:00
Erik Johnston
7321f45457 Clean up _ServiceQueuer 2016-08-17 12:03:04 +01:00
Erik Johnston
b9abf3e4e3 Remove dead appservice code 2016-08-17 11:48:23 +01:00
Erik Johnston
62c5245c87 Measure notify_interested_services 2016-08-17 11:12:29 +01:00
Erik Johnston
949629291c Do it in storage function 2016-08-16 17:05:34 +01:00
David Baker
ad42322257 Add migration script
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
David Baker
0bba2799b6 Merge remote-tracking branch 'origin/develop' into dbkr/contains_display_name_override 2016-08-16 16:46:37 +01:00
Erik Johnston
64a2acb161 Don't update caches replication stream if tokens haven't advanced 2016-08-16 16:44:19 +01:00
David Baker
1594eba29e s/underride/override/ in the rule_id too 2016-08-16 16:44:07 +01:00
Erik Johnston
109a560905 Flake8 2016-08-16 14:57:21 +01:00
Erik Johnston
48b5829aea Fix up preview URL API. Add tests.
This includes:

- Splitting out methods of a class into stand alone functions, to make
  them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston
7c6f4f9427 Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri
Limit number of extremeties in backfill request
2016-08-16 12:55:42 +01:00
Erik Johnston
25c2332071 Merge pull request #1010 from matrix-org/erikj/refactor_deletions
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-16 11:37:53 +01:00
Erik Johnston
2ee1bd124c Limit number of extremeties in backfill request
This works around a bug where if we make a backfill request with too
many extremeties it causes the request URI to be too long.
2016-08-16 11:34:36 +01:00
Erik Johnston
a2427981b7 Use cached get_user_by_access_token in slaves 2016-08-16 11:24:32 +01:00
Erik Johnston
6cbd1b495e Merge branch 'fix_integrity_retry' of https://github.com/Ralith/synapse into Ralith-fix_integrity_retry 2016-08-16 10:50:24 +01:00
David Baker
1c7c317df1 Move display name rule
As per https://github.com/matrix-org/matrix-doc/pull/373 and comment
2016-08-15 18:34:53 +01:00
Erik Johnston
dc3a00f24f Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves. 2016-08-15 17:04:39 +01:00
Erik Johnston
75299af4fc Merge pull request #1009 from matrix-org/erikj/event_split
Split out /events to synchrotron
2016-08-15 15:39:05 +01:00
Erik Johnston
89e786bd85 Doc get_next() context manager usage 2016-08-15 13:45:26 +01:00
Erik Johnston
d9664344ec Rename table. Add docs. 2016-08-15 11:45:57 +01:00
Erik Johnston
784a2d4f2c Remove broken cache stuff 2016-08-15 11:25:48 +01:00
Erik Johnston
0be963472b Use cached version of get_aliases_for_room 2016-08-15 11:24:12 +01:00
Erik Johnston
64e7e11853 Implement cache replication stream 2016-08-15 11:16:45 +01:00
Erik Johnston
4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Erik Johnston
99bbd90b0d Always run txn.after_callbacks 2016-08-15 09:45:44 +01:00
Benjamin Saunders
8a57cc3123 Add missing database corruption recovery case
Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
2016-08-14 11:50:22 -07:00
Daniel Ehlers
dfaf0fee31
Log the value which is observed in the first place.
The name 'result' is of bool type and has no len property,
resulting in a TypeError. Futhermore in the flow control
conn.response is observed and hence should be reported.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:49:05 +02:00
Daniel Ehlers
e380538b59
Fix AttributeError when bind_dn is not defined.
In case one does not define bind_dn in ldap configuration, filter
attribute is not declared. Since auth code only uses ldap_filter attribute
when according LDAP mode is selected, it is safe to only declare the
attribute in that case.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:48:33 +02:00
Erik Johnston
4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00