Erik Johnston
c1a133a6b6
Merge pull request #1043 from matrix-org/erikj/backfill_fix
...
Fix None check in backfill
2016-08-26 09:07:05 +01:00
Erik Johnston
778fa85f47
Make sync not pull out full state
2016-08-25 18:59:44 +01:00
Erik Johnston
7356d52e73
Fix up push to use get_current_state_ids
2016-08-25 18:35:49 +01:00
Paul "LeoNerd" Evans
9459137f1e
Just sprintf the 'kind' argument into uri directly
2016-08-25 18:35:38 +01:00
Paul "LeoNerd" Evans
1294d4a329
Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant
2016-08-25 18:34:47 +01:00
Mark Haines
ab34fdecb7
Merge branch 'develop' into markjh/direct_to_device
2016-08-25 18:34:46 +01:00
Mark Haines
b162cb2e41
Add some TODOs
2016-08-25 18:18:53 +01:00
Erik Johnston
0e1900d819
Pull out full state less
2016-08-25 18:15:51 +01:00
Mark Haines
641efb6a39
Fix the deduplication of incoming direct-to-device messages
2016-08-25 18:14:02 +01:00
Paul "LeoNerd" Evans
e7af8be5ae
Root the 3PE lookup API within /_matrix/app/unstable instead of at toplevel
2016-08-25 18:06:29 +01:00
Paul "LeoNerd" Evans
142983b4ea
APP_SERVICE_PREFIX is never used; don't bother
2016-08-25 18:06:05 +01:00
Erik Johnston
721414d98a
Add desc
2016-08-25 17:49:05 +01:00
Mark Haines
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
Erik Johnston
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
Paul Evans
d9dcb2ba3a
Merge pull request #1041 from matrix-org/paul/third-party-lookup
...
Extend 3PE lookup APIs for metadata query
2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans
adf53f04ce
appease pep8
2016-08-25 16:00:31 +01:00
Paul "LeoNerd" Evans
c435bfee9c
Don't need toplevel cache on 3PE lookup metadata any more
2016-08-25 15:57:07 +01:00
Paul "LeoNerd" Evans
db7283cc6b
Implement a ResponseCache around 3PE lookup metadata lookups
2016-08-25 15:56:27 +01:00
Paul "LeoNerd" Evans
d0b8d49f71
Kill PROTOCOL_META since I'm not using it any more
2016-08-25 15:45:28 +01:00
Paul "LeoNerd" Evans
5474824975
Actually query over AS API for 3PE lookup metadata
2016-08-25 15:29:36 +01:00
Erik Johnston
17f4f14df7
Pull out event ids rather than full events for state
2016-08-25 13:42:44 +01:00
Erik Johnston
cd5b264b03
Fix None check in backfill
2016-08-25 10:39:19 +01:00
Erik Johnston
60a015550a
Bump changelog and version
2016-08-24 14:31:13 +01:00
Erik Johnston
90d5983d7a
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.1
2016-08-24 14:28:57 +01:00
Erik Johnston
c20cb5160d
Remove tracer
2016-08-24 13:22:47 +01:00
Erik Johnston
fda97dd58a
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/preserve_log_contexts
2016-08-24 13:22:02 +01:00
Paul "LeoNerd" Evans
8e1ed09dff
Move static knowledge of protocol metadata into AS handler; cache the result
2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans
965f33c901
Declare 'gitter' known protocol, with user lookup
2016-08-24 12:34:03 +01:00
Paul "LeoNerd" Evans
9899824b85
Initial hack at the 3PN protocols metadata lookup API
2016-08-24 12:33:01 +01:00
Erik Johnston
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
Paul "LeoNerd" Evans
63c19e1df9
Move 3PU/3PL lookup APIs into /thirdparty containing entity
2016-08-24 11:55:57 +01:00
Erik Johnston
86bcf4d6a7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/pagination
2016-08-24 10:38:21 +01:00
Erik Johnston
ba07d4a70e
Add None checks to backfill
2016-08-24 10:31:05 +01:00
Kegan Dougal
4b31426a02
Pass through user-supplied content in /join/$room_id
...
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.
If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
Erik Johnston
122c7a43c9
Merge pull request #1038 from matrix-org/erikj/receved_txn_purge
...
Delete old received_transactions rows
2016-08-23 11:02:40 +01:00
Erik Johnston
14047126d8
Bump changelog and version
2016-08-22 18:26:07 +01:00
Erik Johnston
d143f211c8
Merge pull request #1028 from matrix-org/dbkr/notifications_api
...
Add the Notifications API
2016-08-22 18:23:24 +01:00
Erik Johnston
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
Mark Haines
689f4cb914
Update comment
2016-08-22 16:17:31 +01:00
Mark Haines
c8f9b45bc2
Add usage stats to prometheus monitoring
2016-08-22 15:34:38 +01:00
Erik Johnston
e65bc7d315
Merge pull request #1031 from matrix-org/erikj/measure_notifier
...
Add more Measure blocks
2016-08-22 12:13:07 +01:00
Erik Johnston
33f3624ff7
Add exception logging. Fix typo
2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07
Allow request handlers to override metric name
2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253
Use top level measure
2016-08-22 10:18:12 +01:00
Matthew Hodgson
6e80c03d45
Merge branch 'develop' into dbkr/notifications_api
2016-08-20 00:16:18 +01:00
Erik Johnston
27e0178da9
Add a top level measure
2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218
Remove redundant measure
2016-08-19 18:40:31 +01:00
Erik Johnston
8731197e54
Only abort Measure on Exceptions
2016-08-19 18:23:45 +01:00
Erik Johnston
afbf6b33fc
defer.returnValue must not be called within Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
37adde32dc
Move defer.returnValue out of Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
04fc8bbcb0
Update keyring Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
39b900b316
Measure http.server render
2016-08-19 18:23:44 +01:00
Erik Johnston
47dd8f02a1
Measure _get_event_from_row
2016-08-19 18:23:44 +01:00
Erik Johnston
2426c2f21a
Measure keyrings
2016-08-19 18:23:44 +01:00
Erik Johnston
39242090e3
Add measure blocks to notifier
2016-08-19 18:23:44 +01:00
Erik Johnston
e6784daf07
Merge pull request #1030 from matrix-org/erikj/cache_contexts
...
Add concept of cache contexts
2016-08-19 16:29:58 +01:00
Erik Johnston
45fd2c8942
Ensure invalidation list does not grow unboundedly
2016-08-19 16:09:16 +01:00
Erik Johnston
c0d7d9d642
Rename to on_invalidate
2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909
Make cache_context an explicit option
2016-08-19 15:02:38 +01:00
Erik Johnston
f164fd9220
Move _bulk_get_push_rules_for_room to storage layer
2016-08-19 14:29:20 +01:00
Erik Johnston
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
Erik Johnston
4161ff2fc4
Add concept of cache contexts
2016-08-19 14:17:07 +01:00
Erik Johnston
b770435389
Make get_new_events_for_appservice use indices
2016-08-19 10:28:42 +01:00
Paul Evans
5674ea3e6c
Merge pull request #1026 from matrix-org/paul/thirdpartylookup
...
3rd party entity lookup
2016-08-18 20:52:50 +01:00
David Baker
1e4217c90c
Explicit join
2016-08-18 17:53:44 +01:00
David Baker
0acdd0f1ea
Use tuple comparison
...
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans
65201631a4
Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater
2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans
697872cf08
More warnings about invalid results from AS 3PE query
2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans
b515f844ee
Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration
2016-08-18 17:19:55 +01:00
David Baker
602c84cd9c
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans
2a91799fcc
Minor syntax neatenings
2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans
fcf1dec809
Appease pep8
2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans
105ff162d4
Authenticate 3PE lookup requests
2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans
06964c4a0a
Copypasta the 3PU support code to also do 3PL
2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans
f3afd6ef1a
Remove TODO note about request fields being strings - they're always strings
2016-08-18 15:53:01 +01:00
Erik Johnston
bcbd74dc5b
Remove log lines
2016-08-18 15:52:10 +01:00
Paul "LeoNerd" Evans
d7b42afc74
Log a warning if an AS yields an invalid 3PU lookup result
2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans
80f4740c8f
Scattergather the call out to ASes; validate received results
2016-08-18 15:40:41 +01:00
Erik Johnston
07229bbdae
Add appservice worker
2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans
434bbf2cb5
Filter 3PU lookups by only ASes that declare knowledge of that protocol
2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans
d5bf7a4a99
Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup
2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans
718ffcf8bb
Since empty lookups now return 200/empty list not 404, we can safely log failures as exceptions
2016-08-18 14:18:37 +01:00
Paul "LeoNerd" Evans
3856582741
Ensure that 3PU lookup request fields actually get passed in
2016-08-18 14:06:02 +01:00
Paul "LeoNerd" Evans
f0c73a1e7a
Extend individual list results into the main return list, don't append
2016-08-18 13:53:54 +01:00
Paul "LeoNerd" Evans
b3511adb92
Don't catch the return-value-as-exception that @defer.inlineCallbacks will use
2016-08-18 13:45:18 +01:00
Erik Johnston
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
Mark Haines
403ecd8a2c
Missed a s/federation reader/media repository/ in a log message
2016-08-18 10:26:15 +01:00
Mark Haines
47fbff7a05
Merge pull request #1024 from matrix-org/markjh/media_repository
...
Add a media repository worker
2016-08-18 10:12:17 +01:00
Mark Haines
396624864a
Add a media repository worker
2016-08-18 09:38:42 +01:00
Erik Johnston
e73dcb66da
Merge pull request #1022 from matrix-org/erikj/as_notify_perf
...
Make notify_interested_services faster
2016-08-18 09:35:19 +01:00
Paul "LeoNerd" Evans
3ec10dffd6
Actually make 3PU lookup calls out to ASes
2016-08-18 00:39:09 +01:00
Erik Johnston
732cf72b86
Fix push_display_name_rename schema update
2016-08-17 18:12:21 +01:00
Erik Johnston
abcb9aee5b
Make push Measure finer grained
2016-08-17 18:00:18 +01:00
Erik Johnston
2a0d8f8219
Merge pull request #1021 from matrix-org/erikj/mediasecurity_policy
...
Set `Content-Security-Policy` on media repo
2016-08-17 17:22:43 +01:00
Erik Johnston
320dfe523c
Make notify_interested_services faster
2016-08-17 17:20:50 +01:00
Erik Johnston
0af9e1a637
Set Content-Security-Policy on media repo
...
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans
fa87c981e1
Thread 3PU lookup through as far as the AS API object; which currently noöps it
2016-08-17 16:17:28 +01:00
David Baker
0d7cef0943
Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override
...
Move display name rule
2016-08-17 16:15:45 +01:00
Erik Johnston
f90b3d83a3
Add None check to _iterate_over_text
2016-08-17 15:17:17 +01:00
Matrix
f743471380
Change name of metric
2016-08-17 15:05:50 +01:00
Erik Johnston
b9e888858c
Move Measure block inside loop
2016-08-17 14:52:26 +01:00
Erik Johnston
973d67a033
Merge pull request #1019 from matrix-org/erikj/appservice_clean
...
Clean up _ServiceQueuer
2016-08-17 14:37:21 +01:00
Paul "LeoNerd" Evans
e3e3fbc23a
Initial empty implementation that just registers an API endpoint handler
2016-08-17 12:46:49 +01:00
Erik Johnston
7321f45457
Clean up _ServiceQueuer
2016-08-17 12:03:04 +01:00
Erik Johnston
b9abf3e4e3
Remove dead appservice code
2016-08-17 11:48:23 +01:00
Erik Johnston
62c5245c87
Measure notify_interested_services
2016-08-17 11:12:29 +01:00
Erik Johnston
949629291c
Do it in storage function
2016-08-16 17:05:34 +01:00
David Baker
ad42322257
Add migration script
...
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
David Baker
0bba2799b6
Merge remote-tracking branch 'origin/develop' into dbkr/contains_display_name_override
2016-08-16 16:46:37 +01:00
Erik Johnston
64a2acb161
Don't update caches replication stream if tokens haven't advanced
2016-08-16 16:44:19 +01:00
David Baker
1594eba29e
s/underride/override/ in the rule_id too
2016-08-16 16:44:07 +01:00
Erik Johnston
109a560905
Flake8
2016-08-16 14:57:21 +01:00
Erik Johnston
48b5829aea
Fix up preview URL API. Add tests.
...
This includes:
- Splitting out methods of a class into stand alone functions, to make
them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston
7c6f4f9427
Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri
...
Limit number of extremeties in backfill request
2016-08-16 12:55:42 +01:00
Erik Johnston
25c2332071
Merge pull request #1010 from matrix-org/erikj/refactor_deletions
...
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-16 11:37:53 +01:00
Erik Johnston
2ee1bd124c
Limit number of extremeties in backfill request
...
This works around a bug where if we make a backfill request with too
many extremeties it causes the request URI to be too long.
2016-08-16 11:34:36 +01:00
Erik Johnston
a2427981b7
Use cached get_user_by_access_token in slaves
2016-08-16 11:24:32 +01:00
Erik Johnston
6cbd1b495e
Merge branch 'fix_integrity_retry' of https://github.com/Ralith/synapse into Ralith-fix_integrity_retry
2016-08-16 10:50:24 +01:00
David Baker
1c7c317df1
Move display name rule
...
As per https://github.com/matrix-org/matrix-doc/pull/373 and comment
2016-08-15 18:34:53 +01:00
Erik Johnston
dc3a00f24f
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-15 17:04:39 +01:00
Erik Johnston
75299af4fc
Merge pull request #1009 from matrix-org/erikj/event_split
...
Split out /events to synchrotron
2016-08-15 15:39:05 +01:00
Erik Johnston
89e786bd85
Doc get_next() context manager usage
2016-08-15 13:45:26 +01:00
Erik Johnston
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
Erik Johnston
784a2d4f2c
Remove broken cache stuff
2016-08-15 11:25:48 +01:00
Erik Johnston
0be963472b
Use cached version of get_aliases_for_room
2016-08-15 11:24:12 +01:00
Erik Johnston
64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
Erik Johnston
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
Erik Johnston
99bbd90b0d
Always run txn.after_callbacks
2016-08-15 09:45:44 +01:00
Benjamin Saunders
8a57cc3123
Add missing database corruption recovery case
...
Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
2016-08-14 11:50:22 -07:00
Daniel Ehlers
dfaf0fee31
Log the value which is observed in the first place.
...
The name 'result' is of bool type and has no len property,
resulting in a TypeError. Futhermore in the flow control
conn.response is observed and hence should be reported.
Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:49:05 +02:00
Daniel Ehlers
e380538b59
Fix AttributeError when bind_dn is not defined.
...
In case one does not define bind_dn in ldap configuration, filter
attribute is not declared. Since auth code only uses ldap_filter attribute
when according LDAP mode is selected, it is safe to only declare the
attribute in that case.
Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:48:33 +02:00
Erik Johnston
4e1cebd56f
Make synchrotron accept /events
2016-08-12 15:31:44 +01:00
Erik Johnston
866a5320de
Dont invoke get_handlers fromClientV1RestServlet
...
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
Erik Johnston
448ac6cf0d
Only process one local membership event per room at a time
2016-08-12 09:32:19 +01:00
Erik Johnston
832799dbff
Merge pull request #997 from Half-Shot/develop
...
Don't change status_msg on /sync
2016-08-11 14:10:55 +01:00
David Baker
b4ecf0b886
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-11 14:09:13 +01:00
Will Hunt
5b5148b7ec
Synced up synchrotron set_state with PresenceHandler set_state
2016-08-11 11:48:30 +01:00
Erik Johnston
1d40373c9d
Include prev_content in redacted state events
2016-08-11 10:24:41 +01:00
Erik Johnston
c315922b5f
PEP8
2016-08-10 16:34:10 +01:00
Erik Johnston
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
Erik Johnston
5aeadb7414
Merge pull request #999 from matrix-org/erikj/measure_more
...
Measure federation send transaction resources
2016-08-10 14:16:14 +01:00
Erik Johnston
487bc49bf8
Don't stop on 4xx series errors
2016-08-10 13:39:12 +01:00
Erik Johnston
739ea29d1e
Also check if server is in the room
2016-08-10 13:32:23 +01:00
Erik Johnston
ea8c4094db
Also pull out rejected events
2016-08-10 13:26:13 +01:00
Erik Johnston
7f41bcbeec
Correctly auth /event/ requests
2016-08-10 13:22:20 +01:00
Erik Johnston
11fdfaf03b
Only resign our own events
2016-08-10 13:16:58 +01:00
Will Hunt
2510db3e76
Don't change status_msg on /sync
2016-08-10 12:59:59 +01:00
Erik Johnston
f91df1f761
Store if we fail to fetch an event from a destination
2016-08-10 11:31:46 +01:00
Erik Johnston
3bc9629be5
Measure federation send transaction resources
2016-08-10 10:56:38 +01:00
Erik Johnston
d45489474d
Merge pull request #996 from matrix-org/erikj/tls_error
...
Don't print stack traces when failing to get remote keys
2016-08-10 10:56:08 +01:00
