Mark Haines
a662252758
Return the current stream position from add_messages_to_device_inbox
2016-08-31 10:42:52 +01:00
Mark Haines
1aa3e1d287
Add a replication stream for direct to device messages
2016-08-31 10:38:58 +01:00
Erik Johnston
1bb8ec296d
Generate state group ids in state layer
2016-08-31 10:09:46 +01:00
Erik Johnston
d80f64d370
Fix email notifs by adding missing param
2016-08-30 21:46:39 +01:00
Kegan Dougal
c882783535
flake8
2016-08-30 17:20:31 +01:00
Kegan Dougal
572acde483
Use None instead of the empty string
...
Change how we validate the 'url' field as a result.
2016-08-30 17:16:00 +01:00
Erik Johnston
5dc2a702cf
Make _state_groups_id_gen a normal IdGenerator
2016-08-30 16:55:11 +01:00
Erik Johnston
3e784eff74
Remove state replication stream
2016-08-30 16:51:36 +01:00
Kegan Dougal
16b652f0a3
Flake8
2016-08-30 16:30:12 +01:00
Kegan Dougal
e82247f990
Allow application services to have an optional 'url'
...
If 'url' is not specified, they will not be pushed for events or queries. This
is useful for bots who simply wish to reserve large chunks of user/alias
namespace, and don't care about being pushed for events.
2016-08-30 16:21:16 +01:00
Erik Johnston
c7f665d700
Merge pull request #1055 from matrix-org/erikj/occaisonally_persist
...
Occaisonally persist unpersisted presence updates
2016-08-30 15:59:00 +01:00
Erik Johnston
097330bae8
Check correct variable
2016-08-30 15:50:20 +01:00
Erik Johnston
21b977ccfe
Occaisonally persist unpersisted presence updates
2016-08-30 15:39:50 +01:00
Erik Johnston
bc1a8b1f7a
Don't notify for online -> online transitions.
...
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
2016-08-30 15:05:32 +01:00
Erik Johnston
67f0c990f8
Merge pull request #1051 from matrix-org/erikj/fix_push_names
...
Fix push room names for rooms with only an alias
2016-08-30 11:50:34 +01:00
Erik Johnston
c8cd87b21b
Comment about message deletion
2016-08-30 11:23:26 +01:00
Erik Johnston
55e17d3697
Fix push room names for rooms with only an alias
2016-08-30 11:19:59 +01:00
Erik Johnston
1ee6285905
Fix check
2016-08-30 11:17:46 +01:00
Erik Johnston
68e1a872fd
Noop get_new_messages_for_device if token hasn't changed
2016-08-30 10:58:46 +01:00
Erik Johnston
55fc17cf4b
Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
...
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
Erik Johnston
41788bba50
Add to slave store
2016-08-30 09:55:17 +01:00
Erik Johnston
873f870e5a
Add new direct message storage functions to slave store
2016-08-30 09:40:32 +01:00
Erik Johnston
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
Mark Haines
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
Erik Johnston
a234e895cf
Fix room name in email notifs
2016-08-26 14:10:21 +01:00
Erik Johnston
4daa397a00
Add is_host_joined to slave storage
2016-08-26 13:10:56 +01:00
Erik Johnston
c7cd35d682
Typo
2016-08-26 11:23:58 +01:00
Erik Johnston
54cc69154e
Make None optional
2016-08-26 11:20:59 +01:00
Erik Johnston
11faa4296d
Measure _filter_events_for_server
2016-08-26 11:15:40 +01:00
Erik Johnston
f6338d6a3e
Don't pull out full state for _filter_events_for_server
2016-08-26 11:13:16 +01:00
Erik Johnston
1ccdc1e93a
Cache check_host_in_room
2016-08-26 10:59:40 +01:00
Erik Johnston
25414b44a2
Add measure on check_host_in_room
2016-08-26 10:47:00 +01:00
Erik Johnston
50943ab942
Add new state storage funcs to replication
2016-08-26 09:57:32 +01:00
Erik Johnston
30961182f2
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids
2016-08-26 09:48:13 +01:00
Erik Johnston
c1a133a6b6
Merge pull request #1043 from matrix-org/erikj/backfill_fix
...
Fix None check in backfill
2016-08-26 09:07:05 +01:00
Erik Johnston
778fa85f47
Make sync not pull out full state
2016-08-25 18:59:44 +01:00
Erik Johnston
7356d52e73
Fix up push to use get_current_state_ids
2016-08-25 18:35:49 +01:00
Paul "LeoNerd" Evans
9459137f1e
Just sprintf the 'kind' argument into uri directly
2016-08-25 18:35:38 +01:00
Paul "LeoNerd" Evans
1294d4a329
Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant
2016-08-25 18:34:47 +01:00
Mark Haines
ab34fdecb7
Merge branch 'develop' into markjh/direct_to_device
2016-08-25 18:34:46 +01:00
Mark Haines
b162cb2e41
Add some TODOs
2016-08-25 18:18:53 +01:00
Erik Johnston
0e1900d819
Pull out full state less
2016-08-25 18:15:51 +01:00
Mark Haines
641efb6a39
Fix the deduplication of incoming direct-to-device messages
2016-08-25 18:14:02 +01:00
Paul "LeoNerd" Evans
e7af8be5ae
Root the 3PE lookup API within /_matrix/app/unstable instead of at toplevel
2016-08-25 18:06:29 +01:00
Paul "LeoNerd" Evans
142983b4ea
APP_SERVICE_PREFIX is never used; don't bother
2016-08-25 18:06:05 +01:00
Erik Johnston
721414d98a
Add desc
2016-08-25 17:49:05 +01:00
Mark Haines
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
Erik Johnston
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
Paul Evans
d9dcb2ba3a
Merge pull request #1041 from matrix-org/paul/third-party-lookup
...
Extend 3PE lookup APIs for metadata query
2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans
adf53f04ce
appease pep8
2016-08-25 16:00:31 +01:00
Paul "LeoNerd" Evans
c435bfee9c
Don't need toplevel cache on 3PE lookup metadata any more
2016-08-25 15:57:07 +01:00
Paul "LeoNerd" Evans
db7283cc6b
Implement a ResponseCache around 3PE lookup metadata lookups
2016-08-25 15:56:27 +01:00
Paul "LeoNerd" Evans
d0b8d49f71
Kill PROTOCOL_META since I'm not using it any more
2016-08-25 15:45:28 +01:00
Paul "LeoNerd" Evans
5474824975
Actually query over AS API for 3PE lookup metadata
2016-08-25 15:29:36 +01:00
Erik Johnston
17f4f14df7
Pull out event ids rather than full events for state
2016-08-25 13:42:44 +01:00
Erik Johnston
cd5b264b03
Fix None check in backfill
2016-08-25 10:39:19 +01:00
Erik Johnston
60a015550a
Bump changelog and version
2016-08-24 14:31:13 +01:00
Erik Johnston
90d5983d7a
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.1
2016-08-24 14:28:57 +01:00
Erik Johnston
c20cb5160d
Remove tracer
2016-08-24 13:22:47 +01:00
Erik Johnston
fda97dd58a
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/preserve_log_contexts
2016-08-24 13:22:02 +01:00
Paul "LeoNerd" Evans
8e1ed09dff
Move static knowledge of protocol metadata into AS handler; cache the result
2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans
965f33c901
Declare 'gitter' known protocol, with user lookup
2016-08-24 12:34:03 +01:00
Paul "LeoNerd" Evans
9899824b85
Initial hack at the 3PN protocols metadata lookup API
2016-08-24 12:33:01 +01:00
Erik Johnston
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
Paul "LeoNerd" Evans
63c19e1df9
Move 3PU/3PL lookup APIs into /thirdparty containing entity
2016-08-24 11:55:57 +01:00
Erik Johnston
86bcf4d6a7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/pagination
2016-08-24 10:38:21 +01:00
Erik Johnston
ba07d4a70e
Add None checks to backfill
2016-08-24 10:31:05 +01:00
Kegan Dougal
4b31426a02
Pass through user-supplied content in /join/$room_id
...
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.
If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
Erik Johnston
122c7a43c9
Merge pull request #1038 from matrix-org/erikj/receved_txn_purge
...
Delete old received_transactions rows
2016-08-23 11:02:40 +01:00
Erik Johnston
14047126d8
Bump changelog and version
2016-08-22 18:26:07 +01:00
Erik Johnston
d143f211c8
Merge pull request #1028 from matrix-org/dbkr/notifications_api
...
Add the Notifications API
2016-08-22 18:23:24 +01:00
Erik Johnston
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
Mark Haines
689f4cb914
Update comment
2016-08-22 16:17:31 +01:00
Mark Haines
c8f9b45bc2
Add usage stats to prometheus monitoring
2016-08-22 15:34:38 +01:00
Erik Johnston
e65bc7d315
Merge pull request #1031 from matrix-org/erikj/measure_notifier
...
Add more Measure blocks
2016-08-22 12:13:07 +01:00
Erik Johnston
33f3624ff7
Add exception logging. Fix typo
2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07
Allow request handlers to override metric name
2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253
Use top level measure
2016-08-22 10:18:12 +01:00
Matthew Hodgson
6e80c03d45
Merge branch 'develop' into dbkr/notifications_api
2016-08-20 00:16:18 +01:00
Erik Johnston
27e0178da9
Add a top level measure
2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218
Remove redundant measure
2016-08-19 18:40:31 +01:00
Erik Johnston
8731197e54
Only abort Measure on Exceptions
2016-08-19 18:23:45 +01:00
Erik Johnston
afbf6b33fc
defer.returnValue must not be called within Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
37adde32dc
Move defer.returnValue out of Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
04fc8bbcb0
Update keyring Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
39b900b316
Measure http.server render
2016-08-19 18:23:44 +01:00
Erik Johnston
47dd8f02a1
Measure _get_event_from_row
2016-08-19 18:23:44 +01:00
Erik Johnston
2426c2f21a
Measure keyrings
2016-08-19 18:23:44 +01:00
Erik Johnston
39242090e3
Add measure blocks to notifier
2016-08-19 18:23:44 +01:00
Erik Johnston
e6784daf07
Merge pull request #1030 from matrix-org/erikj/cache_contexts
...
Add concept of cache contexts
2016-08-19 16:29:58 +01:00
Erik Johnston
45fd2c8942
Ensure invalidation list does not grow unboundedly
2016-08-19 16:09:16 +01:00
Erik Johnston
c0d7d9d642
Rename to on_invalidate
2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909
Make cache_context an explicit option
2016-08-19 15:02:38 +01:00
Erik Johnston
f164fd9220
Move _bulk_get_push_rules_for_room to storage layer
2016-08-19 14:29:20 +01:00
Erik Johnston
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
Erik Johnston
4161ff2fc4
Add concept of cache contexts
2016-08-19 14:17:07 +01:00
Erik Johnston
b770435389
Make get_new_events_for_appservice use indices
2016-08-19 10:28:42 +01:00
Paul Evans
5674ea3e6c
Merge pull request #1026 from matrix-org/paul/thirdpartylookup
...
3rd party entity lookup
2016-08-18 20:52:50 +01:00
David Baker
1e4217c90c
Explicit join
2016-08-18 17:53:44 +01:00
David Baker
0acdd0f1ea
Use tuple comparison
...
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans
65201631a4
Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater
2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans
697872cf08
More warnings about invalid results from AS 3PE query
2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans
b515f844ee
Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration
2016-08-18 17:19:55 +01:00
David Baker
602c84cd9c
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans
2a91799fcc
Minor syntax neatenings
2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans
fcf1dec809
Appease pep8
2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans
105ff162d4
Authenticate 3PE lookup requests
2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans
06964c4a0a
Copypasta the 3PU support code to also do 3PL
2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans
f3afd6ef1a
Remove TODO note about request fields being strings - they're always strings
2016-08-18 15:53:01 +01:00
Erik Johnston
bcbd74dc5b
Remove log lines
2016-08-18 15:52:10 +01:00
Paul "LeoNerd" Evans
d7b42afc74
Log a warning if an AS yields an invalid 3PU lookup result
2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans
80f4740c8f
Scattergather the call out to ASes; validate received results
2016-08-18 15:40:41 +01:00
Erik Johnston
07229bbdae
Add appservice worker
2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans
434bbf2cb5
Filter 3PU lookups by only ASes that declare knowledge of that protocol
2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans
d5bf7a4a99
Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup
2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans
718ffcf8bb
Since empty lookups now return 200/empty list not 404, we can safely log failures as exceptions
2016-08-18 14:18:37 +01:00
Paul "LeoNerd" Evans
3856582741
Ensure that 3PU lookup request fields actually get passed in
2016-08-18 14:06:02 +01:00
Paul "LeoNerd" Evans
f0c73a1e7a
Extend individual list results into the main return list, don't append
2016-08-18 13:53:54 +01:00
Paul "LeoNerd" Evans
b3511adb92
Don't catch the return-value-as-exception that @defer.inlineCallbacks will use
2016-08-18 13:45:18 +01:00
Erik Johnston
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
Mark Haines
403ecd8a2c
Missed a s/federation reader/media repository/ in a log message
2016-08-18 10:26:15 +01:00
Mark Haines
47fbff7a05
Merge pull request #1024 from matrix-org/markjh/media_repository
...
Add a media repository worker
2016-08-18 10:12:17 +01:00
Mark Haines
396624864a
Add a media repository worker
2016-08-18 09:38:42 +01:00
Erik Johnston
e73dcb66da
Merge pull request #1022 from matrix-org/erikj/as_notify_perf
...
Make notify_interested_services faster
2016-08-18 09:35:19 +01:00
Paul "LeoNerd" Evans
3ec10dffd6
Actually make 3PU lookup calls out to ASes
2016-08-18 00:39:09 +01:00
Erik Johnston
732cf72b86
Fix push_display_name_rename schema update
2016-08-17 18:12:21 +01:00
Erik Johnston
abcb9aee5b
Make push Measure finer grained
2016-08-17 18:00:18 +01:00
Erik Johnston
2a0d8f8219
Merge pull request #1021 from matrix-org/erikj/mediasecurity_policy
...
Set `Content-Security-Policy` on media repo
2016-08-17 17:22:43 +01:00
Erik Johnston
320dfe523c
Make notify_interested_services faster
2016-08-17 17:20:50 +01:00
Erik Johnston
0af9e1a637
Set Content-Security-Policy on media repo
...
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans
fa87c981e1
Thread 3PU lookup through as far as the AS API object; which currently noöps it
2016-08-17 16:17:28 +01:00
David Baker
0d7cef0943
Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override
...
Move display name rule
2016-08-17 16:15:45 +01:00
Erik Johnston
f90b3d83a3
Add None check to _iterate_over_text
2016-08-17 15:17:17 +01:00
Matrix
f743471380
Change name of metric
2016-08-17 15:05:50 +01:00
Erik Johnston
b9e888858c
Move Measure block inside loop
2016-08-17 14:52:26 +01:00
Erik Johnston
973d67a033
Merge pull request #1019 from matrix-org/erikj/appservice_clean
...
Clean up _ServiceQueuer
2016-08-17 14:37:21 +01:00
Paul "LeoNerd" Evans
e3e3fbc23a
Initial empty implementation that just registers an API endpoint handler
2016-08-17 12:46:49 +01:00
Erik Johnston
7321f45457
Clean up _ServiceQueuer
2016-08-17 12:03:04 +01:00
Erik Johnston
b9abf3e4e3
Remove dead appservice code
2016-08-17 11:48:23 +01:00
Erik Johnston
62c5245c87
Measure notify_interested_services
2016-08-17 11:12:29 +01:00
Erik Johnston
949629291c
Do it in storage function
2016-08-16 17:05:34 +01:00
David Baker
ad42322257
Add migration script
...
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
David Baker
0bba2799b6
Merge remote-tracking branch 'origin/develop' into dbkr/contains_display_name_override
2016-08-16 16:46:37 +01:00
Erik Johnston
64a2acb161
Don't update caches replication stream if tokens haven't advanced
2016-08-16 16:44:19 +01:00
David Baker
1594eba29e
s/underride/override/ in the rule_id too
2016-08-16 16:44:07 +01:00
Erik Johnston
109a560905
Flake8
2016-08-16 14:57:21 +01:00
Erik Johnston
48b5829aea
Fix up preview URL API. Add tests.
...
This includes:
- Splitting out methods of a class into stand alone functions, to make
them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston
7c6f4f9427
Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri
...
Limit number of extremeties in backfill request
2016-08-16 12:55:42 +01:00
Erik Johnston
25c2332071
Merge pull request #1010 from matrix-org/erikj/refactor_deletions
...
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-16 11:37:53 +01:00
Erik Johnston
2ee1bd124c
Limit number of extremeties in backfill request
...
This works around a bug where if we make a backfill request with too
many extremeties it causes the request URI to be too long.
2016-08-16 11:34:36 +01:00
