Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-10-01 08:25:44 -04:00
Code Issues Packages Projects Releases Wiki Activity
9,892 Commits 4 Branches 609 Tags 60 MiB
8716eb4920
Commit Graph

7176 Commits

Author SHA1 Message Date
Richard van der Hoff
979eed4362 Fix user-interactive password auth 
this got broken in the previous commit
 2017-11-01 17:03:20 +00:00
Richard van der Hoff
bc8a5c0330 Notify auth providers on logout 
Provide a hook by which auth providers can be notified of logouts.
 2017-11-01 16:51:51 +00:00
Richard van der Hoff
4c8f94ac94 Allow password_auth_providers to return a callback 
... so that they have a way to record access tokens.
 2017-11-01 16:51:03 +00:00
Richard van der Hoff
846a94fbc9
Merge pull request #2620 from matrix-org/rav/auth_non_password 
Let password auth providers handle arbitrary login types
 2017-11-01 16:45:33 +00:00
Richard van der Hoff
3cd6b22c7b Let password auth providers handle arbitrary login types 
Provide a hook where password auth providers can say they know about other
login types, and get passed the relevant parameters
 2017-11-01 16:43:57 +00:00
David Baker
c9b9ef575b
Merge pull request #2621 from matrix-org/rav/refactor_accesstoken_delete 
Move access token deletion into auth handler
 2017-11-01 16:26:06 +00:00
Matthew Hodgson
275826f234
Merge pull request #2617 from matrix-org/matthew/auto-displayname 
automatically set default displayname on register
 2017-11-01 16:21:16 +00:00
David Baker
4f0488b307 Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete 2017-11-01 16:20:19 +00:00
David Baker
e5e930aec3
Merge pull request #2615 from matrix-org/rav/break_auth_device_dep 
Break dependency of auth_handler on device_handler
 2017-11-01 16:06:31 +00:00
David Baker
fbbacb284e
Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens 
Remove the last vestiges of refresh_tokens
 2017-11-01 15:57:35 +00:00
Matthew Hodgson
9f7a555b4e switch to setting default displayname in the storage layer 
to avoid clobbering guest user displaynames on registration
 2017-11-01 15:51:30 +00:00
Richard van der Hoff
dd13310fb8 Move access token deletion into auth handler 
Also move duplicated deactivation code into the auth handler.

I want to add some hooks when we deactivate an access token, so let's bring it
all in here so that there's somewhere to put it.
 2017-11-01 15:46:22 +00:00
David Baker
0bb253f37b Apparently this is python 2017-11-01 14:02:52 +00:00
David Baker
59e7e62c4b Log login requests 
Carefully though, to avoid logging passwords
 2017-11-01 13:58:01 +00:00
Matthew Hodgson
f8420d6279 automatically set default displayname on register 
to avoid leaking ugly MXIDs and cluttering up the timeline with
displayname changes as well as membership joins for autojoin rooms
(e.g. the status autojoin rooms), automatically set the displayname
to match the localpart of the mxid upon registration.
 2017-11-01 13:15:41 +00:00
Luke Barnard
99354b430e
Merge pull request #2612 from matrix-org/luke/groups-room-relationship-is-public 
Modify group room association API to allow modification of is_public
 2017-11-01 11:08:36 +00:00
Richard van der Hoff
74c56f794c Break dependency of auth_handler on device_handler 
I'm going to need to make the device_handler depend on the auth_handler, so I
need to break this dependency to avoid a cycle.

It turns out that the auth_handler was only using the device_handler in one
place which was an edge case which we can more elegantly handle by throwing an
error rather than fixing it up.
 2017-11-01 10:27:06 +00:00
Luke Barnard
318a249c8b Leave is_public as required argument of update_room_group_association 2017-11-01 09:36:01 +00:00
Luke Barnard
207fabbc6a Update docs for updating room group association 2017-11-01 09:35:15 +00:00
Richard van der Hoff
356bcafc44 Remove the last vestiges of refresh_tokens 2017-10-31 20:35:58 +00:00
Richard van der Hoff
3e0aaad190 Let auth providers get to the database 
Somewhat open to abuse, but also somewhat unavoidable :/
 2017-10-31 17:22:29 +00:00
Luke Barnard
13b3d7b4a0 Flake8 2017-10-31 17:20:11 +00:00
Luke Barnard
20fe347906 Modify group room association API to allow modification of is_public 
also includes renamings to make things more consistent.
 2017-10-31 17:04:28 +00:00
Richard van der Hoff
1650eb5847 DB schema interface for password auth providers 
Provide an interface by which password auth providers can register db schema
files to be run at startup
 2017-10-31 14:01:53 +00:00
David Baker
c31a7c3ff6
Merge pull request #2609 from matrix-org/rav/refactor_login 
Refactor some logic from LoginRestServlet into AuthHandler
 2017-10-31 13:51:36 +00:00
Richard van der Hoff
1b65ae00ac Refactor some logic from LoginRestServlet into AuthHandler 
I'm going to need some more flexibility in handling login types in password
auth providers, so as a first step, move some stuff from LoginRestServlet into
AuthHandler.

In particular, we pass everything other than SAML, JWT and token logins down to
the AuthHandler, which now has responsibility for checking the login type and
fishing the password out of the login dictionary, as well as qualifying the
user_id if need be. Ideally SAML, JWT and token would go that way too, but
there's no real need for it right now and I'm trying to minimise impact.

This commit *should* be non-functional.
 2017-10-31 10:48:41 +00:00
Richard van der Hoff
ffc574a6f9 Clean up backwards-compat hacks for ldap 
try to make the backwards-compat flows follow the same code paths as the modern
impl.

This commit should be non-functional.
 2017-10-31 10:47:02 +00:00
Luke Barnard
9bc17fc5fb Fix wording on group creation error 2017-10-30 15:17:23 +00:00
Matthew Hodgson
208a6647f1 fix typo 2017-10-29 20:54:20 +00:00
Erik Johnston
d0abb4e8e6 Fix typo when checking if user is invited to group 2017-10-27 16:57:19 +01:00
Erik Johnston
977078f06d Fix bad merge 2017-10-27 15:10:50 +01:00
Erik Johnston
6980c4557e Merge branch 'erikj/attestation_jitter' of github.com:matrix-org/synapse into develop 2017-10-27 15:09:05 +01:00
Erik Johnston
af92f5b00f Revert "Add jitter to validity period of attestations" 2017-10-27 15:07:21 +01:00
Erik Johnston
4ab8abbc2b Merge branch 'erikj/attestation_local_fix' of github.com:matrix-org/synapse into develop 2017-10-27 15:07:08 +01:00
Erik Johnston
6af3656deb Merge pull request #2595 from matrix-org/erikj/attestation_commnet 
Add comment about attestations
 2017-10-27 14:20:19 +01:00
Richard van der Hoff
4d83632009 Merge pull request #2591 from matrix-org/rav/device_delete_auth 
Device deletion: check UI auth matches access token
 2017-10-27 12:30:10 +01:00
Richard van der Hoff
110b373e9c Merge pull request #2589 from matrix-org/rav/as_deactivate_account 
Allow ASes to deactivate their own users
 2017-10-27 12:29:32 +01:00
Erik Johnston
ca571b0ec3 Add jitter to validity period of attestations 
This helps ensure that the renewals of attestations are spread out more
evenly.
 2017-10-27 11:57:27 +01:00
Luke Barnard
d8c26162a1 Merge pull request #2582 from matrix-org/luke/group-is-public 
Add is_public to groups table to allow for private groups
 2017-10-27 11:41:13 +01:00
Erik Johnston
c067088747 Add comment about attestations 2017-10-27 11:35:41 +01:00
Luke Barnard
5451cc7792 Request is_public from database 2017-10-27 11:27:43 +01:00
Luke Barnard
124314672f group is dict 2017-10-27 11:08:19 +01:00
Luke Barnard
6362298fa5 Create groups with is_public = True 2017-10-27 11:04:20 +01:00
Richard van der Hoff
8b56977b6f Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header 
Front-end proxy: pass through auth header
 2017-10-27 11:01:50 +01:00
Richard van der Hoff
173567a7f2 Docstring for post_urlencoded_get_json 2017-10-27 10:59:50 +01:00
Luke Barnard
c7d9f25d22 Fix create_group to pass requester_user_id 2017-10-27 10:57:20 +01:00
Erik Johnston
e27b76d117 Import logger 2017-10-27 10:54:02 +01:00
Richard van der Hoff
8854c039f2 Merge pull request #2585 from matrix-org/rav/unstable_to_r0 
Support /keys/upload on /r0 as well as /unstable
 2017-10-27 10:53:48 +01:00
Richard van der Hoff
14f581abc2 Merge pull request #2584 from matrix-org/rav/fix_httpclient_logcontexts 
Fix logcontext leaks in httpclient
 2017-10-27 10:53:29 +01:00
Luke Barnard
2ca46c7afc Correct logic for checking private group membership 2017-10-27 10:48:01 +01:00
Erik Johnston
82d8c1bacb Fixup 2017-10-27 10:30:21 +01:00
Erik Johnston
2fd9831f7c Merge pull request #2574 from matrix-org/erikj/room_list_fixes 
Add logging and fix log contexts for publicRooms
 2017-10-27 10:01:23 +01:00
Erik Johnston
195abfe7a5 Remove incorrect attestations 2017-10-27 09:58:13 +01:00
Erik Johnston
d8dde19f04 Log if we try to do attestations for our own user and group 2017-10-27 09:55:01 +01:00
Erik Johnston
585972b51a Don't generate group attestations for local users 2017-10-27 09:46:56 +01:00
Richard van der Hoff
7a6546228b Device deletion: check UI auth matches access token 
(otherwise there's no point in the UI auth)
 2017-10-27 00:04:31 +01:00
Richard van der Hoff
785bd7fd75 Allow ASes to deactivate their own users 2017-10-27 00:01:00 +01:00
Richard van der Hoff
c89e6aadff Merge pull request #2581 from matrix-org/rav/fix_init_with_no_logfile 
Fix error when running synapse with no logfile
 2017-10-26 22:16:57 +01:00
Richard van der Hoff
54a2525133 Front-end proxy: pass through auth header 
So that access-token-in-an-auth-header works.
 2017-10-26 18:19:01 +01:00
Richard van der Hoff
0a5866bec9 Support /keys/upload on /r0 as well as /unstable 
(So that we can stop riot relying on it in /unstable)
 2017-10-26 18:18:23 +01:00
Richard van der Hoff
0d8e3ad48b Fix logcontext leaks in httpclient 
`preserve_context_over_fn` is borked
 2017-10-26 18:17:10 +01:00
Richard van der Hoff
12ef02dc3d SimpleHTTPClient: add support for headers 
Sometimes we need to pass headers into these methods
 2017-10-26 17:59:50 +01:00
Luke Barnard
69e8a05f35 Make it work 2017-10-26 17:55:58 +01:00
Luke Barnard
007cd48af6 Recreate groups table instead of adding column 
Adding a column with non-constant default not possible in sqlite3
 2017-10-26 17:55:22 +01:00
Luke Barnard
713e60b9b6 Awful hack to get default true 2017-10-26 17:38:14 +01:00
Luke Barnard
cfa4e658e0 Bump schema version to 46 2017-10-26 17:23:49 +01:00
Luke Barnard
595fe67f01 delint 2017-10-26 17:20:24 +01:00
Luke Barnard
9b2feef9eb Add is_public to groups table to allow for private groups 
Prevent group API access to non-members for private groups

Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.
 2017-10-26 16:51:32 +01:00
Richard van der Hoff
f7f90e0c8d Fix error when running synapse with no logfile 
Fixes 'UnboundLocalError: local variable 'sighup' referenced before assignment'
 2017-10-26 16:45:20 +01:00
Krombel
8299b323ee add release endpoints for /thirdparty 2017-10-26 16:58:20 +02:00
Krombel
9b436c8b4c register some /unstable endpoints in /r0 as well 2017-10-26 15:22:50 +02:00
Erik Johnston
566e21eac8 Update room_list.py 2017-10-26 11:39:54 +01:00
Maxime Vaillancourt
5287e57c86 Ignore noscript tags when generating URL previews 2017-10-25 20:44:34 -04:00
Erik Johnston
2a7e9faeec Do logcontexts outside ResponseCache 2017-10-25 15:21:08 +01:00
Erik Johnston
1ad1ba9e6a Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-10-25 10:27:23 +01:00
Erik Johnston
33a9026cdf Add logging and fix log contexts for publicRooms 2017-10-25 10:26:06 +01:00
Matthew Hodgson
efd0f5a3c5 tip for generating tls_fingerprints 2017-10-24 18:49:49 +01:00
Erik Johnston
6ba4fabdb9 Bump version and changelog 2017-10-24 14:15:27 +01:00
Erik Johnston
39dc52157d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile 2017-10-24 09:16:20 +01:00
Richard van der Hoff
0be99858f3 fix vars named l 
E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".
 2017-10-23 15:56:38 +01:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:' 
what could possibly go wrong
 2017-10-23 15:52:32 +01:00
Erik Johnston
ce6d4914f4 Correctly wire in update group profile over federation 2017-10-23 15:21:24 +01:00
Richard van der Hoff
ecf198aab8 Merge pull request #2566 from matrix-org/rav/media_logcontext_leak 
Fix a logcontext leak in the media repo
 2017-10-23 14:47:49 +01:00
Richard van der Hoff
3267b81b81 Merge pull request #2561 from matrix-org/rav/id_checking 
Updates to ID checking
 2017-10-23 14:39:20 +01:00
Richard van der Hoff
d03cfc4258 Fix a logcontext leak in the media repo 2017-10-23 14:34:27 +01:00
Erik Johnston
1de557975f Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-10-23 13:18:12 +01:00
Erik Johnston
13e16cf302 Bump version and changelog 2017-10-23 13:13:31 +01:00
Richard van der Hoff
1135193dfd Validate group ids when parsing 
May as well do it whenever we parse a Group ID. We check the sigil and basic
structure here so it makes sense to check the grammar in the same place.
 2017-10-21 00:30:39 +01:00
Richard van der Hoff
29812c628b Allow = in mxids and groupids 
... because the spec says we should.
 2017-10-20 23:42:53 +01:00
Richard van der Hoff
58fbbe0f1d Disallow capital letters in userids 
Factor out a common function for checking user ids and group ids, which forbids
capitals.
 2017-10-20 23:37:22 +01:00
Richard van der Hoff
631d7b87b5 Remove pointless create() method 
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
 2017-10-20 22:14:55 +01:00
Erik Johnston
6070647774 Correctly bump version 2017-10-19 16:40:20 +01:00
Erik Johnston
0ef0aeceac Bump version and changelog 2017-10-19 13:48:33 +01:00
Erik Johnston
c7d46510d7 Flake8 2017-10-19 13:36:06 +01:00
Erik Johnston
ffd3f1a783 Add missing file... 2017-10-19 12:17:30 +01:00
Erik Johnston
29bafe2f7e Add config to enable group creation 2017-10-19 12:13:44 +01:00
Erik Johnston
513c23bfd9 Enforce sensible group IDs 2017-10-19 12:01:01 +01:00
Erik Johnston
011d03a0f6 Fix typo 2017-10-19 11:22:48 +01:00
Erik Johnston
9ab859f27b Fix typo in group attestation handling 2017-10-19 10:55:52 +01:00
Erik Johnston
bd5718d0ad Fix typo in thumbnail generation 2017-10-19 10:27:18 +01:00
Erik Johnston
161a862ffb Fix typo 2017-10-19 10:17:43 +01:00
Richard van der Hoff
69994c385a Merge pull request #2553 from matrix-org/rav/fix_500_on_event_send 
Fix 500 error when we get an error handling a PDU
 2017-10-18 11:25:44 +01:00
Richard van der Hoff
b5dbbac308 Merge pull request #2552 from matrix-org/rav/fix_500_on_dodgy_powerlevels 
Fix 500 error when fields missing from power_levels event
 2017-10-17 20:53:30 +01:00
Richard van der Hoff
582bd19ee9 Fix 500 error when we get an error handling a PDU 
FederationServer doesn't have a send_failure (and nor does its subclass,
ReplicationLayer), so this was failing.

I'm not really sure what the idea behind send_failure is, given (a) we don't do
anything at the other end with it except log it, and (b) we also send back the
failure via the transaction response. I suspect there's a whole lot of dead
code around it, but for now I'm just removing the broken bit.
 2017-10-17 20:52:40 +01:00
Richard van der Hoff
74f99f227c Doc some more dynamic Homeserver methods 2017-10-17 20:51:29 +01:00
Richard van der Hoff
c2bd177ea0 Fix 500 error when fields missing from power_levels event 
If the users or events keys were missing from a power_levels event, then
we would throw 500s when trying to auth them.
 2017-10-17 17:05:42 +01:00
Erik Johnston
fe6e9f580b Merge pull request #2550 from krombel/fix_thumbnail_2548 
fix thumbnailing (#2548)
 2017-10-17 15:35:18 +01:00
Richard van der Hoff
7216c76654 Improve error handling for missing files (#2551) 
`os.path.exists` doesn't allow us to distinguish between permissions errors and
the path actually not existing, which repeatedly confuses people. It also means
that we try to overwrite existing key files, which is super-confusing. (cf
issues #2455, #2379). Use os.stat instead.

Also, don't recomemnd the the use of --generate-config, which screws everything
up if you're using debian (cf #2455).
 2017-10-17 14:46:17 +01:00
Richard van der Hoff
4cc8bb0767 Merge pull request #2549 from matrix-org/rav/event_persist_logcontexts 
Fix logcontext handling for persist_events
 2017-10-17 13:20:35 +01:00
David Baker
4e242b3e20 Merge pull request #2545 from matrix-org/dbkr/auto_join_rooms 
Add config option to auto-join new users to rooms
 2017-10-17 11:45:49 +01:00
Krombel
a6245478c8 fix thumbnailing (#2548) 
in commit 0e28281a the code for thumbnailing got refactored and the
renaming of this variables was not done correctly.

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
 2017-10-17 12:45:33 +02:00
Richard van der Hoff
2e9f5ea31a Fix logcontext handling for persist_events 
* don't use preserve_context_over_deferred, which is known broken.

* remove a redundant preserve_fn.

* add/improve some comments
 2017-10-17 10:59:30 +01:00
David Baker
a9c2e930ac pep8 2017-10-17 10:13:13 +01:00
David Baker
c05e6015cc Add config option to auto-join new users to rooms 
New users who register on the server will be dumped into all rooms in
auto_join_rooms in the config.
 2017-10-16 17:57:27 +01:00
Luke Barnard
85f5674e44 Delint 2017-10-16 15:52:17 +01:00
Luke Barnard
c43e8a9736 Make it work. Warn about lack of user profile 2017-10-16 15:50:39 +01:00
Luke Barnard
a3ac4f6b0a _create_rererouter for get_invited_users_in_group 2017-10-16 15:41:03 +01:00
Luke Barnard
5dfd0350c7 Merge branch 'develop' into luke/groups-invited-users 2017-10-16 15:34:36 +01:00
Luke Barnard
ca96d609e4 Merge pull request #2543 from matrix-org/luke/fix-on-group-invite-no-profile 
Log a warning when no profile for invited member
 2017-10-16 16:33:53 +02:00
Luke Barnard
2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Luke Barnard
6079d0027a Log a warning when no profile for invited member 
And return empty profile
 2017-10-16 14:20:45 +01:00
David Baker
9342bcfce0 Omit the *s for @room notifications 
They're just redundant
 2017-10-16 13:38:10 +01:00
Erik Johnston
1b6b0b1e66 Add try/finally block to close t_byte_source 2017-10-13 15:34:08 +01:00
Erik Johnston
6b725cf56a Remove old comment 2017-10-13 15:23:41 +01:00
Erik Johnston
2b24416e90 Don't reuse source but instead copy from primary media store to backup 2017-10-13 14:11:34 +01:00
Erik Johnston
b92a8e6e4a PEP8 2017-10-13 13:58:57 +01:00
Erik Johnston
31aa7bd8d1 Move type into key 2017-10-13 13:47:38 +01:00
Erik Johnston
ad1911bbf4 Comment 2017-10-13 13:47:05 +01:00
Erik Johnston
c021c39cbd Remove spurious addition 2017-10-13 13:46:53 +01:00
Erik Johnston
1f43d22397 Don't needlessly rename variable 2017-10-13 11:42:07 +01:00
Erik Johnston
a675bd08bd Add paths back in... 2017-10-13 11:41:06 +01:00
Erik Johnston
4d7e1dde70 Remove unnecessary diff 2017-10-13 11:36:32 +01:00
Erik Johnston
ae5d18617a Make things be absolute paths again 2017-10-13 11:35:44 +01:00
Erik Johnston
9732ec6797 s/write_to_file/write_to_file_and_backup/ 2017-10-13 11:34:41 +01:00
Erik Johnston
0e28281a02 Fix up 2017-10-13 11:33:49 +01:00
Erik Johnston
505371414f Fix up thumbnailing function 2017-10-13 11:23:53 +01:00
Erik Johnston
e3428d26ca Fix typo 2017-10-13 10:39:59 +01:00
Erik Johnston
35332298ef Fix up comments 2017-10-13 10:39:32 +01:00
Erik Johnston
64db043a71 Move makedirs to thread 2017-10-13 10:25:01 +01:00
Erik Johnston
b60859d6cc Use make_deferred_yieldable 2017-10-13 10:24:19 +01:00
Erik Johnston
d76621a47b Fix comments 2017-10-12 18:16:25 +01:00
Erik Johnston
4ae85ae121 Don't close prematurely.. 2017-10-12 17:57:31 +01:00
Erik Johnston
cc505b4b5e getvalue closes buffer 2017-10-12 17:52:30 +01:00
Erik Johnston
1259a76047 Get len before close 2017-10-12 17:39:23 +01:00
Erik Johnston
802ca12d05 Don't close file prematurely 2017-10-12 17:37:21 +01:00
Erik Johnston
e283b555b1 Copy everything to backup 2017-10-12 17:31:24 +01:00
Erik Johnston
b77a13812c Typo 2017-10-12 15:32:32 +01:00
Erik Johnston
6dfde6d485 Remove dead code 2017-10-12 15:30:26 +01:00
Erik Johnston
c8eeef6947 Fix typos 2017-10-12 15:28:24 +01:00
Erik Johnston
67cb89fbdf Fix typo 2017-10-12 15:23:41 +01:00
Erik Johnston
bf4fb1fb40 Basic implementation of backup media store 2017-10-12 15:20:59 +01:00
hera
f807f7f804 log when we get an exception handling replication updates 2017-10-12 11:51:24 +01:00
David Baker
b8d8ed1ba9 Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages 
Allow error strings from spam checker
 2017-10-12 10:31:03 +01:00
Richard van der Hoff
cc794d60e7 Merge pull request #2532 from matrix-org/rav/fix_linearizer 
Fix stackoverflow and logcontexts from linearizer
 2017-10-11 17:29:32 +01:00
Richard van der Hoff
8dd0c85ac5 Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling 
log pdu_failures from incoming transactions
 2017-10-11 17:29:14 +01:00
Richard van der Hoff
76fa695241 Merge pull request #2515 from matrix-org/rav/fix_receipt_logcontext 
A logformatter which includes the stack when the exception was caught when
logging exceptions.
 2017-10-11 17:28:01 +01:00
Richard van der Hoff
f30c4ed2bc logformatter: fix AttributeError 
make sure we have the relevant fields before we try to log them.
 2017-10-11 17:26:17 +01:00
Erik Johnston
b752507b48 Fix fetching remote summaries 2017-10-11 16:59:18 +01:00
Erik Johnston
818b08d0e4 peeeeeeeeep8888888888888888888888888888 2017-10-11 15:54:00 +01:00
Erik Johnston
ea18996f54 Fix group stream replication 
The stream update functions expect the storage function to return a list
of tuples.
 2017-10-11 15:44:39 +01:00
Richard van der Hoff
68fd82e840 Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext 
fix a logcontext leak in read receipt handling
 2017-10-11 15:08:53 +01:00
Richard van der Hoff
4fad8efbfb Fix stackoverflow and logcontexts from linearizer 
1. make it not blow out the stack when there are more than 50 things waiting
   for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505.

2. Make it not mess up the log contexts.
 2017-10-11 15:05:05 +01:00
David Baker
b78bae2d51 fix isinstance 2017-10-11 14:49:09 +01:00
Erik Johnston
271f5601f3 Fix typo in invite to group 2017-10-11 14:45:33 +01:00
David Baker
c3b7a45e84 Allow error strings from spam checker 2017-10-11 14:39:22 +01:00
Richard van der Hoff
c3e190ce67 fix a logcontext leak in read receipt handling 2017-10-11 14:37:20 +01:00
Richard van der Hoff
b75d443caf log pdu_failures from incoming transactions 
... even if we have no EDUs.

This appears to have been introduced in
476899295f.
 2017-10-11 14:36:13 +01:00
Erik Johnston
27e727a146 Fix typo 2017-10-11 14:32:40 +01:00
Erik Johnston
4ce4379235 Fix attestations to check correct server name 2017-10-11 14:11:43 +01:00
Erik Johnston
c2c47550f9 Fix schema delta versions 2017-10-11 13:23:15 +01:00
Erik Johnston
535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged 
Initial Group Implementation
 2017-10-11 13:20:07 +01:00
Erik Johnston
dfbf73408c Merge pull request #2501 from matrix-org/dbkr/channel_notifications 
Support for channel notifications
 2017-10-11 13:19:29 +01:00
Erik Johnston
bc7f3eb32f Merge pull request #2483 from jeremycline/unfreeze-ujson-dump 
Unfreeze event before serializing with ujson
 2017-10-11 13:18:52 +01:00
Erik Johnston
ec954f47fb Validate room ids 2017-10-11 13:15:44 +01:00
David Baker
81a5e0073c pep8 2017-10-10 15:53:34 +01:00
David Baker
ab1bc9bf5f Don't KeyError if no power_levels event 2017-10-10 15:34:05 +01:00
David Baker
0f1eb3e914 Use notification levels in power_levels 
Rather than making the condition directly require a specific power
level. This way the level require to notify a room can be configured
per room.
 2017-10-10 15:23:00 +01:00
Erik Johnston
84e27a592d Merge pull request #2490 from matrix-org/erikj/drop_left_room_events 
Ignore incoming events for rooms that we have left
 2017-10-10 11:58:32 +01:00
David Baker
c9f034b4ac There was already a constant for this 
also update copyright
 2017-10-10 11:47:10 +01:00
David Baker
a9f9d68631 More optimisation 2017-10-10 11:38:31 +01:00
David Baker
707374d5dc What year is it!? Who's the president!? 2017-10-10 11:21:41 +01:00
David Baker
89fa00ddff Merge branch 'develop' into dbkr/channel_notifications 2017-10-10 11:20:17 +01:00
Richard van der Hoff
6a6cc27aee fed server: process PDUs for different rooms in parallel 
With luck, this will give a real-time improvement when there are many rooms and
the server ends up calling out to fetch missing events.
 2017-10-09 18:30:31 +01:00
Richard van der Hoff
4c7c4d4061 Fed server: use a linearizer for ongoing transactions 
We don't want to process the same transaction multiple times concurrently, so
use a linearizer.
 2017-10-09 18:30:10 +01:00
Richard van der Hoff
ba5b9b80a5 fed server: refactor on_incoming_transaction 
Move as much as possible to after the have_responded check, and reduce the
number of times we iterate over the pdu list.
 2017-10-09 18:10:53 +01:00
Richard van der Hoff
a6e3222fe5 Fed server: Move origin-check code to _handle_received_pdu 
The response-building code expects there to be an entry in the `results` list
for each entry in the pdu_list, so the early `continue` was messing this
up. That doesn't really matter, because all that the federation client does is
log any errors, but it's pretty poor form.
 2017-10-09 17:53:32 +01:00
Richard van der Hoff
3cc852d339 Fancy logformatter to format exceptions better 
This is a bit of an experimental change at this point; the idea is to see if it
helps us track down where our stack overflows are coming from by logging the
stack when the exception was caught and turned into a Failure. (We'll also need
edf2704420).

If we deploy this, we'll be able to enable it via the log config yaml.
 2017-10-09 17:44:42 +01:00
Richard van der Hoff
0eeaa25694 Merge pull request #2508 from matrix-org/rav/federation_queue_logcontexts 
Fix up logcontext handling in (federation) TransactionQueue
 2017-10-09 17:43:48 +01:00
Richard van der Hoff
aa3fac8057 Merge pull request #2507 from matrix-org/rav/execute_concurrently_log_contexts 
Fix logcontext handling for concurrently_execute
 2017-10-09 17:43:32 +01:00
Erik Johnston
e8496efe84 Fix up comment 2017-10-09 15:17:34 +01:00
Richard van der Hoff
01bbacf3c4 Fix up logcontext handling in (federation) TransactionQueue 
Avoid using preserve_context_over_function, which has problems with respect to
logcontexts.
 2017-10-06 22:39:25 +01:00
Richard van der Hoff
148428ce76 Fix logcontext handling for concurrently_execute 
Avoid preserve_context_over_deferred, which is broken.
 2017-10-06 22:24:28 +01:00
Richard van der Hoff
c8f568ddf9 Fix up deferred handling in federation.py 
* Avoid preserve_context_over_deferred, which is broken

* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
  failure" erros
 2017-10-06 22:14:24 +01:00
Richard van der Hoff
3ddda939d3 some comments in the state res code 2017-10-05 14:58:17 +01:00
David Baker
f878e6f8af Spam checking: add the invitee to user_may_invite 2017-10-05 14:02:28 +01:00
David Baker
269af961e9 Make be faster 2017-10-05 13:27:12 +01:00
David Baker
ed80c6b6cc Add fastpath optimisation 2017-10-05 13:20:22 +01:00
David Baker
e433393c4f pep8 2017-10-05 13:08:02 +01:00
David Baker
985ce80375 They're called rooms 2017-10-05 13:03:44 +01:00
David Baker
b9b9714fd5 Get rule type right 2017-10-05 13:02:19 +01:00
David Baker
fa969cfdde Support for channel notifications 
Add condition type to check the sender's power level and add a base
rule using it for @channel notifications.
 2017-10-05 12:39:18 +01:00
David Baker
0c8da8b519 Use better method for word boundary searching 
From ebc95667b8
 2017-10-05 11:57:43 +01:00
Erik Johnston
eaaa837e00 Don't corrupt cache 2017-10-05 11:43:22 +01:00
David Baker
cbe3c3fdd4 pep8 2017-10-05 11:43:10 +01:00
David Baker
6748f0a579 Fix notif kws that start/end with non-word chars 
Only prepend / append word bounary characters if the search
expression starts or ends with a word character, otherwise they
don't work because there's no word bounary between whitespace and
a non-word char.
 2017-10-05 11:33:30 +01:00
David Baker
d8ce68b09b spam check room publishing 2017-10-04 14:29:33 +01:00
David Baker
78d4ced829 un-double indent 2017-10-04 12:44:27 +01:00
David Baker
197c14dbcf Add room creation checks to spam checker 
Lets the spam checker deny attempts to create rooms and add aliases
to them.
 2017-10-04 10:47:54 +01:00
David Baker
1e2ac54351 s/roomid/room_id/ 2017-10-03 17:41:38 +01:00
David Baker
1e375468de pass room id too 2017-10-03 17:13:14 +01:00
David Baker
c2c188b699 Federation was passing strings anyway 
so pass string everywhere
 2017-10-03 15:46:19 +01:00
David Baker
c46a0d7eb4 this shouldn't be debug 2017-10-03 15:20:14 +01:00
David Baker
bd769a81e1 better logging 2017-10-03 15:16:40 +01:00
David Baker
537088e7dc Actually write warpper function 2017-10-03 14:28:12 +01:00
David Baker
41fd9989a2 Skip spam check for admin users 2017-10-03 14:17:44 +01:00
Erik Johnston
11d62f43c9 Invalidate cache 2017-10-03 14:12:28 +01:00
Erik Johnston
e4ab96021e Update comments 2017-10-03 14:10:41 +01:00
David Baker
2a7ed700d5 Fix param name & lint 2017-10-03 14:04:10 +01:00
David Baker
84716d267c Allow spam checker to reject invites too 2017-10-03 13:56:43 +01:00
Erik Johnston
f2da6df568 Remove spurious line feed 2017-10-03 11:31:06 +01:00
Erik Johnston
30848c0fcd Ignore incoming events for rooms that we have left 
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.

However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.

This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
 2017-10-03 11:18:21 +01:00
Erik Johnston
e585c83209 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-10-02 18:11:24 +01:00
Erik Johnston
6c1bb1601e Bump version and changelog 2017-10-02 18:05:17 +01:00
Erik Johnston
ea87cb1ba5 Make 'affinity' package optional 2017-10-02 18:03:59 +01:00
Erik Johnston
3fed5bb25f Move quit_with_error 2017-10-02 17:59:34 +01:00
David Baker
27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Erik Johnston
90d70af269 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-10-02 16:20:23 +01:00
Erik Johnston
e4a709eda3 Bump version and change log 2017-10-02 13:51:38 +01:00
Jeremy Cline
cafb8de132
Unfreeze event before serializing with ujson 
In newer versions of https://github.com/esnme/ultrajson, ujson does not
serialize frozendicts (introduced in esnme/ultrajson@53f85b1). Although
the PyPI version is still 1.35, Fedora ships with a build from commit
esnme/ultrajson@2f1d487. This causes the serialization to fail if the
distribution-provided package is used.

This runs the event through the unfreeze utility before serializing it.

Thanks to @ignatenkobrain for tracking down the root cause.

fixes #2351

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
 2017-09-30 11:22:37 -04:00
Richard van der Hoff
d5325d7ef1 Merge pull request #2480 from matrix-org/rav/federation_client_logging 
Improve logging of failures in matrixfederationclient
 2017-09-29 17:32:53 +01:00
Erik Johnston
d5694ac5fa Only log if we've removed media 2017-09-28 16:08:08 +01:00
Richard van der Hoff
e43de3ae4b Improve logging of failures in matrixfederationclient 
* don't log exception types twice
* not all exceptions have a meaningful 'message'. Use the repr rather than
  attempting to build a string ourselves.
 2017-09-28 15:38:09 +01:00
Richard van der Hoff
75e67b9ee4 Handle SERVFAILs when doing AAAA lookups for federation (#2477) 
... to cope with people with broken dnssec setups, mostly
 2017-09-28 15:24:00 +01:00
Erik Johnston
768f00dedb Up the limits on number of url cache entries to delete at one time 2017-09-28 14:27:27 +01:00
Erik Johnston
4dc07e93a8 Add old indices 2017-09-28 14:10:33 +01:00
Erik Johnston
7cc483aa0e Clear up expired url cache every 10s 2017-09-28 13:56:53 +01:00
Erik Johnston
e1e7d76cf1 Actually assign result to variable 2017-09-28 13:55:29 +01:00
Erik Johnston
93247a424a Only pull out local media that were for url cache 2017-09-28 13:48:14 +01:00
Erik Johnston
5f501ec7e2 Fix typo in url cache expiry timer 2017-09-28 12:59:01 +01:00
Erik Johnston
761d255fdf Merge pull request #2479 from matrix-org/erikj/expire_url_cache_thumbnails 
Support new and old style media id formats
 2017-09-28 12:58:13 +01:00
Erik Johnston
ace8079086 Support new and old style media id formats 2017-09-28 12:52:51 +01:00
Erik Johnston
7a44c01d89 Fix typo 2017-09-28 12:46:04 +01:00
Erik Johnston
ae79764fe5 Change expires column to expires_ts 2017-09-28 12:37:53 +01:00
Erik Johnston
77f1d24de3 More brackets 2017-09-28 12:23:15 +01:00
Erik Johnston
9ccb4226ba Delete expired url cache data 2017-09-28 12:18:06 +01:00
Erik Johnston
8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston
adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
David Baker
ef3a5ae787 Don't test is spam_checker not None 
Sometimes it's a Mock object which is not none but is still not
what we're after
 2017-09-27 11:24:19 +01:00
David Baker
8c06dd6071 Remove unintentional debugging 2017-09-27 10:31:14 +01:00
David Baker
60c78666ab pep8 2017-09-27 10:26:13 +01:00
David Baker
1786b0e768 Forgot the new file again :( 2017-09-27 10:22:54 +01:00
David Baker
6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
David Baker
9fd086e506 unnecessary parens 2017-09-26 17:59:46 +01:00
David Baker
0b03a97708 Add module_loader.py 2017-09-26 17:56:41 +01:00
David Baker
4824a33c31 Factor out module loading to a separate place 
So it can be reused
 2017-09-26 17:51:26 +01:00
Erik Johnston
17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston
a8e2a3df32 Add unique index to group_rooms table 2017-09-26 15:39:21 +01:00
Erik Johnston
95298783bb Add is_publicised to group summary 2017-09-26 11:04:37 +01:00
Erik Johnston
1a398b19fd Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.23.0 2017-09-26 10:08:59 +01:00
Erik Johnston
f4c8cd5e85 Bump changelog and version 2017-09-26 10:02:48 +01:00
Erik Johnston
e3edca3b5d Refactor to speed up incremental syncs 2017-09-25 17:35:39 +01:00
Erik Johnston
44edac0497 Merge branch 'release-v0.23.0' of github.com:matrix-org/synapse into develop 2017-09-25 14:52:46 +01:00
Richard van der Hoff
94133d7ce8 Merge branch 'develop' into develop 2017-09-25 11:50:11 +01:00
Erik Johnston
ba8fdc925c Bump version and changes 2017-09-25 11:01:31 +01:00
Richard van der Hoff
79b3cf3e02 Fix logcontxt leak in keyclient (#2465) 
preserve_context_over_function doesn't do what you want it to do.
 2017-09-25 09:51:39 +01:00
Richard van der Hoff
f65e31d22f Do an AAAA lookup on SRV record targets (#2462) 
Support SRV records which point at AAAA records, as well as A records.

Fixes https://github.com/matrix-org/synapse/issues/2405
 2017-09-22 20:26:47 +01:00
Matthew Hodgson
f496399ac4 fix thinko'd docstring 2017-09-22 15:34:14 +01:00
Erik Johnston
3166ed55b2 Fix device list when rejoining room (#2461) 2017-09-22 14:44:17 +01:00
Erik Johnston
e1dec2f1a7 Remove user from group summary when the leave the group 2017-09-21 16:09:57 +01:00
Erik Johnston
bb746a9de1 Revert: Keep room_id's in group summary 2017-09-21 15:57:22 +01:00
Erik Johnston
ae8d4bb0f0 Keep room_id's in group summary 2017-09-21 15:55:18 +01:00
Erik Johnston
197d82dc07 Correctly return next token 2017-09-20 11:12:11 +01:00
Erik Johnston
069ae2df12 Fix initial sync 2017-09-20 10:52:12 +01:00
Richard van der Hoff
6de74ea6d7 Fix logcontexts in _check_sigs_and_hashes 2017-09-20 01:32:42 +01:00
Richard van der Hoff
c5c24c239b Fix logcontext handling in verify_json_objects_for_server 
preserve_context_over_fn is essentially broken, because (a) it pointlessly
drops the current logcontext before calling its wrapped function, which means
we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the
resulting deferred in a _PreservingContextDeferred, which is very dangerous
because you then can't yield on it without leaking context back into the
reactor.

Instead, let's specify that the resultant deferreds call their callbacks with
no logcontext.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
c5b0e9f485 Turn _start_key_lookups into an inlineCallbacks function 
... which means that logcontexts can be correctly preserved for the stuff it
does.

get_server_verify_keys is now called with the logcontext, so needs to
preserve_fn when it fires off its nested inlineCallbacks function.

Also renames get_server_verify_keys to reflect the fact it's meant to be
private.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
abdefb8a01 Fix potential race in _start_key_lookups 
If the verify_request.deferred has already completed, then `remove_deferreds`
will be called immediately. It therefore might resolve the server_to_deferred
deferred while there are still other requests for that server in flight.

To avoid that, we should build the complete list of requests, and *then* add the
callbacks.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
afbd773dc6 Add some comments to _start_key_lookups 2017-09-20 01:32:42 +01:00
Richard van der Hoff
2a4b9ea233 Consistency for how verify_request.deferred is called 
Define that it is run with no log context, and make sure that happens.

If we aren't careful to reset the logcontext, we can't bung the deferreds into
defer.gatherResults etc. We don't actually do that directly, but we *do*
resolve other deferreds from affected callbacks (notably the server_to_deferred
map in _start_key_lookups), and those *do* get passed into
defer.gatherResults. It turns out that this way ends up being least confusing.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
3b98439eca Factor out _start_key_lookups 
... to make it easier to see what's going on.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
fde63b880d Replace server_and_json with verify_requests 
This is a precursor to factoring some of this code out.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
2d511defd9 pull out handle_key_deferred to top level 
There's no need for this to be a nested definition; pulling it out not only
makes it more efficient, but makes it easier to check that it's not accessing
any local variables it shouldn't be.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
dd1ea9763a Fix incorrect key_ids in error message 2017-09-20 01:32:42 +01:00
Richard van der Hoff
e76d1135dd Invalidate signing key cache when we gat an update 
This might make the cache slightly more efficient.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
fcf2c0fd1a Remove redundant preserve_fn 
preserve_fn is a no-op unless the wrapped function returns a
Deferred. verify_json_objects_for_server returns a list, so this is doing
nothing.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
9864efa532 Fix concurrent server_key requests (#2458) 
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
 2017-09-19 23:25:44 +01:00
Richard van der Hoff
aa620d09a0 Add a config option to block all room invites (#2457) 
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
 2017-09-19 16:08:14 +01:00
Richard van der Hoff
2eabdf3f98 add some comments to on_exchange_third_party_invite_request 2017-09-19 12:20:36 +01:00
Richard van der Hoff
5ed109d59f PoC for filtering spammy events (#2456) 
Demonstration of how you might add some hooks to filter out spammy events.
 2017-09-19 12:20:11 +01:00
Erik Johnston
93e504d04e Ensure that creator of group sees group down /sync 2017-09-19 11:08:16 +01:00
Erik Johnston
b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Richard van der Hoff
3f405b34e9 Fix overzealous kicking of guest users (#2453) 
We should only kick guest users if the guest access event is authorised.
 2017-09-19 08:52:52 +01:00
Richard van der Hoff
290777b3d9 Clean up and document handling of logcontexts in Keyring (#2452) 
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
 2017-09-18 18:31:01 +01:00
Erik Johnston
77c81ca6ea Merge pull request #2451 from matrix-org/erikj/add_state_to_timeline 
Don't filter out current state events from timeline
 2017-09-18 17:22:33 +01:00
Erik Johnston
2d1b7955ae Don't filter out current state events from timeline 2017-09-18 17:13:03 +01:00
David Baker
862c8da560 Merge pull request #2450 from matrix-org/dbkr/push_event_id_only 
Add support for event_id_only push format
 2017-09-18 16:41:29 +01:00
David Baker
436ee0a2ea Also include the room_id 
as really it's part of the event ID
 2017-09-18 15:58:38 +01:00
David Baker
b393f5db51 Use .get - it's much shorter 2017-09-18 15:50:26 +01:00
David Baker
a2562f9d74 Add support for event_id_only push format 
Param in the data dict of a pusher that tells an HTTP pusher to
send just the event_id of the event it's notifying about and the
notification counts. For clients that want to go & fetch the body
of the event themselves anyway.
 2017-09-18 15:39:39 +01:00
Erik Johnston
d6dadd95ac Correctly handle leaving room in /key/changes 2017-09-18 15:38:22 +01:00