Commit Graph

8763 Commits

Author SHA1 Message Date
Erik Johnston
3e6e94fe9f
Merge pull request #3872 from matrix-org/hawkowl/timeouts-2
timeouts 2: electric boogaloo
2018-09-14 16:58:44 +01:00
Amber Brown
bc9af88a2d fix 2018-09-15 00:26:00 +10:00
Erik Johnston
d0f6c1ce21 Remove spurious comment 2018-09-14 15:12:36 +01:00
Erik Johnston
9e2f9a7b57 Measure outbound requests 2018-09-14 15:11:26 +01:00
Erik Johnston
0a81038ea0 Add in flight real time metrics for Measure blocks 2018-09-14 15:08:37 +01:00
Neil Johnson
7de1989ea2 fix link for case that config.email_riot_base_url is set 2018-09-13 22:43:50 +01:00
Amber Brown
c971aa7b9d fix 2018-09-14 03:57:02 +10:00
Amber Brown
8f08d848f5 fix 2018-09-14 03:53:56 +10:00
Travis Ralston
f1a7264663
Fix minor typo in exception 2018-09-13 11:51:12 -06:00
Amber Brown
7c33ab76da redact better 2018-09-14 03:45:34 +10:00
Amber Brown
63755fa4c2 we do that higher up 2018-09-14 03:21:47 +10:00
Amber Brown
73884ebac5 Merge remote-tracking branch 'origin/develop' into hawkowl/timeouts-2 2018-09-14 03:11:25 +10:00
Amber Brown
7c27c4d51c
merge (#3576) 2018-09-14 03:11:11 +10:00
Amber Brown
1c3f4d9ca5 buffer? 2018-09-14 03:09:13 +10:00
Erik Johnston
6c0f8d9d50
Merge pull request #3856 from matrix-org/erikj/speed_up_purge
Make purge history slightly faster
2018-09-13 16:14:46 +01:00
Erik Johnston
ed5331a627 comment 2018-09-13 16:10:56 +01:00
Erik Johnston
89a76d1889 Fix handling of redacted events from federation
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.

This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
2018-09-13 15:44:12 +01:00
Amber Brown
bfa0b759e0
Attempt to figure out what's going on with timeouts (#3857) 2018-09-14 00:15:51 +10:00
Erik Johnston
9cbd0094f0 pep8 2018-09-13 15:15:35 +01:00
Erik Johnston
9dbe38ea7d Create indices after insertion 2018-09-13 15:05:52 +01:00
Erik Johnston
93139a1fb8 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_purge 2018-09-13 12:57:09 +01:00
Erik Johnston
c857f5ef9b Make purge history slightly faster
Don't pull out events that are outliers and won't be deleted, as nothing
should happen to them.
2018-09-13 12:48:10 +01:00
Amber Brown
b7d2fb5eb9
Remove some superfluous logging (#3855) 2018-09-13 19:59:32 +10:00
Neil Johnson
f30a303590
Merge pull request #3846 from matrix-org/neilj/expose-registered-users
expose number of real reserved users
2018-09-12 17:14:04 +01:00
Matthew Hodgson
2ac1abbc7e
show heroes if a room has a 'deleted' name/canonical_alias (#3851) 2018-09-12 17:11:05 +01:00
Erik Johnston
fa0d464fa4
Merge pull request #3853 from matrix-org/erikj/log_outbound_each_time
Log outbound requests when we retry
2018-09-12 16:55:40 +01:00
Matthew Hodgson
0403cf0783 argh pep8 2018-09-12 16:54:28 +01:00
Matthew Hodgson
0e200e366d correctly log gappy sync metrics 2018-09-12 16:47:20 +01:00
Matthew Hodgson
11bfc2af1c fix logline 2018-09-12 16:45:42 +01:00
Neil Johnson
8decd6233d improve naming 2018-09-12 16:22:15 +01:00
Erik Johnston
8c5b84441b Log outbound requests when we retry 2018-09-12 16:22:14 +01:00
Erik Johnston
54f8616d2c
Merge pull request #3841 from matrix-org/erikj/manhole_key_length
Change the manhole SSH key to have more bits
2018-09-12 14:33:39 +01:00
Amber Brown
7ca097f77e
Port federation/ to py3 (#3847) 2018-09-12 23:23:32 +10:00
Neil Johnson
0ddf486724 expose number of real reserved users 2018-09-12 11:58:52 +01:00
Amber Brown
546aee7e52
Merge pull request #3835 from krombel/fix_3821
fix VOIP crashes under Python 3
2018-09-12 20:44:18 +10:00
Amber Brown
33716c4aea
Merge pull request #3826 from matrix-org/rav/logging_for_keyring
add some logging for the keyring queue
2018-09-12 20:43:47 +10:00
Amber Brown
bc635026c5
Merge pull request #3824 from matrix-org/rav/fix_jwt_import
Fix jwt import check
2018-09-12 20:41:57 +10:00
Amber Brown
02aa41809b
Port rest/ to Python 3 (#3823) 2018-09-12 20:41:31 +10:00
Amber Brown
8fd93b5eea
Port crypto/ to Python 3 (#3822) 2018-09-12 20:16:31 +10:00
Erik Johnston
4084a774a8 Timeout reading body for outbound HTTP requests 2018-09-12 10:10:20 +01:00
Matthew Hodgson
b041115415
Speed up lazy loading (#3827)
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats  (#3840)
2018-09-12 00:50:39 +01:00
Erik Johnston
9e05c8d309 Change the manhole SSH key to have more bits
Newer versions of openssh client refuse to connect to the old key due to
its length.
2018-09-11 10:42:10 +01:00
Krombel
7bc22539ff fix VOIP crashes under Python 3 (#3821) 2018-09-10 14:30:08 +02:00
Amber Brown
9a5ea511b5
Merge pull request #3810 from matrix-org/erikj/send_tags_down_sync_on_join
Send existing room tags down sync on join
2018-09-07 23:28:42 +10:00
Richard van der Hoff
b8ad756bd0 Fix jwt import check
This handy code attempted to check that we could import jwt, but utterly failed
to check it was the right jwt.

Fixes https://github.com/matrix-org/synapse/issues/3793
2018-09-07 14:20:54 +01:00
Amber Brown
771d213ac5 Merge branch 'master' into develop 2018-09-07 21:45:38 +10:00
Amber Brown
6febd8e8f7 version 2018-09-07 21:40:57 +10:00
Richard van der Hoff
cd7ef43872 clearer logging when things fail, too 2018-09-06 23:56:47 +01:00
Richard van der Hoff
806964b5de add some logging for the keyring queue
why is it so damn slow?
2018-09-06 18:51:06 +01:00
Neil Johnson
c5440b2ca0
Merge pull request #3800 from matrix-org/neilj/remove-guests-from-mau-count
guest users should not be part of mau total
2018-09-06 17:45:55 +01:00
Neil Johnson
84a750e0c3 ensure guests never enter mau list 2018-09-06 17:22:53 +01:00
Erik Johnston
f60c9e2a01 Don't send empty tags list down sync 2018-09-06 17:01:41 +01:00
Erik Johnston
7baf66ef5d Send existing room tags down sync on join
When a user joined a room any existing tags were not sent down the sync
stream. Ordinarily this isn't a problem because the user needs to be in
the room to have set tags in it, however synapse will sometimes add tags
for a user to a room, e.g. for server notices, which need to come down
sync.
2018-09-06 16:46:51 +01:00
Amber Brown
654324eded
Merge pull request #3805 from matrix-org/erikj/limit_transaction_pdus_edus
Limit the number of PDUs/EDUs per fedreation transaction
2018-09-07 01:33:31 +10:00
Amber Brown
70fd75cd1d
Merge pull request #3788 from matrix-org/erikj/remove_conn_id
Remove conn_id from repl prometheus metrics
2018-09-07 00:48:19 +10:00
Amber Brown
ee7c8bd2b5
Merge pull request #3795 from matrix-org/erikj/faster_sync_state
User iter* during sync state calculations
2018-09-07 00:24:43 +10:00
Erik Johnston
6707a3212c Limit the number of PDUs/EDUs per fedreation transaction 2018-09-06 15:23:55 +01:00
Amber Brown
135f3b4390
Merge pull request #3804 from matrix-org/rav/fix_openssl_dep
bump dep on pyopenssl to 16.x
2018-09-07 00:23:39 +10:00
Amber Brown
2608ebc04c
Port handlers/ to Python 3 (#3803) 2018-09-07 00:22:23 +10:00
Erik Johnston
417e7077aa Bump version and changelog 2018-09-06 14:08:55 +01:00
Erik Johnston
d64b24dfe6 Synapse 0.33.3.1 (2018-09-06)
=============================
 
 SECURITY FIXES
 --------------
 
 - Fix an issue where event signatures were not always correctly validated ([\#3796](https://github.com/matrix-org/synapse/issues/3796))
 - Fix an issue where server_acls could be circumvented for incoming events ([\#3796](https://github.com/matrix-org/synapse/issues/3796))
 
 Internal Changes
 ----------------
 
 - Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802))
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQEcBAABCAAGBQJbkPLrAAoJEIofk9V1tejV7K4IAItYIX98DKN9x3FNs9Hd69Pw
 mZtqmuJ12YAyiVZoR7IJ5GfyctHCYUUXcqmNR1O+2/IRvezFnU6ZTcPW3OfNfnuD
 vlnMiK53F21T96ul3Wu47Z2wyO+WkoKeXdvlqt3Wa8HpnPbU6y5CSac2vK57ppTU
 DcMZKXaDae6vA7bsjryYgTRRplH6eFWZEexjbuZudbDvqkySi2zGUTs6SUesMC5B
 FRPgI6p6sCeQgGMJY+d1i+ZdzWkZhH5OLH2icf2MI1hnhSnRSJ/scSGm5OYDCwYy
 03hOy0ZVMr8SB5/j0TjSaFXonKsmPKqxnU2g7crJmvFP4BfJrEbym0y9YhYXnVQ=
 =8PDU
 -----END PGP SIGNATURE-----

Merge tag 'v0.33.3.1' into release-v0.33.4

Synapse 0.33.3.1 (2018-09-06)
=============================

SECURITY FIXES
--------------

- Fix an issue where event signatures were not always correctly validated ([\#3796](https://github.com/matrix-org/synapse/issues/3796))
- Fix an issue where server_acls could be circumvented for incoming events ([\#3796](https://github.com/matrix-org/synapse/issues/3796))

Internal Changes
----------------

- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802))
2018-09-06 14:08:33 +01:00
Richard van der Hoff
4f8baab0c4 Merge branch 'master' into develop 2018-09-06 13:05:22 +01:00
Richard van der Hoff
625542878d bump dep on pyopenssl to 16.x 2018-09-06 12:53:15 +01:00
Richard van der Hoff
80189ed27c prepare v0.33.3.1 2018-09-06 10:26:23 +01:00
Neil Johnson
61b05727fa guest users should not be part of mau total 2018-09-05 22:30:36 +01:00
Richard van der Hoff
dfba1d843d
Merge pull request #3790 from matrix-org/rav/respect_event_format_in_filter
Implement 'event_format' filter param in /sync
2018-09-05 16:24:14 +01:00
Erik Johnston
7419764351 User iter* during sync state calculations 2018-09-05 16:19:50 +01:00
Amber Brown
2d2828dcbc
Port http/ to Python 3 (#3771) 2018-09-06 00:10:47 +10:00
Richard van der Hoff
c127c8d042 Fix origin handling for pushed transactions
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
2018-09-05 13:08:07 +01:00
Richard van der Hoff
804dd41e18 Check that signatures on events are valid
We should check that both the sender's server, and the server which created the
event_id (which may be different from whatever the remote server has told us
the origin is), have signed the event.
2018-09-05 13:08:07 +01:00
Richard van der Hoff
87c18d12ee Implement 'event_format' filter param in /sync
This has been specced and part-implemented; let's implement it for /sync (but
no other endpoints yet :/).
2018-09-04 15:20:09 +01:00
Neil Johnson
a6cf7d9d9a
Merge pull request #3789 from matrix-org/neilj/improve_threepid_error_strings
improve human readable error messages
2018-09-04 13:16:00 +00:00
Amber Brown
7e9ced4178 version and towncrier 2018-09-04 21:12:04 +10:00
Neil Johnson
bae37cd811 improve human readable error message 2018-09-04 12:07:00 +01:00
Neil Johnson
c42f7fd7b9 improve human readable error messages 2018-09-04 12:03:17 +01:00
Erik Johnston
3e242dc149 Remove conn_id 2018-09-04 11:45:52 +01:00
Erik Johnston
b13836da7f Remove conn_id from repl prometheus metrics
`conn_id` gets set to a random string, and so we end up filling up
prometheus with tonnes of data series, which is bad.
2018-09-03 17:22:49 +01:00
Erik Johnston
567363e497
Merge pull request #3737 from matrix-org/erikj/remove_redundant_state_func
Remove unnecessary resolve_events_with_state_map
2018-09-03 16:19:41 +01:00
Neil Johnson
301cb60d0b assert rather than warn 2018-08-31 17:29:35 +01:00
Neil Johnson
0b01281e77 move threepid checker to config, add missing yields 2018-08-31 17:11:11 +01:00
Neil Johnson
e8e540630e fix reference to is_threepid_reserved 2018-08-31 16:09:15 +01:00
Neil Johnson
09f3cf1a7e ensure post registration auth checks do not fail erroneously 2018-08-31 15:42:51 +01:00
Neil Johnson
ea068d6f3c fix bug where preserved threepid user comes to sign up and server is mau blocked 2018-08-31 10:49:14 +01:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 (#3725) 2018-08-31 00:19:58 +10:00
Richard van der Hoff
475253a88e
Merge pull request #3764 from matrix-org/rav/close_db_conn_after_init
Make sure that we close db connections opened during init
2018-08-30 10:47:27 +01:00
Krombel
79a8a347a6 fix #3445
itervalues(d) calls d.itervalues() [PY2] and d.values() [PY3]
but SortedDict only implements d.values()
2018-08-29 16:28:25 +02:00
Matthew Hodgson
b1580f50fe
don't return non-LL-member state in incremental sync state blocks (#3760)
don't return non-LL-member state in incremental sync state blocks
2018-08-28 23:25:58 +01:00
Richard van der Hoff
32eb1dedd2 use abc.abstractproperty
This gives clearer messages when someone gets it wrong
2018-08-28 17:10:43 +01:00
Richard van der Hoff
0b07f02e19 Make sure that we close db connections opened during init
We should explicitly close any db connections we open, because failing to do so
can block other transactions as per
https://github.com/matrix-org/synapse/issues/3682.

Let's also try to factor out some of the boilerplate by having server classes
define their datastore class rather than duplicating the whole of `setup`.
2018-08-28 13:39:49 +01:00
Erik Johnston
9db2476991 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_contact 2018-08-24 17:00:37 +01:00
Erik Johnston
05077e06fa Change admin_uri to admin_contact in config and errors 2018-08-24 16:51:27 +01:00
Erik Johnston
01a5a8b9e3 Fix checking if service notice room is already tagged
This manifested in synapse repeatedly setting the tag for the room
2018-08-24 16:22:37 +01:00
Erik Johnston
cdd24449ee Ensure we wake up /sync when we add tag to notice room 2018-08-24 14:50:03 +01:00
Erik Johnston
14d49c51db Make content of tag an empty object rather than null 2018-08-24 14:44:16 +01:00
Erik Johnston
84b4e76fed
Merge pull request #3754 from matrix-org/erikj/fix_whitelist
Allow federation_domain_whitelist to be emtpy list
2018-08-24 12:23:39 +01:00
Erik Johnston
1d67b13674 Fix bug when federation_domain_whitelist is an emtpy list
Outbound federation were incorrectly allowed when the config option was
set to an empty list
2018-08-24 12:13:12 +01:00
Erik Johnston
92d50e3c2a
Merge pull request #3753 from matrix-org/erikj/fix_no_server_noticse
Fix bug where we broke sync when using limit_usage_by_mau
2018-08-24 11:56:08 +01:00
Richard van der Hoff
e94cdbaecf
Merge pull request #3751 from matrix-org/rav/twisted_17
Pin to twisted 17.1 or later
2018-08-24 11:55:52 +01:00
Erik Johnston
7f147d623b Fix bug where we broke sync when using limit_usage_by_mau
We assumed that we always had service notices configured, but that is
not always true
2018-08-24 11:33:50 +01:00
Richard van der Hoff
f584d6108f Pin to twisted 17.1 or later
Fixes https://github.com/matrix-org/synapse/issues/3741.
2018-08-24 10:02:31 +01:00