Commit Graph

2100 Commits

Author SHA1 Message Date
Richard van der Hoff
95ab2eb4a1
Fix notes about well-known and acme (#5357)
fixes #4951
2019-06-05 15:12:33 +01:00
Richard van der Hoff
e2dfb922e1 Validate federation server TLS certificates by default. 2019-06-05 14:17:50 +01:00
Erik Johnston
d53faa40e9
Merge pull request #5317 from matrix-org/erikj/make_do_auth_non_essential
Fix handling of failures when calling /event_auth.
2019-06-05 14:01:32 +01:00
Brendan Abolivier
a4cf2c1184
Rewrite changelog 2019-06-05 14:00:18 +01:00
Neil Johnson
4650526b5e
Neilj/changelog clean up (#5356)
* group together key validity refactors
2019-06-05 13:47:03 +01:00
Brendan Abolivier
40596aec0e Merge branch 'develop' into m-heroes-empty-room-name 2019-06-05 13:38:01 +01:00
Neil Johnson
26713515de
Neilj/mau tracking config explainer (#5284)
Improve documentation of monthly active user blocking and mau_trial_days
2019-06-05 13:16:23 +01:00
Brendan Abolivier
804f26a9ff
Properly format the changelog 2019-06-05 12:03:01 +01:00
Brendan Abolivier
a412be2bc7
Changelog 2019-06-05 11:53:50 +01:00
Richard van der Hoff
d18e4ea0d4 Implement room v5 which enforces signing key validity
Implements [MSC2077](https://github.com/matrix-org/matrix-doc/pull/2077) and
fixes #5247 and #4364.
2019-06-05 11:00:47 +01:00
Richard van der Hoff
cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
Richard van der Hoff
14f13babb0
Add a test room version where we enforce key validity (#5348) 2019-06-05 10:38:25 +01:00
Richard van der Hoff
2615c6bd9e
Clean up debug logging (#5347)
Remove some spurious stuff, clarify some other stuff
2019-06-05 10:35:40 +01:00
Richard van der Hoff
016af01598
Rename VerifyKeyRequest.deferred field (#5343)
it's a bit confusing
2019-06-05 10:35:13 +01:00
Richard van der Hoff
aa530e6800
Call RetryLimiter correctly (#5340)
Fixes a regression introduced in #5335.
2019-06-04 22:02:53 +01:00
Richard van der Hoff
dae224a73f
Fix failure to fetch batches of PDUs (#5342)
FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A
failure to fetch one should not result in a failure of the whole batch. Add the
missing `continue`.
2019-06-04 18:05:06 +01:00
Richard van der Hoff
b4189b112f
Rename get_events->get_events_from_store_or_dest (#5344)
We have too many things called get_event, and it's hard to figure out what we
mean. Also remove some unused params from the signature, and add some logging.
2019-06-04 18:01:09 +01:00
Brendan Abolivier
f6dd12d1e2
Merge pull request #5341 from matrix-org/babolivier/email_config
Make account validity renewal emails work when email notifs are disabled
2019-06-04 14:49:06 +01:00
Erik Johnston
d1d38081a7
Merge pull request #5324 from matrix-org/erikj/ignore_null
Ignore room state with null bytes in for room stats
2019-06-04 14:20:08 +01:00
Brendan Abolivier
ac3cc32367
Make account validity renewal emails work when email notifs are disabled 2019-06-04 13:47:44 +01:00
Richard van der Hoff
df9c100542
Avoid rapidly backing-off a server if we ignore the retry interval (#5335) 2019-06-04 11:53:29 +01:00
Richard van der Hoff
4d08b8f30c
Don't do long retries when calling the key notary server. (#5334)
It takes at least 20 minutes to work through the long_retries schedule (11
attempts, each with a 60 second timeout, and 60 seconds between each request),
so if the notary server isn't returning within the timeout, we'll just end up
blocking whatever request is happening for 20 minutes.

Ain't nobody got time for that.
2019-06-04 11:53:07 +01:00
Richard van der Hoff
cb683d3e3c
Merge pull request #5333 from matrix-org/rav/server_keys/09_improve_notary_server
Fixes for the key-notary server
2019-06-04 11:48:18 +01:00
Richard van der Hoff
5bdb189f86
Improve docstrings on MatrixFederationClient. (#5332) 2019-06-04 11:14:16 +01:00
Brendan Abolivier
aeb2263320 Merge branch 'develop' into babolivier/port_db_account_validity 2019-06-04 09:13:42 +01:00
Amber Brown
b2b90b7d34
Hawkowl/fix missing auth (#5328) 2019-06-04 15:54:27 +10:00
Richard van der Hoff
a3f2d000e0 changelog 2019-06-04 00:16:56 +01:00
Richard van der Hoff
dce6e9e0c1 Avoid rapidly backing-off a server if we ignore the retry interval 2019-06-03 23:58:42 +01:00
Richard van der Hoff
fec2dcb1a5
Enforce validity period on server_keys for fed requests. (#5321)
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.

We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
Erik Johnston
0d67a8cd9d Newsfile 2019-06-03 17:17:57 +01:00
Erik Johnston
fe2294ec8d Revert "Newsfile"
This reverts commit 4bd67db100.
2019-06-03 17:17:35 +01:00
Brendan Abolivier
deca87ddf2
Changelog 2019-06-03 17:11:28 +01:00
Erik Johnston
4bd67db100 Newsfile 2019-06-03 17:08:33 +01:00
Amber Brown
6f9f08005c Merge remote-tracking branch 'origin/develop' into hawkowl/full-schema-v1 2019-06-04 01:16:01 +10:00
Amber Brown
ed6138461b more fix 2019-06-03 22:29:19 +10:00
Richard van der Hoff
862b2f9ad5
Merge pull request #5307 from matrix-org/rav/server_keys/07-fix-notary-cache-poison
Stop overwriting server keys with other keys
2019-06-03 13:19:20 +01:00
Amber Brown
2889b05554
Unify v1 and v2 REST client APIs (#5226) 2019-06-03 21:28:59 +10:00
Erik Johnston
fde37e4e98 Newsfile 2019-06-03 10:22:03 +01:00
Ike Johnson
0df5b41759
Create 5313.misc 2019-06-02 23:23:58 +08:00
Richard van der Hoff
d828d1dc57
Merge pull request #5309 from matrix-org/rav/limit_displayname_length
Limit displaynames and avatar URLs
2019-06-01 11:34:50 +01:00
Richard van der Hoff
d16c6375fe Limit displaynames and avatar URLs
These end up in join events everywhere, so let's limit them.

Fixes #5079
2019-06-01 10:44:36 +01:00
Richard van der Hoff
37b165620d
Merge pull request #5299 from matrix-org/rav/server_keys/05-rewrite-gsvk-again
Rewrite get_server_verify_keys, again.
2019-05-31 17:07:31 +01:00
Richard van der Hoff
3600f5568b Stop overwriting server keys with other keys
Fix a bug where we would discard a key result which the origin server is no
longer returning. Fixes #5305.
2019-05-31 15:58:35 +01:00
Erik Johnston
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
Richard van der Hoff
c605da97bf Merge remote-tracking branch 'origin/develop' into rav/server_keys/05-rewrite-gsvk-again 2019-05-31 11:38:13 +01:00
Richard van der Hoff
fe79b5e521
Merge pull request #5300 from matrix-org/rav/server_keys/06-fix-serverkeys-handling
Remove some pointless exception handling
2019-05-31 11:35:29 +01:00
Richard van der Hoff
2ae3cc287e
Merge pull request #5296 from matrix-org/rav/server_keys/04-use-attrs-for_verify-request
use attr.s for VerifyKeyRequest
2019-05-31 11:34:09 +01:00
Erik Johnston
e9981d58ca
Merge pull request #5293 from Kagamihime/messages-federation-format
Fix ignored filter field in `/messages` endpoint
2019-05-31 10:52:59 +01:00
Erik Johnston
31d44ec4bd
Merge pull request #5294 from matrix-org/erikj/speed_up_room_stats
Speed up room stats background update
2019-05-31 10:48:51 +01:00
Erik Johnston
39bbf6a4a5 Newsfile 2019-05-31 10:26:59 +01:00
Travis Ralston
3e1af5109c Clarify that the admin change password endpoint logs them out (#5303) 2019-05-31 09:45:46 +01:00
Richard van der Hoff
8ea2f756a9 Remove some pointless exception handling
The verify_request deferred already returns a suitable SynapseError, so I don't
really know what we expect to achieve by doing more wrapping, other than log
spam.

Fixes #4278.
2019-05-30 18:29:56 +01:00
Richard van der Hoff
a82c96b87f Rewrite get_server_verify_keys, again.
Attempt to simplify the logic in get_server_verify_keys by splitting it into
two methods.
2019-05-30 18:20:40 +01:00
Richard van der Hoff
099829d5a9 use attr.s for VerifyKeyRequest
because namedtuple is awful
2019-05-30 17:39:28 +01:00
Erik Johnston
c831748f4d 0.99.5.2 2019-05-30 16:29:47 +01:00
Erik Johnston
f5c7f90d72 Newsfile 2019-05-30 16:18:40 +01:00
Erik Johnston
9f5268388a Newsfile 2019-05-30 16:15:37 +01:00
Erik Johnston
df9d900544 Correctly filter out extremities with soft failed prevs (#5274)
When we receive a soft failed event we, correctly, *do not* update the
forward extremity table with the event. However, if we later receive an
event that references the soft failed event we then need to remove the
soft failed events prev events from the forward extremities table,
otherwise we just build up forward extremities.

Fixes #5269
2019-05-30 16:12:50 +01:00
Eisha Chen-yen-su
0b6bc36402 Add changelog 2019-05-30 17:07:21 +02:00
Erik Johnston
06675db684 Newsfile 2019-05-30 15:05:26 +01:00
Erik Johnston
e9e5d3392d
Merge pull request #5278 from matrix-org/erikj/cleanup_bad_extremities
Add DB bg update to cleanup extremities.
2019-05-30 14:28:26 +01:00
Erik Johnston
45f5d8f3fd
Merge pull request #5256 from aaronraimist/logout-correct-error
Show correct error when logging out and access token is missing
2019-05-30 13:33:44 +01:00
Richard van der Hoff
8d92329214
Remove spurious debug from MatrixFederationHttpClient.get_json (#5287)
This is just unhelpful spam
2019-05-29 19:31:52 +01:00
Richard van der Hoff
3dcf2feba8
Improve logging for logcontext leaks. (#5288) 2019-05-29 19:27:50 +01:00
Erik Johnston
8541db741a
Merge pull request #5283 from aaronraimist/captcha-docs
Specify the type of reCAPTCHA key to use (#5013)
2019-05-29 19:02:27 +01:00
Amber Brown
46c8f7a517
Implement the SHHS complexity API (#5216) 2019-05-30 01:47:16 +10:00
Erik Johnston
67e0631f8f Newsfile 2019-05-29 15:59:10 +01:00
Amber Brown
532b825ed9
Serve CAS login over r0 (#5286) 2019-05-30 00:55:18 +10:00
Aaron Raimist
30858ff461 Fix error when downloading thumbnail with width/height param missing (#5258)
Fix error when downloading thumbnail with width/height param missing

Fixes #2748

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-29 14:27:41 +01:00
Erik Johnston
58c8ed5b0d
Correctly filter out extremities with soft failed prevs (#5274)
When we receive a soft failed event we, correctly, *do not* update the
forward extremity table with the event. However, if we later receive an
event that references the soft failed event we then need to remove the
soft failed events prev events from the forward extremities table,
otherwise we just build up forward extremities.

Fixes #5269
2019-05-29 11:56:24 +01:00
Richard van der Hoff
f76d407ef3
Fix dropped logcontexts during high outbound traffic. (#5277)
Fixes #5271.
2019-05-29 09:17:33 +01:00
Amber Brown
ecaa299cab
Rename 5282.misc to 5282.doc 2019-05-29 16:32:30 +10:00
Aaron Raimist
2ec2809460
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-28 22:05:53 -05:00
Aaron Raimist
878b00c395
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-28 20:58:22 -05:00
Richard van der Hoff
540f40f0cd
Merge pull request #5251 from matrix-org/rav/server_keys/01-check_sig
Ensure that server_keys fetched via a notary server are correctly signed.
2019-05-28 21:32:17 +01:00
Richard van der Hoff
5726378ece
Fix "db txn 'update_presence' from sentinel context" log messages (#5275)
Fixes #4414.
2019-05-28 21:20:11 +01:00
Brendan Abolivier
4aba561c65
Config and changelog 2019-05-28 16:55:10 +01:00
Brendan Abolivier
a97d4e218a
Merge pull request #5268 from matrix-org/babolivier/account_validity_fix_schema
Fix schema update for account validity
2019-05-28 10:30:07 +01:00
Brendan Abolivier
ddd30f44a0
Changelog 2019-05-28 10:14:21 +01:00
Erik Johnston
d0bba35197
Merge pull request #5260 from matrix-org/travis/fix-room-bg-task
Fix logging for room stats background update
2019-05-25 19:59:42 +01:00
Travis Ralston
4ccdbfcdb1 Changelog 2019-05-25 12:21:21 -06:00
Aaron Raimist
0b4f4cb0b4
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-25 11:39:18 -05:00
Erik Johnston
338dca58c0
Merge pull request #5257 from aaronraimist/fix-error-code-publicrooms
Fix error code for invalid parameter
2019-05-25 14:09:27 +01:00
Aaron Raimist
6dac0e738c
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-24 17:15:30 -05:00
Richard van der Hoff
fa1b293da2
Simplification to Keyring.wait_for_previous_lookups. (#5250)
The list of server names was redundant, since it was equivalent to the keys on
the server_to_deferred map. This reduces the number of large lists being passed
around, and has the benefit of deduplicating the entries in `wait_on`.
2019-05-24 22:17:18 +01:00
Richard van der Hoff
cbcfd642a0 changelog 2019-05-24 15:47:30 +01:00
Tulir Asokan
dd64b9dbdd Fix appservice timestamp massaging (#5233)
Signed-off-by: Tulir Asokan <tulir@maunium.net>
2019-05-24 14:44:04 +01:00
Richard van der Hoff
dba9152d15
Add missing blank line in config (#5249) 2019-05-24 14:12:38 +01:00
Erik Johnston
d16f5574b6
Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events
Don't bundle aggregations with events in /sync or /events or state queries
2019-05-24 10:36:31 +01:00
Erik Johnston
8c41c04ee4
Merge pull request #5244 from matrix-org/rav/server_keys/00-factor-out-fetchers
Factor out KeyFetchers from KeyRing
2019-05-23 16:28:51 +01:00
Andrew Morgan
6368150a74
Add config option for setting homeserver's default room version (#5223)
Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present.

That hardcoded value is now located in the server.py config file.
2019-05-23 15:00:20 +01:00
Richard van der Hoff
895b79ac2e Factor out KeyFetchers from KeyRing
Rather than have three methods which have to have the same interface,
factor out a separate interface which is provided by three implementations.

I find it easier to grok the code this way.
2019-05-23 13:46:47 +01:00
Richard van der Hoff
b75537beaf Store key validity time in the storage layer
This is a first step to checking that the key is valid at the required moment.

The idea here is that, rather than passing VerifyKey objects in and out of the
storage layer, we instead pass FetchKeyResult objects, which simply wrap the
VerifyKey and add a valid_until_ts field.
2019-05-23 11:52:22 +01:00
Richard van der Hoff
84660d91b2
Simplify process_v2_response (#5236)
* Pass time_added_ms into process_v2_response

* Simplify process_v2_response

We can merge old_verify_keys into verify_keys, and reduce the number of dicts
flying around.
2019-05-23 11:51:39 +01:00
Richard van der Hoff
cc187f9337
Remove unused VerifyKey.expired and .time_added fields (#5235)
These were never used, and poking arbitary data into objects from other
packages seems confusing at best.
2019-05-23 11:46:05 +01:00
Richard van der Hoff
2e052110ee
Rewrite store_server_verify_key to store several keys at once (#5234)
Storing server keys hammered the database a bit. This replaces the
implementation which stored a single key, with one which can do many updates at
once.
2019-05-23 11:45:39 +01:00
Richard van der Hoff
85d1e03b9d
Simplifications and comments in do_auth (#5227)
I was staring at this function trying to figure out wtf it was actually
doing. This is (hopefully) a non-functional refactor which makes it a bit
clearer.
2019-05-23 11:17:42 +01:00
Richard van der Hoff
1a94de60e8
Run black on synapse.crypto.keyring (#5232) 2019-05-22 18:39:33 +01:00
Marcus Hoffmann
62388a1e44 remove urllib3 pin (#5230)
requests 2.22.0 as been released supporting urllib3 1.25.2

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
2019-05-22 16:48:12 +01:00
Neil Johnson
66b75e2d81
Neilj/ensure get profileinfo available in client reader slaved store (#5213)
* expose SlavedProfileStore to ClientReaderSlavedStore
2019-05-22 13:55:32 +01:00
Richard van der Hoff
b898a5600a Merge branch 'master' into develop 2019-05-22 11:38:27 +01:00
Richard van der Hoff
e26e6b3230 update changelog 2019-05-21 17:37:19 +01:00
Amber Brown
4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
Richard van der Hoff
f3ff64e000 Merge commit 'f4c80d70f' into release-v0.99.5 2019-05-21 17:35:31 +01:00
Erik Johnston
f4c80d70f8
Merge pull request #5203 from matrix-org/erikj/aggregate_by_sender
Only count aggregations from distinct senders
2019-05-21 17:10:48 +01:00
Erik Johnston
9526aa96a6
Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactions
Block attempts to annotate the same event twice
2019-05-21 17:08:14 +01:00
Erik Johnston
9259cd4bee Newsfile 2019-05-21 17:06:21 +01:00
Richard van der Hoff
959550b645 0.99.5rc1 2019-05-21 16:51:49 +01:00
Richard van der Hoff
17f6804837
Introduce room v4 which updates event ID format. (#5217)
Implements https://github.com/matrix-org/matrix-doc/pull/2002.
2019-05-21 16:22:54 +01:00
Richard van der Hoff
c4aef549ad
Exclude soft-failed events from fwd-extremity candidates. (#5146)
When considering the candidates to be forward-extremities, we must exclude soft
failures.

Hopefully fixes #5090.
2019-05-21 16:10:54 +01:00
Richard van der Hoff
bab3eddac4
Pin eliot to <1.8 on python 3.5.2 (#5218)
* Pin eliot to <1.8 on python 3.5.2

Fixes https://github.com/matrix-org/synapse/issues/5199

* Add support for 'markers' to python_dependencies

* tell xargs not to strip quotes
2019-05-21 15:58:01 +01:00
Erik Johnston
c448f35de2 Newsfile 2019-05-21 15:35:13 +01:00
Brendan Abolivier
6a5a70edf0
Merge pull request #5204 from matrix-org/babolivier/account_validity_expiration_date
Add startup background job for account validity
2019-05-21 14:55:15 +01:00
Erik Johnston
ef13dc4846 Newsfile 2019-05-21 13:59:09 +01:00
Richard van der Hoff
04d53794d6
Fix error handling for rooms whose versions are unknown. (#5219)
If we remove support for a particular room version, we should behave more
gracefully. This should make client requests fail with a 400 rather than a 500,
and will ignore individiual PDUs in a federation transaction, rather than the
whole transaction.
2019-05-21 13:47:25 +01:00
Erik Johnston
0620dd49db Newsfile 2019-05-20 17:40:24 +01:00
Richard van der Hoff
24b93b9c76 Revert "expose SlavedProfileStore to ClientReaderSlavedStore (#5200)"
This reverts commit ce5bcefc60.

This caused:

```
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/synapse/src/synapse/app/client_reader.py", line 32, in <module>
    from synapse.replication.slave.storage import SlavedProfileStore
ImportError: cannot import name 'SlavedProfileStore' from 'synapse.replication.slave.storage' (/home/synapse/src/synapse/replication/slave/storage/__init__.py)
error starting synapse.app.client_reader('/home/synapse/config/workers/client_reader.yaml') (exit code: 1); see above for logs
```
2019-05-20 16:21:34 +01:00
Richard van der Hoff
5206648a4a
Add a test room version which updates event ID format (#5210)
Implements MSC1884
2019-05-20 15:54:42 +01:00
Erik Johnston
d642178654 Newsfile 2019-05-20 14:32:16 +01:00
Erik Johnston
57ba3451b6
Merge pull request #5209 from matrix-org/erikj/reactions_base
Land basic reaction and edit support.
2019-05-20 14:06:40 +01:00
Erik Johnston
06671057b6 Newsfile 2019-05-20 12:39:07 +01:00
ReidAnderson
3787133c9e Limit UserIds to a length that fits in a state key (#5198) 2019-05-20 11:20:08 +01:00
Brendan Abolivier
99c4ec1eef
Changelog 2019-05-17 19:38:41 +01:00
bytepoets-blo
291e1eea5e fix mapping of return values for get_or_register_3pid_guest (#5177)
* fix mapping of return values for get_or_register_3pid_guest
2019-05-17 17:27:14 +01:00
Erik Johnston
85ece3df46
Merge pull request #5191 from matrix-org/erikj/refactor_pagination_bounds
Make generating SQL bounds for pagination generic
2019-05-17 17:24:36 +01:00
Neil Johnson
ce5bcefc60
expose SlavedProfileStore to ClientReaderSlavedStore (#5200)
* expose SlavedProfileStore to ClientReaderSlavedStore
2019-05-17 13:27:19 +01:00
Richard van der Hoff
da5ef0bb42 Merge remote-tracking branch 'origin/master' into develop 2019-05-17 12:39:48 +01:00
Richard van der Hoff
7ce1f97a13
Stop telling people to install the optional dependencies. (#5197)
* Stop telling people to install the optional dependencies.

They're optional.

Also update the postgres docs a bit for clarity(?)
2019-05-17 12:38:03 +01:00
Brendan Abolivier
fdeac1e984
Merge pull request #5196 from matrix-org/babolivier/per_room_profiles
Add an option to disable per-room profiles
2019-05-17 12:10:49 +01:00
PauRE
f89f688a55 Fix image orientation when generating thumbnail (#5039) 2019-05-16 19:04:26 +01:00
David Baker
07cff7b121
Merge pull request #5174 from matrix-org/dbkr/add_dummy_flow_to_recaptcha_only
Re-order registration stages to do msisdn & email auth last
2019-05-16 17:27:39 +01:00
Brendan Abolivier
a5fe16c5a7
Changelog + sample config 2019-05-16 15:11:37 +01:00
Matthew Hodgson
4a6d5de98c Make /sync attempt to return device updates for both joined and invited users (#3484) 2019-05-16 13:23:43 +01:00
David Baker
fafb936de5
Merge pull request #5187 from matrix-org/dbkr/only_check_threepid_not_in_use_if_actually_registering
Only check 3pids not in use when registering
2019-05-16 10:58:09 +01:00
David Baker
cd0faba7cd
Make newsfile clearer 2019-05-15 20:53:48 +01:00
Amber Brown
f1e5b41388
Make all the rate limiting options more consistent (#5181) 2019-05-15 12:06:04 -05:00
Richard van der Hoff
5f027a315f
Drop support for v2_alpha API prefix (#5190) 2019-05-15 17:37:46 +01:00
Erik Johnston
5fb72e6888 Newsfile 2019-05-15 13:36:51 +01:00
Erik Johnston
7155162844 Newsfile 2019-05-15 11:33:22 +01:00
Erik Johnston
0aba6c8251
Merge pull request #5183 from matrix-org/erikj/async_serialize_event
Allow client event serialization to be async
2019-05-15 10:36:30 +01:00
Erik Johnston
d94544051b
Merge pull request #5184 from matrix-org/erikj/expose_get_events_as_array
Expose DataStore._get_events as get_events_as_list
2019-05-15 10:17:38 +01:00
David Baker
efefb5bda2 Have I got newsfile for you 2019-05-14 19:18:42 +01:00
Erik Johnston
db3046f565 Newsfile 2019-05-14 14:39:27 +01:00
Erik Johnston
53788a447f Newsfile 2019-05-14 13:41:36 +01:00
Erik Johnston
a80e6b53f9 Newsfile 2019-05-14 13:12:23 +01:00
Amber Brown
df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171) 2019-05-13 15:01:14 -05:00
Andrew Morgan
5a4b328f52 Add ability to blacklist ip ranges for federation traffic (#5043) 2019-05-13 19:05:06 +01:00
Brendan Abolivier
2e1129b5f7 0.99.4rc1 2019-05-13 16:11:21 +01:00
David Baker
516a5fb64b Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only 2019-05-13 15:54:25 +01:00
David Baker
9e99143c47 Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only 2019-05-13 15:37:03 +01:00
Brendan Abolivier
2725cd2290 Fix changelog 2019-05-13 15:32:07 +01:00
Brendan Abolivier
1a536699fd Changelog 2019-05-13 15:21:23 +01:00
Andrew Morgan
2f48c4e1ae
URL preview blacklisting fixes (#5155)
Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.
2019-05-10 10:32:44 -07:00
David Baker
c9f811c5d4 Update changelog 2019-05-10 14:01:19 +01:00
David Baker
7a3eb8657d Thanks, automated grammar pedantry. 2019-05-10 11:18:35 +01:00
David Baker
a18f93279e Add changelog entry 2019-05-10 11:11:59 +01:00
Christoph Müller
ee90c06e38 Set syslog identifiers in systemd units (#5023) 2019-05-10 09:09:25 +01:00
Amber Brown
b36c82576e
Run Black on the tests again (#5170) 2019-05-10 00:12:11 -05:00
Quentin Dufour
11ea16777f Limit the number of EDUs in transactions to 100 as expected by receiver (#5138)
Fixes #3951.
2019-05-09 11:01:41 +01:00
Brendan Abolivier
d216a36b37
Fix bogus imports in tests (#5154) 2019-05-08 21:57:03 +01:00
Matthew Hodgson
c0e0740bef add options to require an access_token to GET /profile and /publicRooms on CS API (#5083)
This commit adds two config options:

* `restrict_public_rooms_to_local_users`

Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.

* `require_auth_for_profile_requests`

When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.

MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.

Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
Erik Johnston
c8c069db92
Merge pull request #5037 from matrix-org/erikj/limit_inflight_dns
Limit in flight DNS requests
2019-05-08 17:11:03 +01:00
Brendan Abolivier
1473058b5e
Do checks on aliases for incoming m.room.aliases events (#5128)
Follow-up to #5124

Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
2019-05-08 17:01:30 +01:00
Erik Johnston
de655e669a
Merge pull request #5104 from matrix-org/erikj/ratelimit_3pid_invite
Ratelimit 3pid invites
2019-05-07 10:12:49 +01:00
Richard van der Hoff
59e2d2694d
Remove the requirement to authenticate for /admin/server_version. (#5122)
This endpoint isn't much use for its intended purpose if you first need to get
yourself an admin's auth token.

I've restricted it to the `/_synapse/admin` path to make it a bit easier to
lock down for those concerned about exposing this information. I don't imagine
anyone is using it in anger currently.
2019-05-07 09:29:30 +01:00
Travis Ralston
3fdff14207 Fix spelling in server notices admin API docs (#5142) 2019-05-06 22:15:02 +01:00
Richard van der Hoff
12f9d51e82
Add admin api for sending server_notices (#5121) 2019-05-02 11:59:16 +01:00
Brendan Abolivier
c193b39134
Merge pull request #5124 from matrix-org/babolivier/aliases
Add some limitations to alias creation
2019-05-02 11:22:40 +01:00
Brendan Abolivier
84196cb231 Add some limitations to alias creation 2019-05-02 11:05:11 +01:00
Richard van der Hoff
0836cbb9f5
Factor out an "assert_requester_is_admin" function (#5120)
Rather than copying-and-pasting the same four lines hundreds of times
2019-05-02 10:45:52 +01:00
Richard van der Hoff
03ad6bd483 changelog 2019-05-01 15:44:30 +01:00
Brendan Abolivier
c1799b0f85
Merge pull request #5116 from matrix-org/babolivier/account_expiration
Fix path in account validity admin route's doc
2019-05-01 11:59:56 +01:00
Brendan Abolivier
6aad81ec0c Rename changelog file 2019-05-01 11:50:15 +01:00
Brendan Abolivier
803a28fd1d Add changelog 2019-05-01 11:43:31 +01:00
Travis Ralston
8c5b1e30d4
Add a default .m.rule.tombstone push rule (#4867)
* Add a default .m.rule.tombstone push rule

In support of MSC1930: https://github.com/matrix-org/matrix-doc/pull/1930

* changelog

* Appease the changelog linter
2019-04-29 15:40:31 -06:00
Richard van der Hoff
b31cc1c613
Merge pull request #5100 from matrix-org/rav/verification_hackery
Improve logging when event-signature checking fails
2019-04-29 13:19:32 +01:00
Erik Johnston
d6118c5be6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ratelimit_3pid_invite 2019-04-26 18:14:23 +01:00
Erik Johnston
19f0722b2c Newsfile 2019-04-26 18:08:33 +01:00
Richard van der Hoff
bd0d45ca69 Fix infinite loop in presence handler
Fixes #5102
2019-04-26 11:14:49 +01:00
Richard van der Hoff
0962d3cdff changelog 2019-04-25 23:05:06 +01:00
Richard van der Hoff
2ebf7d56fa
Merge pull request #5098 from matrix-org/rav/fix_pep_517
Workarounds for pep-517 errors
2019-04-25 15:11:27 +01:00
Richard van der Hoff
e86d74d748 Changelog 2019-04-25 14:56:06 +01:00
*=0=1=4=*
4a9a118a94 Fix handling of SYNAPSE_NO_TLS in docker image (#5005) 2019-04-25 14:47:22 +01:00
Andrew Morgan
6824ddd93d Config option for verifying federation certificates (MSC 1711) (#4967) 2019-04-25 14:22:49 +01:00
Michael Kaye
788163e204 Remove log error for .well-known/matrix/client (#4972) 2019-04-24 17:44:06 +01:00
Amber Brown
6b2b9a58c4 Prevent "producer not unregistered" message (#5009) 2019-04-24 17:37:32 +01:00
Katie Wolfe
60041eac4b
Add full stop to 5084.bugfix
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:51 -04:00
Katie Wolfe
0a4c135f68
Add changelog.d/5084.bugfix
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:47 -04:00
Brendan Abolivier
f8826d31cd Don't crash on lack of expiry templates 2019-04-18 14:50:05 +01:00
Erik Johnston
ca90336a69 Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration 2019-04-17 19:44:40 +01:00
Brendan Abolivier
eaf41a943b Add management endpoints for account validity 2019-04-17 19:34:45 +01:00
Brendan Abolivier
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier
20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00
Brendan Abolivier
49ff74da9b
Merge pull request #5071 from matrix-org/babolivier/3pid-check
Make sure we're not registering the same 3pid twice
2019-04-17 14:37:42 +01:00
Brendan Abolivier
600ec04739 Make sure we're not registering the same 3pid twice 2019-04-17 14:23:01 +01:00
Erik Johnston
fd2fcb817c
Merge pull request #5070 from matrix-org/erikj/postpath
Remove usage of request.postpath
2019-04-17 09:03:26 +01:00
Erik Johnston
a1eb4c6d2f
Merge pull request #5065 from matrix-org/erikj/fix_versions
VersionRestServlet doesn't take a param
2019-04-16 17:52:36 +01:00
Erik Johnston
14d5ad7d2b Newsfile 2019-04-16 17:52:00 +01:00
Travis Ralston
3f22e993f0
Use packages.matrix.org for packages (#5067)
* Use packages.matrix.org for packages

See https://github.com/vector-im/riot-web/issues/9497 (applies to more than just Olm)

* changelog
2019-04-16 08:31:59 -06:00
Silke Hofstra
a137f4eac0 Add systemd-python to optional dependencies (#4339)
Using systemd-python allows for logging to the systemd journal,
as is documented in: `synapse/contrib/systemd/log_config.yaml`.

Signed-off-by: Silke Hofstra <silke@slxh.eu>
2019-04-16 20:41:17 +10:00
Erik Johnston
468b2bcb2e Newsfile 2019-04-15 19:41:25 +01:00
Erik Johnston
6e27a8620f
Merge pull request #5063 from matrix-org/erikj/move_endpoints
Move some rest endpoints to client reader
2019-04-15 18:55:01 +01:00
Erik Johnston
208251956d Newsfile 2019-04-15 17:21:08 +01:00
Andrew Morgan
caa76e6021
Remove periods from copyright headers (#5046) 2019-04-11 17:08:13 +01:00
Erik Johnston
42e1aa5b2c Newsfile 2019-04-10 10:43:47 +01:00
Erik Johnston
e446921def
Merge pull request #5033 from matrix-org/erikj/fix_schema_delta
Fix schema upgrade when dropping tables
2019-04-10 10:22:35 +01:00
Richard van der Hoff
329688c161
Fix disappearing exceptions in manhole. (#5035)
Avoid sending syntax errors from the manhole to sentry.
2019-04-10 07:23:48 +01:00
Erik Johnston
02491e009d Newsfile 2019-04-09 17:23:46 +01:00
Brendan Abolivier
bfc8fdf1fc
Merge pull request #5027 from matrix-org/babolivier/account_expiration
Add time-based account expiration
2019-04-09 17:02:41 +01:00
Brendan Abolivier
747aa9f8ca Add account expiration feature 2019-04-09 16:46:04 +01:00
Erik Johnston
5e45b558b0 Newsfile 2019-04-09 14:39:36 +01:00
Richard van der Hoff
644b86677f
Merge pull request #5030 from matrix-org/rav/rewrite_g_s_v_k
Rewrite Datastore.get_server_verify_keys
2019-04-09 11:30:23 +01:00
Richard van der Hoff
4abf5aa81a
Bump psycopg requirement (#5032) 2019-04-09 11:29:50 +01:00
Richard van der Hoff
1f1e8dd8ec changelog 2019-04-09 00:00:10 +01:00
Richard van der Hoff
3352baac4b
Remove unused server_tls_certificates functions (#5028)
These have been unused since #4120, and with the demise of perspectives, it is
unlikely that they will ever be used again.
2019-04-08 21:50:18 +01:00
Neil Johnson
b25e387c0d
add context to phonehome stats (#5020)
add context to phonehome stats
2019-04-08 15:47:39 +01:00
Richard van der Hoff
67d7b44784
Merge pull request #5024 from matrix-org/rav/record_correct_server_in_serverkeys
Fix from_server buglet in get_keys_from_perspectives
2019-04-08 15:40:37 +01:00
Neil Johnson
2d951686a7
drop tables listed in #1830 (#4992)
Tables dropped: 
 * application_services, 
 * application_services_regex, 
 * transaction_id_to_pdu, 
 * stats_reporting
 * current_state_resets
 * event_content_hashes
 * event_destinations
 * event_edge_hashes
 * event_signatures
 * feedback
 * room_hosts
 * state_forward_extremities
2019-04-08 15:37:26 +01:00
Richard van der Hoff
7d2a0c848e Fix from_server buglet in get_keys_from_perspectives
make sure we store the name of the server the keys came from, rather than the
origin server, after doing a fetch-from-perspectives.
2019-04-08 12:51:16 +01:00
Richard van der Hoff
7fc1e17f4c
Merge pull request #5001 from matrix-org/rav/keyring_cleanups
Cleanups in the Keyring
2019-04-08 12:47:09 +01:00
Richard van der Hoff
b78aac5582 changelog 2019-04-04 19:12:54 +01:00
Brendan Abolivier
8e85493b0c
Add config option to block users from looking up 3PIDs (#5010) 2019-04-04 17:25:47 +01:00
Amber Brown
a33a5abc4c
Clean up the database pagination code (#5007)
* rewrite & simplify

* changelog

* cleanup potential sql injection
2019-04-05 00:21:16 +11:00
Erik Johnston
616e6a10bd
Merge pull request #5002 from matrix-org/erikj/delete_group
Add delete group admin API
2019-04-04 14:15:41 +01:00
Andrew Morgan
db265f0642
Prevent kicking users who aren't in the room (#4999)
Prevent kick events from succeeding if the user is not currently in the room.
2019-04-04 13:05:51 +01:00
Marcel Krüger
9f5d206c4a Avoid redundant URL encoding (#4555)
* Do not double encode fallback redirect URL

Signed-off-by: Marcel Fabian Krüger <zauguin@gmail.com>
2019-04-04 12:05:56 +01:00
Erik Johnston
43c707a010
Merge pull request #5004 from ajensenwaud/develop
Fix issue #4596
2019-04-04 10:44:46 +01:00
*=0=1=4=*
40810b81d2 Correct default POSTGRES_USER in Docker README (#4987)
Correct default POSTGRES_USER in Docker README
2019-04-04 10:38:16 +01:00
Anders
2a59e8e429 Fix issue #4596
Make synapse_port_db --curses work with Python 3.
Signed-off-by: Anders Jensen-Waud <anders@jensenwaud.com>
2019-04-03 21:59:48 +00:00
Erik Johnston
bd3435e982 Newsfile 2019-04-03 16:35:33 +01:00
Andrew Morgan
4a4d5c4fd6
Fix grammar and document get_current_users_in_room (#4998) 2019-04-03 14:32:20 +01:00
Richard van der Hoff
e4d473d855
Rewrite KeyringTestCase as a HomeServerTestCase (#4986)
This is a bit fiddly due to the keyring doing weird things with logcontexts.
2019-04-03 14:11:27 +01:00
Neil Johnson
e8419554ff
Remove presence lists (#4989)
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
Erik Johnston
8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915
Implement MSC1915 - 3PID unbind APIs
2019-04-03 11:07:09 +01:00
Amber Brown
7efd1d87c2 Run black on the rest of the storage module (#4996) 2019-04-03 10:07:29 +01:00
Erik Johnston
3039d61baf
Merge pull request #4991 from matrix-org/erikj/stagger_push_startup
Make starting pushers faster during start up
2019-04-02 18:23:32 +01:00
Erik Johnston
6f226eed42 s/misc/feature/ 2019-04-02 18:22:28 +01:00
Andrew Morgan
66e78700a2
Transfer related groups on room upgrade (#4990)
Transfers the m.room.related_groups state event on room upgrade.
2019-04-02 17:15:24 +01:00
Erik Johnston
ac45b0df0b Newsfile 2019-04-02 17:00:18 +01:00
Richard van der Hoff
8530090b16
Add config.signing_key_path. (#4974)
As requested by @andrewshadura
2019-04-02 16:59:27 +01:00
Richard van der Hoff
297bf2547e
Fix sync bug when accepting invites (#4956)
Hopefully this time we really will fix #4422.

We need to make sure that the cache on
`get_rooms_for_user_with_stream_ordering` is invalidated *before* the
SyncHandler is notified for the new events, and we can now do so reliably via
the `events` stream.
2019-04-02 12:42:39 +01:00
Neil Johnson
4c552ed78a
Neilj/fix threepid auth check (with tests) (#4474)
test threepid checking
2019-04-01 17:42:18 +01:00
Erik Johnston
39fb971e85 Newsfile 2019-04-01 15:31:47 +01:00
manuroe
d461c65465
Merge pull request #4981 from matrix-org/manuroe/demo_bypass_account_rate_limiting
start.sh: Fix the --no-rate-limit option for messages
2019-04-01 16:00:08 +02:00
Neil Johnson
62988f73fd Merge branch 'master' into develop 2019-04-01 14:08:53 +01:00
manuroe
bb925b1bd7 start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. 2019-04-01 14:27:28 +02:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Jurrie Overgoor
50b5f08740 Add changelog.d entry 2019-03-29 10:30:24 +01:00
Richard van der Hoff
902cdc63b6
Merge pull request #4955 from matrix-org/rav/merge_state_into_events
Combine the CurrentStateDeltaStream into the EventStream
2019-03-28 18:32:13 +00:00
Richard van der Hoff
d688a51736
Merge pull request #4954 from matrix-org/rav/refactor_parse_row
Refactors to replication stream row update/parsing
2019-03-28 18:31:17 +00:00
Neil Johnson
c7296bcb98
remove log line for password (#4965)
Remove log line for password.
2019-03-28 17:38:01 +00:00
Andrew Morgan
7a91b9d81c
Allow password providers to bind emails (#4947)
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston
248014379e
Merge pull request #4942 from matrix-org/erikj/fix_presence
Use event streams to calculate presence
2019-03-28 14:38:31 +00:00
Richard van der Hoff
d035d62f6b Merge remote-tracking branch 'origin/develop' into rav/refactor_parse_row 2019-03-28 13:45:14 +00:00
Richard van der Hoff
4eeb2c2f07
Merge pull request #4953 from matrix-org/rav/refactor_replication_streams
Split up replication.tcp.streams into smaller files
2019-03-28 13:43:25 +00:00
Amber Brown
2e060774ad
Run black on some storage modules that the stats branch touches (#4959) 2019-03-29 00:37:16 +11:00
Richard van der Hoff
17d7bacbcf changelog 2019-03-27 22:08:39 +00:00
Richard van der Hoff
a65763a5d6 changelog 2019-03-27 22:04:01 +00:00
Richard van der Hoff
91c3513668 changelog 2019-03-27 21:30:01 +00:00
Richard van der Hoff
acaa18f7dd
Fix/improve some docstrings in the replication code. (#4949) 2019-03-27 21:12:36 +00:00
Erik Johnston
d5a5d1c632 Newsfile 2019-03-27 13:41:36 +00:00
Neil Johnson
6a69bf67db 0.99.3rc1 2019-03-27 10:24:24 +00:00
Andrew Morgan
bbd244c7b2
Support 3PID login in password providers (#4931)
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown
903f04c21f
Use the state event amount for userdir import batching, not room count (#4944) 2019-03-27 02:49:28 +11:00
Andrew Morgan
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 (#4935)
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
Richard van der Hoff
8cbbedaa2b
Fix ClientReplicationStreamProtocol.__str__ (#4929)
`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.

We can calculate the peer addr from the transport, so there is no need for addr
anyway.
2019-03-25 16:41:51 +00:00
Richard van der Hoff
9bde730ef8
Fix bug where read-receipts lost their timestamps (#4927)
Make sure that they are sent correctly over the replication stream.

Fixes: #4898
2019-03-25 16:38:05 +00:00
Richard van der Hoff
a54a44734f
Use an explicit dbname for postgres connections in the tests. (#4928)
I don't have a database with the same name as my user, so leaving the database
name unset fails.

While we're at it, clear out some unused stuff in the test setup.
2019-03-25 16:36:56 +00:00
Richard van der Hoff
7105057cf2
Fix nginx example in ACME doc. (#4923) 2019-03-25 09:59:36 +00:00
Amber Brown
ac396a0d32
Refactor out state delta handling into its own class (#4917) 2019-03-25 20:37:08 +11:00
Erik Johnston
5fee9d8067
Merge pull request #4869 from matrix-org/erikj/yaml_load
Fix yaml warnings by using safe_load
2019-03-22 11:58:13 +00:00
Erik Johnston
d21a4d6be6 Newsfile 2019-03-22 10:24:58 +00:00
Richard van der Hoff
224783a73f
Allow newsfragments to end with exclamation marks! (#4912) 2019-03-21 15:28:19 +00:00
Amber Brown
a68e00fca8
Some more porting to HomeserverTestCase and remove old RESTHelper (#4913) 2019-03-22 02:10:21 +11:00
Andrew Morgan
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 (#4840)
As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
Erik Johnston
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Richard van der Hoff
27813b4ca1
Update changelog.d/4908.bugfix
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-03-21 14:05:59 +00:00
Colin W
ab4e4c6c2f Update Apache Setup To Remove Location Syntax (#4870)
This one should close #4841. Many thanks to @dev4223 for bringing it up and finding a solution.

Signed-off-by: Colin White
2019-03-21 14:05:56 +00:00
Erik Johnston
017ed9d423 Newsfile 2019-03-21 11:26:47 +00:00
Erik Johnston
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Richard van der Hoff
a6f2d3053d
Log requests which are simulated by the unit tests. (#4905)
Rather than stubbing out the access_log, make it actually log the requests,
which makes it a lot more obvious what is going on during tests.
2019-03-20 18:00:02 +00:00
Erik Johnston
30e69ff9b6 Newsfile 2019-03-20 16:56:55 +00:00
Erik Johnston
263f2c9ce1
Merge pull request #4895 from matrix-org/erikj/disable_user_search
Add option to disable searching in the user dir
2019-03-20 16:47:15 +00:00
Amber Brown
4d53017432
Batching in the user directory import (#4900) 2019-03-21 03:06:36 +11:00
Richard van der Hoff
cdb8036161
Add a config option for torture-testing worker replication. (#4902)
Setting this to 50 or so makes a bunch of sytests fail in worker mode.
2019-03-20 16:04:35 +00:00
Richard van der Hoff
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. (#4890)
Rate-limit outgoing read-receipts as per #4730.
2019-03-20 16:02:25 +00:00
Erik Johnston
bf5876990f Newsfile 2019-03-19 17:10:54 +00:00
Erik Johnston
0891202629 Newsfile 2019-03-19 16:41:57 +00:00
Erik Johnston
38ae23d5c2 Newsfile 2019-03-19 16:07:07 +00:00
Erik Johnston
4aa0b707d2
Merge pull request #4879 from matrix-org/erikj/test_old_deps
Add py27-old test case to buildkite
2019-03-19 13:05:39 +00:00
Richard van der Hoff
07f057ac80 changelog 2019-03-19 11:44:43 +00:00
Richard van der Hoff
45c4e19c74 Merge remote-tracking branch 'origin/develop' into HEAD 2019-03-19 11:43:46 +00:00
Neil Johnson
88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Richard van der Hoff
0dbfae03f9 Enforce hs_disabled_message correctly
Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests
if there was no server_notices_mxid configured.
2019-03-19 11:30:54 +00:00
Erik Johnston
e9eeca1314 Fix user directory background update (#4887) 2019-03-19 22:13:53 +11:00
Erik Johnston
cfc5a442ac Update newsfile 2019-03-19 10:56:11 +00:00
Richard van der Hoff
d2a537ea60 Merge remote-tracking branch 'origin/master' into develop 2019-03-19 10:37:50 +00:00
Michael Kaye
9482a84c0a Repoint docs for federation (#4881) 2019-03-19 10:37:18 +00:00
Richard van der Hoff
fd463b4f5d
Comment out most options in the generated config. (#4863)
Make it so that most options in the config are optional, and commented out in
the generated config.

The reasons this is a good thing are as follows:

* If we decide that we should change the default for an option, we can do so,
  and only those admins that have deliberately chosen to override that option
  will be stuck on the old setting.

* It moves us towards a point where we can get rid of the super-surprising
  feature of synapse where the default settings for the config come from the
  generated yaml.

* It makes setting up a test config for unit testing an order of magnitude
  easier (see forthcoming PR).

* It makes the generated config more consistent, and hopefully easier for users
  to understand.
2019-03-19 10:06:40 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Erik Johnston
9ef1107b33 Newsfile 2019-03-18 17:19:49 +00:00
Brendan Abolivier
651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login (#4821)
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff
3b7ceb2c69
Merge pull request #4855 from matrix-org/rav/refactor_transaction_queue
Split TransactionQueue up
2019-03-15 12:32:11 +00:00
Richard van der Hoff
2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston
b0fa3f6ff3
Merge pull request #4853 from matrix-org/erikj/worker_docker_ci
Allow passing --daemonize to workers
2019-03-15 10:35:38 +00:00
Luca Corbatto
a6d84190eb Add systemd setup that supports workers (#4662)
This setup is a way to manage workers with systemd. It does however not
require workers. You can use this setup without workers. You just have
to make sure that the homeserver is forking and writes its PID file
to the location the service is looking in.

The currently distributed setup in the debian package does not work in
conjunction with workers.

* Adds changelog

* Lets systemd handle the forking

Sets all services to `type=simple` and disables daemonizing on the
synapse side.

* Formats readme to 80 columns per line

* Allows for full restart of all workers

* Changes README to reflect the new setup

* Adds dot to end of changelog file

* Removes surplus word

Co-Authored-By: targodan <targodan@users.noreply.github.com>

* Adds missing word

Co-Authored-By: targodan <targodan@users.noreply.github.com>

* Fixes linebreak

Co-Authored-By: targodan <targodan@users.noreply.github.com>

* Fixes unit type
2019-03-15 09:51:46 +00:00
Aaron Raimist
2fb4ff8c89 Add some stuff back to the .gitignore (#4843)
* Add some stuff back to the .gitignore

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Reorder and remove old items from .gitignore

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-03-15 09:22:29 +00:00
Erik Johnston
271cb1998b Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)"
This reverts commit 290552fd83.
2019-03-14 14:30:54 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Richard van der Hoff
ec3a59de50 changelog 2019-03-13 22:08:37 +00:00
Erik Johnston
f5d57d4848 Newsfile 2019-03-13 17:38:58 +00:00
Richard van der Hoff
6accbd25bc changelog 2019-03-13 17:24:10 +00:00
Andrew Morgan
7998ca3a66
Document using a certificate with a full chain (#4849) 2019-03-13 15:26:29 +00:00
Neil Johnson
332b60ec68 Merge branch 'master' of github.com:matrix-org/synapse into develop 2019-03-12 17:15:21 +00:00
Andrew Morgan
d42c81d724
Transfer local user's push rules on room upgrade (#4838)
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Neil Johnson
8b692bf7c2
Neilj/improved delegation doc 2 (#4832)
Improved federation configuration docs.  Specifically detailing  .well-known and SRV based delegation methods. 

Inspiration Valentin Lab <valentin.lab@kalysto.org> for https://github.com/matrix-org/synapse/pull/4781
2019-03-12 14:23:28 +00:00
Aaron Raimist
8ea1b41a0e Clarify what registration_shared_secret allows for (#2885) (#4844)
* Clarify what registration_shared_secret allows for (#2885)

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-03-11 18:21:52 +00:00
Andrew Morgan
f18dca26da Merge branch 'develop' into anoa/trailing_slashes_client 2019-03-11 17:44:29 +00:00
Andrew Morgan
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)
Server side of a solution towards #3622.
2019-03-11 17:44:03 +00:00
Andrew Morgan
a8a028dfce Merge branch 'develop' into anoa/trailing_slashes_client 2019-03-11 17:07:31 +00:00
Erik Johnston
33dfd9a288 Newsfile 2019-03-11 14:13:58 +00:00
Amber Brown
41a5ba1682 changelog 2019-03-12 00:38:17 +11:00
Andrew Morgan
b61ac9660a
Merge pull request #4839 from matrix-org/anoa/no_captcha_tests
Disable captcha registration by default in tests
2019-03-11 09:58:43 +00:00
Matthew Hodgson
2326e00bc4 fix incorrect encoding of filenames with spaces in (#2090)
fixes https://github.com/vector-im/riot-web/issues/3155
2019-03-11 09:53:45 +00:00
Andrew Morgan
f8740d57de Add changelog 2019-03-08 18:23:54 +00:00
Andrew Morgan
50924ee34a Add changelog 2019-03-08 16:49:16 +00:00
Erik Johnston
cac4723afe Newsfile 2019-03-08 15:08:34 +00:00
Erik Johnston
8c4896668f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl 2019-03-08 11:44:20 +00:00
Erik Johnston
469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston
cef80da903 Newsfile 2019-03-07 16:18:02 +00:00
Erik Johnston
7404fb3cdb Newsfile 2019-03-07 16:00:09 +00:00
Richard van der Hoff
fde26e47d6
Merge pull request #4824 from matrix-org/rav/docker_docs
Document the `generate` option for the docker image.
2019-03-07 14:12:44 +00:00
Richard van der Hoff
32471d63b7
Update example_log_config.yaml (#4820) 2019-03-07 14:12:10 +00:00
Richard van der Hoff
685704536f
Fix check-newsfragment for debian-only changes. (#4825) 2019-03-07 14:03:05 +00:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Richard van der Hoff
5580616235 Document the generate option for the docker image. 2019-03-07 07:35:42 +00:00
Matthew Hodgson
8f4b9f5210 Reword the sample config header to be less scary (#4801) 2019-03-07 07:09:01 +00:00
Travis Ralston
06cbf79fe3
Merge pull request #4779 from matrix-org/travis/icons
Use static locations for Riot icons
2019-03-06 15:11:41 -07:00
Richard van der Hoff
898378c9b5
Update changelog.d/4779.misc
Co-Authored-By: turt2live <travpc@gmail.com>
2019-03-06 14:29:56 -07:00
Erik Johnston
1f85c2c0ce
Merge pull request #4818 from matrix-org/erikj/prefill_client_ips
Prefill client IPs cache on workers
2019-03-06 19:17:39 +00:00
Erik Johnston
366877c579 Update changelog 2019-03-06 19:04:52 +00:00
Erik Johnston
0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston
c665b637de
Merge pull request #4816 from matrix-org/erikj/4422_debug
Port #4422 debug logging from hotfixes
2019-03-06 18:59:12 +00:00
Erik Johnston
7791c5194e Newsfile 2019-03-06 17:40:51 +00:00
Erik Johnston
03dce32019 Newsfile 2019-03-06 17:38:19 +00:00
Erik Johnston
4238f63545 Newsfile 2019-03-06 17:32:48 +00:00
Erik Johnston
9c50074c21 Newsfile 2019-03-06 17:24:53 +00:00
Erik Johnston
4c473ba088 Newsfile 2019-03-06 16:24:03 +00:00
Andrew Morgan
7b8a157b79
Merge pull request #4792 from matrix-org/anoa/replication_tokens
Support batch updates in the worker sender
2019-03-06 15:48:29 +00:00
Brendan Abolivier
6fcecb4859 Add changelog 2019-03-05 18:55:29 +00:00
Erik Johnston
16c8b4ecbd
Merge pull request #4772 from jbweston/jbweston/server-version-api
Add 'server_version' endpoint to admin API
2019-03-05 16:31:00 +00:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration (#4735)
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
2019-03-05 14:25:33 +00:00
Andrew Morgan
3887e0cd80
Merge pull request #4795 from matrix-org/anoa/configinatoractoring
Remove reference to that no longer exists in README
2019-03-05 09:47:14 +00:00
Andrew Morgan
ae90531036
Merge pull request #4794 from matrix-org/anoa/erroneous_dollahs
Remove unnecessary dollar signs
2019-03-05 09:46:59 +00:00
Erik Johnston
b050a10871
Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston
9e8bca5667
Merge pull request #4799 from matrix-org/rav/clean_up_replication_code
Clean ups in replication notifier
2019-03-05 09:19:48 +00:00
Erik Johnston
aa06d26ae0 clarify comments 2019-03-05 09:16:35 +00:00
Erik Johnston
c3c542bb4a
Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Richard van der Hoff
eaa9f43603 changelog 2019-03-04 19:01:19 +00:00
Erik Johnston
00b0e8b7df Newsfile 2019-03-04 18:30:07 +00:00
Richard van der Hoff
2db49ea476 Add some debug about processing read receipts.
I'm hoping to establish which rooms are having lots of RRs sent for them, and
how old the events are when they are sent.
2019-03-04 18:19:40 +00:00
Richard van der Hoff
b29693a30b Clean up read-receipt handling.
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.

We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Richard van der Hoff
8e28bc5eee
Include a default configuration file in the 'docs' directory. (#4791) 2019-03-04 17:14:58 +00:00
Andrew Morgan
be18073692 Add changelog 2019-03-04 16:01:17 +00:00
Andrew Morgan
3a438c24a6 Add changelog 2019-03-04 15:28:22 +00:00
Andrew Morgan
0bc50fb60a Add changelog 2019-03-04 14:05:16 +00:00
Seebi
aba5eeabd5 Fix v4v6 option in HAProxy example config (#4790)
The v4v6 option only has a usage one ipv6 socket: https://serverfault.com/q/747895

Signed-off-by: Flakebi <flakebi@t-online.de>
2019-03-04 13:19:41 +00:00
Richard van der Hoff
856c83f5f8
Avoid rebuilding Edu objects in worker mode (#4770)
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston
fbc047f2a5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room 2019-03-04 11:54:58 +00:00
Richard van der Hoff
2c3548d9d8
Update test_typing to use HomeserverTestCase. (#4771) 2019-03-04 10:05:39 +00:00
Travis Ralston
13c18853b9 Changelog 2019-03-01 15:15:00 -07:00
Richard van der Hoff
3064952939
Fix incorrect log about not persisting duplicate state event. (#4776)
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Joseph Weston
d3dcb64501
Add changelog and AUTHORS file entry
Signed-off-by: Joseph Weston <joseph@weston.cloud>
2019-03-01 10:44:40 +01:00
Andrew Morgan
ac61b45a75 Minor docstring fixes for MatrixFederationAgent (#4765) 2019-02-28 16:24:01 +00:00
Amber Brown
b131cc77df
Make 'event_id' a required parameter in federated state requests (#4741)
* make 'event_id' a required parameter in federated state requests

As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40

Signed-off-by: Joseph Weston <joseph@weston.cloud>

* add changelog entry for bugfix

Signed-off-by: Joseph Weston <joseph@weston.cloud>

* Update server.py
2019-02-27 14:35:47 -08:00
Richard van der Hoff
68f47d6744 Fix parsing of Content-Disposition headers (#4763)
* Fix parsing of Content-Disposition headers

TIL: filenames in content-dispostion headers can contain semicolons, and aren't
%-encoded.

* fix python2 incompatibility

* Fix docstrings
2019-02-27 14:29:10 -08:00
Amber Brown
f2a753ea38
Move from TravisCI to BuildKite (#4752) 2019-02-27 13:03:14 -08:00
Erik Johnston
76550c58d2
Merge pull request #4759 from matrix-org/erikj/3pid_client_reader
Move /account/3pid to client_reader
2019-02-27 16:11:21 +00:00
Erik Johnston
8267034a63
Merge pull request #4758 from matrix-org/erikj/use_presence_replication
When presence is disabled don't send over replication
2019-02-27 15:46:26 +00:00
Richard van der Hoff
3134964054
Update changelog.d/4759.feature
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-02-27 15:32:21 +00:00
Erik Johnston
95840d84d4 Newsfile 2019-02-27 14:28:52 +00:00
Erik Johnston
1b2940b3bd Newsfile 2019-02-27 13:54:45 +00:00
Erik Johnston
b5c13df0c4 Newsfile 2019-02-27 13:46:29 +00:00
Erik Johnston
71ef5fc411
Update newsfile to have a full stop 2019-02-27 13:22:23 +00:00
Erik Johnston
7590e9fa28
Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff
Fix tightloop over connecting to replication server
2019-02-27 11:00:59 +00:00
Richard van der Hoff
44a4d65586 0.99.2rc1 2019-02-27 10:48:34 +00:00
Erik Johnston
6bb1c028f1 Limit cache invalidation replication line length (#4748) 2019-02-27 10:28:37 +00:00
Richard van der Hoff
f191be822b
Add database version to phonehome stats. (#4753) 2019-02-27 10:21:49 +00:00
Richard van der Hoff
57426ec6a3 Fix check-newsfragment script (#4750)
* Fix check-newsfragment script

I previously broke this so that it always succeeded...

* more fixes

* fix newsfiles
2019-02-26 13:13:41 -08:00
Erik Johnston
09fc34c935 Newsfile 2019-02-26 15:13:55 +00:00
Andrew Morgan
802884d4ee Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop 2019-02-26 14:23:40 +00:00
Andrew Morgan
b8e6ed36c1 Add changelog 2019-02-26 14:21:20 +00:00
Erik Johnston
899a119c2b Don't log stack trace when client has gone away during media download (#4738)
* Don't log stack trace when client has gone away during media download

* Newsfile

* Fixup newsfile
2019-02-25 11:17:22 -08:00
Richard van der Hoff
641c409e4e Fix ACME config for python 2. (#4717)
Fixes #4675.
2019-02-25 11:16:33 -08:00
Erik Johnston
65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston
1d0f2ec812 Newsfile 2019-02-25 17:02:31 +00:00
Erik Johnston
69efe6fb16
Merge pull request #4718 from matrix-org/erikj/fix_backfill_state_shred
Fix backfill storing incorrect state for events
2019-02-25 16:48:52 +00:00
Erik Johnston
108d5fb20d Fixup changelog 2019-02-25 16:32:19 +00:00
Erik Johnston
b1a90da82e
Merge pull request #4721 from matrix-org/erikj/msc_1866
MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API
2019-02-25 15:53:10 +00:00
Erik Johnston
16c7afa94c
Merge pull request #4722 from matrix-org/erikj/correctly_handle_keyring_exceptions
Handle errors when fetching remote server keys
2019-02-25 15:53:02 +00:00
Erik Johnston
f5050e148c Newsfile 2019-02-25 10:02:58 +00:00
Erik Johnston
21d3f82344 Newsfile 2019-02-23 15:18:38 +00:00
Erik Johnston
65bf9f1119 Newsfile 2019-02-23 15:10:59 +00:00
Erik Johnston
71304bfc8d Newsfile 2019-02-23 14:53:15 +00:00
Erik Johnston
d14e94bae4
Merge pull request #4716 from matrix-org/erikj/pusher_logging
Fix up pusher logging a bit
2019-02-22 15:29:16 +00:00
Erik Johnston
5d3e3c051d Newsfile 2019-02-22 14:51:33 +00:00
Erik Johnston
e28ef831e6 Newsfile 2019-02-22 14:40:08 +00:00
Richard van der Hoff
e07384c4e1
Add prometheus metrics for number of badge update pushes. (#4709)
We're counting the number of push notifications, but not the number of badges;
I'd like to see if they are significant.
2019-02-22 10:57:15 +00:00
Richard van der Hoff
e1666af9be
Better checks on newsfragments (#4698)
* You need an entry in the debian changelog (and not a regular newsfragment)
  for debian packaging changes.
* Regular newsfragments must end in full stops.
2019-02-22 10:56:59 +00:00
Richard van der Hoff
fcd6f01dc7
Minor tweaks to acme docs (#4689) 2019-02-22 10:56:42 +00:00
Richard van der Hoff
0abb094f1a
bail out early in on_new_receipts if no pushers (#4706) 2019-02-21 17:51:21 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job (#4707)
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Benoît S
16e0680498 Added HAProxy example (#4660)
* Added HAProxy example

Proposal of an example with HAProxy. Asked by #4541.

Signed-off-by: Benoît S. (“Benpro”) <gitlab@benpro.fr>

* Following suggestions of @richvdh
2019-02-21 17:44:10 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Eric
8184ae8a09 Consider e2e_room_keys.is_verified column as boolean
This column was considered as an int, crashing the whole
migration process

Signed-off-by: Eric <eric@pedr0.net>
2019-02-20 23:18:00 +01:00
Erik Johnston
56f4ece778 Newsfile 2019-02-20 18:15:13 +00:00
Erik Johnston
82fca11fc1
Merge pull request #4694 from matrix-org/erikj/fix_sentry_config_format
Fixup generated metrics config
2019-02-20 14:13:38 +00:00
Richard van der Hoff
82ca6d1f9f
Add metrics for number of outgoing EDUs, by type (#4695) 2019-02-20 14:13:14 +00:00
Erik Johnston
633e5c933b Newsfile 2019-02-20 13:41:43 +00:00
Amber Brown
a06614bd2a
UPSERT many functionality (#4644) 2019-02-20 23:03:30 +11:00
Richard van der Hoff
b2200a8690
Logging improvements for the pusher (#4691) 2019-02-20 11:36:50 +00:00
Richard van der Hoff
c88bc53903
Fix TaskStopped exceptions when outbound requests time out (#4690) 2019-02-20 11:35:52 +00:00
Richard van der Hoff
8d98dc8ffe
Clean up gitignores (#4688)
I just got bitten by a file being caught by the .gitignore, which shouldn't
have been, and am now pissed off with the .gitignore. I have basically declared
bankruptcy on it and started again.
2019-02-20 11:10:50 +00:00
Richard van der Hoff
86920ac266
Merge pull request #4678 from matrix-org/rav/tls_install_instructions
Attempt to clarify installation/config instructions
2019-02-20 11:10:33 +00:00
Erik Johnston
dbdc565dfd Fix registration on workers (#4682)
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
2019-02-20 18:47:31 +11:00
Richard van der Hoff
c594cc8076 Run unit tests against python 3.7 (#4677)
* Run unit tests against python 3.7

... so that we span the full range of our supported python versions

* Switch to xenial

* fix psql fail

* pep8 etc want python 3.6
2019-02-20 18:42:24 +11:00
Richard van der Hoff
ae753fed8c changelog 2019-02-19 13:56:44 +00:00
Erik Johnston
c003450057
Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation
Batch cache invalidation over replication
2019-02-19 13:14:30 +00:00
Richard van der Hoff
1c0eb8bbb2
Merge pull request #4676 from matrix-org/rav/pg95
Test against Postgres 9.5 as well as 9.4
2019-02-19 11:19:21 +00:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Erik Johnston
32590b7139
Merge pull request #4670 from matrix-org/erikj/register_login_split
Split /login into client_reader
2019-02-19 10:46:22 +00:00
Richard van der Hoff
7c70b8f8a6
Try and make TLS federation client code faster (#4674)
* fix to use makeContext so that we don't need to rebuild the certificateoptions each time
2019-02-19 10:19:16 +00:00
Richard van der Hoff
107aeb6915 misc->feature 2019-02-19 10:18:48 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Richard van der Hoff
0869f01e74 Test against Postgres 9.5 as well as 9.4
Postgres 9.5 is the first to support UPSERTs, so we should really run against
it as well as 9.4.
2019-02-19 09:54:31 +00:00
Amber Brown
2b2466f78b changelog 2019-02-19 16:18:48 +11:00
Erik Johnston
92e6fb5c89 Newsfile 2019-02-18 17:58:17 +00:00
Andrew Morgan
f8b9ca53ce Move member event processing and changelog fix 2019-02-18 17:28:52 +00:00
Erik Johnston
d154f5a055
Merge pull request #4632 from matrix-org/erikj/basic_sentry
Add basic optional sentry.io integration
2019-02-18 17:22:45 +00:00
Erik Johnston
f3ab0b2390 Newsfile 2019-02-18 17:22:01 +00:00
Erik Johnston
128902d60a Update worker docs 2019-02-18 17:21:51 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston
028267acd2
Merge pull request #4669 from matrix-org/erikj/log_exception
Cleanup top level request exception logging
2019-02-18 16:41:41 +00:00
Erik Johnston
d08bac4136
Merge pull request #4651 from matrix-org/matthew/well-known-cors
set CORS on .well-known URI to unbreak modular
2019-02-18 16:38:11 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Erik Johnston
092b541401 Fixup 2019-02-18 15:52:26 +00:00
Erik Johnston
12ae64ce0d Newsfile 2019-02-18 15:23:10 +00:00
Erik Johnston
e85aabb030 Newsfile 2019-02-18 15:06:22 +00:00
Matthew Hodgson
d9713e916e changelog 2019-02-18 16:00:22 +01:00
Erik Johnston
2f16857ca9 Newsfile 2019-02-18 14:55:11 +00:00
Brendan Abolivier
68a53f825f Merge branch 'develop' into babolivier/acme-delegated 2019-02-18 14:52:23 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan
7033b05cad Add changelog 2019-02-18 13:52:56 +00:00
Will Hunt
e83a190643
Update changelog.d/4666.feature
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-02-18 13:46:13 +00:00
Erik Johnston
91c8a7f9f4 Newsfile 2019-02-18 12:15:27 +00:00
Andrew Morgan
5b68e12fd8
Typo in changelog
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:44 +00:00
Travis Ralston
da95867d30 Changelog 2019-02-15 22:24:39 -07:00
Erik Johnston
bd4505f765
Merge pull request #4647 from matrix-org/erikj/add_room_publishing_rules
Add configurable room list publishing rules
2019-02-15 22:11:01 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Erik Johnston
4074c8b968 Newsfile 2019-02-14 16:04:12 +00:00
Richard van der Hoff
b02465b9db Merge branch 'master' into develop 2019-02-14 14:42:03 +00:00
Erik Johnston
0927adb012 Newsfile 2019-02-14 14:02:04 +00:00
Erik Johnston
6cb415b63f Fixup comments and add warning 2019-02-13 16:15:11 +00:00
Richard van der Hoff
c6e75c9f2d
Merge pull request #4450 from 14mRh4X0r/fix-dependency-message
Fix error message for optional dependencies
2019-02-13 16:12:49 +00:00
Richard van der Hoff
3bc238629e 0.99.1rc2 2019-02-13 14:46:18 +00:00
Richard van der Hoff
c1dfd6a18a Merge remote-tracking branch 'origin/release-v0.99.1' into develop 2019-02-13 14:27:45 +00:00
Amber Brown
bb4fd8f927
Run black on user directory code (#4635) 2019-02-13 23:05:32 +11:00
Richard van der Hoff
2d0e0a4044 changelog 2019-02-13 12:00:34 +00:00
Erik Johnston
dc70789056 Newsfile 2019-02-12 16:07:43 +00:00
Erik Johnston
d2fa7b7e99 Update changelog and version 2019-02-12 13:22:25 +00:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Erik Johnston
b18cd25e42 Fixup changelog entries 2019-02-12 13:05:31 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze
Analyze user_ips before running deduplication
2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe
Reduce user_ips bloat during dedupe background update
2019-02-12 13:02:58 +00:00
Erik Johnston
b2327eb9cb Newsfile 2019-02-12 11:58:36 +00:00
Erik Johnston
218cc071c5 Newsfile 2019-02-12 11:39:36 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings
fix self-signed cert notice from generate-config
2019-02-12 11:24:45 +00:00
Richard van der Hoff
2418b91bb7
README updates (#4621)
Lots of updates to the README/INSTALL.md.

Fixes #4601.
2019-02-12 10:53:28 +00:00
Richard van der Hoff
a4ce91396b
Disable TLS by default (#4614) 2019-02-12 10:52:08 +00:00
Richard van der Hoff
32b781bfe2
Fix error when loading cert if tls is disabled (#4618)
If TLS is disabled, it should not be an error if no cert is given.

Fixes #4554.
2019-02-12 10:51:31 +00:00
Richard van der Hoff
dfc846a316 fix self-signed cert notice from generate-config
fixes #4620
2019-02-12 10:37:59 +00:00
Erik Johnston
46b8a79b3a
Merge pull request #4619 from matrix-org/rav/remove_docker_no_tls_hacks
Remove redundant entries from docker config
2019-02-12 10:00:38 +00:00
Richard van der Hoff
91f8cd3307 Remove redundant entries from docker config
* no_tls is now redundant (#4613)
* we don't need a dummy cert any more (#4618)
2019-02-11 22:16:44 +00:00
Richard van der Hoff
4fddf8fc77 Infer no_tls from presence of TLS listeners
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
2019-02-11 21:39:14 +00:00
Richard van der Hoff
15272f837c Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work 2019-02-11 21:34:19 +00:00
Richard van der Hoff
9645728619 Don't create server contexts when TLS is disabled
we aren't going to use them anyway.
2019-02-11 21:32:01 +00:00
Richard van der Hoff
be794c7cf7 Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work 2019-02-11 21:16:00 +00:00
Richard van der Hoff
2129dd1a02 Fail cleanly if listener config lacks a 'port'
... otherwise we would fail with a mysterious KeyError or something later.
2019-02-11 21:15:01 +00:00
Richard van der Hoff
086f6f27d4 Logging improvements around TLS certs
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
2019-02-11 21:02:06 +00:00
Richard van der Hoff
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates (#4611)
It's nothing to do with refreshing the certificates. No idea why it was here.
2019-02-11 18:03:30 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating
add updating of backup versions
2019-02-11 13:45:49 +00:00
Richard van der Hoff
24b7f3916d
Clean up default listener configuration (#4586)
Rearrange the comments to try to clarify them, and expand on what some of it
means.

Use a sensible default 'bind_addresses' setting.

For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
2019-02-11 12:50:30 +00:00
Richard van der Hoff
c475275926
Clarifications for reverse proxy docs (#4607)
Factor out the reverse proxy info to a separate file, add some more info on
reverse-proxying the federation port.
2019-02-11 11:44:28 +00:00
Andrew Morgan
eff2042217 Changelog 2019-02-11 11:41:57 +00:00
Amber Brown
6e2a5aa050 ACME Reprovisioning (#4522) 2019-02-11 10:36:26 +00:00
Amber Brown
4ffd10f46d Be tolerant of blank TLS fingerprints config (#4589) 2019-02-11 10:04:27 +00:00
Erik Johnston
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Valentin Anger
2dc2b6e9f1 Allow "unavailable" presence status for /sync (#4592)
* Allow "unavailable" presence status for /sync

Closes #3772, closes #3779

Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>

* Add changelog for PR 4592
2019-02-08 21:09:56 +00:00
Richard van der Hoff
56710c7df5
Fix 'no unique or exclusion constraint' error (#4591)
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
2019-02-08 18:30:46 +00:00
Amber Brown
9cd33d2f4b
Deduplicate some code in synapse.app (#4567) 2019-02-08 17:25:57 +00:00
Erik Johnston
acb2ac5863 Update MSC1711 FAQ to be explicit about well-known (#4584)
A surprising number of people are using the well-known method, and are
simply copying the example configuration. This is problematic as the
example includes an explicit port, which causes inbound federation
requests to have the HTTP Host header include the port, upsetting some
reverse proxies.

Given that, we update the well-known example to be more explicit about
the various ways you can set it up, and the consequence of using an
explict port.
2019-02-07 19:30:32 +00:00
Richard van der Hoff
7a22a645b5 Merge branch 'master' into develop 2019-02-07 19:18:48 +00:00
Richard van der Hoff
624b172e08 Merge remote-tracking branch 'origin/release-v0.99.0' 2019-02-07 19:18:26 +00:00
Andrew Morgan
c17b128b83 Update ACME docs to include port instructions (#4578) 2019-02-07 19:18:08 +00:00
Hubert Chathi
51b73be63b add changelog entry 2019-02-06 21:39:56 -05:00
Richard van der Hoff
7fe407a87a Merge branch 'master' into develop 2019-02-06 09:50:54 +00:00
Richard van der Hoff
d8e63846e2
Fix docker upload job to push -py2 images (#4576) 2019-02-06 09:41:54 +00:00
Richard van der Hoff
2475434080 Merge branch 'master' into develop 2019-02-05 18:44:49 +00:00
Richard van der Hoff
3bd9daf4b8 v0.99.0 2019-02-05 18:33:02 +00:00
Neil Johnson
6585ef4799
Neilj/1711faq (#4572)
MSC1711 certificates FAQ
2019-02-05 17:19:28 +00:00
Richard van der Hoff
40b35fb875
Enable ACME support in the docker image (#4566)
Also:

* Fix wrapping in docker readme
* Clean up some docs on the docker image
* a workaround for #4554
2019-02-05 13:42:21 +00:00
Andrew Morgan
627ecd358e Filter user directory state query to a subset of state events (#4462)
* Filter user directory state query to a subset of state events

* Add changelog
2019-02-05 12:16:28 +00:00
Richard van der Hoff
3ef71a6ea0
Docker: only copy what we need to the build image (#4562)
There are two reasons this is a good thing:

 * first, it means that you don't end up with stuff kicking around your working
   copy ending up in the build image by mistake (which can upset the pip
   install process)

 * second: it means that the docker image cache is more effective, and we can
   reuse docker images when iterating on the docker stuff.
2019-02-05 11:44:40 +00:00
Richard van der Hoff
bf1e4d96ad
Fix default ACME config for py2 (#4564)
Fixes #4559
2019-02-05 11:37:33 +00:00
Richard van der Hoff
9a75c0b52e
switch docker image to py3 by default (#4558)
Switch the matrixdotorg/synapse:latest Docker image to use python 3
2019-02-05 11:33:26 +00:00
Richard van der Hoff
d7e27a1f08
fix typo in config comments (#4557) 2019-02-05 11:32:45 +00:00
Andrew Morgan
bcc78bb0b8 Merge branch 'release-v0.99.0' of github.com:matrix-org/synapse into anoa/acme_docs 2019-02-01 15:57:32 +00:00
Richard van der Hoff
30fd2f89db 0.99.0rc4 2019-02-01 15:52:28 +00:00
Andrew Morgan
c5fc09322c Add changelog 2019-02-01 15:05:10 +00:00
Richard van der Hoff
ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
2019-02-01 14:37:22 +00:00
Richard van der Hoff
f8db967d5a Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-02-01 13:20:15 +00:00
Richard van der Hoff
9763a73af0 Merge branch 'release-v0.99.0' into develop 2019-02-01 12:30:22 +00:00
Richard van der Hoff
f0ba34f581 Fix noisy "twisted.internet.task.TaskStopped" errors in logs
Fixes #4003
2019-02-01 12:22:57 +00:00
Richard van der Hoff
8a21b03fba Treat an invalid .well-known the same as an absent one
... basically, carry on and fall back to SRV etc.
2019-02-01 11:37:31 +00:00
Richard van der Hoff
0390c961ac changelog 2019-02-01 09:40:58 +00:00
Richard van der Hoff
d428b46346 Update federation routing logic to check .well-known before SRV 2019-01-31 23:14:18 +00:00
Richard van der Hoff
85129d7068 v0.99.0rc3 2019-01-31 18:35:38 +00:00
Richard van der Hoff
07dfe148de
Add some debug for membership syncing issues (#4538)
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan
bbb97a35fd Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable 2019-01-31 18:11:00 +00:00
Richard van der Hoff
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room (#4535) 2019-01-31 15:34:17 +00:00
Andrew Morgan
563f6a832b Reject large transactions on federation (#4513)
* Reject large transactions on federation

* Add changelog

* lint

* Simplify large transaction handling
2019-01-31 11:44:04 +00:00
Richard van der Hoff
35f544410a update debian installation instructions (#4526)
* update debian installation instructions

* docs PR is docs
2019-01-31 10:29:15 +00:00
Andrew Morgan
cf9a2676d0 Add changelog 2019-01-30 19:04:48 +00:00
Richard van der Hoff
6fba9fd20c Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-01-30 17:02:32 +00:00
Richard van der Hoff
b8b898666e v0.99.0rc2 2019-01-30 16:31:07 +00:00
Matthew Hodgson
ad7ac8853c by default include m.room.encryption on invites (#3902)
* by default include m.room.encryption on invites

* fix constant

* changelog
2019-01-30 16:26:13 +00:00
Richard van der Hoff
c74b96755c Merge remote-tracking branch 'origin/develop' into release-v0.99.0 2019-01-30 16:23:28 +00:00
Amber Brown
fbaee26c68 ACME Upgrade Docs (#4528) 2019-01-30 16:22:37 +00:00
Erik Johnston
e25ab58c5e Newsfile 2019-01-30 15:50:28 +00:00
Richard van der Hoff
a5d0c771a3 0.99.0rc1 2019-01-30 15:11:18 +00:00
Erik Johnston
6587b0b89b
Merge pull request #4472 from matrix-org/neilj/room_capabilities
Server capabilities support
2019-01-30 14:26:56 +00:00
Erik Johnston
a4f52a33fe Fix replication for room v3 (#4523)
* Fix replication for room v3

We were not correctly quoting the path fragments over http replication,
which meant that it exploded when the event IDs had a slash in them

* Newsfile
2019-01-30 14:19:52 +00:00
Richard van der Hoff
7615a8ced1 ACME config cleanups (#4525)
* Handle listening for ACME requests on IPv6 addresses

the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.

* Clean up the default ACME config

make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.

* newsfile
2019-01-30 14:17:55 +00:00
Erik Johnston
e6a7a15f93 Newsfile 2019-01-30 12:17:38 +00:00
Richard van der Hoff
a79034aedf
Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups
Tiny .well-known fixes
2019-01-30 11:47:24 +00:00
Richard van der Hoff
c7b24ac3d0
Follow redirects on .well-known (#4520) 2019-01-30 11:43:33 +00:00
Amber Brown
f6813919e8
SIGHUP for TLS cert reloading (#4495) 2019-01-30 11:00:02 +00:00
Richard van der Hoff
283753c33a newsfile 2019-01-30 10:59:21 +00:00
Richard van der Hoff
bc5f6e1797
Add a caching layer to .well-known responses (#4516) 2019-01-30 10:55:25 +00:00
Amber Brown
3f189c902e
Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Neil Johnson
ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities 2019-01-30 10:28:08 +00:00
Richard van der Hoff
457fbfaf22
Merge pull request #4486 from xperimental/workaround-4216
Implement workaround for login error.
2019-01-30 07:06:11 +00:00
Robert Jacob
2a7f0b8953 Implement workaround for login error.
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
2019-01-30 01:06:39 +01:00
Erik Johnston
ebcffbc3eb Newsfile 2019-01-29 23:09:10 +00:00
Erik Johnston
67b82f1336 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:58:38 +00:00
Erik Johnston
7740eddd04
Merge pull request #4514 from matrix-org/erikj/remove_event_id
Remove usages of event ID's domain
2019-01-29 22:54:25 +00:00
Erik Johnston
a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston
6f9cdc2d47
Merge pull request #4483 from matrix-org/erikj/event_v2
Implement event format V2
2019-01-29 21:40:00 +00:00
Andrew Morgan
03b086647f
Merge pull request #4512 from matrix-org/anoa/consent_dir
Check consent dir path on startup
2019-01-29 20:08:18 +00:00
Erik Johnston
3680bc18e9 Newsfile 2019-01-29 18:06:11 +00:00
Erik Johnston
b40abe0724 Newsfile 2019-01-29 18:02:26 +00:00
Richard van der Hoff
cc2d650ef7
Relax requirement for a content-type on .well-known (#4511) 2019-01-29 16:49:17 +00:00
Travis Ralston
d02c5ccb11
Merge pull request #4498 from matrix-org/travis/fix-docs-public_baseurl
Don't recommend :8448 to people on public_baseurl
2019-01-29 09:06:16 -07:00
Neil Johnson
19259d903c update to reflect broadening scope 2019-01-29 16:01:46 +00:00
Andrew Morgan
9adbc912b3 Add changelog 2019-01-29 15:34:06 +00:00
Amber Brown
6bd4374636
Do not generate self-signed TLS certificates by default. (#4509) 2019-01-29 14:09:10 +00:00
Erik Johnston
b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
Richard van der Hoff
99e36d5e24
Implement MSC1708 (.well-known lookups for server routing) (#4489) 2019-01-29 13:53:02 +00:00
Erik Johnston
2562319821
Merge pull request #4510 from matrix-org/erikj/fixup_compute_event_signature
Fixup calls to `comput_event_signature`
2019-01-29 13:35:19 +00:00
Richard van der Hoff
5488cadaae
Enable configuring test log level via env var (#4506)
I got fed up with always adding '@unittest.DEBUG' every time I needed to debug a test.
2019-01-29 12:07:00 +00:00
Erik Johnston
7d1024d574 Newsfile 2019-01-29 11:58:16 +00:00
Erik Johnston
0c55b7701c Newsfile 2019-01-29 11:42:33 +00:00
Erik Johnston
ff37acb8ce
Merge pull request #4496 from matrix-org/erikj/invite_fallback
Implement fallback for V2 invite API
2019-01-29 11:28:23 +00:00
Erik Johnston
a388d59d44 Newsfile 2019-01-29 11:13:08 +00:00
Amber Brown
f815bd7feb
Make linearizer more quiet (#4507) 2019-01-29 11:05:31 +00:00
Erik Johnston
073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Amber Brown
94fb63e44f
Fix typo in upserts code (#4505)
* fix obvious problem :|

* changelog
2019-01-29 10:04:23 +00:00
Erik Johnston
17709f8f9c
Merge pull request #4493 from matrix-org/erikj/refactor_event_signing
Refactor event signing to work on dicts
2019-01-29 09:52:54 +00:00
Richard van der Hoff
f2b553d656 Use SimpleResolverComplexifier in tests (#4497)
two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes
unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we
rely on to give consistent behaviour between python 2 and 3.
2019-01-29 09:38:29 +00:00
Erik Johnston
f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Travis Ralston
c4045647eb
Create 4498.misc 2019-01-28 12:16:39 -07:00
Erik Johnston
8cbc99cc19 Newsfile 2019-01-28 17:36:06 +00:00
Amber Brown
5d976c0c7c
Fix worker TLS (#4492)
* load cert

* changelog

* fix
2019-01-28 17:18:33 +00:00
Erik Johnston
1977a9b006 Newsfile 2019-01-28 17:05:04 +00:00
Erik Johnston
b8bea3424f Newsfile 2019-01-28 16:47:12 +00:00
Amber Brown
7072fe3084
Fix UPSERTs on SQLite 3.24+ (#4477) 2019-01-28 15:43:32 +00:00
Andrew Morgan
4026d555fa Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-28 14:08:24 +00:00
Aaron Raimist
57a3e96e8e Remove --process-dependency-links from UPGRADE.rst (#4485)
* Remove --process-dependency-links from UPGRADE.rst

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-28 13:54:25 +00:00
Richard van der Hoff
ff05ad147a changelog 2019-01-28 10:34:30 +00:00
Richard van der Hoff
d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487)
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.

There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.

The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
2019-01-28 09:56:59 +00:00
Erik Johnston
f01c7488ab Newsfile 2019-01-25 18:32:50 +00:00
Erik Johnston
b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Erik Johnston
431e485914 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert 2019-01-25 14:11:17 +00:00
Richard van der Hoff
4a3f138832
Fix quoting for allowed_local_3pids example config (#4476)
If you use double-quotes here, you have to escape your backslashes. It's much
easier with single-quotes.

(Note that the existing double-backslashes are already interpreted by python's
""" parsing.)
2019-01-25 13:57:52 +00:00
Richard van der Hoff
8520bc3109
Fix Host header sent by MatrixFederationAgent (#4468)
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
Andrew Morgan
0b3fd1401f Don't require sqlite3 when using postgres (#4466) 2019-01-25 22:25:02 +11:00
Neil Johnson
a3f0556bea towncrier 2019-01-25 11:15:41 +00:00
Andrew Morgan
821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Andrew Morgan
b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston
8dcfa6e75c Newsfile 2019-01-25 10:48:40 +00:00
Erik Johnston
5d881cbcb8 Newsfile 2019-01-25 10:37:13 +00:00
Erik Johnston
62514bb81b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 2019-01-25 10:07:08 +00:00
Erik Johnston
edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston
80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston
664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2
Add support for persisting event format versions
2019-01-24 18:32:30 +00:00
Andrew Morgan
50c396a7ee
Merge pull request #4461 from matrix-org/anoa/room_dir_quick_fix
Add a 60s timeout to filtered room directory queries
2019-01-24 15:24:35 +00:00
Richard van der Hoff
4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup
MatrixFederationAgent: Look up the right SRV record
2019-01-24 13:57:51 +00:00
Richard van der Hoff
8c58c10697
Generate the debian config during build (#4444)
Rather than hardcoding a config which we always forget to update, generate it
from the default config.
2019-01-24 13:39:01 +00:00
Richard van der Hoff
f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json (#4460)
This could cause exceptions if the id server returned 4xx responses.
2019-01-24 13:38:29 +00:00
Richard van der Hoff
afd69a0920 Look up the right SRV record 2019-01-24 13:31:43 +00:00
Neil Johnson
10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check
Neilj/fix threepid auth check
2019-01-24 13:02:50 +00:00
Andrew Morgan
2a360e834f Add changelog 2019-01-24 12:47:35 +00:00
Neil Johnson
5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc
Clarify docs for public_baseurl
2019-01-24 12:36:09 +00:00
Amber Brown
0e27501ee5
Fix UPSERT check (#4459) 2019-01-24 22:57:41 +11:00
David Baker
1f2058fca5 Changelog 2019-01-24 10:57:12 +00:00
Amber Brown
58f6c48183
Use native UPSERTs where possible (#4306) 2019-01-24 21:31:54 +11:00
Richard van der Hoff
97fd29c019
Don't send IP addresses as SNI (#4452)
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
2019-01-24 09:34:44 +00:00
Erik Johnston
183738f469 Newsfile 2019-01-23 20:04:57 +00:00
Willem Mulder
d528406cb8 Fix error message for optional dependencies
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
2019-01-23 18:44:57 +01:00
Erik Johnston
c5a125b24f Update newsfile 2019-01-23 17:26:06 +00:00
Erik Johnston
4a8b715679 Newsfile 2019-01-23 17:25:27 +00:00
Erik Johnston
a4ef8d8dd5 Newsfile 2019-01-23 17:24:44 +00:00
Andrew Morgan
6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport
Add metric for user dir current event stream position
2019-01-23 15:58:40 +00:00
Richard van der Hoff
a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent
Move SRV magic into an Agent-like thing
2019-01-23 15:50:25 +00:00
Andrew Morgan
cb0e637a94 Add changelog 2019-01-23 15:38:27 +00:00
Richard van der Hoff
2f88881c93
debian package: symlink to python-3.X (#4433)
In the debian package, make the virtualenv symlink python to /usr/bin/python3.X
rather than /usr/bin/python3. Also make sure we depend on the right python3.x
package.

This might help a bit with subtle failures when people install a package from
the wrong distro (https://github.com/matrix-org/synapse/issues/4431).
2019-01-23 11:43:04 +00:00
Erik Johnston
4cd50d983d Newsfile 2019-01-23 11:30:24 +00:00
Erik Johnston
90743c9d89 Fixup removal of duplicate user_ips rows (#4432)
* Remove unnecessary ORDER BY clause

* Add logging

* Newsfile
2019-01-23 19:45:18 +11:00
Amber Brown
6129e52f43
Support ACME for certificate provisioning (#4384) 2019-01-23 19:39:06 +11:00
Jason Robinson
db33634b1d Collapse changelog to one line
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson
1d2c69fee8 Add changelog for openid resource addition
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Richard van der Hoff
c66f4bf7f1 changelog 2019-01-22 20:34:35 +00:00
Neil Johnson
c99c2d58d7 move guard out of is_threepid_reserved and into register.py 2019-01-22 17:47:00 +00:00
Neil Johnson
d619b113ed Fix None guard in config.server.is_threepid_reserved 2019-01-22 16:52:29 +00:00
Erik Johnston
12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup
Fix bug when removing duplicate rows from user_ips
2019-01-22 16:51:57 +00:00
Neil Johnson
388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration
Config option to disable requesting MSISDN on registration
2019-01-22 16:23:08 +00:00
Erik Johnston
c658425e6f Newsfile 2019-01-22 13:34:10 +00:00
Andrew Morgan
117bc94cd2 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-22 13:16:51 +00:00
Richard van der Hoff
6bfa735a69
Make key fetches use regular federation client (#4426)
All this magic is redundant.
2019-01-22 11:04:20 +00:00
Andrew Morgan
25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Andrew Morgan
1f18c7cfc9 Add changelog 2019-01-22 11:00:04 +00:00
Richard van der Hoff
33a55289cb
Refactor and bugfix for resove_service (#4427) 2019-01-22 10:59:27 +00:00
Amber Brown
23b0813599
Require ECDH key exchange & remove dh_params (#4429)
* remove dh_params and set better cipher string
2019-01-22 21:58:50 +11:00
Neil Johnson
5349262302 Config option to disable requesting MSISDN on registration 2019-01-21 14:59:37 +00:00
Erik Johnston
35e1d67b4e Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_v2_invite_server 2019-01-21 14:04:19 +00:00
Erik Johnston
5f54765587
Merge pull request #4390 from matrix-org/erikj/versioned_fed_apis
Add groundwork for new versions of federation APIs
2019-01-21 11:44:05 +00:00
Andrew Morgan
702c4b750c Migrate encryption state on room upgrade (#4411)
* Migrate encryption state on room upgrade

Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>

* Add changelog file
2019-01-21 20:42:58 +11:00
Erik Johnston
25dd56ace3 Fix race when persisting create event (#4404)
* Fix race when persisting create event

When persisting a chunk of DAG it is sometimes requried to do a state
resolution, which requires knowledge of the room version. If this
happens while we're persisting the create event then we need to use that
event rather than attempting to look it up in the database.
2019-01-18 23:17:04 +11:00
Erik Johnston
71b94eac46 Tweak code coverage settings (#4400)
* Tweak code coverage settings

* Fix manifest

* Newsfile

* Fix commit status?
2019-01-18 23:13:14 +11:00
Richard van der Hoff
de6888e7ce Remove redundant WrappedConnection (#4409)
* Remove redundant WrappedConnection

The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
2019-01-18 23:07:38 +11:00
Andrew Morgan
cb80db8941 Add changelog 2019-01-18 11:22:00 +00:00
Richard van der Hoff
676cf2ee26
Fix incorrect logcontexts after a Deferred was cancelled (#4407) 2019-01-17 14:00:23 +00:00
Richard van der Hoff
9feb5d0b71
sign_request -> build_auth_headers (#4408)
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
Richard van der Hoff
3982a6ee07
Changing macaroon_secret_key no longer logs you out (#4387) 2019-01-16 23:14:41 +00:00
Richard van der Hoff
05e1296649
don't store more remote device lists if they have more than 1K devices (#4397) 2019-01-16 23:14:11 +00:00
Erik Johnston
f788c9eb70 Newsfile 2019-01-16 13:46:36 +00:00
Andrej Shadura
3b31a54a6e
Add a changelog entry
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:54:43 +01:00
Neil Johnson
9ec56d6935 ALL_USER_TYPES should be a tuple 2019-01-15 14:38:15 +00:00
Erik Johnston
1a8f4139a5 Newsfile 2019-01-15 11:21:52 +00:00
Richard van der Hoff
34b25dcc8e Silence travis-ci build warnings by removing non-functional python3.6 (#4377)
* Remove non-functional python3.6 in travis env

* changelog
2019-01-12 06:22:56 +11:00
Amber Brown
a35c66a00b
Remove duplicates in the user_ips table and add an index (#4370) 2019-01-12 06:21:50 +11:00
Richard van der Hoff
8c818af38e
Merge pull request #4342 from aaronraimist/new-virtualenv
Update README to use new virtualenv (#4328)
2019-01-10 14:59:33 +00:00
Richard van der Hoff
d566e6b17a Merge branch 'master' into develop 2019-01-09 15:00:46 +00:00
Richard van der Hoff
998f5225c1 0.34.1 2019-01-09 14:53:54 +00:00
Erik Johnston
55c3e853c5
Merge pull request #4368 from matrix-org/erikj/better_errors
Fixup docstrings for matrixfederationclient
2019-01-09 11:39:28 +00:00
Erik Johnston
484867d35d Merge branch 'develop' into release-v0.34.1 2019-01-09 11:38:13 +00:00
Amber Brown
7960c26fda
Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. (#4369) 2019-01-09 22:26:25 +11:00
Erik Johnston
0dce21ba77 Newsfile 2019-01-09 09:27:03 +00:00
Erik Johnston
d91b99abe0 Newsfile 2019-01-08 14:24:05 +00:00
Erik Johnston
83c50bf752 Newsfile 2019-01-08 12:28:35 +00:00
Richard van der Hoff
bc1a4b5576 changelog 2019-01-08 11:14:59 +00:00
Erik Johnston
b970cb0e96 Refactor request sending to have better excpetions (#4358)
* Correctly retry and back off if we get a HTTPerror response

* Refactor request sending to have better excpetions

MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).

This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
Amber Brown
1dcb086f33
Fix synapse.config.__main__ on python 3 (#4356) 2019-01-08 10:03:09 +11:00
Jason Robinson
26e5abf20d Fix command hint to generate a config file (#4353)
* Fix command hint to generate a config file

When trying to start Synapse without a config file, it will complain
and give a hint towards what command to run. This hinted command
is missing the "report_stats" parameter, which is required with either
yes or no value. Add this to the command.

Not an ideal situation but makes the given command work without the
user getting another error, even though it might be unclear what
"report_stats" represents.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-07 16:28:40 +00:00
Travis Ralston
bc1fa8cd01 Add GET account data routes (#4303)
As per https://github.com/matrix-org/matrix-doc/issues/1339
2019-01-07 10:27:54 +00:00
Richard van der Hoff
b7c0218812
Check jinja version for consent resource (#4327)
* Raise a ConfigError if an invalid resource is specified

* Require Jinja 2.9 for the consent resource

* changelog
2019-01-07 10:14:31 +00:00
Michael Telatynski
a27e501b09 fix the check for whether is_url to match all the other ones in codebase (#3405)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2019-01-06 21:43:25 +00:00
Richard van der Hoff
29f20a8a1a
Update debian Conflicts specifications (#4349)
...  to allow installation alongside our matrix-synapse transitional package.
2019-01-04 17:24:13 +00:00
Matthew Hodgson
27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Aaron Raimist
37f8bdc1d5 Update README to not lie about required restart (#4343)
* Update README to not lie about required restart

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-02 16:32:37 +00:00
Neil Johnson
84b6fae1f5 Ensure synchrotrons can access is_support_user in the storage layer 2019-01-02 10:19:59 +00:00
Aaron Raimist
252c0c81fa Update PR template to use absolute links (#4341)
* Update PR template to use absolute links

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-02 07:33:13 +00:00
Richard van der Hoff
e9cdfedff3
Avoid packaging _trial_temp directory (#4326)
Make sure we don't put the _trial_temp directory in the package target
directory.

Fixes https://github.com/matrix-org/synapse/issues/4322
2019-01-02 07:30:31 +00:00
Richard van der Hoff
7134832c01
Install the optional dependencies into the debian package (#4325)
since #4298, the optional dependencies are no longer installed with a simple
`pip install .`, which meant that they were not being included in the debian
package.

The easy fix to that is dh_virtualenv --extras, but that needs dh_virtualenv
1.1...
2019-01-02 07:17:39 +00:00
Aaron Raimist
7975d39cbd Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-01 15:40:55 -06:00
Amber Brown
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00
Krithin Sitaram
98df67a8de Remove mention of lt-cred-mech in the sample coturn config. (#4333)
* Remove mention of lt-cred-mech in the sample coturn config.

See https://github.com/coturn/coturn/pull/262 for more context.
Also clean up some minor formatting issues while I'm here.

* Add changelog.

Signed-off-by: Krithin Sitaram <krithin@gmail.com>
2018-12-28 23:31:49 +00:00
Will Hunt
0708f437cc Log roomid along with Unknown room (#4297) 2018-12-24 10:49:42 +01:00
Richard van der Hoff
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms
Add v2 room version
2018-12-24 10:48:59 +01:00
Richard van der Hoff
f85676cc93
Return well_known in /login response (#4319)
... as per MSC1730.
2018-12-24 10:44:33 +01:00
Richard van der Hoff
3355c1a9ec
Merge pull request #4317 from de-vri-es/test-metric-prometheus-0.5
Fix test_metrics.py compatibility prometheus_client 0.5
2018-12-23 00:14:47 +01:00
Richard van der Hoff
597dafb5e8
Merge pull request #4316 from matrix-org/rav/fix_docker_upload
Fix circleci config for synapse:latest docker upload
2018-12-21 17:33:34 +01:00
Richard van der Hoff
9c2af7b2c5 Add a script to generate a clean config file (#4315) 2018-12-22 02:04:57 +11:00
Richard van der Hoff
f3561f8d86 Fix indentation in default config (#4313)
These settings are not supposed to be under 'listeners'.
2018-12-22 02:01:43 +11:00
Amber Brown
ea6abf6724
Fix IP URL previews on Python 3 (#4215) 2018-12-22 01:56:13 +11:00
Amber Brown
c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools (#4298) 2018-12-22 01:37:26 +11:00
Maarten de Vries
48b7ff7a35 Fix test_metrics.py compatibility prometheus_client 0.5
prometheus_client 0.5 has a named-tuple Sample type with more member
than the old plain tuple had. This commit makes sure the unit test
detects this and changes the way it reads the sample.

Signed-off-by: Maarten de Vries <maarten@de-vri.es>
2018-12-21 01:53:57 +01:00
Richard van der Hoff
d9aaf26539 Fix circleci config for synapse:latest docker upload
Give the image the right tag, so that we can push it.
2018-12-21 00:22:31 +00:00
Nathan Pennie
da8628ba2c
Create 4309.bugfix 2018-12-19 14:42:49 -05:00
Erik Johnston
1dc7492ce5 Newsfile 2018-12-18 18:11:56 +00:00
Neil Johnson
7e22cd90f5
ensure can report mau stats when hs.config.mau_stats_only is set (#4305)
* ensure can report mau stats when hs.config.mau_stats_only is set
2018-12-18 14:36:11 +00:00
Neil Johnson
d2f7c4e6b1
create support user (#4141)
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Amber Brown
e93a0ebf50
Settings Fix deleting e2e room keys on xenial (#4295) 2018-12-14 05:10:31 +11:00
Nad Chishtie
cef8ae272a Improved welcome page (#4294)
* Improved static/index.html look, feel and content.

* Added accessibility tags to in-line SVG.

* Grammar.
2018-12-14 02:42:51 +11:00
Richard van der Hoff
a83826ae99
Merge pull request #4274 from matrix-org/michaelkaye/update_kernel_dco_link
Update link to kernel.org DCO usage
2018-12-11 20:55:43 +01:00
Richard van der Hoff
f208f608cb Merge branch 'release-v0.34.0' into develop 2018-12-11 15:43:20 +00:00
Richard van der Hoff
eb835bf65b 0.34.0rc2 2018-12-11 15:35:47 +00:00
Richard van der Hoff
03116da984 remove changelog files 2018-12-11 15:35:10 +00:00
Richard van der Hoff
188945713e
Merge pull request #4290 from matrix-org/rav/remove_webclient
Stop installing Matrix Console by default
2018-12-11 16:24:15 +01:00
Richard van der Hoff
bd52978fd7 Merge branch 'rav/disable_pager_in_ci' into rav/welcome_page 2018-12-11 14:13:02 +00:00
Richard van der Hoff
2755a0d48a Disable pager for ci script
... otherwise it hangs
2018-12-11 14:12:22 +00:00
Richard van der Hoff
f537432ef9 Add a welcome page to the static resources
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.

The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.

It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.

It's based on the work done by @krombel for #2601.
2018-12-11 13:26:22 +00:00
Richard van der Hoff
df96177ca7 Stop installing Matrix Console by default
This is based on the work done by @krombel in #2601.
2018-12-11 13:20:33 +00:00
David Baker
89ac2a5bdb Add 'sandbox' to CSP for media repo (#4284)
* Add 'sandbox' to the CSP for media repo

* Changelog
2018-12-11 04:05:02 +11:00
Will Hunt
989f1167af
Merge pull request #4279 from matrix-org/hs/fix-config-cors
Make /config more CORS-y
2018-12-10 15:24:03 +00:00
Erik Johnston
74c3606c53 Newsfile 2018-12-10 14:15:43 +00:00
Will Hunt
91206e09f2 changelog & isort 2018-12-09 17:39:44 +00:00
Michael Kaye
912a843294
changelog 2018-12-07 17:50:52 +00:00
Richard van der Hoff
30da50a5b8
Initialise user displayname from SAML2 data (#4272)
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
Richard van der Hoff
35e13477cf
Update the example systemd config to use a virtualenv (#4273)
If you're installing as a system package, the system package should have set up
the systemd config, so it's more useful to give an example of running in a
virtualenv here.
2018-12-07 14:43:41 +01:00
Richard van der Hoff
c7401a697f
Implement SAML2 authentication (#4267)
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.

What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
2018-12-07 13:11:11 +01:00
Richard van der Hoff
c588b9b9e4
Factor SSO success handling out of CAS login (#4264)
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
rkfg
ae19a7db8c Prevent crash on pagination. 2018-12-06 13:32:05 +03:00
Richard van der Hoff
b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265)
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
Richard van der Hoff
9a3e24a13d drop undocumented dependency on dateutil (#4266)
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 04:52:42 +11:00
Richard van der Hoff
e8d98466b0
Implement .well-known handling (#4262)
Sometimes it's useful for synapse to generate its own .well-known file.
2018-12-05 14:38:58 +01:00
Richard van der Hoff
a2ed0f287e
Merge pull request #4260 from matrix-org/rav/python3
Notes on upgrading to python3, and README updates.
2018-12-04 14:46:31 +01:00
Richard van der Hoff
75937e9033 Remove obsolete settings from docker homeserver.yaml
These aren't used, because we have a `log_config` setting.
2018-12-04 12:31:00 +00:00
Richard van der Hoff
4acd1a3549 Notes on upgrading to python3, and README updates. 2018-12-04 12:28:24 +00:00
Travis Ralston
1737753a62 Add an option to enable recording IPs for appservice users (#3831) 2018-12-04 12:44:41 +01:00
Amber Brown
fd96dd75a3 Fix non-ASCII pushrules (#4248) 2018-12-04 12:44:02 +01:00
Travis Ralston
158ffb92f1 Add an option to disable search for homeservers which may not be interested in it (#4230)
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
Aaron Raimist
512e94d230 Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers (#4224)
* Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-04 11:59:09 +01:00
Ben Parsons
b5ac0ffa0a add more detail to logging regarding "More than one row matched" error (#4234) 2018-12-04 11:57:39 +01:00
Richard van der Hoff
ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english (#4253)
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
2018-12-04 11:55:52 +01:00
Richard van der Hoff
f144c0a210
Merge pull request #4244 from aaronraimist/drop-sent-txt
Drop sent_transactions
2018-12-04 11:41:28 +01:00
Richard van der Hoff
48972ce9d1
Patch defer.inlineCallbacks to check logcontexts in tests (#4205) 2018-12-04 11:30:32 +01:00
Richard van der Hoff
a077e710a3
Merge pull request #4250 from matrix-org/hawkowl/pusher-remove-py3
Fix removing pushers on python 3
2018-12-04 11:22:46 +01:00
Richard van der Hoff
a484735bb0
Merge pull request #4257 from aaronraimist/add-editorconfig
Add a basic .editorconfig
2018-12-04 11:10:02 +01:00
Richard van der Hoff
52e87fbfbe
Run the AS senders as background processes (#4189)
This should fix some "Starting db connection from sentinel context" warnings,
and will mean we get metrics for these processes.
2018-12-04 10:53:49 +01:00
Aaron Raimist
3518c28aa8
Add a basic .editorconfig
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-03 22:38:47 -06:00
Amber Brown
998ba41493 changelog 2018-12-03 22:28:12 +11:00
Richard van der Hoff
c03324294d Workaround for non-ascii event ids (#4241)
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.

Fixes #4226
2018-12-03 21:47:48 +11:00
Aaron Raimist
44dc4c365b
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:10:21 -06:00
Neil Johnson
7039ece8fb Neilj/fix autojoin (#4223)
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
Amber Brown
8ca53fb53e
Report combined coverage to codecov (#4225) 2018-11-28 20:59:31 +11:00
Neil Johnson
f9b136a886 Neilj/fix mau initial reserved users (#4211)
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail

* towncrier

* white space
2018-11-28 20:33:41 +11:00