Commit Graph

9492 Commits

Author SHA1 Message Date
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze
Analyze user_ips before running deduplication
2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe
Reduce user_ips bloat during dedupe background update
2019-02-12 13:02:58 +00:00
Erik Johnston
495ea92350 Fix pep8 2019-02-12 12:40:42 +00:00
Erik Johnston
483ba85c7a Analyze user_ips before running deduplication
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
Erik Johnston
362d80b770 Reduce user_ips bloat during dedupe background update
The background update to remove duplicate rows naively deleted and
reinserted the duplicates. For large tables with a large number of
duplicates this causes a lot of bloat (with postgres), as the inserted
rows are appended to the table, since deleted rows will not be
overwritten until a VACUUM has happened.

This should hopefully also help ensure that the query in the last batch
uses the correct index, as inserting a large number of new rows without
analyzing will upset the query planner.
2019-02-12 11:39:34 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings
fix self-signed cert notice from generate-config
2019-02-12 11:24:45 +00:00
Richard van der Hoff
a4ce91396b
Disable TLS by default (#4614) 2019-02-12 10:52:08 +00:00
Richard van der Hoff
32b781bfe2
Fix error when loading cert if tls is disabled (#4618)
If TLS is disabled, it should not be an error if no cert is given.

Fixes #4554.
2019-02-12 10:51:31 +00:00
Richard van der Hoff
dfc846a316 fix self-signed cert notice from generate-config
fixes #4620
2019-02-12 10:37:59 +00:00
Richard van der Hoff
0ca2908653 fix tests 2019-02-11 22:01:27 +00:00
Richard van der Hoff
4fddf8fc77 Infer no_tls from presence of TLS listeners
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
2019-02-11 21:39:14 +00:00
Richard van der Hoff
15272f837c Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work 2019-02-11 21:34:19 +00:00
Richard van der Hoff
9645728619 Don't create server contexts when TLS is disabled
we aren't going to use them anyway.
2019-02-11 21:32:01 +00:00
Richard van der Hoff
be794c7cf7 Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work 2019-02-11 21:16:00 +00:00
Richard van der Hoff
2129dd1a02 Fail cleanly if listener config lacks a 'port'
... otherwise we would fail with a mysterious KeyError or something later.
2019-02-11 21:15:01 +00:00
Richard van der Hoff
086f6f27d4 Logging improvements around TLS certs
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
2019-02-11 21:02:06 +00:00
Richard van der Hoff
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates (#4611)
It's nothing to do with refreshing the certificates. No idea why it was here.
2019-02-11 18:03:30 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating
add updating of backup versions
2019-02-11 13:45:49 +00:00
Richard van der Hoff
24b7f3916d
Clean up default listener configuration (#4586)
Rearrange the comments to try to clarify them, and expand on what some of it
means.

Use a sensible default 'bind_addresses' setting.

For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
2019-02-11 12:50:30 +00:00
Andrew Morgan
a126f86eec Transfer Server ACLs on room upgrade 2019-02-11 11:30:37 +00:00
Amber Brown
6e2a5aa050 ACME Reprovisioning (#4522) 2019-02-11 10:36:26 +00:00
Amber Brown
4ffd10f46d Be tolerant of blank TLS fingerprints config (#4589) 2019-02-11 10:04:27 +00:00
Erik Johnston
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Valentin Anger
2dc2b6e9f1 Allow "unavailable" presence status for /sync (#4592)
* Allow "unavailable" presence status for /sync

Closes #3772, closes #3779

Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>

* Add changelog for PR 4592
2019-02-08 21:09:56 +00:00
Richard van der Hoff
56710c7df5
Fix 'no unique or exclusion constraint' error (#4591)
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
2019-02-08 18:30:46 +00:00
Amber Brown
9cd33d2f4b
Deduplicate some code in synapse.app (#4567) 2019-02-08 17:25:57 +00:00
Hubert Chathi
afae8442b5 make sure version is in body and wrap in linearizer queue
also add tests
2019-02-08 01:32:45 -05:00
Hubert Chathi
d9e424bf64 re-try to make isort happy 2019-02-06 22:18:41 -05:00
Hubert Chathi
9ff620a518 fix import to make isort happy 2019-02-06 21:32:52 -05:00
Hubert Chathi
8248637173 add new endpoint to update backup versions 2019-02-06 17:57:10 -05:00
Hubert Chathi
664c81e8b7 return proper error codes for some 404s 2019-02-06 17:47:22 -05:00
Richard van der Hoff
2475434080 Merge branch 'master' into develop 2019-02-05 18:44:49 +00:00
Richard van der Hoff
3bd9daf4b8 v0.99.0 2019-02-05 18:33:02 +00:00
Andrew Morgan
627ecd358e Filter user directory state query to a subset of state events (#4462)
* Filter user directory state query to a subset of state events

* Add changelog
2019-02-05 12:16:28 +00:00
Richard van der Hoff
bf1e4d96ad
Fix default ACME config for py2 (#4564)
Fixes #4559
2019-02-05 11:37:33 +00:00
Richard van der Hoff
d7e27a1f08
fix typo in config comments (#4557) 2019-02-05 11:32:45 +00:00
Richard van der Hoff
30fd2f89db 0.99.0rc4 2019-02-01 15:52:28 +00:00
Richard van der Hoff
ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
2019-02-01 14:37:22 +00:00
Richard van der Hoff
f8db967d5a Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-02-01 13:20:15 +00:00
Richard van der Hoff
fa794980ec
Merge pull request #4544 from matrix-org/rav/skip_invalid_well_known
Treat an invalid .well-known the same as an absent one
2019-02-01 13:18:36 +00:00
Richard van der Hoff
e9779a6f8f Fix b'ab' noise in logs 2019-02-01 12:34:31 +00:00
Richard van der Hoff
9763a73af0 Merge branch 'release-v0.99.0' into develop 2019-02-01 12:30:22 +00:00
Richard van der Hoff
f0ba34f581 Fix noisy "twisted.internet.task.TaskStopped" errors in logs
Fixes #4003
2019-02-01 12:22:57 +00:00
Richard van der Hoff
8a21b03fba Treat an invalid .well-known the same as an absent one
... basically, carry on and fall back to SRV etc.
2019-02-01 11:37:31 +00:00
Richard van der Hoff
3c8a41140e Cache failures to parse .well-known
Also add a Measure block around the .well-known fetch
2019-02-01 00:37:52 +00:00
Richard van der Hoff
24d59c7568 better logging for federation connections 2019-01-31 23:18:20 +00:00
Richard van der Hoff
d428b46346 Update federation routing logic to check .well-known before SRV 2019-01-31 23:14:18 +00:00
Andrew Morgan
c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
Richard van der Hoff
625385d684 Merge branch 'release-v0.99.0' into develop 2019-01-31 18:43:20 +00:00
Richard van der Hoff
85129d7068 v0.99.0rc3 2019-01-31 18:35:38 +00:00
Andrew Morgan
d239f67c25 Raise an exception instead of returning None 2019-01-31 18:34:15 +00:00
Richard van der Hoff
07dfe148de
Add some debug for membership syncing issues (#4538)
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan
3ed3cb4339 New function for getting room's create event 2019-01-31 18:21:39 +00:00
Andrew Morgan
bbb97a35fd Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable 2019-01-31 18:11:00 +00:00
Richard van der Hoff
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room (#4535) 2019-01-31 15:34:17 +00:00
Andrew Morgan
563f6a832b Reject large transactions on federation (#4513)
* Reject large transactions on federation

* Add changelog

* lint

* Simplify large transaction handling
2019-01-31 11:44:04 +00:00
Andrew Morgan
fb50934b8f lint 2019-01-31 11:34:45 +00:00
Richard van der Hoff
6fba9fd20c Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-01-30 17:02:32 +00:00
Andrew Morgan
d621c5562e Copy over non-federatable trait on room upgrade 2019-01-30 16:33:51 +00:00
Richard van der Hoff
b8b898666e v0.99.0rc2 2019-01-30 16:31:07 +00:00
Matthew Hodgson
ad7ac8853c by default include m.room.encryption on invites (#3902)
* by default include m.room.encryption on invites

* fix constant

* changelog
2019-01-30 16:26:13 +00:00
Richard van der Hoff
c74b96755c Merge remote-tracking branch 'origin/develop' into release-v0.99.0 2019-01-30 16:23:28 +00:00
Neil Johnson
81b7e7eed3
Update constants.py
remove trailing ,
2019-01-30 16:11:36 +00:00
Erik Johnston
ed8c5e4cda Fix remote invite rejections not comming down sync
This was broken in PR #4405, commit 886e5ac, where we changed remote
rejections to be outliers.

The fix is to explicitly add the leave event in when we know its an out
of band invite. We can't always add the event as if the server is/was in
the room there might be more events to send down the sync than just the
leave.
2019-01-30 15:46:27 +00:00
Richard van der Hoff
a5d0c771a3 0.99.0rc1 2019-01-30 15:11:18 +00:00
Erik Johnston
6587b0b89b
Merge pull request #4472 from matrix-org/neilj/room_capabilities
Server capabilities support
2019-01-30 14:26:56 +00:00
Erik Johnston
a4f52a33fe Fix replication for room v3 (#4523)
* Fix replication for room v3

We were not correctly quoting the path fragments over http replication,
which meant that it exploded when the event IDs had a slash in them

* Newsfile
2019-01-30 14:19:52 +00:00
Richard van der Hoff
7615a8ced1 ACME config cleanups (#4525)
* Handle listening for ACME requests on IPv6 addresses

the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.

* Clean up the default ACME config

make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.

* newsfile
2019-01-30 14:17:55 +00:00
Erik Johnston
e87d7a4b0f Raise ConfigError instead 2019-01-30 12:48:09 +00:00
Erik Johnston
270f212a2a _listener_http should return a list 2019-01-30 12:14:50 +00:00
Richard van der Hoff
a79034aedf
Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups
Tiny .well-known fixes
2019-01-30 11:47:24 +00:00
Richard van der Hoff
c7b24ac3d0
Follow redirects on .well-known (#4520) 2019-01-30 11:43:33 +00:00
Amber Brown
f6813919e8
SIGHUP for TLS cert reloading (#4495) 2019-01-30 11:00:02 +00:00
Richard van der Hoff
09a1a6b55e fix exception text 2019-01-30 10:58:52 +00:00
Richard van der Hoff
928c50b59a Also jitter the invalid cache period 2019-01-30 10:58:52 +00:00
Neil Johnson
b37e8c9572 Merge branch 'neilj/room_capabilities' of github.com:matrix-org/synapse into neilj/room_capabilities 2019-01-30 10:56:47 +00:00
Neil Johnson
f834d98402 isort 2019-01-30 10:55:42 +00:00
Richard van der Hoff
bc5f6e1797
Add a caching layer to .well-known responses (#4516) 2019-01-30 10:55:25 +00:00
Amber Brown
3f189c902e
Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Neil Johnson
ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities 2019-01-30 10:28:08 +00:00
Neil Johnson
c5a0f82cca define room dispositions for use in exposing room capabilities 2019-01-30 10:24:24 +00:00
Neil Johnson
9c850d9d5e formatting and use constants where available 2019-01-30 10:23:26 +00:00
Neil Johnson
2f46804055 Populate default room version from Constants 2019-01-30 09:39:10 +00:00
Neil Johnson
c7837dce24 reflect that rooms v3 is a stable room version 2019-01-30 09:33:30 +00:00
Richard van der Hoff
457fbfaf22
Merge pull request #4486 from xperimental/workaround-4216
Implement workaround for login error.
2019-01-30 07:06:11 +00:00
Robert Jacob
2a7f0b8953 Implement workaround for login error.
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
2019-01-30 01:06:39 +01:00
Erik Johnston
0b24d58e05 No vdh tests! 2019-01-29 23:11:48 +00:00
Erik Johnston
a1b0e1879b Enable room version v3 2019-01-29 23:09:10 +00:00
Erik Johnston
67b82f1336 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:58:38 +00:00
Erik Johnston
afeea319df Fixup comment 2019-01-29 22:55:29 +00:00
Erik Johnston
7740eddd04
Merge pull request #4514 from matrix-org/erikj/remove_event_id
Remove usages of event ID's domain
2019-01-29 22:54:25 +00:00
Erik Johnston
ff2f65d737 Update comment 2019-01-29 22:35:36 +00:00
Erik Johnston
655ce037fd check event format version not room version 2019-01-29 22:33:43 +00:00
Erik Johnston
f46a818ce5 kill vdh test some more 2019-01-29 22:02:58 +00:00
Erik Johnston
a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston
c21b7cbc09
Update synapse/storage/events_worker.py 2019-01-29 21:53:48 +00:00
Erik Johnston
4db252c073 Check redaction state when event is pulled out of the database 2019-01-29 21:48:36 +00:00
Erik Johnston
6d23ec2111 Fix typo 2019-01-29 21:45:53 +00:00
Erik Johnston
b5d510ad64 Remove unused arg 2019-01-29 21:45:28 +00:00