Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-10-01 08:25:44 -04:00
Code Issues Packages Projects Releases Wiki Activity
13,572 Commits 4 Branches 609 Tags 60 MiB
4abc988c6a
Commit Graph

9640 Commits

Author SHA1 Message Date
Erik Johnston
9342cc6ab1 Add comments and paranoia 2019-02-25 10:02:12 +00:00
Erik Johnston
47a7e3928d Correctly proxy exception in frontend_proxy worker 2019-02-23 15:17:57 +00:00
Erik Johnston
41285ffe5b Handle errors when fetching remote server keys 2019-02-23 15:09:39 +00:00
Erik Johnston
59e0112209 MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API 2019-02-23 14:50:43 +00:00
Erik Johnston
d14e94bae4
Merge pull request #4716 from matrix-org/erikj/pusher_logging 
Fix up pusher logging a bit
 2019-02-22 15:29:16 +00:00
Erik Johnston
b82c9cf462 Add missing return 2019-02-22 15:27:40 +00:00
Erik Johnston
f2891d2487 Correctly handle PusherConfigException 2019-02-22 15:18:19 +00:00
Erik Johnston
a164134a53 Drop logging level of creating a pusher 2019-02-22 14:48:06 +00:00
Erik Johnston
1d9df51ff1 Correctly handle null data in HttpPusher 2019-02-22 14:47:48 +00:00
Erik Johnston
80467bbac3 Fix state cache invalidation on workers 2019-02-22 14:38:14 +00:00
Erik Johnston
7b288826b7 Fix backfill storing incorrect state for events 2019-02-22 11:33:51 +00:00
Richard van der Hoff
e07384c4e1
Add prometheus metrics for number of badge update pushes. (#4709) 
We're counting the number of push notifications, but not the number of badges;
I'd like to see if they are significant.
 2019-02-22 10:57:15 +00:00
Richard van der Hoff
0abb094f1a
bail out early in on_new_receipts if no pushers (#4706) 2019-02-21 17:51:21 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job (#4707) 
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
 2019-02-21 17:50:30 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop 
Prevent crash on pagination.
 2019-02-21 17:42:15 +00:00
Erik Johnston
71b625d808 Stop backpaginating when events not visible 2019-02-20 18:14:12 +00:00
Erik Johnston
82fca11fc1
Merge pull request #4694 from matrix-org/erikj/fix_sentry_config_format 
Fixup generated metrics config
 2019-02-20 14:13:38 +00:00
Richard van der Hoff
82ca6d1f9f
Add metrics for number of outgoing EDUs, by type (#4695) 2019-02-20 14:13:14 +00:00
Erik Johnston
3d672fec51 Fixup generated metrics config 2019-02-20 13:39:37 +00:00
Amber Brown
a06614bd2a
UPSERT many functionality (#4644) 2019-02-20 23:03:30 +11:00
Richard van der Hoff
b2200a8690
Logging improvements for the pusher (#4691) 2019-02-20 11:36:50 +00:00
Richard van der Hoff
c88bc53903
Fix TaskStopped exceptions when outbound requests time out (#4690) 2019-02-20 11:35:52 +00:00
Richard van der Hoff
86920ac266
Merge pull request #4678 from matrix-org/rav/tls_install_instructions 
Attempt to clarify installation/config instructions
 2019-02-20 11:10:33 +00:00
Erik Johnston
dbdc565dfd Fix registration on workers (#4682) 
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
 2019-02-20 18:47:31 +11:00
Richard van der Hoff
5f9bdf90fe Attempt to make default config more consistent 
The general idea here is that config examples should just have a hash and no
extraneous whitespace, both to make it easier for people who don't understand
yaml, and to make the examples stand out from the comments.
 2019-02-19 13:54:29 +00:00
Erik Johnston
c003450057
Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation 
Batch cache invalidation over replication
 2019-02-19 13:14:30 +00:00
Erik Johnston
62175a20e5 Docs 2019-02-19 11:38:40 +00:00
Erik Johnston
1bb35e3a83 Use itertools 2019-02-19 11:34:40 +00:00
Erik Johnston
bc8fa1509d Documentation 2019-02-19 11:24:59 +00:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated 
Support .well-known delegation when issuing certificates through ACME
 2019-02-19 11:15:38 +00:00
Brendan Abolivier
5a707a2f9a Improve config documentation 2019-02-19 10:59:26 +00:00
Brendan Abolivier
a8626901cd Fetch ACME domain into an instance member 2019-02-19 10:54:33 +00:00
Erik Johnston
32590b7139
Merge pull request #4670 from matrix-org/erikj/register_login_split 
Split /login into client_reader
 2019-02-19 10:46:22 +00:00
Richard van der Hoff
7c70b8f8a6
Try and make TLS federation client code faster (#4674) 
* fix to use makeContext so that we don't need to rebuild the certificateoptions each time
 2019-02-19 10:19:16 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade 
Transfer bans on room upgrade
 2019-02-19 10:16:42 +00:00
Amber Brown
561eebe170 fix to use makeContext so that we don't need to rebuild the certificateoptions each time 2019-02-19 16:18:05 +11:00
Andrew Morgan
34ac75ce2c lint 2019-02-18 18:23:37 +00:00
Erik Johnston
a9b5ea6fc1 Batch cache invalidation over replication 
Currently whenever the current state changes in a room invalidate a lot
of caches, which cause *a lot* of traffic over replication. Instead,
lets batch up all those invalidations and send a single poke down
the replication streams.

Hopefully this will reduce load on the master process by substantially
reducing traffic.
 2019-02-18 17:53:31 +00:00
Andrew Morgan
f8b9ca53ce Move member event processing and changelog fix 2019-02-18 17:28:52 +00:00
Erik Johnston
d154f5a055
Merge pull request #4632 from matrix-org/erikj/basic_sentry 
Add basic optional sentry.io integration
 2019-02-18 17:22:45 +00:00
Erik Johnston
4cc4400b4d Split /login into client_reader 2019-02-18 17:19:01 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split 
Split out registration to worker
 2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions 
Correctly handle HttpResponseException when handling device updates
 2019-02-18 16:55:43 +00:00
Erik Johnston
d328a93b51 Fixup error handling and message 2019-02-18 16:53:56 +00:00
Erik Johnston
af691e415c Move register_device into handler 2019-02-18 16:49:38 +00:00
Erik Johnston
028267acd2
Merge pull request #4669 from matrix-org/erikj/log_exception 
Cleanup top level request exception logging
 2019-02-18 16:41:41 +00:00
Erik Johnston
d08bac4136
Merge pull request #4651 from matrix-org/matthew/well-known-cors 
set CORS on .well-known URI to unbreak modular
 2019-02-18 16:38:11 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers 
Fix kicking guest users in worker mode
 2019-02-18 15:55:48 +00:00
Erik Johnston
092b541401 Fixup 2019-02-18 15:52:26 +00:00
Brendan Abolivier
45bb55c6de Use a configuration parameter to give the domain to generate a certificate for 2019-02-18 15:46:23 +00:00
Erik Johnston
8b9ae6d3a6 Update docs 2019-02-18 15:26:13 +00:00
Erik Johnston
94960cef03 pep8 2019-02-18 15:24:13 +00:00
Erik Johnston
fe725f7e45 Cleanup top level request exception logging 
Firstly, we always logged that the request was being handled via
`JsonResource._async_render`, so we change that to use the servlet name
we add to the request.

Secondly, we pass the exception information to the logger rather than
formatting it manually. This makes it consistent with other exception
logging, allwoing logging hooks and formatters to access the exception
information.
 2019-02-18 15:11:04 +00:00
Matthew Hodgson
04dad5ac16 fix missig import 2019-02-18 15:59:23 +01:00
Erik Johnston
e07cc31cb8 Correctly handle HttpResponseException 2019-02-18 14:55:09 +00:00
Brendan Abolivier
68a53f825f Merge branch 'develop' into babolivier/acme-delegated 2019-02-18 14:52:23 +00:00
Erik Johnston
32e54b472a Fix kicking guest users in worker mode 
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.

Presumably broken in #4141
 2019-02-18 14:16:07 +00:00
Andrew Morgan
915421065b Membership events are done later 2019-02-18 14:02:09 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions 
Correctly handle RequestSendFailed exceptions
 2019-02-18 13:53:24 +00:00
Andrew Morgan
9caab0c364 Transfer bans on room upgrade 2019-02-18 13:52:51 +00:00
Erik Johnston
dc5efc92a8 Fixup 2019-02-18 13:52:49 +00:00
Erik Johnston
eb2b8523ae Split out registration to worker 
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.

Note: due to the in-memory session map all registration requests must be
handled by the same worker.
 2019-02-18 12:12:57 +00:00
Andrew Morgan
6d02a13d81
Typo in info log 
Co-Authored-By: babolivier <contact@brendanabolivier.com>
 2019-02-18 11:36:34 +00:00
Juuso "Linda" Lapinlampi
68d2869c8d config: Remove a repeated word from a logger warning 
The warning for missing macaroon_secret_key was "missing missing".
 2019-02-15 22:24:53 -07:00
Erik Johnston
bd4505f765
Merge pull request #4647 from matrix-org/erikj/add_room_publishing_rules 
Add configurable room list publishing rules
 2019-02-15 22:11:01 +00:00
Brendan Abolivier
f86b695cbd
Various cosmetics to make TravisCI happy 2019-02-15 12:29:34 +00:00
Brendan Abolivier
af8a2f679b
Remove unused import 2019-02-15 12:27:43 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Erik Johnston
b99c532c1c Move defaults up into code 2019-02-15 10:53:39 +00:00
Erik Johnston
02c729d6b0 Hoist up checks to reduce overall work 2019-02-15 10:20:02 +00:00
Erik Johnston
02c46acc6a Fixup comments 2019-02-15 10:17:13 +00:00
Matthew Hodgson
bfcefbb230 tabs 2019-02-14 19:53:12 +00:00
Matthew Hodgson
6f47bc3fb2 set CORS on .well-known URI to unbreak modular 
otherwise a riot/web running on foo.riot.im can't query
the .well-known on foo.modular.im...
 2019-02-14 19:51:41 +00:00
Erik Johnston
8e32f26cb8 Clarify comments 2019-02-14 18:21:24 +00:00
Erik Johnston
cb12a37708 Clarify and fix behaviour when there are multiple aliases 2019-02-14 18:16:32 +00:00
Erik Johnston
f61b2068e6 Only fetch aliases when publishing rooms 2019-02-14 18:08:09 +00:00
Erik Johnston
f666fe36d7 Fixup comments 2019-02-14 18:07:24 +00:00
Richard van der Hoff
bf4fd14806 Merge branch 'master' into develop 2019-02-14 17:34:09 +00:00
Richard van der Hoff
f595d6ac57 0.99.1.1 2019-02-14 17:20:02 +00:00
Richard van der Hoff
f311018823
Fix errors in acme provisioning (#4648) 
* Better logging for errors on startup

* Fix "TypeError: '>' not supported" when starting without an existing
  certificate
* Fix a bug where an existing certificate would be reprovisoned every day
 2019-02-14 17:10:36 +00:00
Erik Johnston
eaf4d11af9 Add configurable room list publishing rules 
This allows specifying who and what is allowed to be published onto the
public room list
 2019-02-14 16:02:23 +00:00
Richard van der Hoff
b02465b9db Merge branch 'master' into develop 2019-02-14 14:42:03 +00:00
Richard van der Hoff
06cd757ae7 0.99.1 2019-02-14 14:24:24 +00:00
Erik Johnston
7fc1196a36 Correctly handle RequestSendFailed exceptions 
This mainly reduces the number of exceptions we log.
 2019-02-14 14:01:04 +00:00
Erik Johnston
6cb415b63f Fixup comments and add warning 2019-02-13 16:15:11 +00:00
Richard van der Hoff
c6e75c9f2d
Merge pull request #4450 from 14mRh4X0r/fix-dependency-message 
Fix error message for optional dependencies
 2019-02-13 16:12:49 +00:00
Richard van der Hoff
3bc238629e 0.99.1rc2 2019-02-13 14:46:18 +00:00
Richard van der Hoff
c1dfd6a18a Merge remote-tracking branch 'origin/release-v0.99.1' into develop 2019-02-13 14:27:45 +00:00
Erik Johnston
309f3bb322
Update synapse/app/_base.py 
Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>
 2019-02-13 13:24:27 +00:00
Amber Brown
bb4fd8f927
Run black on user directory code (#4635) 2019-02-13 23:05:32 +11:00
Richard van der Hoff
767686af48 Use listen_tcp for the replication listener 
Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with
what we do elsewhere.
 2019-02-13 11:59:04 +00:00
Richard van der Hoff
2a5a15aff8 Improve logging around listening services 
I wanted to bring listen_tcp into line with listen_ssl in terms of returning a
list of ports, and wanted to check that was a safe thing to do - hence the
logging in `refresh_certificate`.

Also, pull the 'Synapse now listening' message up to homeserver.py, because it
was being duplicated everywhere else.
 2019-02-13 11:58:54 +00:00
Richard van der Hoff
e3a0300431 Special-case the default bind_addresses for metrics listener 
turns out it doesn't really support ipv6, so let's hack around that by only
listening on ipv4 by default.
 2019-02-13 11:48:56 +00:00
Erik Johnston
93f7d2df3e Comments 2019-02-12 16:03:40 +00:00
Erik Johnston
6a8f902edb Raise an appropriate error message if sentry_sdk missing 2019-02-12 16:01:41 +00:00
Erik Johnston
ef2228c890 Basic sentry integration 2019-02-12 13:55:58 +00:00
Erik Johnston
d2fa7b7e99 Update changelog and version 2019-02-12 13:22:25 +00:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade 
Transfer Server ACLs on room upgrade
 2019-02-12 13:20:06 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze 
Analyze user_ips before running deduplication
 2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe 
Reduce user_ips bloat during dedupe background update
 2019-02-12 13:02:58 +00:00
Erik Johnston
495ea92350 Fix pep8 2019-02-12 12:40:42 +00:00
Erik Johnston
483ba85c7a Analyze user_ips before running deduplication 
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
 2019-02-12 11:55:27 +00:00
Erik Johnston
362d80b770 Reduce user_ips bloat during dedupe background update 
The background update to remove duplicate rows naively deleted and
reinserted the duplicates. For large tables with a large number of
duplicates this causes a lot of bloat (with postgres), as the inserted
rows are appended to the table, since deleted rows will not be
overwritten until a VACUUM has happened.

This should hopefully also help ensure that the query in the last batch
uses the correct index, as inserting a large number of new rows without
analyzing will upset the query planner.
 2019-02-12 11:39:34 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings 
fix self-signed cert notice from generate-config
 2019-02-12 11:24:45 +00:00
Richard van der Hoff
a4ce91396b
Disable TLS by default (#4614) 2019-02-12 10:52:08 +00:00
Richard van der Hoff
32b781bfe2
Fix error when loading cert if tls is disabled (#4618) 
If TLS is disabled, it should not be an error if no cert is given.

Fixes #4554.
 2019-02-12 10:51:31 +00:00
Richard van der Hoff
dfc846a316 fix self-signed cert notice from generate-config 
fixes #4620
 2019-02-12 10:37:59 +00:00
Richard van der Hoff
0ca2908653 fix tests 2019-02-11 22:01:27 +00:00
Richard van der Hoff
4fddf8fc77 Infer no_tls from presence of TLS listeners 
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
 2019-02-11 21:39:14 +00:00
Richard van der Hoff
15272f837c Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work 2019-02-11 21:34:19 +00:00
Richard van der Hoff
9645728619 Don't create server contexts when TLS is disabled 
we aren't going to use them anyway.
 2019-02-11 21:32:01 +00:00
Richard van der Hoff
be794c7cf7 Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work 2019-02-11 21:16:00 +00:00
Richard van der Hoff
2129dd1a02 Fail cleanly if listener config lacks a 'port' 
... otherwise we would fail with a mysterious KeyError or something later.
 2019-02-11 21:15:01 +00:00
Richard van der Hoff
086f6f27d4 Logging improvements around TLS certs 
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
 2019-02-11 21:02:06 +00:00
Richard van der Hoff
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates (#4611) 
It's nothing to do with refreshing the certificates. No idea why it was here.
 2019-02-11 18:03:30 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating 
add updating of backup versions
 2019-02-11 13:45:49 +00:00
Richard van der Hoff
24b7f3916d
Clean up default listener configuration (#4586) 
Rearrange the comments to try to clarify them, and expand on what some of it
means.

Use a sensible default 'bind_addresses' setting.

For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
 2019-02-11 12:50:30 +00:00
Andrew Morgan
a126f86eec Transfer Server ACLs on room upgrade 2019-02-11 11:30:37 +00:00
Amber Brown
6e2a5aa050 ACME Reprovisioning (#4522) 2019-02-11 10:36:26 +00:00
Amber Brown
4ffd10f46d Be tolerant of blank TLS fingerprints config (#4589) 2019-02-11 10:04:27 +00:00
Erik Johnston
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener 
New listener resource for the federation API "openid/userinfo" endpoint
 2019-02-11 09:44:00 +00:00
Valentin Anger
2dc2b6e9f1 Allow "unavailable" presence status for /sync (#4592) 
* Allow "unavailable" presence status for /sync

Closes #3772, closes #3779

Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>

* Add changelog for PR 4592
 2019-02-08 21:09:56 +00:00
Richard van der Hoff
56710c7df5
Fix 'no unique or exclusion constraint' error (#4591) 
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
 2019-02-08 18:30:46 +00:00
Amber Brown
9cd33d2f4b
Deduplicate some code in synapse.app (#4567) 2019-02-08 17:25:57 +00:00
Hubert Chathi
afae8442b5 make sure version is in body and wrap in linearizer queue 
also add tests
 2019-02-08 01:32:45 -05:00
Hubert Chathi
d9e424bf64 re-try to make isort happy 2019-02-06 22:18:41 -05:00
Hubert Chathi
9ff620a518 fix import to make isort happy 2019-02-06 21:32:52 -05:00
Hubert Chathi
8248637173 add new endpoint to update backup versions 2019-02-06 17:57:10 -05:00
Hubert Chathi
664c81e8b7 return proper error codes for some 404s 2019-02-06 17:47:22 -05:00
Richard van der Hoff
2475434080 Merge branch 'master' into develop 2019-02-05 18:44:49 +00:00
Richard van der Hoff
3bd9daf4b8 v0.99.0 2019-02-05 18:33:02 +00:00
Andrew Morgan
627ecd358e Filter user directory state query to a subset of state events (#4462) 
* Filter user directory state query to a subset of state events

* Add changelog
 2019-02-05 12:16:28 +00:00
Richard van der Hoff
bf1e4d96ad
Fix default ACME config for py2 (#4564) 
Fixes #4559
 2019-02-05 11:37:33 +00:00
Richard van der Hoff
d7e27a1f08
fix typo in config comments (#4557) 2019-02-05 11:32:45 +00:00
Richard van der Hoff
30fd2f89db 0.99.0rc4 2019-02-01 15:52:28 +00:00
Richard van der Hoff
ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation 
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
 2019-02-01 14:37:22 +00:00
Richard van der Hoff
f8db967d5a Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-02-01 13:20:15 +00:00
Richard van der Hoff
fa794980ec
Merge pull request #4544 from matrix-org/rav/skip_invalid_well_known 
Treat an invalid .well-known the same as an absent one
 2019-02-01 13:18:36 +00:00
Richard van der Hoff
e9779a6f8f Fix b'ab' noise in logs 2019-02-01 12:34:31 +00:00
Richard van der Hoff
9763a73af0 Merge branch 'release-v0.99.0' into develop 2019-02-01 12:30:22 +00:00
Richard van der Hoff
f0ba34f581 Fix noisy "twisted.internet.task.TaskStopped" errors in logs 
Fixes #4003
 2019-02-01 12:22:57 +00:00
Richard van der Hoff
8a21b03fba Treat an invalid .well-known the same as an absent one 
... basically, carry on and fall back to SRV etc.
 2019-02-01 11:37:31 +00:00
Richard van der Hoff
3c8a41140e Cache failures to parse .well-known 
Also add a Measure block around the .well-known fetch
 2019-02-01 00:37:52 +00:00
Richard van der Hoff
24d59c7568 better logging for federation connections 2019-01-31 23:18:20 +00:00
Richard van der Hoff
d428b46346 Update federation routing logic to check .well-known before SRV 2019-01-31 23:14:18 +00:00
Andrew Morgan
c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable 
Copy over non-federatable trait on room upgrade
 2019-01-31 20:28:37 +00:00
Richard van der Hoff
625385d684 Merge branch 'release-v0.99.0' into develop 2019-01-31 18:43:20 +00:00
Richard van der Hoff
85129d7068 v0.99.0rc3 2019-01-31 18:35:38 +00:00
Andrew Morgan
d239f67c25 Raise an exception instead of returning None 2019-01-31 18:34:15 +00:00
Richard van der Hoff
07dfe148de
Add some debug for membership syncing issues (#4538) 
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
 2019-01-31 18:30:40 +00:00