Amber Brown
f2a753ea38
Move from TravisCI to BuildKite ( #4752 )
2019-02-27 13:03:14 -08:00
Erik Johnston
76550c58d2
Merge pull request #4759 from matrix-org/erikj/3pid_client_reader
...
Move /account/3pid to client_reader
2019-02-27 16:11:21 +00:00
Erik Johnston
8267034a63
Merge pull request #4758 from matrix-org/erikj/use_presence_replication
...
When presence is disabled don't send over replication
2019-02-27 15:46:26 +00:00
Erik Johnston
54f9ce11a7
Move /account/3pid to client_reader
2019-02-27 14:26:08 +00:00
Erik Johnston
1e315017d3
When presence is enabled don't send over replication
2019-02-27 13:53:46 +00:00
Erik Johnston
4cff9376f7
Move server key queries to federation reader
2019-02-27 13:43:53 +00:00
Erik Johnston
b183fef9ac
Update comments
2019-02-27 13:06:10 +00:00
Erik Johnston
7590e9fa28
Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff
...
Fix tightloop over connecting to replication server
2019-02-27 11:00:59 +00:00
Richard van der Hoff
44a4d65586
0.99.2rc1
2019-02-27 10:48:34 +00:00
Erik Johnston
6bb1c028f1
Limit cache invalidation replication line length ( #4748 )
2019-02-27 10:28:37 +00:00
Erik Johnston
6870fc496f
Move connecting logic into ClientReplicationStreamProtocol
2019-02-27 10:23:51 +00:00
Richard van der Hoff
f191be822b
Add database version to phonehome stats. ( #4753 )
2019-02-27 10:21:49 +00:00
Erik Johnston
25814921f1
Increase the max delay between retry attempts
...
Otherwise if you have many workers they can easily take out master with
their connection attempts
2019-02-26 15:12:33 +00:00
Erik Johnston
313987187e
Fix tightloop over connecting to replication server
...
If the client failed to process incoming commands during the initial set
up of the replication connection it would immediately disconnect and
reconnect, resulting in a tightloop.
This can happen, for example, when subscribing to a stream that has a
row that is too long in the backlog.
The fix here is to not consider the connection successfully set up until
the client has succesfully subscribed and caught up with the streams.
This ensures that the retry logic timers aren't reset until then,
meaning that if an error does happen during start up the client will
continue backing off before retrying again.
2019-02-26 15:05:41 +00:00
Andrew Morgan
802884d4ee
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop
2019-02-26 14:23:40 +00:00
Andrew Morgan
6fcb25202f
Put function def back to the way it was
2019-02-26 14:17:38 +00:00
Andrew Morgan
7a4632af9c
Prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:37:24 +00:00
Andrew Morgan
c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms"
2019-02-26 13:20:38 +00:00
Andrew Morgan
a1a6473293
Merge pull request #4736 from matrix-org/anoa/public_rooms_federate
...
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:07:15 +00:00
Andrew Morgan
c4414768af
Cleaner chunk logic
2019-02-26 12:22:34 +00:00
Andrew Morgan
a712aa3a9c
Correct indent
2019-02-26 12:13:55 +00:00
Andrew Morgan
16565e67db
Correct docstring types and chunk logic
2019-02-26 12:12:48 +00:00
Andrew Morgan
40c2271680
Clean up room chunk logic
2019-02-26 12:04:34 +00:00
Andrew Morgan
6728bf3940
Make not showing non-federated rooms the default
2019-02-26 11:52:52 +00:00
Andrew Morgan
6946c20111
Result may be None
2019-02-26 11:27:19 +00:00
Andrew Morgan
71669a0fba
Address rich comments
2019-02-26 11:25:00 +00:00
Erik Johnston
899a119c2b
Don't log stack trace when client has gone away during media download ( #4738 )
...
* Don't log stack trace when client has gone away during media download
* Newsfile
* Fixup newsfile
2019-02-25 11:17:22 -08:00
Richard van der Hoff
641c409e4e
Fix ACME config for python 2. ( #4717 )
...
Fixes #4675 .
2019-02-25 11:16:33 -08:00
Matthew Hodgson
70ea2f4e1d
switch from google.com to recaptcha.net for reCAPTCHA ( #4731 )
...
* add trivial clarification about jemalloc
* switch from google.com to recaptcha.net
because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally
2019-02-25 11:15:36 -08:00
Andrew Morgan
96c408273e
Fix group's call to generate_room_entry
2019-02-25 18:00:17 +00:00
Andrew Morgan
1330aa4a8f
elif not else if
2019-02-25 17:28:19 +00:00
Erik Johnston
65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
...
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston
c7b333c545
Log tracebacks correctly
2019-02-25 16:56:41 +00:00
Erik Johnston
69efe6fb16
Merge pull request #4718 from matrix-org/erikj/fix_backfill_state_shred
...
Fix backfill storing incorrect state for events
2019-02-25 16:48:52 +00:00
Erik Johnston
9c598dddcb
Fix typo
2019-02-25 16:32:02 +00:00
Erik Johnston
b1a90da82e
Merge pull request #4721 from matrix-org/erikj/msc_1866
...
MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API
2019-02-25 15:53:10 +00:00
Erik Johnston
16c7afa94c
Merge pull request #4722 from matrix-org/erikj/correctly_handle_keyring_exceptions
...
Handle errors when fetching remote server keys
2019-02-25 15:53:02 +00:00
Andrew Morgan
8aaf7ffc44
syntax derp
2019-02-25 15:27:17 +00:00
Andrew Morgan
84c0a20dfe
Simplify call to generate_room_entry
2019-02-25 15:23:27 +00:00
Erik Johnston
4b9e5076c4
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate
2019-02-25 15:08:18 +00:00
Andrew Morgan
07493607a8
Docs and arg name clarification
2019-02-25 15:04:46 +00:00
Andrew Morgan
bd398b874e
Don't restrict non-fed rooms over client APIs
2019-02-25 15:04:46 +00:00
Andrew Morgan
e4b078a600
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-25 15:04:46 +00:00
Erik Johnston
d730c2c22b
More comments
2019-02-25 14:45:02 +00:00
Erik Johnston
890cb048fd
Assert rather than clobber the values
2019-02-25 14:42:11 +00:00
Erik Johnston
65d1003d01
raise_from already raises
2019-02-25 14:34:03 +00:00
Erik Johnston
9342cc6ab1
Add comments and paranoia
2019-02-25 10:02:12 +00:00
Erik Johnston
47a7e3928d
Correctly proxy exception in frontend_proxy worker
2019-02-23 15:17:57 +00:00
Erik Johnston
41285ffe5b
Handle errors when fetching remote server keys
2019-02-23 15:09:39 +00:00
Erik Johnston
59e0112209
MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API
2019-02-23 14:50:43 +00:00
Erik Johnston
d14e94bae4
Merge pull request #4716 from matrix-org/erikj/pusher_logging
...
Fix up pusher logging a bit
2019-02-22 15:29:16 +00:00
Erik Johnston
b82c9cf462
Add missing return
2019-02-22 15:27:40 +00:00
Erik Johnston
f2891d2487
Correctly handle PusherConfigException
2019-02-22 15:18:19 +00:00
Erik Johnston
a164134a53
Drop logging level of creating a pusher
2019-02-22 14:48:06 +00:00
Erik Johnston
1d9df51ff1
Correctly handle null data in HttpPusher
2019-02-22 14:47:48 +00:00
Erik Johnston
80467bbac3
Fix state cache invalidation on workers
2019-02-22 14:38:14 +00:00
Erik Johnston
7b288826b7
Fix backfill storing incorrect state for events
2019-02-22 11:33:51 +00:00
Richard van der Hoff
e07384c4e1
Add prometheus metrics for number of badge update pushes. ( #4709 )
...
We're counting the number of push notifications, but not the number of badges;
I'd like to see if they are significant.
2019-02-22 10:57:15 +00:00
Richard van der Hoff
0abb094f1a
bail out early in on_new_receipts if no pushers ( #4706 )
2019-02-21 17:51:21 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job ( #4707 )
...
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop
...
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston
71b625d808
Stop backpaginating when events not visible
2019-02-20 18:14:12 +00:00
Erik Johnston
82fca11fc1
Merge pull request #4694 from matrix-org/erikj/fix_sentry_config_format
...
Fixup generated metrics config
2019-02-20 14:13:38 +00:00
Richard van der Hoff
82ca6d1f9f
Add metrics for number of outgoing EDUs, by type ( #4695 )
2019-02-20 14:13:14 +00:00
Erik Johnston
3d672fec51
Fixup generated metrics config
2019-02-20 13:39:37 +00:00
Amber Brown
a06614bd2a
UPSERT many functionality ( #4644 )
2019-02-20 23:03:30 +11:00
Richard van der Hoff
b2200a8690
Logging improvements for the pusher ( #4691 )
2019-02-20 11:36:50 +00:00
Richard van der Hoff
c88bc53903
Fix TaskStopped exceptions when outbound requests time out ( #4690 )
2019-02-20 11:35:52 +00:00
Richard van der Hoff
86920ac266
Merge pull request #4678 from matrix-org/rav/tls_install_instructions
...
Attempt to clarify installation/config instructions
2019-02-20 11:10:33 +00:00
Erik Johnston
dbdc565dfd
Fix registration on workers ( #4682 )
...
* Move RegistrationHandler init to HomeServer
* Move post registration actions to RegistrationHandler
* Add post regisration replication endpoint
* Newsfile
2019-02-20 18:47:31 +11:00
Richard van der Hoff
5f9bdf90fe
Attempt to make default config more consistent
...
The general idea here is that config examples should just have a hash and no
extraneous whitespace, both to make it easier for people who don't understand
yaml, and to make the examples stand out from the comments.
2019-02-19 13:54:29 +00:00
Erik Johnston
c003450057
Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation
...
Batch cache invalidation over replication
2019-02-19 13:14:30 +00:00
Erik Johnston
62175a20e5
Docs
2019-02-19 11:38:40 +00:00
Erik Johnston
1bb35e3a83
Use itertools
2019-02-19 11:34:40 +00:00
Erik Johnston
bc8fa1509d
Documentation
2019-02-19 11:24:59 +00:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
...
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Brendan Abolivier
5a707a2f9a
Improve config documentation
2019-02-19 10:59:26 +00:00
Brendan Abolivier
a8626901cd
Fetch ACME domain into an instance member
2019-02-19 10:54:33 +00:00
Erik Johnston
32590b7139
Merge pull request #4670 from matrix-org/erikj/register_login_split
...
Split /login into client_reader
2019-02-19 10:46:22 +00:00
Richard van der Hoff
7c70b8f8a6
Try and make TLS federation client code faster ( #4674 )
...
* fix to use makeContext so that we don't need to rebuild the certificateoptions each time
2019-02-19 10:19:16 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
...
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Amber Brown
561eebe170
fix to use makeContext so that we don't need to rebuild the certificateoptions each time
2019-02-19 16:18:05 +11:00
Andrew Morgan
34ac75ce2c
lint
2019-02-18 18:23:37 +00:00
Erik Johnston
a9b5ea6fc1
Batch cache invalidation over replication
...
Currently whenever the current state changes in a room invalidate a lot
of caches, which cause *a lot* of traffic over replication. Instead,
lets batch up all those invalidations and send a single poke down
the replication streams.
Hopefully this will reduce load on the master process by substantially
reducing traffic.
2019-02-18 17:53:31 +00:00
Andrew Morgan
f8b9ca53ce
Move member event processing and changelog fix
2019-02-18 17:28:52 +00:00
Erik Johnston
d154f5a055
Merge pull request #4632 from matrix-org/erikj/basic_sentry
...
Add basic optional sentry.io integration
2019-02-18 17:22:45 +00:00
Erik Johnston
4cc4400b4d
Split /login into client_reader
2019-02-18 17:19:01 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
...
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
...
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston
d328a93b51
Fixup error handling and message
2019-02-18 16:53:56 +00:00
Erik Johnston
af691e415c
Move register_device into handler
2019-02-18 16:49:38 +00:00
Erik Johnston
028267acd2
Merge pull request #4669 from matrix-org/erikj/log_exception
...
Cleanup top level request exception logging
2019-02-18 16:41:41 +00:00
Erik Johnston
d08bac4136
Merge pull request #4651 from matrix-org/matthew/well-known-cors
...
set CORS on .well-known URI to unbreak modular
2019-02-18 16:38:11 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
...
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Erik Johnston
092b541401
Fixup
2019-02-18 15:52:26 +00:00
Brendan Abolivier
45bb55c6de
Use a configuration parameter to give the domain to generate a certificate for
2019-02-18 15:46:23 +00:00
Erik Johnston
8b9ae6d3a6
Update docs
2019-02-18 15:26:13 +00:00
Erik Johnston
94960cef03
pep8
2019-02-18 15:24:13 +00:00
Erik Johnston
fe725f7e45
Cleanup top level request exception logging
...
Firstly, we always logged that the request was being handled via
`JsonResource._async_render`, so we change that to use the servlet name
we add to the request.
Secondly, we pass the exception information to the logger rather than
formatting it manually. This makes it consistent with other exception
logging, allwoing logging hooks and formatters to access the exception
information.
2019-02-18 15:11:04 +00:00
Matthew Hodgson
04dad5ac16
fix missig import
2019-02-18 15:59:23 +01:00
Erik Johnston
e07cc31cb8
Correctly handle HttpResponseException
2019-02-18 14:55:09 +00:00
Brendan Abolivier
68a53f825f
Merge branch 'develop' into babolivier/acme-delegated
2019-02-18 14:52:23 +00:00
Erik Johnston
32e54b472a
Fix kicking guest users in worker mode
...
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.
Presumably broken in #4141
2019-02-18 14:16:07 +00:00
Andrew Morgan
915421065b
Membership events are done later
2019-02-18 14:02:09 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
...
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan
9caab0c364
Transfer bans on room upgrade
2019-02-18 13:52:51 +00:00
Erik Johnston
dc5efc92a8
Fixup
2019-02-18 13:52:49 +00:00
Erik Johnston
eb2b8523ae
Split out registration to worker
...
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.
Note: due to the in-memory session map all registration requests must be
handled by the same worker.
2019-02-18 12:12:57 +00:00
Andrew Morgan
6d02a13d81
Typo in info log
...
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:34 +00:00
Juuso "Linda" Lapinlampi
68d2869c8d
config: Remove a repeated word from a logger warning
...
The warning for missing macaroon_secret_key was "missing missing".
2019-02-15 22:24:53 -07:00
Erik Johnston
bd4505f765
Merge pull request #4647 from matrix-org/erikj/add_room_publishing_rules
...
Add configurable room list publishing rules
2019-02-15 22:11:01 +00:00
Brendan Abolivier
f86b695cbd
Various cosmetics to make TravisCI happy
2019-02-15 12:29:34 +00:00
Brendan Abolivier
af8a2f679b
Remove unused import
2019-02-15 12:27:43 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME
2019-02-15 12:05:08 +00:00
Erik Johnston
b99c532c1c
Move defaults up into code
2019-02-15 10:53:39 +00:00
Erik Johnston
02c729d6b0
Hoist up checks to reduce overall work
2019-02-15 10:20:02 +00:00
Erik Johnston
02c46acc6a
Fixup comments
2019-02-15 10:17:13 +00:00
Matthew Hodgson
bfcefbb230
tabs
2019-02-14 19:53:12 +00:00
Matthew Hodgson
6f47bc3fb2
set CORS on .well-known URI to unbreak modular
...
otherwise a riot/web running on foo.riot.im can't query
the .well-known on foo.modular.im...
2019-02-14 19:51:41 +00:00
Erik Johnston
8e32f26cb8
Clarify comments
2019-02-14 18:21:24 +00:00
Erik Johnston
cb12a37708
Clarify and fix behaviour when there are multiple aliases
2019-02-14 18:16:32 +00:00
Erik Johnston
f61b2068e6
Only fetch aliases when publishing rooms
2019-02-14 18:08:09 +00:00
Erik Johnston
f666fe36d7
Fixup comments
2019-02-14 18:07:24 +00:00
Richard van der Hoff
bf4fd14806
Merge branch 'master' into develop
2019-02-14 17:34:09 +00:00
Richard van der Hoff
f595d6ac57
0.99.1.1
2019-02-14 17:20:02 +00:00
Richard van der Hoff
f311018823
Fix errors in acme provisioning ( #4648 )
...
* Better logging for errors on startup
* Fix "TypeError: '>' not supported" when starting without an existing
certificate
* Fix a bug where an existing certificate would be reprovisoned every day
2019-02-14 17:10:36 +00:00
Erik Johnston
eaf4d11af9
Add configurable room list publishing rules
...
This allows specifying who and what is allowed to be published onto the
public room list
2019-02-14 16:02:23 +00:00
Richard van der Hoff
b02465b9db
Merge branch 'master' into develop
2019-02-14 14:42:03 +00:00
Richard van der Hoff
06cd757ae7
0.99.1
2019-02-14 14:24:24 +00:00
Erik Johnston
7fc1196a36
Correctly handle RequestSendFailed exceptions
...
This mainly reduces the number of exceptions we log.
2019-02-14 14:01:04 +00:00
Erik Johnston
6cb415b63f
Fixup comments and add warning
2019-02-13 16:15:11 +00:00
Richard van der Hoff
c6e75c9f2d
Merge pull request #4450 from 14mRh4X0r/fix-dependency-message
...
Fix error message for optional dependencies
2019-02-13 16:12:49 +00:00
Richard van der Hoff
3bc238629e
0.99.1rc2
2019-02-13 14:46:18 +00:00
Richard van der Hoff
c1dfd6a18a
Merge remote-tracking branch 'origin/release-v0.99.1' into develop
2019-02-13 14:27:45 +00:00
Erik Johnston
309f3bb322
Update synapse/app/_base.py
...
Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>
2019-02-13 13:24:27 +00:00
Amber Brown
bb4fd8f927
Run black
on user directory code ( #4635 )
2019-02-13 23:05:32 +11:00
Richard van der Hoff
767686af48
Use listen_tcp
for the replication listener
...
Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with
what we do elsewhere.
2019-02-13 11:59:04 +00:00
Richard van der Hoff
2a5a15aff8
Improve logging around listening services
...
I wanted to bring listen_tcp into line with listen_ssl in terms of returning a
list of ports, and wanted to check that was a safe thing to do - hence the
logging in `refresh_certificate`.
Also, pull the 'Synapse now listening' message up to homeserver.py, because it
was being duplicated everywhere else.
2019-02-13 11:58:54 +00:00
Richard van der Hoff
e3a0300431
Special-case the default bind_addresses for metrics listener
...
turns out it doesn't really support ipv6, so let's hack around that by only
listening on ipv4 by default.
2019-02-13 11:48:56 +00:00
Erik Johnston
93f7d2df3e
Comments
2019-02-12 16:03:40 +00:00
Erik Johnston
6a8f902edb
Raise an appropriate error message if sentry_sdk missing
2019-02-12 16:01:41 +00:00
Erik Johnston
ef2228c890
Basic sentry integration
2019-02-12 13:55:58 +00:00
Erik Johnston
d2fa7b7e99
Update changelog and version
2019-02-12 13:22:25 +00:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
...
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze
...
Analyze user_ips before running deduplication
2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe
...
Reduce user_ips bloat during dedupe background update
2019-02-12 13:02:58 +00:00
Erik Johnston
495ea92350
Fix pep8
2019-02-12 12:40:42 +00:00
Erik Johnston
483ba85c7a
Analyze user_ips before running deduplication
...
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
Erik Johnston
362d80b770
Reduce user_ips bloat during dedupe background update
...
The background update to remove duplicate rows naively deleted and
reinserted the duplicates. For large tables with a large number of
duplicates this causes a lot of bloat (with postgres), as the inserted
rows are appended to the table, since deleted rows will not be
overwritten until a VACUUM has happened.
This should hopefully also help ensure that the query in the last batch
uses the correct index, as inserting a large number of new rows without
analyzing will upset the query planner.
2019-02-12 11:39:34 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings
...
fix self-signed cert notice from generate-config
2019-02-12 11:24:45 +00:00