Commit Graph

12166 Commits

Author SHA1 Message Date
Richard van der Hoff
0a86850ba3
Stop the parent process flushing the logs on exit (#8012)
This solves the problem that the first few lines are logged twice on matrix.org. Hopefully the comments explain it.
2020-08-05 09:35:17 +01:00
Patrick Cloke
8ff2deda72
Fix async/await calls for broken media providers. (#8027) 2020-08-04 09:44:25 -04:00
Patrick Cloke
88a3ff12f0
Convert the SimpleHttpClient to async. (#8016) 2020-08-04 07:22:04 -04:00
Patrick Cloke
e19de43eb5
Convert streams to async. (#8014) 2020-08-04 07:21:47 -04:00
Richard van der Hoff
916cf2d439
re-implement daemonize (#8011)
This has long been something I've wanted to do. Basically the `Daemonize` code
is both too flexible and not flexible enough, in that it offers a bunch of
features that we don't use (changing UID, closing FDs in the child, logging to
syslog) and doesn't offer a bunch that we could do with (redirecting stdout/err
to a file instead of /dev/null; having the parent not exit until the child is
running).

As a first step, I've lifted the Daemonize code and removed the bits we don't
use. This should be a non-functional change. Fixing everything else will come
later.
2020-08-04 10:03:41 +01:00
Andrew Morgan
481f76c7aa
Remove signature check on v1 identity server lookups (#8001)
We've [decided](https://github.com/matrix-org/synapse/issues/5253#issuecomment-665976308) to remove the signature check for v1 lookups.

The signature check has been removed in v2 lookups. v1 lookups are currently deprecated. As mentioned in the above linked issue, this verification was causing deployments for the vector.im and matrix.org IS deployments, and this change is the simplest solution, without being unjustified.

Implementations are encouraged to use the v2 lookup API as it has [increased privacy benefits](https://github.com/matrix-org/matrix-doc/pull/2134).
2020-08-03 21:56:43 +01:00
Andrew Morgan
5d92a1428c
Prevent join->join membership transitions changing member count (#7977)
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.

However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.

This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.

Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).

Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
2020-08-03 21:54:24 +01:00
Patrick Cloke
6812509807
Implement handling of HTTP HEAD requests. (#7999) 2020-08-03 08:45:42 -04:00
Patrick Cloke
2a89ce8cd4
Convert the crypto module to async/await. (#8003) 2020-08-03 08:29:01 -04:00
Michael Albert
b6c6fb7950
Allow guests to operate in encrypted rooms (#7314)
Signed-off-by: Michael Albert <michael.albert@awesome-technologies.de>
2020-08-03 12:13:49 +01:00
Patrick Cloke
3b415e23a5
Convert replication code to async/await. (#7987) 2020-08-03 07:12:55 -04:00
Patrick Cloke
db5970ac6d
Convert ACME code to async/await. (#7989) 2020-08-03 07:09:33 -04:00
Patrick Cloke
d1008fe949
Fix some comments and types in service notices (#7996) 2020-07-31 16:22:06 -04:00
Erik Johnston
faba873d4b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins 2020-07-31 15:07:01 +01:00
Erik Johnston
18de00adb4 Add ratelimiting on joins 2020-07-31 15:06:56 +01:00
Richard van der Hoff
0a7fb24716
Fix invite rejection when we have no forward-extremeties (#7980)
Thanks to some slightly overzealous cleanup in the
`delete_old_current_state_events`, it's possible to end up with no
`event_forward_extremities` in a room where we have outstanding local
invites. The user would then get a "no create event in auth events" when trying
to reject the invite.

We can hack around it by using the dangling invite as the prev event.
2020-07-30 16:58:57 +01:00
Olivier Wilkinson (reivilibre)
3aa36b782c Merge branch 'master' into develop 2020-07-30 15:18:36 +01:00
Patrick Cloke
c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Patrick Cloke
4cce8ef74e
Convert appservice to async. (#7973) 2020-07-30 07:27:39 -04:00
Patrick Cloke
b3a97d6dac
Convert some of the data store to async. (#7976) 2020-07-30 07:20:41 -04:00
Patrick Cloke
3950ae51ef
Ensure that remove_pusher is always async (#7981) 2020-07-30 06:56:55 -04:00
Olivier Wilkinson (reivilibre)
a9631b7b4b 1.18.0 2020-07-30 10:56:54 +01:00
Erik Johnston
2c1b9d6763
Update worker docs with recent enhancements (#7969) 2020-07-29 23:22:13 +01:00
Patrick Cloke
a53e0160a2
Ensure the msg property of HttpResponseException is a string. (#7979) 2020-07-29 13:56:06 -04:00
Patrick Cloke
d90087cffa
Remove from the event_relations table when purging historical events. (#7978) 2020-07-29 13:55:01 -04:00
Patrick Cloke
3a00bd1378
Add additional logging for SAML sessions. (#7971) 2020-07-29 13:54:44 -04:00
Brendan Abolivier
8dff4a1242
Re-implement unread counts (#7736) 2020-07-29 18:26:55 +01:00
Aaron Raimist
2184f61fae
Various improvements to the docs (#7899) 2020-07-29 10:35:44 -04:00
Patrick Cloke
3345c166a4
Convert storage layer to async/await. (#7963) 2020-07-28 16:09:53 -04:00
Dirk Klimpel
e866e3b896
Add an option to disable purge in delete room admin API (#7964)
Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete```
Fixes: #3761

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-28 20:08:23 +01:00
Andrew Morgan
8a25332d94
Move some log lines from default logger to sql/transaction loggers (#7952)
Idea from matrix-org/synapse-dinsic#49
2020-07-28 18:52:13 +01:00
Patrick Cloke
2c1e1b153d
Use the JSON module from the std library instead of simplejson. (#7936) 2020-07-28 10:28:59 -04:00
lugino-emeritus
3857de2194
Option to allow server admins to join complex rooms (#7902)
Fixes #7901.

Signed-off-by: Niklas Tittjung <nik_t.01@web.de>
2020-07-28 13:41:44 +01:00
Richard van der Hoff
349119a340 Synapse 1.18.0rc2 (2020-07-28)
==============================
 
 Bugfixes
 --------
 
 - Fix an `AssertionError` exception introduced in v1.18.0rc1. ([\#7876](https://github.com/matrix-org/synapse/issues/7876))
 - Fix experimental support for moving typing off master when worker is restarted, which is broken in v1.18.0rc1. ([\#7967](https://github.com/matrix-org/synapse/issues/7967))
 
 Internal Changes
 ----------------
 
 - Further optimise queueing of inbound replication commands. ([\#7876](https://github.com/matrix-org/synapse/issues/7876))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl8f/f8ACgkQOSor00I9
 eP8/Uwf8CiVWvrBsmFZMvxJDkUWm0/f1kN4IQdm8ibDtyNyvFUx+Y1K8KOQS+VwG
 a3bZqSC2Vv2sO9O9kR+V2tk831l+ujO0Nlaohuqyvhcl9lzh04rRYI9x9IHlAq2H
 WPb0NMLwMufL6YkXDBwZT/G9TVW1vLRGASu4f7X2rXqek34VNVgYbg1hB2dp4dDa
 wjKk3iBZ6h34IhKPgu0sLBUcyvX4U5xdOHjEG3HXvNnvDNO0HMD8rGB7065vFMD6
 PH4nUK/h+RL0UBs2sJOMK1ZazFUODdURwANJQNAQ6pNvf9/RWgw2okka2bYIcmQQ
 UT7tiwMsBvKdy4PER5fcDX3COY16qw==
 =Q+bI
 -----END PGP SIGNATURE-----

Merge tag 'v1.18.0rc2' into develop

Synapse 1.18.0rc2 (2020-07-28)
==============================

Bugfixes
--------

- Fix an `AssertionError` exception introduced in v1.18.0rc1. ([\#7876](https://github.com/matrix-org/synapse/issues/7876))
- Fix experimental support for moving typing off master when worker is restarted, which is broken in v1.18.0rc1. ([\#7967](https://github.com/matrix-org/synapse/issues/7967))

Internal Changes
----------------

- Further optimise queueing of inbound replication commands. ([\#7876](https://github.com/matrix-org/synapse/issues/7876))
2020-07-28 11:31:31 +01:00
Richard van der Hoff
7000a215e6 1.18.0rc2 2020-07-28 11:22:32 +01:00
Erik Johnston
a8f7ed28c6
Typing worker needs to handle stream update requests (#7967)
IIRC this doesn't break tests because its only hit on reconnection, or something.

Basically, when a process needs to fetch missing updates for the `typing` stream it needs to query the writer instance via HTTP (as we don't write typing notifications to the DB), the problem was that the endpoint (`streams`) was only registered on master and specifically not on the typing writer worker.
2020-07-28 11:04:53 +01:00
Patrick Cloke
68626ff8e9
Convert the remaining media repo code to async / await. (#7947) 2020-07-27 14:40:11 -04:00
Richard van der Hoff
f57b99af22
Handle replication commands synchronously where possible (#7876)
Most of the stuff we do for replication commands can be done synchronously. There's no point spinning up background processes if we're not going to need them.
2020-07-27 18:54:43 +01:00
Patrick Cloke
8553f46498
Convert a synapse.events to async/await. (#7949) 2020-07-27 13:40:22 -04:00
Patrick Cloke
5f65e62681
Convert groups and visibility code to async / await. (#7951) 2020-07-27 12:32:08 -04:00
Patrick Cloke
8144bc26a7
Convert push to async/await. (#7948) 2020-07-27 12:21:34 -04:00
Richard van der Hoff
f88c48f3b8 1.18.0rc1 2020-07-27 16:57:40 +01:00
Erik Johnston
1ef9efc1e0
Fix error reporting when using opentracing.trace (#7961) 2020-07-27 16:20:24 +01:00
Erik Johnston
84d099ae11
Fix typing replication not being handled on master (#7959)
Handling of incoming typing stream updates from replication was not
hooked up on master, effecting set ups where typing was handled on a
different worker.

This is really only a problem if the master process is also handling
sync requests, which is unlikely for those that are at the stage of
moving typing off.

The other observable effect is that if a worker restarts or a
replication connect drops then the typing worker will issue a
`POSITION typing`, triggering master process to try and stream *all*
typing updates from position 0.

Fixes #7907
2020-07-27 14:10:53 +01:00
Patrick Cloke
d8a9cd8d3e
Remove hacky error handling for inlineDeferreds. (#7950) 2020-07-27 08:35:56 -04:00
Patrick Cloke
3fc8fdd150
Support oEmbed for media previews. (#7920)
Fixes previews of Twitter URLs by using their oEmbed endpoint to grab content.
2020-07-27 07:50:44 -04:00
Patrick Cloke
b975fa2e99
Convert state resolution to async/await (#7942) 2020-07-24 10:59:51 -04:00
Patrick Cloke
e739b20588
Fix up types and comments that refer to Deferreds. (#7945) 2020-07-24 10:53:25 -04:00
Patrick Cloke
53f7b49f5b
Do not convert async functions to Deferreds in the interactive_auth_handler (#7944) 2020-07-24 09:43:49 -04:00
Patrick Cloke
5ea29d7f85
Convert more of the media code to async/await (#7873) 2020-07-24 09:39:02 -04:00