Commit Graph

17388 Commits

Author SHA1 Message Date
Christopher May-Townsend
a5545cf86d
Switch to Debian:Slim from Alpine for the docker image (#7839)
As mentioned in #7397, switching to a debian base should help with multi-arch work to save time on compiling. This is unashamedly based on #6373, but without the extra functionality. Switch python version back to generic 3.7 to always pull the latest. Essentially, keeping this as small as possible. The image is bigger though unfortunately.
2020-07-17 17:40:53 +01:00
Erik Johnston
2d2acc1cf2
Stop using 'device_max_stream_id' (#7882)
It serves no purpose and updating everytime we write to the device inbox
stream means all such transactions will conflict, causing lots of
transaction failures and retries.
2020-07-17 17:03:27 +01:00
Erik Johnston
a3ad045286
Fix TypeError in synapse.notifier (#7880)
Fixes #7774
2020-07-17 14:11:05 +01:00
Patrick Cloke
852930add7
Add a default limit (of 100) to get/sync operations. (#7858) 2020-07-17 07:59:23 -04:00
Erik Johnston
4642fd66df
Change "unknown room ver" logging to warning. (#7881)
It's somewhat expected for us to have unknown room versions in the
database due to room version experiments.
2020-07-17 12:10:43 +01:00
Patrick Cloke
6b3ac3b8cd
Convert device handler to async/await (#7871) 2020-07-17 07:09:25 -04:00
Patrick Cloke
00e57b755c
Convert synapse.app to async/await. (#7868) 2020-07-17 07:08:56 -04:00
Patrick Cloke
6fca1b3506
Convert _base, profile, and _receipts handlers to async/await (#7860) 2020-07-17 07:08:30 -04:00
Michael Albert
fff483ea96
Add admin endpoint to get members in a room. (#7842) 2020-07-16 16:43:23 -04:00
Patrick Cloke
f460da6031
Consistently use db_to_json to convert from database values to JSON objects. (#7849) 2020-07-16 11:32:19 -04:00
Luke Faraone
b0f031f92a
Combine nginx federation server blocks (#7823)
I'm pretty sure there's no technical reason these have to be distinct server blocks, so collapse into one and go with the more terse location block.

Signed-off-by: Luke W Faraone <luke@faraone.cc>
2020-07-16 16:01:45 +01:00
Richard van der Hoff
e5300063ed
Optimise queueing of inbound replication commands (#7861)
When we get behind on replication, we tend to stack up background processes
behind a linearizer. Bg processes are heavy (particularly with respect to
prometheus metrics) and linearizers aren't terribly efficient once the queue
gets long either.

A better approach is to maintain a queue of requests to be processed, and
nominate a single process to work its way through the queue.

Fixes: #7444
2020-07-16 15:49:37 +01:00
Richard van der Hoff
346476df21
Reject attempts to join empty rooms over federation (#7859)
We shouldn't allow others to make_join through us if we've left the room;
reject such attempts with a 404.

Fixes #7835. Fixes #6958.
2020-07-16 15:17:31 +01:00
Erik Johnston
f2e38ca867
Allow moving typing off master (#7869) 2020-07-16 15:12:54 +01:00
Erik Johnston
649a7ead5c
Add ability to run multiple pusher instances (#7855)
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
Richard van der Hoff
a827838706
Merge pull request #7866 from matrix-org/rav/fix_guest_user_id
Fix guest user registration with lots of client readers
2020-07-16 13:54:45 +01:00
Richard van der Hoff
a973bcb8a4
Add some tiny type annotations (#7870)
I found these made pycharm have more of a clue as to what was going on in other places.
2020-07-16 13:52:29 +01:00
Richard van der Hoff
16368c8a34 changelog 2020-07-16 13:01:11 +01:00
Richard van der Hoff
c445bc0cad Use a postgres sequence to generate guest user IDs 2020-07-16 13:00:25 +01:00
Richard van der Hoff
3c36ae17a5 Use SequenceGenerator for state group ID allocation 2020-07-16 11:25:08 +01:00
Richard van der Hoff
42509b8fb6 Use PostgresSequenceGenerator from MultiWriterIdGenerator
partly just to show it works, but alwo to remove a bit of code duplication.
2020-07-16 11:25:08 +01:00
Richard van der Hoff
90b0cdda42 Add some helper classes for generating ID sequences 2020-07-16 11:25:08 +01:00
Olivier Wilkinson (reivilibre)
12528dc42f Remove obsolete comment.
It was correct at the time of our friend Jorik writing it (checking
git blame), but the world has moved now and it is no longer a
generator.

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2020-07-16 11:12:48 +01:00
Patrick Cloke
35450519de
Ensure that calls to json.dumps are compatible with the standard library json. (#7836) 2020-07-15 13:40:54 -04:00
Richard van der Hoff
a57df9b827
Avoid brand new rooms in delete_old_current_state_events (#7854)
When considering rooms to clean up in `delete_old_current_state_events`, skip
rooms which we are creating, which otherwise look a bit like rooms we have
left.

Fixes #7834.
2020-07-15 18:33:03 +01:00
Erik Johnston
97e1159ac1 Merge branch 'erikj/faster_typing' of github.com:matrix-org/synapse into develop 2020-07-15 16:54:30 +01:00
Patrick Cloke
8c7d0f163d
Allow accounts to be re-activated from the admin APIs. (#7847) 2020-07-15 11:00:21 -04:00
Erik Johnston
9006e125af Fix tests 2020-07-15 15:47:27 +01:00
Erik Johnston
62352c3a1b Fix typo 2020-07-15 15:46:16 +01:00
Erik Johnston
3032b54ac9 Newsfile 2020-07-15 15:45:19 +01:00
Erik Johnston
3a3a618460 Use get_users_in_room rather than state handler in typing for speed 2020-07-15 15:42:07 +01:00
Erik Johnston
f13061d515
Fix client reader sharding tests (#7853)
* Fix client reader sharding tests

* Newsfile

* Fix typing

* Update changelog.d/7853.misc

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Move mocking of http_client to tests

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-07-15 15:27:35 +01:00
Patrick Cloke
b11450dedc
Convert E2E key and room key handlers to async/await. (#7851) 2020-07-15 08:48:58 -04:00
Patrick Cloke
111e70d75c
Return the proper 403 Forbidden error during errors with JWT logins. (#7844) 2020-07-15 07:10:21 -04:00
Richard van der Hoff
1d9dca02f9
remove retry_on_integrity_error wrapper for persist_events (#7848)
As far as I can tell from the sentry logs, the only time this has actually done
anything in the last two years is when we had two master workers running at
once, and even then, it made a bit of a mess of it (see
https://github.com/matrix-org/synapse/issues/7845#issuecomment-658238739).

Generally I feel like this code is doing more harm than good.
2020-07-15 10:34:53 +01:00
Patrick Cloke
8d0097bef1
Fix bug in per-room message retention policies. (#7850) 2020-07-14 15:51:13 -04:00
Brendan Abolivier
85223106f3
Allow email subjects to be customised through Synapse's configuration (#7846) 2020-07-14 19:10:42 +01:00
Dirk Klimpel
491f0dab1b
Add delete room admin endpoint (#7613)
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API.

Fixes: #6425 

It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105)

Related to:
- #5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-14 12:36:23 +01:00
Patrick Cloke
77d2c05410
Add the option to validate the iss and aud claims for JWT logins. (#7827) 2020-07-14 07:16:43 -04:00
Patrick Cloke
4db1509516
Improve the type hints of synapse.api.errors. (#7820) 2020-07-14 07:03:58 -04:00
Luke Faraone
93c8b077ed
Clearly state built-in ACME no longer works (#7824)
I'm tempted to remove this section entirely, but it's helpful for admins who are trying to figure out why their Synapse is crashing on start with ACME errors.

Signed-off-by: Luke W Faraone <luke@faraone.cc>
2020-07-14 10:49:10 +01:00
Erik Johnston
f886a69916
Correctly pass app_name to all email templates. (#7829)
We didn't do this for e.g. registration emails.
2020-07-14 10:00:53 +01:00
Patrick Cloke
457096e6df
Support handling registration requests across multiple client readers. (#7830) 2020-07-13 13:31:46 -04:00
Brendan Abolivier
504c8f3483
Fix handling of "off" in encryption_enabled_by_default_for_room_type (#7822)
Fixes https://github.com/matrix-org/synapse/issues/7821, introduced in https://github.com/matrix-org/synapse/pull/7639

Turns out PyYAML translates `off` into a `False` boolean if it's
unquoted (see https://stackoverflow.com/questions/36463531/pyyaml-automatically-converting-certain-keys-to-boolean-values),
which seems to be a liberal interpretation of this bit of the YAML spec: https://yaml.org/spec/1.1/current.html#id864510

An alternative fix would be to implement the solution mentioned in the
SO post linked above, but I'm aware it might break existing setups
(which might use these values in the configuration file) so it's
probably better just to add an extra check for this one. We should be
aware that this is a thing for the next times we do that though.

I didn't find any other occurrence of this bug elsewhere in the
codebase.
2020-07-13 17:14:42 +01:00
Richard van der Hoff
fa361c8f65 Update grafana dashboard 2020-07-13 14:48:21 +01:00
Richard van der Hoff
59e64b6d5b Merge branch 'master' into develop 2020-07-13 11:42:52 +01:00
Richard van der Hoff
29df3d0e9f 1.17.0 2020-07-13 10:20:36 +01:00
Patrick Cloke
66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802) 2020-07-10 14:30:08 -04:00
Patrick Cloke
d9e47af617
Add types to the server code and remove unused parameter (#7813) 2020-07-10 14:28:42 -04:00
Sorunome
1bca21e1da
Include room states on invite events sent to ASes (#6455) 2020-07-10 18:44:56 +01:00