Commit Graph

2643 Commits

Author SHA1 Message Date
Patrick Cloke
380122866f
Cast a coroutine into a Deferred in the federation base (#6996)
Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
2020-02-26 11:32:13 -05:00
Erik Johnston
1f773eec91
Port PresenceHandler to async/await (#6991) 2020-02-26 15:33:26 +00:00
Uday Bansal
7728d87fd7
Updated warning for incorrect database collation/ctype (#6985)
Signed-off-by: Uday Bansal <43824981+udaybansal19@users.noreply.github.com>
2020-02-26 15:17:03 +00:00
Andrew Morgan
8c75b621bf
Ensure 'deactivated' parameter is a boolean on user admin API, Fix error handling of call to deactivate user (#6990) 2020-02-26 12:22:55 +00:00
Richard van der Hoff
e66f099ca9
Sanity-check database before running upgrades (#6982)
Some of the database deltas rely on `config.server_name` being set correctly,
so we should check that it is before running the deltas.

Fixes #6870.
2020-02-25 17:46:00 +00:00
Erik Johnston
bbf8886a05
Merge worker apps into one. (#6964) 2020-02-25 16:56:55 +00:00
Fridtjof Mund
4aea0bd292
contrib/docker: remove quotes for POSTGRES_INITDB_ARGS (#6984)
I made a mistake in https://github.com/matrix-org/synapse/pull/6921 - the quotes break the postgres container's startup script (or docker-compose), which makes initdb fail: https://github.com/matrix-org/synapse/pull/6921#issuecomment-590657154

Signed-off-by: Fridtjof Mund <fridtjof@das-labor.org>
2020-02-25 10:48:13 +00:00
Richard van der Hoff
691659568f
Remove redundant store_room call (#6979)
`_process_received_pdu` is only called by `on_receive_pdu`, which ignores any
events for unknown rooms, so this is redundant.
2020-02-24 17:20:44 +00:00
Richard van der Hoff
a301934f46
Upsert room version when we join over federation (#6968)
This is intended as a precursor to storing room versions when we receive an
invite over federation, but has the happy side-effect of fixing #3374 at last.

In short: change the store_room with try/except to a proper upsert which
updates the right columns.
2020-02-24 15:46:41 +00:00
Richard van der Hoff
4c2ed3f20e
Fix minor issues with email config (#6962)
* Give `notif_template_html`, `notif_template_text` default values (fixes #6960)
 * Don't complain if `smtp_host` and `smtp_port` are unset, since they have sensible defaults (fixes #6961)
 * Set the example for `enable_notifs` to `True`, for consistency and because it's more useful
 * Raise errors as ConfigError rather than RuntimeError for nicer formatting
2020-02-24 15:18:38 +00:00
Patrick Cloke
af6c389501
No longer use room alias events to calculate room names for push notifications. (#6966) 2020-02-21 12:50:48 -05:00
Dirk Klimpel
7b0e2d961c
Change displayname of user as admin in rooms (#6876) 2020-02-21 17:44:03 +00:00
Patrick Cloke
fcf4599488
Stop returning aliases as part of the room list. (#6970) 2020-02-21 12:40:23 -05:00
Patrick Cloke
7936d2a96e
Publishing/removing from the directory requires a power level greater than canonical aliases. 2020-02-21 07:18:33 -05:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff
1fcb9a1a7a changelog 2020-02-21 09:06:18 +00:00
Richard van der Hoff
9c1b83b007 1.11.0 2020-02-21 08:56:04 +00:00
Andrew Morgan
8f6d9c4cf0
Small grammar fixes to the ACME v1 deprecation notice (#6944)
Some small fixes to the copy in #6907.
2020-02-21 08:53:01 +00:00
Patrick Cloke
99eed85a77
Do not send alias events when creating / upgrading a room (#6941)
Stop emitting room alias update events during room creation/upgrade.
2020-02-20 16:24:04 -05:00
Hubert Chathi
a90d0dc5c2
don't insert into the device table for remote cross-signing keys (#6956) 2020-02-20 09:59:00 -05:00
Ruben Barkow-Kuder
4fb5f4d0ce
Add some clarifications to README.md in the database schema directory. (#6615)
Signed-off-by: Ruben Barkow-Kuder <github@r.z11.de>
2020-02-20 10:37:57 +00:00
Erik Johnston
7b7c3cedf2 Minor perf fixes to get_auth_chain_ids. 2020-02-19 15:47:11 +00:00
Erik Johnston
fc87d2ffb3
Freeze allocated objects on startup. (#6953)
This may make gc go a bit faster as the gc will know things like
caches/data stores etc. are frozen without having to check.
2020-02-19 15:09:00 +00:00
Erik Johnston
2b37eabca1
Reduce auth chains fetched during v2 state res. (#6952)
The state res v2 algorithm only cares about the difference between auth
chains, so we can pass in the known common state to the `get_auth_chain`
storage function so that it can ignore those events.
2020-02-19 15:04:47 +00:00
Richard van der Hoff
197b08de35 1.11.0rc1 2020-02-19 13:48:32 +00:00
Erik Johnston
099c96b89b
Revert get_auth_chain_ids changes (#6951) 2020-02-19 11:37:35 +00:00
Richard van der Hoff
2fb7794e60
Merge pull request #6949 from matrix-org/rav/list_room_aliases_peekable
Make room alias lists peekable
2020-02-19 11:19:11 +00:00
Brendan Abolivier
bbe39f808c
Merge pull request #6940 from matrix-org/babolivier/federate.md
Clean up and update federation docs
2020-02-19 10:58:59 +00:00
Richard van der Hoff
880aaac1d8
Move MSC2432 stuff onto unstable prefix (#6948)
it's not in the spec yet, so needs to be unstable. Also add a feature flag for it. Also add a test for admin users.
2020-02-19 10:40:27 +00:00
Richard van der Hoff
abf1e5c526
Tiny optimisation for _get_handler_for_request (#6950)
we have hundreds of path_regexes (see #5118), so let's not convert the same
bytes to str for each of them.
2020-02-19 10:38:20 +00:00
Erik Johnston
0d0bc35792
Increase DB/CPU perf of _is_server_still_joined check. (#6936)
* Increase DB/CPU perf of `_is_server_still_joined` check.

For rooms with large amount of state a single user leaving could cause
us to go and load a lot of membership events and then pull out
membership state in a large number of batches.

* Newsfile

* Update synapse/storage/persist_events.py

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Fix adding if too soon

* Update docstring

* Review comments

* Woops typo

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-02-19 10:15:49 +00:00
Brendan Abolivier
5e4a438556
Merge pull request #6945 from matrix-org/babolivier/fix-retention-debug-log
Fix log in message retention purge jobs
2020-02-19 10:12:55 +00:00
Brendan Abolivier
fa64f836ec
Update changelog.d/6945.bugfix
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-02-19 09:54:13 +00:00
Erik Johnston
5a5abd55e8
Limit size of get_auth_chain_ids query (#6947) 2020-02-19 09:39:26 +00:00
Richard van der Hoff
603618c002 changelog 2020-02-19 08:53:32 +00:00
Brendan Abolivier
771d70e89c
Changelog 2020-02-18 17:31:02 +00:00
Richard van der Hoff
adfaea8c69
Implement GET /_matrix/client/r0/rooms/{roomId}/aliases (#6939)
per matrix-org/matrix-doc#2432
2020-02-18 16:23:25 +00:00
Richard van der Hoff
3f1cd14791
Merge pull request #6872 from matrix-org/rav/dictproperty
Rewrite _EventInternalMetadata to back it with a dict
2020-02-18 16:21:02 +00:00
Brendan Abolivier
d484126bf7
Merge pull request #6907 from matrix-org/babolivier/acme-config
Add mention and warning about ACME v1 deprecation to the TLS config
2020-02-18 16:11:31 +00:00
Erik Johnston
8a380d0fe2
Increase perf of get_auth_chain_ids used in state res v2. (#6937)
We do this by moving the recursive query to be fully in the DB.
2020-02-18 15:39:09 +00:00
Erik Johnston
818def8248
Fix worker docs to point /publicised_groups API correctly. (#6938) 2020-02-18 15:27:45 +00:00
Brendan Abolivier
7a782c32a2
Merge pull request #6909 from matrix-org/babolivier/acme-install
Update INSTALL.md to recommend reverse proxying and warn about ACMEv1 deprecation
2020-02-18 15:06:06 +00:00
Brendan Abolivier
b1255077f5 Changelog 2020-02-18 14:35:51 +00:00
Patrick Cloke
fe3941f6e3
Stop sending events when creating or deleting aliases (#6904)
Stop sending events when creating or deleting associations (room aliases). Send an updated canonical alias event if one of the alt_aliases is deleted.
2020-02-18 07:29:44 -05:00
Richard van der Hoff
3be2abd0a9
Kill off deprecated "config-on-the-fly" docker mode (#6918)
Lots of people seem to get confused by this mode, and it's been deprecated
since Synapse 1.1.0. It's time for it to go.
2020-02-18 11:41:53 +00:00
Richard van der Hoff
bc831d1d9a #6924 has been released in 1.10.1 2020-02-17 16:34:13 +00:00
Andrew Morgan
3404ad289b
Raise the default power levels for invites, tombstones and server acls (#6834) 2020-02-17 13:23:37 +00:00
Richard van der Hoff
46fa66bbfd
wait for current_state_events_membership before delete_old_current_state_events (#6924) 2020-02-17 11:30:50 +00:00
Patrick Cloke
10027c80b0
Add type hints to the spam check module (#6915)
Add typing information to the spam checker modules.
2020-02-14 12:49:40 -05:00
Richard van der Hoff
5a78f47f6e changelog 2020-02-14 16:42:40 +00:00
Fridtjof Mund
32873efa87
contrib/docker: Ensure correct encoding and locale settings on DB creation (#6921)
Signed-off-by: Fridtjof Mund <fridtjof@das-labor.org>
2020-02-14 16:27:29 +00:00
Richard van der Hoff
97a42bbc3a
Add a warning about indentation to generated config (#6920)
Fixes #6916.
2020-02-14 16:22:30 +00:00
Patrick Cloke
02e89021f5
Convert the directory handler tests to use HomeserverTestCase (#6919)
Convert directory handler tests to use HomeserverTestCase.
2020-02-14 09:05:43 -05:00
Patrick Cloke
49f877d32e
Filter the results of user directory searching via the spam checker (#6888)
Add a method to the spam checker to filter the user directory results.
2020-02-14 07:17:54 -05:00
Brendan Abolivier
79460ce9c9
Changelog 2020-02-13 17:24:14 +00:00
Brendan Abolivier
df1c98c22a
Update changelog for #6905 to group it with upcoming PRs 2020-02-13 16:12:20 +00:00
Brendan Abolivier
f3f142259e
Changelog 2020-02-13 16:10:16 +00:00
Patrick Cloke
361de49c90
Add documentation for the spam checker module (#6906)
Add documentation for the spam checker.
2020-02-13 07:40:57 -05:00
Brendan Abolivier
f48bf4febd
Merge pull request #6905 from matrix-org/babolivier/acme.md
Update ACME.md to mention ACME v1 deprecation
2020-02-13 12:13:18 +00:00
Aaron Raimist
dc3f998706
Remove m.lazy_load_members from unstable features since it is in CS r0.5.0 (#6877)
Fixes #5528
2020-02-13 12:02:32 +00:00
Brendan Abolivier
e88a5dd108
Changelog 2020-02-12 20:15:41 +00:00
Brendan Abolivier
6cd34da8b1
Merge pull request #6891 from matrix-org/babolivier/retention-doc-amend
Spell out that the last event sent to a room won't be deleted by a purge
2020-02-12 20:12:20 +00:00
Andrew Morgan
d8994942f2
Return a 404 for admin api user lookup if user not found (#6901) 2020-02-12 18:14:10 +00:00
Patrick Cloke
ba547ec3a9
Use BSD-compatible in-place editing for sed. (#6887) 2020-02-12 07:02:19 -05:00
Brendan Abolivier
a0c4769f1a
Update the changelog file 2020-02-11 17:56:42 +00:00
Brendan Abolivier
705c978366
Changelog 2020-02-11 17:38:27 +00:00
Patrick Cloke
a92e703ab9
Reject device display names that are too long (#6882)
* Reject device display names that are too long.

Too long is currently defined as 100 characters in length.

* Add a regression test for rejecting a too long device display name.
2020-02-10 16:35:26 -05:00
Patrick Cloke
3a3118f4ec
Add an additional test to the SyTest blacklist for worker mode. (#6883) 2020-02-10 11:47:18 -05:00
Richard van der Hoff
db0fee738d Synapse 1.10.0rc3 (2020-02-10)
==============================
 
 Features
 --------
 
 - Filter out m.room.aliases from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878))
 
 Internal Changes
 ----------------
 
 - Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl5BLBwACgkQOSor00I9
 eP/9Ogf9HZ+XD5tt1uYkGqDSvUBS4sHdNcvxpNf1UMo6VaY4K5b9kF28MjY07ur2
 nGq/nnLyLZZzwVXtV3q8upIdIK1hcMp2HCU0qDhEljepOkGmyQTqZooxrdrLTjPS
 232bbifB6VFdK8P4gPIGopCRQI9a+TII4jODBobiJR43e0gEPfSREJKAnJs7DYaD
 Quf3Svzu+jB+X8ymWLVbCFXuk4A4UqPzrRPb8TQBDN3w35G4JBDAgIpdxhQdJpdR
 RIS1HgSm95P03doS23RLUjXuIpBfYveyStuGfPx4P9Lf30M2o6bxDeTPo2+IGbST
 yhl1jB0l8Ve5/6veuTieA6vuXWU55g==
 =2rgX
 -----END PGP SIGNATURE-----

Merge tag 'v1.10.0rc3' into develop

Synapse 1.10.0rc3 (2020-02-10)
==============================

Features
--------

- Filter out m.room.aliases from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878))

Internal Changes
----------------

- Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880))
2020-02-10 10:15:32 +00:00
Richard van der Hoff
3de57e7062 1.10.0rc3 2020-02-10 09:56:42 +00:00
Matthew Hodgson
8e64c5a24c filter out m.room.aliases from the CS API until a better solution is specced (#6878)
We're in the middle of properly mitigating spam caused by malicious aliases being added to a room. However, until this work fully lands, we temporarily filter out all m.room.aliases events from /sync and /messages on the CS API, to remove abusive aliases. This is considered acceptable as m.room.aliases events were never a reliable record of the given alias->id mapping and were purely informational, and in their current state do more harm than good.
2020-02-10 09:36:23 +00:00
Richard van der Hoff
cc0800ebfc Merge remote-tracking branch 'origin/release-v1.10.0' into develop 2020-02-10 00:41:49 +00:00
Richard van der Hoff
fe73f0d533
Update setuptools for python 3.5 tests (#6880)
Workaround for jaraco/zipp#40
2020-02-10 00:41:20 +00:00
Erik Johnston
21db35f77e
Add support for putting fed user query API on workers (#6873) 2020-02-07 15:45:39 +00:00
Richard van der Hoff
e1d858984d
Remove unused get_room_stats_state method. (#6869) 2020-02-07 15:30:26 +00:00
Richard van der Hoff
799001f2c0
Add a make_event_from_dict method (#6858)
... and use it in places where it's trivial to do so.

This will make it easier to pass room versions into the FrozenEvent
constructors.
2020-02-07 15:30:04 +00:00
Erik Johnston
b08b0a22d5
Add typing to synapse.federation.sender (#6871) 2020-02-07 13:56:38 +00:00
Erik Johnston
de2d267375
Allow moving group read APIs to workers (#6866) 2020-02-07 11:14:19 +00:00
Dirk Klimpel
56ca93ef59
Admin api to add an email address (#6789) 2020-02-07 10:29:36 +00:00
Richard van der Hoff
f4884444c3
remove unused room_version_to_event_format (#6857) 2020-02-07 09:26:57 +00:00
Richard van der Hoff
e1b240329e
Merge pull request #6856 from matrix-org/rav/redact_changes/6
Pass room_version into `event_from_pdu_json`
2020-02-07 09:22:15 +00:00
Patrick Cloke
7765bf3989
Limit the number of events that can be requested when backfilling events (#6864)
Limit the maximum number of events requested when backfilling events.
2020-02-06 13:25:24 -05:00
Richard van der Hoff
928edef979 Pass room_version into event_from_pdu_json
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
Richard van der Hoff
b0c8bdd49d
pass room version into FederationClient.send_join (#6854)
... which allows us to sanity-check the create event.
2020-02-06 15:50:39 +00:00
timfi
bce557175b
Allow empty federation_certificate_verification_whitelist (#6849) 2020-02-06 14:45:01 +00:00
PeerD
99fcc96289
Third party event rules Update (#6781) 2020-02-06 14:15:29 +00:00
Erik Johnston
ed630ea17c
Reduce amount of logging at INFO level. (#6862)
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.

Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
Richard van der Hoff
9bcd37146e
Merge pull request #6823 from matrix-org/rav/redact_changes/5
pass room versions around
2020-02-06 11:32:33 +00:00
Erik Johnston
6a7e90ad78 1.10.0rc2 2020-02-06 10:40:08 +00:00
Patrick Cloke
5e019069ab
Merge pull request #6855 from matrix-org/clokep/readme-pip-install
Add quotes around the pip install target to avoid my shell complaining
2020-02-05 13:29:52 -05:00
Patrick Cloke
39c2d26e0b Add quotes around pip install target (my shell complained without them). 2020-02-05 12:53:18 -05:00
Richard van der Hoff
ff70ec0a00 Newsfile 2020-02-05 17:43:57 +00:00
Richard van der Hoff
577f460369
Merge pull request #6840 from matrix-org/rav/federation_client_async
Port much of `synapse.federation.federation_client` to async/await
2020-02-05 16:56:39 +00:00
Erik Johnston
a58860e480
Check sender_key matches on inbound encrypted events. (#6850)
If they don't then the device lists are probably out of sync.
2020-02-05 14:02:39 +00:00
Hubert Chathi
60d0672426 Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
add device signatures to device key query results
2020-02-05 10:54:49 +00:00
Michael Kaye
a831d2e4e3
Reduce performance logging to DEBUG (#6833)
* Reduce tnx performance logging to DEBUG
* Changelog.d
2020-02-05 08:57:37 +00:00
Richard van der Hoff
d88e0ec080
Database updates to populate rooms.room_version (#6847)
We're going to need this so that we can figure out how to handle redactions when fetching events from the database.
2020-02-04 21:31:08 +00:00
Erik Johnston
6475382d80
Fix detecting unknown devices from remote encrypted events. (#6848)
We were looking at the wrong event type (`m.room.encryption` vs
`m.room.encrypted`).

Also fixup the duplicate `EvenTypes` entries.

Introduced in #6776.
2020-02-04 17:25:54 +00:00
Hubert Chathi
74bf3fdbb9
Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
add device signatures to device key query results
2020-02-04 12:03:54 -05:00
Michael Kaye
c87572d6e4
Update CONTRIBUTING.md about merging PRs. (#6846) 2020-02-04 16:21:09 +00:00