Commit Graph

483 Commits

Author SHA1 Message Date
Andrew Morgan
f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595)
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
Olof Johansson
fe434cd3c9
Fix a bug in automatic user creation with m.login.jwt. (#7585) 2020-06-01 12:55:07 -04:00
Dirk Klimpel
901b1fa561
Email notifications for new users when creating via the Admin API. (#7267) 2020-06-01 15:34:33 +01:00
Erik Johnston
06a02bc1ce
Convert sending mail to async/await. (#7557)
Mainly because sometimes the email push code raises exceptions where the
stack traces have gotten lost, which is hopefully fixed by this.
2020-05-22 13:41:11 +01:00
Andrew Morgan
225c165087
Allow expired accounts to logout (#7443) 2020-05-14 16:32:49 +01:00
Richard van der Hoff
dede23ff1e Synapse 1.13.0rc2 (2020-05-14)
==============================
 
 Bugfixes
 --------
 
 - Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. ([\#7376](https://github.com/matrix-org/synapse/issues/7376))
 - Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. ([\#7483](https://github.com/matrix-org/synapse/issues/7483))
 
 Internal Changes
 ----------------
 
 - Fix linting errors in new version of Flake8. ([\#7470](https://github.com/matrix-org/synapse/issues/7470))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl69IQ8ACgkQOSor00I9
 eP87lAf8DK+v6cs2U0BoD5opzQ7ZazJT6JYTmnMBaTzHU6Wx20V2ttkF7Vpwm3WU
 Zsz0048tdYtHFyYBQ1kF5RNIBBJwV8SA/QUcPkR7FVpwZMLR2q4aJn0EE7kC9OMf
 tYsmdbHeBdyfLXpXzazxWlgHquLyEIt52ykAcCphjx/Jl2fAExFEhtfsxpECoJ2f
 8Dqhjg3WFjd6QWU6AFkElbwHUYCdIWdJOcsC8N1p8OvBmDz5QXv/RlYipHE00Cpx
 QQQOgEjdRc6dlz2mbetMklnfII3p2kO9bzNdmEpOzT0Zt7nFaGdntW4I1QA0yJfa
 gows9bYMzhqYk7YSiyTYOZ4qyavVtw==
 =N/zZ
 -----END PGP SIGNATURE-----

Merge tag 'v1.13.0rc2' into develop

Synapse 1.13.0rc2 (2020-05-14)
==============================

Bugfixes
--------

- Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. ([\#7376](https://github.com/matrix-org/synapse/issues/7376))
- Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. ([\#7483](https://github.com/matrix-org/synapse/issues/7483))

Internal Changes
----------------

- Fix linting errors in new version of Flake8. ([\#7470](https://github.com/matrix-org/synapse/issues/7470))
2020-05-14 11:46:38 +01:00
Patrick Cloke
5d64fefd6c
Do not validate that the client dict is stable during UI Auth. (#7483)
This backs out some of the validation for the client dictionary and logs if
this changes during a user interactive authentication session instead.
2020-05-13 14:26:44 -04:00
Andrew Morgan
5cf758cdd6 Merge branch 'release-v1.13.0' into develop
* release-v1.13.0:
  Don't UPGRADE database rows
  RST indenting
  Put rollback instructions in upgrade notes
  Fix changelog typo
  Oh yeah, RST
  Absolute URL it is then
  Fix upgrade notes link
  Provide summary of upgrade issues in changelog. Fix )
  Move next version notes from changelog to upgrade notes
  Changelog fixes
  1.13.0rc1
  Documentation on setting up redis (#7446)
  Rework UI Auth session validation for registration (#7455)
  Fix errors from malformed log line (#7454)
  Drop support for redis.dbid (#7450)
2020-05-11 16:46:33 +01:00
Patrick Cloke
0ad6d28b0d
Rework UI Auth session validation for registration (#7455)
Be less strict about validation of UI authentication sessions during
registration to match client expecations.
2020-05-08 16:08:58 -04:00
Manuel Stahl
a4a5ec4096
Add room details admin endpoint (#7317) 2020-05-07 15:33:07 -04:00
Patrick Cloke
627b0f5f27
Persist user interactive authentication sessions (#7302)
By persisting the user interactive authentication sessions to the database, this fixes
situations where a user hits different works throughout their auth session and also
allows sessions to persist through restarts of Synapse.
2020-04-30 13:47:49 -04:00
Richard van der Hoff
c2e1a2110f
Fix limit logic for EventsStream (#7358)
* Factor out functions for injecting events into database

I want to add some more flexibility to the tools for injecting events into the
database, and I don't want to clutter up HomeserverTestCase with them, so let's
factor them out to a new file.

* Rework TestReplicationDataHandler

This wasn't very easy to work with: the mock wrapping was largely superfluous,
and it's useful to be able to inspect the received rows, and clear out the
received list.

* Fix AssertionErrors being thrown by EventsStream

Part of the problem was that there was an off-by-one error in the assertion,
but also the limit logic was too simple. Fix it all up and add some tests.
2020-04-29 12:30:36 +01:00
Manuel Stahl
04dd7d182d
Return total number of users and profile attributes in admin users endpoint (#6881)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-04-28 18:19:36 +01:00
Brendan Abolivier
2e3b9a0fcb
Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token""
This reverts commit 1adf6a5587.
2020-04-23 11:23:53 +02:00
Dirk Klimpel
6b6685db9f
Extend room admin api with additional attributes (#7225) 2020-04-22 13:38:41 +01:00
Andrew Morgan
a48138784e
Allow specifying the value of Accept-Language header for URL previews (#7265) 2020-04-15 13:35:29 +01:00
Zay11Zay
f1097e7720
Fix the parameters of a test fixture (#7243) 2020-04-14 15:37:28 -04:00
Andrew Morgan
ac978ab3da
Default PL100 to enable encryption in a room (#7230) 2020-04-09 18:45:38 +01:00
Patrick Cloke
b85d7652ff
Do not allow a deactivated user to login via SSO. (#7240) 2020-04-09 13:28:13 -04:00
Richard van der Hoff
6486c96b65
Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_tests
Add tests for outbound device pokes
2020-03-30 13:59:07 +01:00
Richard van der Hoff
b7da598a61 Always whitelist the login fallback for SSO (#7153)
That fallback sets the redirect URL to itself (so it can process the login
token then return gracefully to the client). This would make it pointless to
ask the user for confirmation, since the URL the confirmation page would be
showing wouldn't be the client's.
2020-03-27 20:24:52 +00:00
Dirk Klimpel
fb69690761
Admin API to join users to a room. (#7051) 2020-03-27 19:16:43 +00:00
Dirk Klimpel
8327eb9280
Add options to prevent users from changing their profile. (#7096) 2020-03-27 19:15:23 +00:00
Dirk Klimpel
e8e2ddb60a
Allow server admins to define and enforce a password policy (MSC2000). (#7118) 2020-03-26 16:51:13 +00:00
Patrick Cloke
1c1242acba
Validate that the session is not modified during UI-Auth (#7068) 2020-03-26 07:39:34 -04:00
Richard van der Hoff
28d9d6e8a9 Remove spurious "name" parameter to default_config
this is never set to anything other than "test", and is a source of unnecessary
boilerplate.
2020-03-24 18:33:49 +00:00
Richard van der Hoff
39230d2171
Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Patrick Cloke
60724c46b7
Remove special casing of m.room.aliases events (#7034) 2020-03-17 07:37:04 -04:00
Richard van der Hoff
6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)"
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
2020-03-17 11:25:01 +00:00
Brendan Abolivier
54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
dklimpel
39f6595b4a lint, fix tests 2020-03-09 22:13:20 +01:00
dklimpel
885134529f updates after review 2020-03-09 22:09:29 +01:00
dklimpel
50ea178c20 lint 2020-03-09 19:57:04 +01:00
dklimpel
04f4b5f6f8 add tests 2020-03-09 19:51:31 +01:00
Brendan Abolivier
43f874055d
Merge branch 'master' into develop 2020-03-03 15:20:49 +00:00
Patrick Cloke
7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Richard van der Hoff
b68041df3d Add a whitelist for the SSO confirmation step. 2020-03-02 17:05:09 +00:00
Richard van der Hoff
27d099edd6 Merge remote-tracking branch 'origin/release-v1.11.1' into release-v1.11.1 2020-03-02 16:43:33 +00:00
Brendan Abolivier
b2bd54a2e3 Add a confirmation step to the SSO login flow 2020-03-02 16:36:32 +00:00
Dirk Klimpel
bbeee33d63 Fixed set a user as an admin with the new API (#6928)
Fix #6910
2020-03-02 13:28:50 +00:00
Dirk Klimpel
9b06d8f8a6
Fixed set a user as an admin with the new API (#6928)
Fix #6910
2020-02-28 09:58:05 +00:00
Andrew Morgan
8c75b621bf
Ensure 'deactivated' parameter is a boolean on user admin API, Fix error handling of call to deactivate user (#6990) 2020-02-26 12:22:55 +00:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff
2fb7794e60
Merge pull request #6949 from matrix-org/rav/list_room_aliases_peekable
Make room alias lists peekable
2020-02-19 11:19:11 +00:00
Richard van der Hoff
880aaac1d8
Move MSC2432 stuff onto unstable prefix (#6948)
it's not in the spec yet, so needs to be unstable. Also add a feature flag for it. Also add a test for admin users.
2020-02-19 10:40:27 +00:00
Richard van der Hoff
709e81f518 Make room alias lists peekable
As per
https://github.com/matrix-org/matrix-doc/pull/2432#pullrequestreview-360566830,
make room alias lists accessible to users outside world_readable rooms.
2020-02-19 08:53:32 +00:00
Richard van der Hoff
adfaea8c69
Implement GET /_matrix/client/r0/rooms/{roomId}/aliases (#6939)
per matrix-org/matrix-doc#2432
2020-02-18 16:23:25 +00:00
Andrew Morgan
3404ad289b
Raise the default power levels for invites, tombstones and server acls (#6834) 2020-02-17 13:23:37 +00:00
Andrew Morgan
d8994942f2
Return a 404 for admin api user lookup if user not found (#6901) 2020-02-12 18:14:10 +00:00
Dirk Klimpel
56ca93ef59
Admin api to add an email address (#6789) 2020-02-07 10:29:36 +00:00
Richard van der Hoff
184303b865
MSC2260: Block direct sends of m.room.aliases events (#6794)
as per MSC2260
2020-01-30 17:20:55 +00:00
Andrew Morgan
90a28fb475
Admin API to list, filter and sort rooms (#6720) 2020-01-22 13:36:43 +00:00
Brendan Abolivier
6ae0c8db33
Lint + changelog 2020-01-22 12:38:18 +00:00
Brendan Abolivier
d9a8728b11
Remove unused import 2020-01-22 12:30:49 +00:00
Brendan Abolivier
67aa18e8dc
Add tests for thumbnailing 2020-01-22 12:28:07 +00:00
Andrew Morgan
aa9b00fb2f
Fix and add test to deprecated quarantine media admin api (#6756) 2020-01-22 11:05:50 +00:00
Erik Johnston
74b74462f1
Fix /events/:event_id deprecated API. (#6731) 2020-01-20 17:38:09 +00:00
Erik Johnston
ceecedc68b
Fix changing password via user admin API. (#6730) 2020-01-20 17:23:59 +00:00
Erik Johnston
19a1aac48c
Fix purge_room admin API (#6711) 2020-01-15 18:13:47 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership table (#6655)
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Andrew Morgan
1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Manuel Stahl
d2906fe666 Allow admin users to create or modify users without a shared secret (#6495)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-01-09 13:31:00 +00:00
Manuel Stahl
7caaa29daa Fix GET request on /_synapse/admin/v2/users endpoint (#6563)
Fixes #6552
2020-01-08 13:26:40 +00:00
Richard van der Hoff
573fee759c
Back out ill-advised notary server hackery (#6657)
This was ill-advised. We can't modify verify_keys here, because the response
object has already been signed by the requested key.

Furthermore, it's somewhat unnecessary because existing versions of Synapse
(which get upset that the notary key isn't present in verify_keys) will fall
back to a direct fetch via `/key/v2/server`.

Also: more tests for fetching keys via perspectives: it would be nice if we actually tested when our fetcher can't talk to our notary impl.
2020-01-08 13:24:10 +00:00
Richard van der Hoff
18674eebb1
Workaround for error when fetching notary's own key (#6620)
* Kill off redundant SynapseRequestFactory

We already get the Site via the Channel, so there's no need for a dedicated
RequestFactory: we can just use the right constructor.

* Workaround for error when fetching notary's own key

As a notary server, when we return our own keys, include all of our signing
keys in verify_keys.

This is a workaround for #6596.
2020-01-06 12:28:58 +00:00
Richard van der Hoff
6e8f8e14f2 Merge release-v1.7.1 into develop 2019-12-18 09:51:51 +00:00
Will Hunt
bfb95654c9 Add option to allow profile queries without sharing a room (#6523) 2019-12-16 16:11:55 +00:00
Brendan Abolivier
a820069549
Incorporate review 2019-12-16 16:00:18 +00:00
Brendan Abolivier
a29420f9f4
Lint 2019-12-16 14:55:50 +00:00
Brendan Abolivier
596dd9914d
Add test case 2019-12-16 14:53:21 +00:00
Erik Johnston
257ef2c727 Port handlers.account_validity to async/await. 2019-12-10 14:40:15 +00:00
Erik Johnston
a9b393340f
Merge pull request #6484 from matrix-org/erikj/port_sync_handler
Port SyncHandler to async/await
2019-12-09 11:32:44 +00:00
Erik Johnston
2ace775d88 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_database_class 2019-12-06 11:33:34 +00:00
Erik Johnston
b3a4e35ca8 Fixup functions to consistently return deferreds 2019-12-06 10:40:05 +00:00
Erik Johnston
4ca3ef10b9 Fixup tests 2019-12-05 15:57:28 +00:00
Erik Johnston
756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Erik Johnston
ee86abb2d6 Remove underscore from SQLBaseStore functions 2019-12-04 16:23:43 +00:00
Brendan Abolivier
c530f9af4d
Merge pull request #6329 from matrix-org/babolivier/context_filters
Filter state, events_before and events_after in /context requests
2019-12-04 15:24:16 +00:00
Brendan Abolivier
65c6aee621
Un-remove room purge test 2019-12-04 14:37:04 +00:00
Brendan Abolivier
aeda1b3b94
Merge branch 'babolivier/context_filters' of github.com:matrix-org/synapse into babolivier/context_filters 2019-12-04 14:29:03 +00:00
Brendan Abolivier
9dc84b7989
Merge branch 'develop' into babolivier/context_filters 2019-12-04 14:23:44 +00:00
Brendan Abolivier
54dd5dc12b
Add ephemeral messages support (MSC2228) (#6409)
Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). The parts that differ are:

* the feature is hidden behind a configuration flag (`enable_ephemeral_messages`)
* self-destruction doesn't happen for state events
* only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one)
* doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
2019-12-03 19:19:45 +00:00
Erik Johnston
8c9a713f8d Add tests 2019-11-28 11:32:06 +00:00
Brendan Abolivier
d31f69afa0
Merge pull request #6358 from matrix-org/babolivier/message_retention
Implement message retention policies (MSC1763)
2019-11-27 15:04:38 +00:00
Andrew Morgan
ce578031f4
Remove assertion and provide a clear warning on startup for missing public_baseurl (#6379) 2019-11-26 18:42:27 +00:00
Brendan Abolivier
9e937c28ee Merge branch 'develop' into babolivier/message_retention 2019-11-26 17:53:57 +00:00
Brendan Abolivier
f0ef970824
Don't restrict the tests to v1 rooms 2019-11-26 17:49:12 +00:00
Brendan Abolivier
4c1b799e1b
Merge branch 'develop' into babolivier/context_filters 2019-11-26 10:53:48 +00:00
Brendan Abolivier
e2a20326e8
Lint 2019-11-20 15:08:47 +00:00
Brendan Abolivier
6356f2088f
Test if a purge can make /messages return 500 responses 2019-11-20 12:09:06 +00:00
Brendan Abolivier
bf9a11c54d
Lint again 2019-11-19 13:30:04 +00:00
Brendan Abolivier
7c24d0f443
Lint 2019-11-19 13:22:37 +00:00
Andrew Morgan
c350bc2f92
Blacklist PurgeRoomTestCase (#6361) 2019-11-13 19:09:20 +00:00
Erik Johnston
f713c01e2b
Merge pull request #6295 from matrix-org/erikj/split_purge_history
Split purge API into events vs state and add PurgeEventsStorage
2019-11-08 10:19:15 +00:00
Brendan Abolivier
f03c9d3444
Don't apply retention policy based filtering on state events
As per MSC1763, 'Retention is only considered for non-state events.', so don't filter out state events based on the room's retention policy.
2019-11-06 18:40:04 +00:00
Erik Johnston
5c3363233c Fix deleting state groups during room purge.
And fix the tests to actually test that things got deleted.
2019-11-06 17:02:08 +00:00
Richard van der Hoff
55a7da247a
Merge branch 'develop' into rav/url_preview_limit_title 2019-11-05 17:08:07 +00:00
Richard van der Hoff
e78167c94b
Apply suggestions from code review
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-11-05 16:46:39 +00:00
Richard van der Hoff
e9bfe719ba Strip overlong OpenGraph data from url preview
... to stop people causing DoSes with malicious web pages
2019-11-05 15:51:18 +00:00
Brendan Abolivier
a6863da249
Lint 2019-11-05 14:50:19 +00:00