Commit Graph

672 Commits

Author SHA1 Message Date
Patrick Cloke
0b3112123d
Use mock from the stdlib. (#9772) 2021-04-09 13:44:38 -04:00
Jonathan de Jong
2ca4e349e9
Bugbear: Add Mutable Parameter fixes (#9682)
Part of #9366

Adds in fixes for B006 and B008, both relating to mutable parameter lint errors.

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-04-08 22:38:54 +01:00
Jonathan de Jong
e2b8a90897
Update mypy configuration: no_implicit_optional = True (#9742) 2021-04-05 09:10:18 -04:00
Andrew Morgan
7b06f85c0e
Ensure we use a copy of the event content dict before modifying it in serialize_event (#9585)
This bug was discovered by DINUM. We were modifying `serialized_event["content"]`, which - if you've got `USE_FROZEN_DICTS` turned on or are [using a third party rules module](17cd48fe51/synapse/events/third_party_rules.py (L73-L76)) - will raise a 500 if you try to a edit a reply to a message.

`serialized_event["content"]` could be set to the edit event's content, instead of a copy of it, which is bad as we attempt to modify it. Instead, we also end up modifying the original event's content. DINUM uses a third party rules module, which meant the event's content got frozen and thus an exception was raised.

To be clear, the problem is not that the event's content was frozen. In fact doing so helped us uncover the fact we weren't copying event content correctly.
2021-03-17 16:51:55 +00:00
Dirk Klimpel
8000cf1315
Return m.change_password.enabled=false if local database is disabled (#9588)
Instead of if the user does not have a password hash. This allows a SSO
user to add a password to their account, but only if the local password
database is configured.
2021-03-16 11:44:25 -04:00
Richard van der Hoff
dd69110d95
Add support for stable MSC2858 API (#9617)
The stable format uses different brand identifiers, so we need to support two
identifiers for each IdP.
2021-03-16 11:21:26 +00:00
Patrick Cloke
33a02f0f52
Fix additional type hints from Twisted upgrade. (#9518) 2021-03-03 15:47:38 -05:00
Richard van der Hoff
15090de850
SSO: redirect to public URL before setting cookies (#9436)
... otherwise, we don't get the cookie back.
2021-02-26 14:02:06 +00:00
Andrew Morgan
0a363f9ca4
Remove cache for get_shared_rooms_for_users (#9416)
This PR remove the cache for the `get_shared_rooms_for_users` storage method (the db method driving the experimental "what rooms do I share with this user?" feature: [MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)). Currently subsequent requests to the endpoint will return the same result, even if your shared rooms with that user have changed.

The cache was added in https://github.com/matrix-org/synapse/pull/7785, but we forgot to ensure it was invalidated appropriately.

Upon attempting to invalidate it, I found that the cache had to be entirely invalidated whenever a user (remote or local) joined or left a room. This didn't make for a very useful cache, especially for a function that may or may not be called very often. Thus, I've opted to remove it instead of invalidating it.
2021-02-22 16:52:45 +00:00
Rishabh Arya
e17553e185
Parse ui_auth.session_timeout as a duration (instead of treating it as ms) (#9426) 2021-02-18 09:18:14 -05:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke
731e08c63a
Handle missing data in power levels events during room upgrade. (#9395) 2021-02-16 08:31:39 -05:00
Patrick Cloke
74af356baf
Convert additional test-cases to homeserver test case. (#9396)
And convert some inlineDeferreds to async-friendly functions.
2021-02-16 08:04:15 -05:00
Patrick Cloke
2c9b4a5f16 Synapse 1.27.0rc2 (2021-02-11)
==============================
 
 Features
 --------
 
 - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297))
 
 Bugfixes
 --------
 
 - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302))
 - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313))
 
 Improved Documentation
 ----------------------
 
 - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAlX4AACgkQM/xY9qcR
 MEjxhg/+JzOGDmgxy4Vm9oU84htkkJsflU755ykxhwbIjVy2j+07pMgtWMmebXIS
 /tWjinwRB9OgKIK+j5RWdYzz/GRTkLCbafTLhhNOWsnaCO7nnAfZFozLOMd9g2qN
 309cS8efZkxGCISGSRm1QQKjQDx6HHx7nZVAwpqb778Q9TMrBYRAQeCk9yws7FfL
 GsP5YrBI42n84YhvflWA8J8QRYkeJJaggVWzE0XvwdI6raY1BmxMKcDvyl9iKpGb
 /QXsdketD3eJxdOlU92O5ZxRXmL00bdyAFdw2+J3Y+wjoGBnC5njKSFG3j8Z2UcP
 xIdB6w/zyGoPLKd4s7tkDI3axE7qrwFetA4NytannWGEHZ9q5tgOd5PA1kTeKYvn
 ao2os4pKMjWQOHiWLskqZVXgmoW5Kb2zgyZU/vhFAz8ImuidFULOjpEytAWqtt1B
 iuL/fRlM3z0BpwQNd3NBsblYTpZri8gdTp7ULJYtaKpT2MeG0sg3swJAptUjw9L4
 awbQuBFZ8o/E/0xwiGieSxbR6b6Nz5WiCyLaeMz6b3Y4YdomttOAZ6pL6E9y8ygU
 2wKYleSmg+tbaBKKZR5CSCwXOduSYPhWw7oqsfEfvT0NCQLuItfGNFZqxRzsf73H
 lCV0wwxNZBv8kRyMs6KivMvSI6SKinAjXS8AjzHE402ozdwOCv0=
 =NSyp
 -----END PGP SIGNATURE-----

Merge tag 'v1.27.0rc2' into develop

Synapse 1.27.0rc2 (2021-02-11)
==============================

Features
--------

- Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297))

Bugfixes
--------

- Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302))
- Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313))

Improved Documentation
----------------------

- Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))
2021-02-11 11:56:03 -05:00
Patrick Cloke
e40d88cff3
Backout changes for automatically calculating the public baseurl. (#9313)
This breaks some people's configurations (if their Client-Server API
is not accessed via port 443).
2021-02-11 11:16:54 -05:00
Patrick Cloke
8a33d217bd
Convert some test cases to use HomeserverTestCase. (#9377)
This has the side-effect of being able to remove use of `inlineCallbacks`
in the test-cases for cleaner tracebacks.
2021-02-11 10:29:09 -05:00
Richard van der Hoff
e288499c60
Social login UI polish (#9301) 2021-02-03 20:31:23 +00:00
Richard van der Hoff
18ab35284a Merge branch 'social_login' into develop 2021-02-01 17:28:37 +00:00
Richard van der Hoff
4167494c90
Replace username picker with a template (#9275)
There's some prelimiary work here to pull out the construction of a jinja environment to a separate function.

I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.
2021-02-01 15:52:50 +00:00
Richard van der Hoff
9c715a5f19
Fix SSO on workers (#9271)
Fixes #8966.

* Factor out build_synapse_client_resource_tree

Start a function which will mount resources common to all workers.

* Move sso init into build_synapse_client_resource_tree

... so that we don't have to do it for each worker

* Fix SSO-login-via-a-worker

Expose the SSO login endpoints on workers, like the documentation says.

* Update workers config for new endpoints

Add documentation for endpoints recently added (#8942, #9017, #9262)

* remove submit_token from workers endpoints list

this *doesn't* work on workers (yet).

* changelog

* Add a comment about the odd path for SAML2Resource
2021-02-01 15:47:59 +00:00
Richard van der Hoff
f78d07bf00
Split out a separate endpoint to complete SSO registration (#9262)
There are going to be a couple of paths to get to the final step of SSO reg, and I want the URL in the browser to consistent. So, let's move the final step onto a separate path, which we redirect to.
2021-02-01 13:15:51 +00:00
Erik Johnston
f2c1560eca
Ratelimit invites by room and target user (#9258) 2021-01-29 16:38:29 +00:00
Erik Johnston
4b73488e81
Ratelimit 3PID /requestToken API (#9238) 2021-01-28 17:39:21 +00:00
Richard van der Hoff
7fa1346f93 Merge branch 'social_login' into develop 2021-01-27 17:27:24 +00:00
Richard van der Hoff
a737cc2713
Implement MSC2858 support (#9183)
Fixes #8928.
2021-01-27 12:41:24 +00:00
Patrick Cloke
4a55d267ee
Add an admin API for shadow-banning users. (#9209)
This expands the current shadow-banning feature to be usable via
the admin API and adds documentation for it.

A shadow-banned users receives successful responses to their
client-server API requests, but the events are not propagated into rooms.

Shadow-banning a user should be used as a tool of last resort and may lead
to confusing or broken behaviour for the client.
2021-01-25 14:49:39 -05:00
Richard van der Hoff
7447f19702
Prefix idp_id with "oidc-" (#9189)
... to avoid clashes with other SSO mechanisms
2021-01-21 12:25:02 +00:00
Richard van der Hoff
02070c69fa
Fix bugs in handling clientRedirectUrl, and improve OIDC tests (#9127, #9128)
* Factor out a common TestHtmlParser

Looks like I'm doing this in a few different places.

* Improve OIDC login test

Complete the OIDC login flow, rather than giving up halfway through.

* Ensure that OIDC login works with multiple OIDC providers

* Fix bugs in handling clientRedirectUrl

 - don't drop duplicate query-params, or params with no value
 - allow utf-8 in query-params
2021-01-18 14:52:49 +00:00
Richard van der Hoff
b5dea8702d Fix test failure due to bad merge
0dd2649c1 (#9112) changed the signature of `auth_via_oidc`. Meanwhile,
26d10331e (#9091) introduced a new test which relied on the old signature of
`auth_via_oidc`. The two branches were never tested together until they landed
in develop.
2021-01-15 18:03:33 +00:00
Richard van der Hoff
0dd2649c12
Improve UsernamePickerTestCase (#9112)
* make the OIDC bits of the test work at a higher level - via the REST api instead of poking the OIDCHandler directly.
* Move it to test_login.py, where I think it fits better.
2021-01-15 13:45:13 +00:00
Richard van der Hoff
26d10331e5 Add a test for wrong user returned by SSO 2021-01-13 20:22:41 +00:00
Richard van der Hoff
233c8b9fce
Add a test for UI-Auth-via-SSO (#9082)
* Add complete test for UI-Auth-via-SSO.

* review comments
2021-01-13 20:21:55 +00:00
Dirk Klimpel
7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932)
This only applies if the user's data is to be erased.
2021-01-12 16:30:15 -05:00
Richard van der Hoff
8a910f97a4 Add some tests for the IDP picker flow 2021-01-07 14:56:42 +00:00
Richard van der Hoff
bbd04441ed Fix type hints in test_login.py 2021-01-07 14:56:42 +00:00
Richard van der Hoff
3fc2399dbe black-format tests/rest/client/v1/test_login.py
black seems to want to reformat this, despite `black --check` being happy with
it :/
2021-01-07 12:17:37 +00:00
Richard van der Hoff
8d3d264052
Skip unit tests which require optional dependencies (#9031)
If we are lacking an optional dependency, skip the tests that rely on it.
2021-01-07 11:41:28 +00:00
Richard van der Hoff
d2c616a413
Combine the SSO Redirect Servlets (#9015)
* Implement CasHandler.handle_redirect_request

... to make it match OidcHandler and SamlHandler

* Clean up interface for OidcHandler.handle_redirect_request

Make it accept `client_redirect_url=None`.

* Clean up interface for `SamlHandler.handle_redirect_request`

... bring it into line with CAS and OIDC by making it take a Request parameter,
move the magic for `client_redirect_url` for UIA into the handler, and fix the
return type to be a `str` rather than a `bytes`.

* Define a common protocol for SSO auth provider impls

* Give SsoIdentityProvider an ID and register them

* Combine the SSO Redirect servlets

Now that the SsoHandler knows about the identity providers, we can combine the
various *RedirectServlets into a single implementation which delegates to the
right IdP.

* changelog
2021-01-04 18:13:49 +00:00
Patrick Cloke
a802606475
Support PyJWT v2.0.0. (#8986)
Tests were broken due to an API changing. The code used in Synapse
proper should be compatible with both versions already.
2020-12-22 13:00:14 -05:00
Patrick Cloke
5d4c330ed9
Allow re-using a UI auth validation for a period of time (#8970) 2020-12-18 07:33:57 -05:00
Richard van der Hoff
7a332850e6
Merge pull request #8951 from matrix-org/rav/username_picker_2
More preparatory refactoring of the OidcHandler tests
2020-12-16 14:53:26 +00:00
Richard van der Hoff
2dd2e90e2b Test get_extra_attributes fallback
despite the warnings saying "don't implement get_extra_attributes", we had
implemented it, so the tests weren't doing what we thought they were.
2020-12-15 23:10:59 +00:00
Richard van der Hoff
c9dd47d668 lint 2020-12-15 22:35:50 +00:00
Richard van der Hoff
394516ad1b Remove spurious "SynapseRequest" result from `make_request"
This was never used, so let's get rid of it.
2020-12-15 22:35:40 +00:00
Richard van der Hoff
ac2acf1524 Remove redundant reading of SynapseRequest.args
this didn't seem to be doing a lot, so remove it.
2020-12-15 22:35:03 +00:00
Richard van der Hoff
0378581c13 remove 'response' result from _get_shared_rooms 2020-12-15 22:34:20 +00:00
Richard van der Hoff
7eebe4b3fc Replace request.code with channel.code
The two are equivalent, but really we want to check the HTTP result that got
returned to the channel, not the code that the Request object *intended* to
return to the channel.
2020-12-15 22:32:12 +00:00
Erik Johnston
1d55c7b567
Don't ratelimit autojoining of rooms (#8921)
Fixes #8866
2020-12-11 10:17:49 +00:00
Richard van der Hoff
cf3b8156be
Fix errorcode for disabled registration (#8867)
The spec says we should return `M_FORBIDDEN` when someone tries to register and
registration is disabled.
2020-12-03 15:41:19 +00:00
Richard van der Hoff
ed5172852a
Merge pull request #8858 from matrix-org/rav/sso_uia
UIA: offer only available auth flows
2020-12-02 20:06:53 +00:00
Richard van der Hoff
0bac276890 UIA: offer only available auth flows
During user-interactive auth, do not offer password auth to users with no
password, nor SSO auth to users with no SSO.

Fixes #7559.
2020-12-02 18:54:15 +00:00
Patrick Cloke
30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Andrew Morgan
edb3d3f827
Allow specifying room version in 'RestHelper.create_room_as' and add typing (#8854)
This PR adds a `room_version` argument to the `RestHelper`'s `create_room_as` function for tests. I plan to use this for testing knocking, which currently uses an unstable room version.
2020-12-02 10:38:18 +00:00
Richard van der Hoff
4d9496559d
Support "identifier" dicts in UIA (#8848)
The spec requires synapse to support `identifier` dicts for `m.login.password`
user-interactive auth, which it did not (instead, it required an undocumented
`user` parameter.)

To fix this properly, we need to pull the code that interprets `identifier`
into `AuthHandler.validate_login` so that it can be called from the UIA code.

Fixes #5665.
2020-12-01 17:42:26 +00:00
Jonathan de Jong
ca60822b34
Simplify the way the HomeServer object caches its internal attributes. (#8565)
Changes `@cache_in_self` to use underscore-prefixed attributes.
2020-11-30 13:28:44 -05:00
Dirk Klimpel
3f0ff53158
Remove deprecated /_matrix/client/*/admin endpoints (#8785)
These are now only available via `/_synapse/admin/v1`.
2020-11-25 16:26:11 -05:00
Richard van der Hoff
acfe3b3065 Remove redundant HomeserverTestCase.render 2020-11-16 18:24:08 +00:00
Richard van der Hoff
be8fa65d0b Remove redundant calls to render() 2020-11-16 18:24:08 +00:00
Richard van der Hoff
129ae841e5 Make make_request actually render the request
remove the stubbing out of `request.process`, so that `requestReceived` also renders the request via the appropriate resource.

Replace render() with a stub for now.
2020-11-16 18:24:00 +00:00
Richard van der Hoff
bebfb9a97b
Merge branch 'develop' into rav/pass_site_to_make_request 2020-11-16 15:22:40 +00:00
Richard van der Hoff
ebc405446e
Add a custom_headers param to make_request (#8760)
Some tests want to set some custom HTTP request headers, so provide a way to do
that before calling requestReceived().
2020-11-16 14:45:22 +00:00
Richard van der Hoff
cfd895a22e use global make_request() directly where we have a custom Resource
Where we want to render a request against a specific Resource, call the global
make_request() function rather than the one in HomeserverTestCase, allowing us
to pass in an appropriate `Site`.
2020-11-15 23:09:03 +00:00
Richard van der Hoff
9debe657a3 pass a Site into make_request 2020-11-15 23:09:03 +00:00
Richard van der Hoff
d3523e3e97 pass a Site into RestHelper 2020-11-15 23:09:03 +00:00
Erik Johnston
f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Andrew Morgan
74f29284aa
Remove some extraneous @unittest.INFOs on unit tests (#8592) 2020-10-20 11:49:15 +01:00
Richard van der Hoff
898196f1cc guard against accidental modification 2020-10-13 23:24:50 +01:00
Richard van der Hoff
617e8a4653 Allow ThirdPartyRules modules to replace event content
Support returning a new event dict from `check_event_allowed`.
2020-10-13 23:24:50 +01:00
Erik Johnston
b2486f6656
Fix message duplication if something goes wrong after persisting the event (#8476)
Should fix #3365.
2020-10-13 12:07:56 +01:00
Andrew Morgan
66ac4b1e34
Allow modules to create and send events into rooms (#8479)
This PR allows Synapse modules making use of the `ModuleApi` to create and send non-membership events into a room. This can useful to have modules send messages, or change power levels in a room etc. Note that they must send event through a user that's already in the room.

The non-membership event limitation is currently arbitrary, as it's another chunk of work and not necessary at the moment.
2020-10-09 13:46:36 +01:00
Patrick Cloke
c9c0ad5e20
Remove the deprecated Handlers object (#8494)
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Richard van der Hoff
a024461130
Additional tests for third-party event rules (#8468)
* Optimise and test state fetching for 3p event rules

Getting all the events at once is much more efficient than getting them
individually

* Test that 3p event rules can modify events
2020-10-06 16:31:31 +01:00
Richard van der Hoff
9c0b168cff
Merge pull request #8467 from matrix-org/rav/fix_3pevent_rules
Fix third-party event modules for `check_visibility_can_be_modified` check
2020-10-06 11:32:53 +01:00
Richard van der Hoff
785437dc0d
Update default room version to 6 (#8461)
Per https://github.com/matrix-org/matrix-doc/pull/2788
2020-10-05 21:40:51 +01:00
Richard van der Hoff
4cd1448d0e Fix third-party event modules for check_visibility_can_be_modified check
PR #8292 tried to maintain backwards compat with modules which don't provide a
`check_visibility_can_be_modified` method, but the tests weren't being run,
and the check didn't work.
2020-10-05 20:29:52 +01:00
Andrew Morgan
0991a2da93
Allow ThirdPartyEventRules modules to manipulate public room state (#8292)
This PR allows `ThirdPartyEventRules` modules to view, manipulate and block changes to the state of whether a room is published in the public rooms directory.

While the idea of whether a room is in the public rooms list is not kept within an event in the room, `ThirdPartyEventRules` generally deal with controlling which modifications can happen to a room. Public rooms fits within that idea, even if its toggle state isn't controlled through a state event.
2020-10-05 14:57:46 +01:00
Erik Johnston
7941372ec8
Make token serializing/deserializing async (#8427)
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
2020-09-30 20:29:19 +01:00
Erik Johnston
ea70f1c362
Various clean ups to room stream tokens. (#8423) 2020-09-29 21:48:33 +01:00
Andrew Morgan
1c6b8752b8
Only assert valid next_link params when provided (#8417)
Broken in https://github.com/matrix-org/synapse/pull/8275 and has yet to be put in a release. Fixes https://github.com/matrix-org/synapse/issues/8418.

`next_link` is an optional parameter. However, we were checking whether the `next_link` param was valid, even if it wasn't provided. In that case, `next_link` was `None`, which would clearly not be a valid URL.

This would prevent password reset and other operations if `next_link` was not provided, and the `next_link_domain_whitelist` config option was set.
2020-09-29 12:36:44 +01:00
Will Hunt
68c7a6936f
Allow appservice users to /login (#8320)
Add ability for ASes to /login using the `uk.half-shot.msc2778.login.application_service` login `type`.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-09-18 14:55:13 +01:00
Andrew Morgan
a3a90ee031
Show a confirmation page during user password reset (#8004)
This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset.

This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.
2020-09-10 11:45:12 +01:00
reivilibre
a5370072b5
Don't remember enabled of deleted push rules and properly return 404 for missing push rules in .../actions and .../enabled (#7796)
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-09 11:39:39 +01:00
Andrew Morgan
094896a69d
Add a config option for validating 'next_link' parameters against a domain whitelist (#8275)
This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285

They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality.

This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint.

This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.
2020-09-08 16:03:09 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Brendan Abolivier
5a1dd297c3
Re-implement unread counts (again) (#8059) 2020-09-02 17:19:37 +01:00
Will Hunt
b257c788c0
Add /user/{user_id}/shared_rooms/ api (#7785)
* Add shared_rooms api

* Add changelog

* Add .

* Wrap response in {"rooms": }

* linting

* Add unstable_features key

* Remove options from isort that aren't part of 5.x

`-y` and `-rc` are now default behaviour and no longer exist.

`dont-skip` is no longer required

https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020

* Update imports to make isort happy

* Add changelog

* Update tox.ini file with correct invocation

* fix linting again for isort

* Vendor prefix unstable API

* Fix to match spec

* import Codes

* import Codes

* Use FORBIDDEN

* Update changelog.d/7785.feature

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Implement get_shared_rooms_for_users

* a comma

* trailing whitespace

* Handle the easy feedback

* Switch to using runInteraction

* Add tests

* Feedback

* Seperate unstable endpoint from v2

* Add upgrade node

* a line

* Fix style by adding a blank line at EOF.

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Tulir Asokan <tulir@maunium.net>

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update UPGRADE.rst

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Fix UPGRADE/CHANGELOG unstable paths

unstable unstable unstable

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2020-09-02 13:18:40 +01:00
Patrick Cloke
e00816ad98
Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
Andrew Morgan
b8f20e4276
Remove remaining is_guest argument uses from get_room_data calls (#8181)
#8174 removed the `is_guest` parameter from `get_room_data`, at the same time that #8157 was merged using it, colliding together to break unit tests on develop.

This PR removes the `is_guest` parameter from the call in the broken test.

Uses the same changelog as #8174.
2020-08-26 17:26:56 +01:00
Patrick Cloke
6fe12c9512
Do not propagate typing notifications from shadow-banned users. (#8176) 2020-08-26 12:05:36 -04:00
Andrew Morgan
e0d6244beb
Remove unused parameter from, and add safeguard in, get_room_data (#8174)
Small cleanup PR.

* Removed the unused `is_guest` argument
* Added a safeguard to a (currently) impossible code path, fixing static checking at the same time.
2020-08-26 15:07:35 +01:00
Patrick Cloke
2e6c90ff84
Do not propagate profile changes of shadow-banned users into rooms. (#8157) 2020-08-26 08:49:01 -04:00
Patrick Cloke
56efa9ec71
Fix rate limiting unit tests. (#8167)
These were passing on the release-v1.19.1 branch but started failing once merged
to develop.
2020-08-26 07:19:20 -04:00
Brendan Abolivier
6e1c64a668 Synapse 1.19.1rc1 (2020-08-25)
==============================
 
 Bugfixes
 --------
 
 - Fix a bug introduced in v1.19.0 where appservices with ratelimiting disabled would still be ratelimited when joining rooms. ([\#8139](https://github.com/matrix-org/synapse/issues/8139))
 - Fix a bug introduced in v1.19.0 that would cause e.g. profile updates to fail due to incorrect application of rate limits on join requests. ([\#8153](https://github.com/matrix-org/synapse/issues/8153))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAl9FIKkACgkQHgFcFF8Z
 Fs3zIw/9H40ieb73Iay6ecQeOSfHiMVMzvJqYbKgho/a6h5JDHir+NGpwuLFdeGM
 eHR07QkQgUU9+dLNTMOQpCKTIsU70wvzH1vTDINS3ChjnRBdrHKqhAG6ZyEt1dJx
 kxYX54zsQUwiwshMKbJ+DPclHaBFnL+SY5OFfqCNjvaNob59DbHAL3tlSktPc2go
 tGmj81q0dWY6maMCGI3IIYcrW7oLi+4TwosZual5Hz/xgRBiGaKHXRIJnInvkXpl
 R+rSOmpYraapfDPHzPyQgLN4Dt7aAccGho843tt7dAVfd2GRSaUkLGXVXCdoruQG
 CRjY1P3BRBzRBx6o80Uw7Ah0hsoVgpJTSzY008KigJce+IiRWG5sgPjoubhfK0MA
 BqzmCa3/lrR+/WUOf4+w6HSfRncKawgAp7Y7wVj4nQF5fc8mwpFLz4pA/C2YOyjp
 nYXCHf60/KSBDhnr0ZRAhAby4MJoYSf03djFG1oef5SVzOzHD7zho9oBnEz15Tab
 XXkg1iJ7AhNFiQjsY4H1sl2onoF4T7B53NOnUEwD0oll+nXIYGe6hlNuq6x4j6l9
 39ZlMoe9zK28LoKKWa1RDug8z+PmarKRJ2zATlHIb2RGeVX+oFfaKVsIbJtupJVC
 8HSFt7gcgLCdUazk6taKpOHeVyGxK6WUkLnCMHzD2rzPhzpSyws=
 =0rHM
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAl9FJK4ACgkQHgFcFF8Z
 Fs0O2xAAgxqCfAHylVuwunRppwV49Bvq6H0mMM29hYBogGB5cmh1vRnUm5GsPQt0
 9vTKlwMz6XKjLs3TLqYsnZfXqK+lyaN0xFXd7xWCNzFtEXoIvDnq+u3h5WGaKOC9
 PcXb2LSZoHC5yECBpoh58ZQEPKtYILEjo+OSDboIqHz4N5HSjMSPGPvMkUn6xMNG
 roWTAWKPd9juyE2dPzZxIoBWwJGn3D8EMkeTQDlExTTvmnDPyPvJ5MVUY/xaHLgy
 XV8lapFu/SzWAKotc5+9qkVN64obaxwovYTU9JnlqEc5+WlD+Jl+g0258Q1bV1H9
 341aQQJX08iYw3xw13xVgT0zLPRbp82O3/SHC3S1nz27HUWKXqUtsm6woDbgHIz5
 UPvKFQsp2dEN4tFXxkEHiossIVNGuXdRYwEjFQrxOwayCuS4cQwDADhqnzDU4hio
 LSVhtxs9rgLps4iKpcaRAqK8kifTrsomlQfh/7axPJQ43pmBR2PiItetlBW/9Le6
 KTH90ghLQzJwKFkIcFcvPhFMVqSyXI32+g5++YAPmNVy9M/7LdJxuEc9ifTWgwds
 LtV3/F8xlqd0qwl5IbwC6Wf19N06jdlRv/q1zL/Hb6qu3FLQeGd+/1aiC0rsbq15
 grdHVZkZi1iVF/zrOx24ctxQvgLyGHA+M7n/oIaIgxlT1S6+FUI=
 =49ZC
 -----END PGP SIGNATURE-----

Merge tag 'v1.19.1rc1' into develop

Synapse 1.19.1rc1 (2020-08-25)
==============================

Bugfixes
--------

- Fix a bug introduced in v1.19.0 where appservices with ratelimiting disabled would still be ratelimited when joining rooms. ([\#8139](https://github.com/matrix-org/synapse/issues/8139))
- Fix a bug introduced in v1.19.0 that would cause e.g. profile updates to fail due to incorrect application of rate limits on join requests. ([\#8153](https://github.com/matrix-org/synapse/issues/8153))
2020-08-25 15:48:11 +01:00
Patrick Cloke
cbd8d83da7
Stop shadow-banned users from sending non-member events. (#8142) 2020-08-24 13:58:56 -04:00
Brendan Abolivier
420484a334
Allow capping a room's retention policy (#8104) 2020-08-24 18:21:04 +01:00
Patrick Cloke
3f8f96be00
Fix flaky shadow-ban tests. (#8152) 2020-08-24 13:08:33 -04:00
Brendan Abolivier
393a811a41
Fix join ratelimiter breaking profile updates and idempotency (#8153) 2020-08-24 18:06:04 +01:00
Patrick Cloke
e259d63f73
Stop shadow-banned users from sending invites. (#8095) 2020-08-20 15:07:42 -04:00
Patrick Cloke
f594e434c3
Switch the JSON byte producer from a pull to a push producer. (#8116) 2020-08-19 08:07:57 -04:00
Erik Johnston
5dd73d029e
Add type hints to handlers.message and events.builder (#8067) 2020-08-12 15:05:50 +01:00
Brendan Abolivier
2ffd6783c7
Revert #7736 (#8039) 2020-08-06 17:15:35 +01:00
Patrick Cloke
d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Patrick Cloke
66f24449dd
Improve performance of the register endpoint (#8009) 2020-08-06 08:09:55 -04:00
Andrew Morgan
5d92a1428c
Prevent join->join membership transitions changing member count (#7977)
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.

However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.

This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.

Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).

Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
2020-08-03 21:54:24 +01:00
Brendan Abolivier
8dff4a1242
Re-implement unread counts (#7736) 2020-07-29 18:26:55 +01:00
Patrick Cloke
111e70d75c
Return the proper 403 Forbidden error during errors with JWT logins. (#7844) 2020-07-15 07:10:21 -04:00
Patrick Cloke
77d2c05410
Add the option to validate the iss and aud claims for JWT logins. (#7827) 2020-07-14 07:16:43 -04:00
Patrick Cloke
66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802) 2020-07-10 14:30:08 -04:00
Patrick Cloke
38e1fac886
Fix some spelling mistakes / typos. (#7811) 2020-07-09 09:52:58 -04:00
Dirk Klimpel
21a212f8e5
Fix inconsistent handling of upper and lower cases of email addresses. (#7021)
fixes #7016
2020-07-03 14:03:13 +01:00
Sorunome
6920e58136
add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675) 2020-06-24 10:23:55 +01:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Andrew Morgan
f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595)
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
Olof Johansson
fe434cd3c9
Fix a bug in automatic user creation with m.login.jwt. (#7585) 2020-06-01 12:55:07 -04:00
Erik Johnston
06a02bc1ce
Convert sending mail to async/await. (#7557)
Mainly because sometimes the email push code raises exceptions where the
stack traces have gotten lost, which is hopefully fixed by this.
2020-05-22 13:41:11 +01:00
Andrew Morgan
225c165087
Allow expired accounts to logout (#7443) 2020-05-14 16:32:49 +01:00
Patrick Cloke
5d64fefd6c
Do not validate that the client dict is stable during UI Auth. (#7483)
This backs out some of the validation for the client dictionary and logs if
this changes during a user interactive authentication session instead.
2020-05-13 14:26:44 -04:00
Patrick Cloke
0ad6d28b0d
Rework UI Auth session validation for registration (#7455)
Be less strict about validation of UI authentication sessions during
registration to match client expecations.
2020-05-08 16:08:58 -04:00
Patrick Cloke
627b0f5f27
Persist user interactive authentication sessions (#7302)
By persisting the user interactive authentication sessions to the database, this fixes
situations where a user hits different works throughout their auth session and also
allows sessions to persist through restarts of Synapse.
2020-04-30 13:47:49 -04:00
Richard van der Hoff
c2e1a2110f
Fix limit logic for EventsStream (#7358)
* Factor out functions for injecting events into database

I want to add some more flexibility to the tools for injecting events into the
database, and I don't want to clutter up HomeserverTestCase with them, so let's
factor them out to a new file.

* Rework TestReplicationDataHandler

This wasn't very easy to work with: the mock wrapping was largely superfluous,
and it's useful to be able to inspect the received rows, and clear out the
received list.

* Fix AssertionErrors being thrown by EventsStream

Part of the problem was that there was an off-by-one error in the assertion,
but also the limit logic was too simple. Fix it all up and add some tests.
2020-04-29 12:30:36 +01:00
Brendan Abolivier
2e3b9a0fcb
Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token""
This reverts commit 1adf6a5587.
2020-04-23 11:23:53 +02:00
Zay11Zay
f1097e7720
Fix the parameters of a test fixture (#7243) 2020-04-14 15:37:28 -04:00
Andrew Morgan
ac978ab3da
Default PL100 to enable encryption in a room (#7230) 2020-04-09 18:45:38 +01:00
Patrick Cloke
b85d7652ff
Do not allow a deactivated user to login via SSO. (#7240) 2020-04-09 13:28:13 -04:00
Richard van der Hoff
6486c96b65
Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_tests
Add tests for outbound device pokes
2020-03-30 13:59:07 +01:00
Richard van der Hoff
b7da598a61 Always whitelist the login fallback for SSO (#7153)
That fallback sets the redirect URL to itself (so it can process the login
token then return gracefully to the client). This would make it pointless to
ask the user for confirmation, since the URL the confirmation page would be
showing wouldn't be the client's.
2020-03-27 20:24:52 +00:00
Dirk Klimpel
8327eb9280
Add options to prevent users from changing their profile. (#7096) 2020-03-27 19:15:23 +00:00
Dirk Klimpel
e8e2ddb60a
Allow server admins to define and enforce a password policy (MSC2000). (#7118) 2020-03-26 16:51:13 +00:00
Patrick Cloke
1c1242acba
Validate that the session is not modified during UI-Auth (#7068) 2020-03-26 07:39:34 -04:00
Richard van der Hoff
28d9d6e8a9 Remove spurious "name" parameter to default_config
this is never set to anything other than "test", and is a source of unnecessary
boilerplate.
2020-03-24 18:33:49 +00:00
Richard van der Hoff
39230d2171
Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Patrick Cloke
60724c46b7
Remove special casing of m.room.aliases events (#7034) 2020-03-17 07:37:04 -04:00
Richard van der Hoff
6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)"
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
2020-03-17 11:25:01 +00:00
Brendan Abolivier
54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
dklimpel
39f6595b4a lint, fix tests 2020-03-09 22:13:20 +01:00
dklimpel
885134529f updates after review 2020-03-09 22:09:29 +01:00
dklimpel
50ea178c20 lint 2020-03-09 19:57:04 +01:00
dklimpel
04f4b5f6f8 add tests 2020-03-09 19:51:31 +01:00
Brendan Abolivier
43f874055d
Merge branch 'master' into develop 2020-03-03 15:20:49 +00:00
Patrick Cloke
7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Richard van der Hoff
b68041df3d Add a whitelist for the SSO confirmation step. 2020-03-02 17:05:09 +00:00
Brendan Abolivier
b2bd54a2e3 Add a confirmation step to the SSO login flow 2020-03-02 16:36:32 +00:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff
2fb7794e60
Merge pull request #6949 from matrix-org/rav/list_room_aliases_peekable
Make room alias lists peekable
2020-02-19 11:19:11 +00:00
Richard van der Hoff
880aaac1d8
Move MSC2432 stuff onto unstable prefix (#6948)
it's not in the spec yet, so needs to be unstable. Also add a feature flag for it. Also add a test for admin users.
2020-02-19 10:40:27 +00:00
Richard van der Hoff
709e81f518 Make room alias lists peekable
As per
https://github.com/matrix-org/matrix-doc/pull/2432#pullrequestreview-360566830,
make room alias lists accessible to users outside world_readable rooms.
2020-02-19 08:53:32 +00:00
Richard van der Hoff
adfaea8c69
Implement GET /_matrix/client/r0/rooms/{roomId}/aliases (#6939)
per matrix-org/matrix-doc#2432
2020-02-18 16:23:25 +00:00
Andrew Morgan
3404ad289b
Raise the default power levels for invites, tombstones and server acls (#6834) 2020-02-17 13:23:37 +00:00
Richard van der Hoff
184303b865
MSC2260: Block direct sends of m.room.aliases events (#6794)
as per MSC2260
2020-01-30 17:20:55 +00:00
Erik Johnston
74b74462f1
Fix /events/:event_id deprecated API. (#6731) 2020-01-20 17:38:09 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership table (#6655)
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Andrew Morgan
1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Richard van der Hoff
6e8f8e14f2 Merge release-v1.7.1 into develop 2019-12-18 09:51:51 +00:00
Will Hunt
bfb95654c9 Add option to allow profile queries without sharing a room (#6523) 2019-12-16 16:11:55 +00:00
Brendan Abolivier
a820069549
Incorporate review 2019-12-16 16:00:18 +00:00
Brendan Abolivier
a29420f9f4
Lint 2019-12-16 14:55:50 +00:00
Brendan Abolivier
596dd9914d
Add test case 2019-12-16 14:53:21 +00:00
Erik Johnston
257ef2c727 Port handlers.account_validity to async/await. 2019-12-10 14:40:15 +00:00
Erik Johnston
b3a4e35ca8 Fixup functions to consistently return deferreds 2019-12-06 10:40:05 +00:00
Erik Johnston
4ca3ef10b9 Fixup tests 2019-12-05 15:57:28 +00:00
Brendan Abolivier
c530f9af4d
Merge pull request #6329 from matrix-org/babolivier/context_filters
Filter state, events_before and events_after in /context requests
2019-12-04 15:24:16 +00:00
Brendan Abolivier
65c6aee621
Un-remove room purge test 2019-12-04 14:37:04 +00:00
Brendan Abolivier
aeda1b3b94
Merge branch 'babolivier/context_filters' of github.com:matrix-org/synapse into babolivier/context_filters 2019-12-04 14:29:03 +00:00
Brendan Abolivier
9dc84b7989
Merge branch 'develop' into babolivier/context_filters 2019-12-04 14:23:44 +00:00
Brendan Abolivier
54dd5dc12b
Add ephemeral messages support (MSC2228) (#6409)
Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). The parts that differ are:

* the feature is hidden behind a configuration flag (`enable_ephemeral_messages`)
* self-destruction doesn't happen for state events
* only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one)
* doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
2019-12-03 19:19:45 +00:00
Erik Johnston
8c9a713f8d Add tests 2019-11-28 11:32:06 +00:00
Brendan Abolivier
d31f69afa0
Merge pull request #6358 from matrix-org/babolivier/message_retention
Implement message retention policies (MSC1763)
2019-11-27 15:04:38 +00:00
Andrew Morgan
ce578031f4
Remove assertion and provide a clear warning on startup for missing public_baseurl (#6379) 2019-11-26 18:42:27 +00:00
Brendan Abolivier
9e937c28ee Merge branch 'develop' into babolivier/message_retention 2019-11-26 17:53:57 +00:00
Brendan Abolivier
f0ef970824
Don't restrict the tests to v1 rooms 2019-11-26 17:49:12 +00:00
Brendan Abolivier
4c1b799e1b
Merge branch 'develop' into babolivier/context_filters 2019-11-26 10:53:48 +00:00
Brendan Abolivier
e2a20326e8
Lint 2019-11-20 15:08:47 +00:00
Brendan Abolivier
6356f2088f
Test if a purge can make /messages return 500 responses 2019-11-20 12:09:06 +00:00
Brendan Abolivier
bf9a11c54d
Lint again 2019-11-19 13:30:04 +00:00
Brendan Abolivier
7c24d0f443
Lint 2019-11-19 13:22:37 +00:00
Brendan Abolivier
f03c9d3444
Don't apply retention policy based filtering on state events
As per MSC1763, 'Retention is only considered for non-state events.', so don't filter out state events based on the room's retention policy.
2019-11-06 18:40:04 +00:00
Brendan Abolivier
a6863da249
Lint 2019-11-05 14:50:19 +00:00
Brendan Abolivier
8822b33111
Update copyrights 2019-11-05 14:46:35 +00:00
Brendan Abolivier
037360e6cf
Add tests for /search 2019-11-05 14:33:18 +00:00
Brendan Abolivier
c9e4748cb7
Merge labels tests for /context and /messages 2019-11-05 13:47:47 +00:00
Brendan Abolivier
a7c818c79b
Add test case 2019-11-05 13:21:26 +00:00
Brendan Abolivier
09957ce0e4
Implement per-room message retention policies 2019-11-04 17:09:22 +00:00
Brendan Abolivier
988d8d6507
Incorporate review 2019-11-01 16:22:44 +00:00
Brendan Abolivier
c6dbca2422
Incorporate review 2019-11-01 10:30:51 +00:00
Brendan Abolivier
dcc069a2e2
Lint 2019-10-30 18:01:56 +00:00
Brendan Abolivier
d8c9109aee
Add integration tests for /messages 2019-10-30 17:48:22 +00:00
Brendan Abolivier
fe51d6cacf
Add more integration testing 2019-10-30 17:28:41 +00:00
Brendan Abolivier
395683add1
Add integration tests for sync 2019-10-30 16:47:37 +00:00
werner291
b5b03b7079 Add domain validation when creating room with list of invitees (#6121) 2019-10-10 13:05:48 +01:00
krombel
2efd050c9d send 404 as http-status when filter-id is unknown to the server (#2380)
This fixed the weirdness of 400 vs 404 as http status code in the case
the filter id is not known by the server.
As e.g. matrix-js-sdk expects 404 to catch this situation this leads
to unwanted behaviour.
2019-10-10 12:59:55 +01:00
Brendan Abolivier
fbb8ff3088
ok 2019-09-27 15:23:07 +01:00
Brendan Abolivier
873fe7883c
Lint 2019-09-27 15:21:03 +01:00
Brendan Abolivier
e94ff67903
Add test to validate the change 2019-09-27 15:14:02 +01:00
Richard van der Hoff
990928abde
Stop advertising unsupported flows for registration (#6107)
If email or msisdn verification aren't supported, let's stop advertising them
for registration.

Fixes #6100.
2019-09-25 12:10:26 +01:00
Richard van der Hoff
2cd98812ba
Refactor the user-interactive auth handling (#6105)
Pull the checkers out to their own classes, rather than having them lost in a
massive 1000-line class which does everything.

This is also preparation for some more intelligent advertising of flows, as per #6100
2019-09-25 11:33:03 +01:00
Richard van der Hoff
8004d6ca2f
Refactor code for calculating registration flows (#6106)
because, frankly, it looked like it was written by an axe-murderer.

This should be a non-functional change, except that where `m.login.dummy` was
previously advertised *before* `m.login.terms`, it will now be advertised
afterwards. AFAICT that should have no effect, and will be more consistent with
the flows that involve passing a 3pid.
2019-09-25 11:32:05 +01:00
Erik Johnston
57dd41a45b
Fix comments
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2019-09-11 13:54:50 +01:00
Erik Johnston
caa9d6fed7 Add test for admin redaction ratelimiting. 2019-09-11 11:18:04 +01:00
Erik Johnston
6e834e94fc
Fix and refactor room and user stats (#5971)
Previously the stats were not being correctly populated.
2019-09-04 13:04:27 +01:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements (#5931)
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Brendan Abolivier
bc35503528
Add tests 2019-08-01 12:00:08 +02:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Andrew Morgan
b2a382efdb
Remove the ability to query relations when the original event was redacted. (#5629)
Fixes #5594

Forbid viewing relations on an event once it has been redacted.
2019-07-18 14:41:42 +01:00
Richard van der Hoff
9c70a02a9c
Ignore redactions of m.room.create events (#5701) 2019-07-17 19:08:02 +01:00
Richard van der Hoff
2091c91fde
More refactoring in get_events_as_list (#5707)
We can now use `_get_events_from_cache_or_db` rather than going right back to
the database, which means that (a) we can benefit from caching, and (b) it
opens the way forward to more extensive checks on the original event.

We now always require the original event to exist before we will serve up a
redaction.
2019-07-17 17:34:13 +01:00
Richard van der Hoff
5f158ec039
Implement access token expiry (#5660)
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Andrew Morgan
d88421ab03
Include the original event in /relations (#5626)
When asking for the relations of an event, include the original event in the response. This will mostly be used for efficiently showing edit history, but could be useful in other circumstances.
2019-07-09 13:43:08 +01:00
Brendan Abolivier
73cb716b3c
Lint 2019-07-08 17:44:20 +01:00
Brendan Abolivier
5e01e9ac19
Add test case 2019-07-08 17:41:16 +01:00
Andrew Morgan
ad8b909ce9
Add origin_server_ts and sender fields to m.replace (#5613)
Riot team would like some extra fields as part of m.replace, so here you go.

Fixes: #5598
2019-07-05 17:20:02 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Brendan Abolivier
bfe84e051e Split public rooms directory auth config in two 2019-06-24 15:42:31 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Brendan Abolivier
f12e1f029c
Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules
Allow server admins to define implementations of extra rules for allowing or denying incoming events
2019-06-14 19:37:59 +01:00
Brendan Abolivier
f874b16b2e Add plugin APIs for implementations of custom event rules. 2019-06-14 18:16:03 +01:00
Brendan Abolivier
6d56a694f4 Don't send renewal emails to deactivated users 2019-06-14 15:05:56 +01:00
Brendan Abolivier
d0530382ee
Track deactivated accounts in the database (#5378) 2019-06-14 13:18:24 +01:00
Erik Johnston
7e68691ce9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2019-06-11 17:25:16 +01:00
Erik Johnston
a766c41d25 Bump bleach version so that tests can run on old deps. 2019-06-11 12:34:18 +01:00
Erik Johnston
10383e6e6f Change password reset links to /_matrix. 2019-06-11 11:34:33 +01:00
Brendan Abolivier
26b62796c2
Merge pull request #5363 from matrix-org/babolivier/account_validity_send_mail_auth
Don't check whether the user's account is expired on /send_mail requests
2019-06-10 11:57:02 +01:00
Brendan Abolivier
fe13bd52ac
Don't check whether the user's account is expired on /send_mail requests 2019-06-05 16:35:05 +01:00
Amber Brown
2889b05554
Unify v1 and v2 REST client APIs (#5226) 2019-06-03 21:28:59 +10:00
Richard van der Hoff
d828d1dc57
Merge pull request #5309 from matrix-org/rav/limit_displayname_length
Limit displaynames and avatar URLs
2019-06-01 11:34:50 +01:00
Richard van der Hoff
93003aa172 add some tests 2019-06-01 11:14:37 +01:00
Erik Johnston
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
Brendan Abolivier
4d794dae21 Move delta from +10% to -10% 2019-05-31 11:09:39 +01:00
Brendan Abolivier
847b9dcd1c Make max_delta equal to period * 10% 2019-05-31 09:54:46 +01:00
Brendan Abolivier
7e1c7cc274
Typo 2019-05-28 17:13:26 +01:00
Brendan Abolivier
52839886d6
Allow configuring a range for the account validity startup job
When enabling the account validity feature, Synapse will look at startup for registered account without an expiration date, and will set one equals to 'now + validity_period' for them. On large servers, it can mean that a large number of users will have the same expiration date, which means that they will all be sent a renewal email at the same time, which isn't ideal.
In order to mitigate this, this PR allows server admins to define a 'max_delta' so that the expiration date is a random value in the [now + validity_period ; now + validity_period + max_delta] range. This allows renewal emails to be progressively sent over a configured period instead of being sent all in one big batch.
2019-05-28 16:52:45 +01:00
Andrew Morgan
6368150a74
Add config option for setting homeserver's default room version (#5223)
Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present.

That hardcoded value is now located in the server.py config file.
2019-05-23 15:00:20 +01:00
Amber Brown
4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
Erik Johnston
9526aa96a6
Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactions
Block attempts to annotate the same event twice
2019-05-21 17:08:14 +01:00
Brendan Abolivier
6a5a70edf0
Merge pull request #5204 from matrix-org/babolivier/account_validity_expiration_date
Add startup background job for account validity
2019-05-21 14:55:15 +01:00
Erik Johnston
c7ec06e8a6 Block attempts to annotate the same event twice 2019-05-20 17:39:05 +01:00
Erik Johnston
1dff859d6a Rename relation types to match MSC 2019-05-20 14:31:19 +01:00
Erik Johnston
57ba3451b6
Merge pull request #5209 from matrix-org/erikj/reactions_base
Land basic reaction and edit support.
2019-05-20 14:06:40 +01:00
Erik Johnston
2ac9c965dd Fixup comments 2019-05-20 12:32:26 +01:00
Erik Johnston
935af0da38 Correctly update aggregation counts after redaction 2019-05-20 12:09:27 +01:00
Brendan Abolivier
ad5b4074e1
Add startup background job for account validity
If account validity is enabled in the server's configuration, this job will run at startup as a background job and will stick an expiration date to any registered account missing one.
2019-05-17 19:37:31 +01:00
Erik Johnston
d4ca533d70 Make tests use different user for each reaction it sends
As users aren't allowed to react with the same emoji more than once.
2019-05-17 18:03:05 +01:00
Brendan Abolivier
fdeac1e984
Merge pull request #5196 from matrix-org/babolivier/per_room_profiles
Add an option to disable per-room profiles
2019-05-17 12:10:49 +01:00
David Baker
07cff7b121
Merge pull request #5174 from matrix-org/dbkr/add_dummy_flow_to_recaptcha_only
Re-order registration stages to do msisdn & email auth last
2019-05-16 17:27:39 +01:00
Erik Johnston
d46aab3fa8 Add basic editing support 2019-05-16 16:54:45 +01:00
Brendan Abolivier
8f9ce1a8a2
Lint 2019-05-16 15:25:54 +01:00
Brendan Abolivier
cc8c139a39
Lint 2019-05-16 15:20:59 +01:00
Brendan Abolivier
efdc55db75
Forgot copyright 2019-05-16 15:10:24 +01:00
Brendan Abolivier
54a582ed44
Add test case 2019-05-16 15:09:16 +01:00
Erik Johnston
95f3fcda3c Check that event is visible in new APIs 2019-05-16 14:19:06 +01:00
Erik Johnston
a0603523d2 Add aggregations API 2019-05-16 09:37:20 +01:00
Erik Johnston
b50641e357 Add simple pagination API 2019-05-15 13:36:51 +01:00
Erik Johnston
efe3c7977a Add simple send_relation API and track in DB 2019-05-15 13:36:51 +01:00
Amber Brown
df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171) 2019-05-13 15:01:14 -05:00
David Baker
8782bfb783 And now I realise why the test is failing... 2019-05-13 15:34:11 +01:00
Amber Brown
b36c82576e
Run Black on the tests again (#5170) 2019-05-10 00:12:11 -05:00
Brendan Abolivier
d216a36b37
Fix bogus imports in tests (#5154) 2019-05-08 21:57:03 +01:00
Matthew Hodgson
c0e0740bef add options to require an access_token to GET /profile and /publicRooms on CS API (#5083)
This commit adds two config options:

* `restrict_public_rooms_to_local_users`

Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.

* `require_auth_for_profile_requests`

When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.

MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.

Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
Brendan Abolivier
1473058b5e
Do checks on aliases for incoming m.room.aliases events (#5128)
Follow-up to #5124

Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
2019-05-08 17:01:30 +01:00
Richard van der Hoff
12f9d51e82
Add admin api for sending server_notices (#5121) 2019-05-02 11:59:16 +01:00
Richard van der Hoff
40e576e29c Move admin api impl to its own package
It doesn't really belong under rest/client/v1 any more.
2019-05-01 15:44:30 +01:00
Erik Johnston
ca90336a69 Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration 2019-04-17 19:44:40 +01:00
Brendan Abolivier
eaf41a943b Add management endpoints for account validity 2019-04-17 19:34:45 +01:00
Brendan Abolivier
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier
20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00
Brendan Abolivier
bfc8fdf1fc
Merge pull request #5027 from matrix-org/babolivier/account_expiration
Add time-based account expiration
2019-04-09 17:02:41 +01:00
Brendan Abolivier
747aa9f8ca Add account expiration feature 2019-04-09 16:46:04 +01:00
Brendan Abolivier
8e85493b0c
Add config option to block users from looking up 3PIDs (#5010) 2019-04-04 17:25:47 +01:00
Erik Johnston
c6a233a936 Add unit test for deleting groups 2019-04-03 16:29:52 +01:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Amber Brown
a68e00fca8
Some more porting to HomeserverTestCase and remove old RESTHelper (#4913) 2019-03-22 02:10:21 +11:00
Erik Johnston
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Erik Johnston
d3f640f0ac isort 2019-03-21 11:29:48 +00:00
Erik Johnston
cd80cbffea Fix typo and add description 2019-03-21 11:24:04 +00:00
Erik Johnston
536a266520 Deny peeking into rooms that have been blocked 2019-03-21 11:20:13 +00:00
Erik Johnston
4a8a1ac962 Rejig testcase to make it more extensible 2019-03-21 11:02:11 +00:00
Erik Johnston
9c9e618b93 Remove debug 2019-03-21 10:58:56 +00:00
Erik Johnston
5c6f61f81c Add tests 2019-03-21 10:51:21 +00:00
Richard van der Hoff
45bb54a6c6 Fix registration test
* Set allow_guest_access = True, since we rely on it
* config doesn't have a `hostname` attribute; it is `server_name`
2019-03-19 11:44:43 +00:00
Brendan Abolivier
651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login (#4821)
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Erik Johnston
16c8b4ecbd
Merge pull request #4772 from jbweston/jbweston/server-version-api
Add 'server_version' endpoint to admin API
2019-03-05 16:31:00 +00:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration (#4735)
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
2019-03-05 14:25:33 +00:00
Joseph Weston
1e8388b311 Add 'server_version' endpoint to admin API
This is required because the 'Server' HTTP header is not always
passed through proxies.
2019-03-01 09:56:58 +01:00
Joseph Weston
b136ee10df Import 'admin' module rather than 'register_servlets' directly
We will later need also to import 'register_servlets' from the
'login' module, so we un-pollute the namespace now to keep the
logical changes separate.
2019-03-01 09:46:25 +01:00
Erik Johnston
af691e415c Move register_device into handler 2019-02-18 16:49:38 +00:00
Erik Johnston
41c3f21c3b Fix unit tests 2019-02-18 13:43:16 +00:00
Neil Johnson
e4bef9d470 rework format of change password capability 2019-01-29 18:04:56 +00:00
Neil Johnson
f03b3a7a3a support change_password in capabilities end-point 2019-01-29 15:58:37 +00:00
Neil Johnson
4eeb2fb215 isort 2019-01-29 12:44:10 +00:00
Neil Johnson
a124025dab enforce auth for capabilities endpoint 2019-01-29 11:37:56 +00:00
Neil Johnson
95f871fc0d Support room version capabilities in CS API (MSC1804) 2019-01-25 11:16:29 +00:00
Amber Brown
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00
Neil Johnson
d2f7c4e6b1
create support user (#4141)
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Amber Brown
80cac86b2c
Fix fallback auth on Python 3 (#4197) 2018-11-19 12:27:33 -06:00
Richard van der Hoff
c1efcd7c6a Add a test for the public T&Cs form 2018-11-14 10:46:27 +00:00
Amber Brown
e62f7f17b3
Remove some boilerplate in tests (#4156) 2018-11-07 03:00:00 +11:00
Amber Brown
efdcbbe46b
Tests for user consent resource (#4140) 2018-11-06 05:53:44 +11:00
Amber Brown
cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Amber Brown
3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown
52ec6e9dfa
Port tests/ to Python 3 (#3808) 2018-09-07 02:58:18 +10:00