Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-10-01 08:25:44 -04:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work

Browse Source
This commit is contained in:
Richard van der Hoff 2019-09-24 12:57:32 +01:00
commit ed8b92f0d2
51 changed files with 844 additions and 235 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

95
UPGRADE.rst
View File

@ -2,52 +2,78 @@ Upgrading Synapse
=================
Before upgrading check if any special steps are required to upgrade from the
what you currently have installed to current version of synapse. The extra
what you currently have installed to current version of Synapse. The extra
instructions that may be required are listed later in this document.
1. If synapse was installed in a virtualenv then activate that virtualenv before
upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then
run:
* If Synapse was installed using `prebuilt packages
<INSTALL.md#prebuilt-packages>`_, you will need to follow the normal process
for upgrading those packages.
* If Synapse was installed from source, then:
1. Activate the virtualenv before upgrading. For example, if Synapse is
installed in a virtualenv in ``~/synapse/env`` then run:
.. code:: bash
source ~/synapse/env/bin/activate
2. If synapse was installed using pip then upgrade to the latest version by
2. If Synapse was installed using pip then upgrade to the latest version by
running:
.. code:: bash
pip install --upgrade matrix-synapse[all]
pip install --upgrade matrix-synapse
# restart synapse
synctl restart
If synapse was installed using git then upgrade to the latest version by
If Synapse was installed using git then upgrade to the latest version by
running:
.. code:: bash
# Pull the latest version of the master branch.
git pull
pip install --upgrade .
# Update synapse and its python dependencies.
pip install --upgrade .[all]
3. Restart Synapse:
.. code:: bash
# restart synapse
./synctl restart
To check whether your update was successful, you can check the Server header
returned by the Client-Server API:
To check whether your update was successful, you can check the running server
version with:
.. code:: bash
# replace <host.name> with the hostname of your synapse homeserver.
# You may need to specify a port (eg, :8448) if your server is not
# configured on port 443.
curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
# you may need to replace 'localhost:8008' if synapse is not configured
# to listen on port 8008.
curl http://localhost:8008/_synapse/admin/v1/server_version
Rolling back to older versions
------------------------------
Rolling back to previous releases can be difficult, due to database schema
changes between releases. Where we have been able to test the rollback process,
this will be noted below.
In general, you will need to undo any changes made during the upgrade process,
for example:
* pip:
.. code:: bash
source env/bin/activate
# replace `1.3.0` accordingly:
pip install matrix-synapse==1.3.0
* Debian:
.. code:: bash
# replace `1.3.0` and `stretch` accordingly:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.4.0
===================
@ -99,6 +125,31 @@ Synapse will expect these files to exist inside the configured template director
default templates, see `synapse/res/templates
<https://github.com/matrix-org/synapse/tree/master/synapse/res/templates>`_.
Rolling back to v1.3.1
----------------------
If you encounter problems with v1.4.0, it should be possible to roll back to
v1.3.1, subject to the following:
* The 'room statistics' engine was heavily reworked in this release (see
`#5971 <https://github.com/matrix-org/synapse/pull/5971>`_), including
significant changes to the database schema, which are not easily
reverted. This will cause the room statistics engine to stop updating when
you downgrade.
The room statistics are essentially unused in v1.3.1 (in future versions of
Synapse, they will be used to populate the room directory), so there should
be no loss of functionality. However, the statistics engine will write errors
to the logs, which can be avoided by setting the following in `homeserver.yaml`:
.. code:: yaml
stats:
enabled: false
Don't forget to re-enable it when you upgrade again, in preparation for its
use in the room directory!
Upgrading to v1.2.0
===================

2
changelog.d/5893.misc
View File

@ -1 +1 @@
Drop some unused tables.
Stop populating some unused tables.

1
changelog.d/5972.misc Normal file
View File

@ -0,0 +1 @@
Add m.require_identity_server flag to /version's unstable_features.

1
changelog.d/5974.feature Normal file
View File

@ -0,0 +1 @@
Add m.id_access_token to unstable_features in /versions as per MSC2264.

2
changelog.d/5980.feature
View File

@ -1 +1 @@
Add POST /_matrix/client/r0/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.

1
changelog.d/5992.feature Normal file
View File

@ -0,0 +1 @@
Give appropriate exit codes when synctl fails.

1
changelog.d/6000.feature Normal file
View File

@ -0,0 +1 @@
Apply the federation blacklist to requests to identity servers.

1
changelog.d/6028.feature Normal file
View File

@ -0,0 +1 @@
Replace `trust_identity_server_for_password_resets` config option with `account_threepid_delegates`.

1
changelog.d/6042.feature Normal file
View File

@ -0,0 +1 @@
Allow homeserver to handle or delegate email validation when adding an email to a user's account.

1
changelog.d/6043.feature Normal file
View File

@ -0,0 +1 @@
Implement new Client Server API endpoints `/account/3pid/add` and `/account/3pid/bind` as per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290).

1
changelog.d/6044.feature Normal file
View File

@ -0,0 +1 @@
Add an unstable feature flag for separate add/bind 3pid APIs.

2
changelog.d/6047.misc Normal file
View File

@ -0,0 +1,2 @@
Stop populating some unused tables.

1
changelog.d/6049.doc Normal file
View File

@ -0,0 +1 @@
Add some notes on rolling back to v1.3.1.

1
changelog.d/6050.doc Normal file
View File

@ -0,0 +1 @@
Update the upgrade notes.

1
changelog.d/6056.bugfix Normal file
View File

@ -0,0 +1 @@
Remove POST method from password reset submit_token endpoint until we implement submit_url functionality.

1
changelog.d/6058.docker Normal file
View File

@ -0,0 +1 @@
Provide SYNAPSE_WORKER envvar to specify python module.

1
changelog.d/6059.bugfix Normal file
View File

@ -0,0 +1 @@
Fix logcontext spam on non-Linux platforms.

1
changelog.d/6062.bugfix Normal file
View File

@ -0,0 +1 @@
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.

1
changelog.d/6063.bugfix Normal file
View File

@ -0,0 +1 @@
Ensure query parameters in email validation links are URL-encoded.

1
changelog.d/6067.feature Normal file
View File

@ -0,0 +1 @@
Remove `bind` parameter from Client Server POST `/account` endpoint as per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290/).

1
changelog.d/6072.misc Normal file
View File

@ -0,0 +1 @@
Add a 'failure_ts' column to the 'destinations' database table.

1
changelog.d/6073.feature Normal file
View File

@ -0,0 +1 @@
Return a clearer error message when a timeout occurs when attempting to contact an identity server.

1
changelog.d/6074.feature Normal file
View File

@ -0,0 +1 @@
Prevent password reset's submit_token endpoint from accepting trailing slashes.

1
changelog.d/6075.misc Normal file
View File

@ -0,0 +1 @@
Change mailer logging to reflect Synapse doesn't just do chat notifications by email now.

1
changelog.d/6078.feature Normal file
View File

@ -0,0 +1 @@
Add `POST /add_threepid/msisdn/submit_token` endpoint for proxying submitToken on an account_threepid_handler.

1
changelog.d/6079.feature Normal file
View File

@ -0,0 +1 @@
Add `submit_url` response parameter to `*/msisdn/requestToken` endpoints.

1
changelog.d/6082.feature Normal file
View File

@ -0,0 +1 @@
Return 403 on `/register/available` if registration has been disabled.

2
docker/README.md
View File

@ -89,6 +89,8 @@ The following environment variables are supported in run mode:
`/data`.
* `SYNAPSE_CONFIG_PATH`: path to the config file. Defaults to
`<SYNAPSE_CONFIG_DIR>/homeserver.yaml`.
* `SYNAPSE_WORKER`: module to execute, used when running synapse with workers.
Defaults to `synapse.app.homeserver`, which is suitable for non-worker mode.
* `UID`, `GID`: the user and group id to run Synapse as. Defaults to `991`, `991`.
* `TZ`: the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) the container will run with. Defaults to `UTC`.

3
docker/start.py
View File

@ -182,6 +182,7 @@ def main(args, environ):
mode = args[1] if len(args) > 1 else None
desired_uid = int(environ.get("UID", "991"))
desired_gid = int(environ.get("GID", "991"))
synapse_worker = environ.get("SYNAPSE_WORKER", "synapse.app.homeserver")
if (desired_uid == os.getuid()) and (desired_gid == os.getgid()):
ownership = None
else:
@ -245,7 +246,7 @@ def main(args, environ):
log("Starting synapse with config file " + config_path)
args = ["python", "-m", "synapse.app.homeserver", "--config-path", config_path]
args = ["python", "-m", synapse_worker, "--config-path", config_path]
if ownership is not None:
args = ["su-exec", ownership] + args
os.execv("/sbin/su-exec", args)

19
docs/sample_config.yaml
View File

@ -110,6 +110,9 @@ pid_file: DATADIR/homeserver.pid
# blacklist IP address CIDR ranges. If this option is not specified, or
# specified with an empty list, no ip range blacklist will be enforced.
#
# As of Synapse v1.4.0 this option also affects any outbound requests to identity
# servers provided by user input.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
@ -937,8 +940,10 @@ uploads_path: "DATADIR/uploads"
# by the Matrix Identity Service API specification:
# https://matrix.org/docs/spec/identity_service/latest
#
# If a delegate is specified, the config option public_baseurl must also be filled out.
#
account_threepid_delegates:
#email: https://example.com # Delegate email sending to matrix.org
#email: https://example.com # Delegate email sending to example.org
#msisdn: http://localhost:8090 # Delegate SMS sending to this local process
# Users who register on this homeserver will automatically be joined
@ -1295,6 +1300,12 @@ password_config:
# #registration_template_html: registration.html
# #registration_template_text: registration.txt
#
# # Templates for validation emails sent by the homeserver when adding an email to
# # your user account
# #
# #add_threepid_template_html: add_threepid.html
# #add_threepid_template_text: add_threepid.txt
#
# # Templates for password reset success and failure pages that a user
# # will see after attempting to reset their password
# #
@ -1306,6 +1317,12 @@ password_config:
# #
# #registration_template_success_html: registration_success.html
# #registration_template_failure_html: registration_failure.html
#
# # Templates for success and failure pages that a user will see after attempting
# # to add an email or phone to their account
# #
# #add_threepid_success_html: add_threepid_success.html
# #add_threepid_failure_html: add_threepid_failure.html
#password_providers:

36
synapse/config/emailconfig.py
View File

@ -169,12 +169,22 @@ class EmailConfig(Config):
self.email_registration_template_text = email_config.get(
"registration_template_text", "registration.txt"
)
self.email_add_threepid_template_html = email_config.get(
"add_threepid_template_html", "add_threepid.html"
)
self.email_add_threepid_template_text = email_config.get(
"add_threepid_template_text", "add_threepid.txt"
)
self.email_password_reset_template_failure_html = email_config.get(
"password_reset_template_failure_html", "password_reset_failure.html"
)
self.email_registration_template_failure_html = email_config.get(
"registration_template_failure_html", "registration_failure.html"
)
self.email_add_threepid_template_failure_html = email_config.get(
"add_threepid_template_failure_html", "add_threepid_failure.html"
)
# These templates do not support any placeholder variables, so we
# will read them from disk once during setup
@ -184,6 +194,9 @@ class EmailConfig(Config):
email_registration_template_success_html = email_config.get(
"registration_template_success_html", "registration_success.html"
)
email_add_threepid_template_success_html = email_config.get(
"add_threepid_template_success_html", "add_threepid_success.html"
)
# Check templates exist
for f in [
@ -191,9 +204,14 @@ class EmailConfig(Config):
self.email_password_reset_template_text,
self.email_registration_template_html,
self.email_registration_template_text,
self.email_add_threepid_template_html,
self.email_add_threepid_template_text,
self.email_password_reset_template_failure_html,
self.email_registration_template_failure_html,
self.email_add_threepid_template_failure_html,
email_password_reset_template_success_html,
email_registration_template_success_html,
email_add_threepid_template_success_html,
]:
p = os.path.join(self.email_template_dir, f)
if not os.path.isfile(p):
@ -212,6 +230,12 @@ class EmailConfig(Config):
self.email_registration_template_success_html_content = self.read_file(
filepath, "email.registration_template_success_html"
)
filepath = os.path.join(
self.email_template_dir, email_add_threepid_template_success_html
)
self.email_add_threepid_template_success_html_content = self.read_file(
filepath, "email.add_threepid_template_success_html"
)
if self.email_enable_notifs:
required = [
@ -328,6 +352,12 @@ class EmailConfig(Config):
# #registration_template_html: registration.html
# #registration_template_text: registration.txt
#
# # Templates for validation emails sent by the homeserver when adding an email to
# # your user account
# #
# #add_threepid_template_html: add_threepid.html
# #add_threepid_template_text: add_threepid.txt
#
# # Templates for password reset success and failure pages that a user
# # will see after attempting to reset their password
# #
@ -339,6 +369,12 @@ class EmailConfig(Config):
# #
# #registration_template_success_html: registration_success.html
# #registration_template_failure_html: registration_failure.html
#
# # Templates for success and failure pages that a user will see after attempting
# # to add an email or phone to their account
# #
# #add_threepid_success_html: add_threepid_success.html
# #add_threepid_failure_html: add_threepid_failure.html
"""

4
synapse/config/registration.py
View File

@ -293,8 +293,10 @@ class RegistrationConfig(Config):
# by the Matrix Identity Service API specification:
# https://matrix.org/docs/spec/identity_service/latest
#
# If a delegate is specified, the config option public_baseurl must also be filled out.
#
account_threepid_delegates:
#email: https://example.com # Delegate email sending to matrix.org
#email: https://example.com # Delegate email sending to example.org
#msisdn: http://localhost:8090 # Delegate SMS sending to this local process
# Users who register on this homeserver will automatically be joined

2
synapse/config/repository.py
View File

@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
# Copyright 2014, 2015 matrix.org
# Copyright 2014, 2015 OpenMarket Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
synapse/config/server.py
View File

@ -545,6 +545,9 @@ class ServerConfig(Config):
# blacklist IP address CIDR ranges. If this option is not specified, or
# specified with an empty list, no ip range blacklist will be enforced.
#
# As of Synapse v1.4.0 this option also affects any outbound requests to identity
# servers provided by user input.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#

2
synapse/config/workers.py
View File

@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
# Copyright 2016 matrix.org
# Copyright 2016 OpenMarket Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
synapse/federation/transport/server.py
View File

@ -165,7 +165,7 @@ class Authenticator(object):
async def _reset_retry_timings(self, origin):
try:
logger.info("Marking origin %r as up", origin)
await self.store.set_destination_retry_timings(origin, 0, 0)
await self.store.set_destination_retry_timings(origin, None, 0, 0)
except Exception:
logger.exception("Error resetting retry timings on %s", origin)

4
synapse/handlers/deactivate_account.py
View File

@ -73,7 +73,9 @@ class DeactivateAccountHandler(BaseHandler):
# unbinding
identity_server_supports_unbinding = True
threepids = yield self.store.user_get_threepids(user_id)
# Retrieve the 3PIDs this user has bound to an identity server
threepids = yield self.store.user_get_bound_threepids(user_id)
for threepid in threepids:
try:
result = yield self._identity_handler.try_unbind_threepid(

229
synapse/handlers/identity.py
View File

@ -22,6 +22,7 @@ import logging
from canonicaljson import json
from twisted.internet import defer
from twisted.internet.error import TimeoutError
from synapse.api.errors import (
CodeMessageException,
@ -29,6 +30,8 @@ from synapse.api.errors import (
HttpResponseException,
SynapseError,
)
from synapse.config.emailconfig import ThreepidBehaviour
from synapse.http.client import SimpleHttpClient
from synapse.util.stringutils import random_string
from ._base import BaseHandler
@ -40,40 +43,15 @@ class IdentityHandler(BaseHandler):
def __init__(self, hs):
super(IdentityHandler, self).__init__(hs)
self.http_client = hs.get_simple_http_client()
self.http_client = SimpleHttpClient(hs)
# We create a blacklisting instance of SimpleHttpClient for contacting identity
# servers specified by clients
self.blacklisting_http_client = SimpleHttpClient(
hs, ip_blacklist=hs.config.federation_ip_range_blacklist
)
self.federation_http_client = hs.get_http_client()
self.hs = hs
def _extract_items_from_creds_dict(self, creds):
"""
Retrieve entries from a "credentials" dictionary
Args:
creds (dict[str, str]): Dictionary of credentials that contain the following keys:
* client_secret|clientSecret: A unique secret str provided by the client
* id_server|idServer: the domain of the identity server to query
* id_access_token: The access token to authenticate to the identity
server with.
Returns:
tuple(str, str, str|None): A tuple containing the client_secret, the id_server,
and the id_access_token value if available.
"""
client_secret = creds.get("client_secret") or creds.get("clientSecret")
if not client_secret:
raise SynapseError(
400, "No client_secret in creds", errcode=Codes.MISSING_PARAM
)
id_server = creds.get("id_server") or creds.get("idServer")
if not id_server:
raise SynapseError(
400, "No id_server in creds", errcode=Codes.MISSING_PARAM
)
id_access_token = creds.get("id_access_token")
return client_secret, id_server, id_access_token
@defer.inlineCallbacks
def threepid_from_creds(self, id_server, creds):
"""
@ -81,11 +59,10 @@ class IdentityHandler(BaseHandler):
given identity server
Args:
id_server (str|None): The identity server to validate 3PIDs against. If None,
we will attempt to extract id_server creds
id_server (str): The identity server to validate 3PIDs against. Must be a
complete URL including the protocol (http(s)://)
creds (dict[str, str]): Dictionary containing the following keys:
* id_server|idServer: An optional domain name of an identity server
* client_secret|clientSecret: A unique secret str provided by the client
* sid: The ID of the validation session
@ -104,51 +81,59 @@ class IdentityHandler(BaseHandler):
raise SynapseError(
400, "Missing param session_id in creds", errcode=Codes.MISSING_PARAM
)
if not id_server:
# Attempt to get the id_server from the creds dict
id_server = creds.get("id_server") or creds.get("idServer")
if not id_server:
raise SynapseError(
400, "Missing param id_server in creds", errcode=Codes.MISSING_PARAM
)
query_params = {"sid": session_id, "client_secret": client_secret}
url = "https://%s%s" % (
id_server,
"/_matrix/identity/api/v1/3pid/getValidated3pid",
)
url = id_server + "/_matrix/identity/api/v1/3pid/getValidated3pid"
try:
data = yield self.http_client.get_json(url, query_params)
return data if "medium" in data else None
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except HttpResponseException as e:
logger.info(
"%s returned %i for threepid validation for: %s",
id_server,
e.code,
creds,
)
return None
# Old versions of Sydent return a 200 http code even on a failed validation
# check. Thus, in addition to the HttpResponseException check above (which
# checks for non-200 errors), we need to make sure validation_session isn't
# actually an error, identified by the absence of a "medium" key
# See https://github.com/matrix-org/sydent/issues/215 for details
if "medium" in data:
return data
logger.info("%s reported non-validated threepid: %s", id_server, creds)
return None
@defer.inlineCallbacks
def bind_threepid(self, creds, mxid, use_v2=True):
def bind_threepid(
self, client_secret, sid, mxid, id_server, id_access_token=None, use_v2=True
):
"""Bind a 3PID to an identity server
Args:
creds (dict[str, str]): Dictionary of credentials that contain the following keys:
* client_secret|clientSecret: A unique secret str provided by the client
* id_server|idServer: the domain of the identity server to query
* id_access_token: The access token to authenticate to the identity
server with. Required if use_v2 is true
client_secret (str): A unique secret provided by the client
sid (str): The ID of the validation session
mxid (str): The MXID to bind the 3PID to
use_v2 (bool): Whether to use v2 Identity Service API endpoints
id_server (str): The domain of the identity server to query
id_access_token (str): The access token to authenticate to the identity
server with, if necessary. Required if use_v2 is true
use_v2 (bool): Whether to use v2 Identity Service API endpoints. Defaults to True
Returns:
Deferred[dict]: The response from the identity server
"""
logger.debug("binding threepid %r to %s", creds, mxid)
client_secret, id_server, id_access_token = self._extract_items_from_creds_dict(
creds
)
sid = creds.get("sid")
if not sid:
raise SynapseError(
400, "No sid in three_pid_creds", errcode=Codes.MISSING_PARAM
)
logger.debug("Proxying threepid bind request for %s to %s", mxid, id_server)
# If an id_access_token is not supplied, force usage of v1
if id_access_token is None:
@ -164,10 +149,11 @@ class IdentityHandler(BaseHandler):
bind_url = "https://%s/_matrix/identity/api/v1/3pid/bind" % (id_server,)
try:
data = yield self.http_client.post_json_get_json(
# Use the blacklisting http client as this call is only to identity servers
# provided by a client
data = yield self.blacklisting_http_client.post_json_get_json(
bind_url, bind_data, headers=headers
)
logger.debug("bound threepid %r to %s", creds, mxid)
# Remember where we bound the threepid
yield self.store.add_user_bound_threepid(
@ -182,12 +168,17 @@ class IdentityHandler(BaseHandler):
if e.code != 404 or not use_v2:
logger.error("3PID bind failed with Matrix error: %r", e)
raise e.to_synapse_error()
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except CodeMessageException as e:
data = json.loads(e.msg) # XXX WAT?
return data
logger.info("Got 404 when POSTing JSON %s, falling back to v1 URL", bind_url)
return (yield self.bind_threepid(creds, mxid, use_v2=False))
res = yield self.bind_threepid(
client_secret, sid, mxid, id_server, id_access_token, use_v2=False
)
return res
@defer.inlineCallbacks
def try_unbind_threepid(self, mxid, threepid):
@ -263,7 +254,11 @@ class IdentityHandler(BaseHandler):
headers = {b"Authorization": auth_headers}
try:
yield self.http_client.post_json_get_json(url, content, headers)
# Use the blacklisting http client as this call is only to identity servers
# provided by a client
yield self.blacklisting_http_client.post_json_get_json(
url, content, headers
)
changed = True
except HttpResponseException as e:
changed = False
@ -272,7 +267,9 @@ class IdentityHandler(BaseHandler):
logger.warn("Received %d response while unbinding threepid", e.code)
else:
logger.error("Failed to unbind threepid on identity server: %s", e)
raise SynapseError(502, "Failed to contact identity server")
raise SynapseError(500, "Failed to contact identity server")
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
yield self.store.remove_user_bound_threepid(
user_id=mxid,
@ -405,6 +402,8 @@ class IdentityHandler(BaseHandler):
except HttpResponseException as e:
logger.info("Proxied requestToken failed: %r", e)
raise e.to_synapse_error()
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
@defer.inlineCallbacks
def requestMsisdnToken(
@ -453,10 +452,100 @@ class IdentityHandler(BaseHandler):
id_server + "/_matrix/identity/api/v1/validate/msisdn/requestToken",
params,
)
return data
except HttpResponseException as e:
logger.info("Proxied requestToken failed: %r", e)
raise e.to_synapse_error()
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
assert self.hs.config.public_baseurl
# we need to tell the client to send the token back to us, since it doesn't
# otherwise know where to send it, so add submit_url response parameter
# (see also MSC2078)
data["submit_url"] = (
self.hs.config.public_baseurl
+ "_matrix/client/unstable/add_threepid/msisdn/submit_token"
)
return data
@defer.inlineCallbacks
def validate_threepid_session(self, client_secret, sid):
"""Validates a threepid session with only the client secret and session ID
Tries validating against any configured account_threepid_delegates as well as locally.
Args:
client_secret (str): A secret provided by the client
sid (str): The ID of the session
Returns:
Dict[str, str|int] if validation was successful, otherwise None
"""
# XXX: We shouldn't need to keep wrapping and unwrapping this value
threepid_creds = {"client_secret": client_secret, "sid": sid}
# We don't actually know which medium this 3PID is. Thus we first assume it's email,
# and if validation fails we try msisdn
validation_session = None
# Try to validate as email
if self.hs.config.threepid_behaviour_email == ThreepidBehaviour.REMOTE:
# Ask our delegated email identity server
validation_session = yield self.threepid_from_creds(
self.hs.config.account_threepid_delegate_email, threepid_creds
)
elif self.hs.config.threepid_behaviour_email == ThreepidBehaviour.LOCAL:
# Get a validated session matching these details
validation_session = yield self.store.get_threepid_validation_session(
"email", client_secret, sid=sid, validated=True
)
if validation_session:
return validation_session
# Try to validate as msisdn
if self.hs.config.account_threepid_delegate_msisdn:
# Ask our delegated msisdn identity server
validation_session = yield self.threepid_from_creds(
self.hs.config.account_threepid_delegate_msisdn, threepid_creds
)
return validation_session
@defer.inlineCallbacks
def proxy_msisdn_submit_token(self, id_server, client_secret, sid, token):
"""Proxy a POST submitToken request to an identity server for verification purposes
Args:
id_server (str): The identity server URL to contact
client_secret (str): Secret provided by the client
sid (str): The ID of the session
token (str): The verification token
Raises:
SynapseError: If we failed to contact the identity server
Returns:
Deferred[dict]: The response dict from the identity server
"""
body = {"client_secret": client_secret, "sid": sid, "token": token}
try:
return (
yield self.http_client.post_json_get_json(
id_server + "/_matrix/identity/api/v1/validate/msisdn/submitToken",
body,
)
)
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except HttpResponseException as e:
logger.warning("Error contacting msisdn account_threepid_delegate: %s", e)
raise SynapseError(400, "Error contacting the identity server")
def create_id_access_token_header(id_access_token):

23
synapse/handlers/room_member.py
View File

@ -25,11 +25,13 @@ from signedjson.sign import verify_signed_json
from unpaddedbase64 import decode_base64
from twisted.internet import defer
from twisted.internet.error import TimeoutError
from synapse import types
from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import AuthError, Codes, HttpResponseException, SynapseError
from synapse.handlers.identity import LookupAlgorithm, create_id_access_token_header
from synapse.http.client import SimpleHttpClient
from synapse.types import RoomID, UserID
from synapse.util.async_helpers import Linearizer
from synapse.util.distributor import user_joined_room, user_left_room
@ -61,7 +63,11 @@ class RoomMemberHandler(object):
self.auth = hs.get_auth()
self.state_handler = hs.get_state_handler()
self.config = hs.config
self.simple_http_client = hs.get_simple_http_client()
# We create a blacklisting instance of SimpleHttpClient for contacting identity
# servers specified by clients
self.simple_http_client = SimpleHttpClient(
hs, ip_blacklist=hs.config.federation_ip_range_blacklist
)
self.federation_handler = hs.get_handlers().federation_handler
self.directory_handler = hs.get_handlers().directory_handler
@ -756,7 +762,8 @@ class RoomMemberHandler(object):
raise AuthError(401, "No signatures on 3pid binding")
yield self._verify_any_signature(data, id_server)
return data["mxid"]
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except IOError as e:
logger.warning("Error from v1 identity server lookup: %s" % (e,))
@ -777,10 +784,13 @@ class RoomMemberHandler(object):
Deferred[str|None]: the matrix ID of the 3pid, or None if it is not recognised.
"""
# Check what hashing details are supported by this identity server
try:
hash_details = yield self.simple_http_client.get_json(
"%s%s/_matrix/identity/v2/hash_details" % (id_server_scheme, id_server),
{"access_token": id_access_token},
)
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
if not isinstance(hash_details, dict):
logger.warning(
@ -851,6 +861,8 @@ class RoomMemberHandler(object):
},
headers=headers,
)
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except Exception as e:
logger.warning("Error when performing a v2 3pid lookup: %s", e)
raise SynapseError(
@ -873,10 +885,13 @@ class RoomMemberHandler(object):
if server_hostname not in data["signatures"]:
raise AuthError(401, "No signature from server %s" % (server_hostname,))
for key_name, signature in data["signatures"][server_hostname].items():
try:
key_data = yield self.simple_http_client.get_json(
"%s%s/_matrix/identity/api/v1/pubkey/%s"
% (id_server_scheme, server_hostname, key_name)
)
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
if "public_key" not in key_data:
raise AuthError(
401, "No public key named %s from %s" % (key_name, server_hostname)
@ -1051,6 +1066,8 @@ class RoomMemberHandler(object):
invite_config,
{"Authorization": create_id_access_token_header(id_access_token)},
)
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except HttpResponseException as e:
if e.code != 404:
logger.info("Failed to POST %s with JSON: %s", url, e)
@ -1067,6 +1084,8 @@ class RoomMemberHandler(object):
data = yield self.simple_http_client.post_json_get_json(
url, invite_config
)
except TimeoutError:
raise SynapseError(500, "Timed out contacting identity server")
except HttpResponseException as e:
logger.warning(
"Error trying to call /store-invite on %s%s: %s",

13
synapse/logging/context.py
View File

@ -1,4 +1,5 @@
# Copyright 2014-2016 OpenMarket Ltd
# Copyright 2019 The Matrix.org Foundation C.I.C.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -42,13 +43,17 @@ try:
# exception.
resource.getrusage(RUSAGE_THREAD)
is_thread_resource_usage_supported = True
def get_thread_resource_usage():
return resource.getrusage(RUSAGE_THREAD)
except Exception:
# If the system doesn't support resource.getrusage(RUSAGE_THREAD) then we
# won't track resource usage by returning None.
# won't track resource usage.
is_thread_resource_usage_supported = False
def get_thread_resource_usage():
return None
@ -359,7 +364,11 @@ class LoggingContext(object):
# When we stop, let's record the cpu used since we started
if not self.usage_start:
logger.warning("Called stop on logcontext %s without calling start", self)
# Log a warning on platforms that support thread usage tracking
if is_thread_resource_usage_supported:
logger.warning(
"Called stop on logcontext %s without calling start", self
)
return
utime_delta, stime_delta = self._get_cputime()

41
synapse/push/mailer.py
View File

@ -136,10 +136,11 @@ class Mailer(object):
group together multiple email sending attempts
sid (str): The generated session ID
"""
params = {"token": token, "client_secret": client_secret, "sid": sid}
link = (
self.hs.config.public_baseurl
+ "_matrix/client/unstable/password_reset/email/submit_token"
"?token=%s&client_secret=%s&sid=%s" % (token, client_secret, sid)
+ "_matrix/client/unstable/password_reset/email/submit_token?%s"
% urllib.parse.urlencode(params)
)
template_vars = {"link": link}
@ -163,10 +164,11 @@ class Mailer(object):
group together multiple email sending attempts
sid (str): The generated session ID
"""
params = {"token": token, "client_secret": client_secret, "sid": sid}
link = (
self.hs.config.public_baseurl
+ "_matrix/client/unstable/registration/email/submit_token"
"?token=%s&client_secret=%s&sid=%s" % (token, client_secret, sid)
+ "_matrix/client/unstable/registration/email/submit_token?%s"
% urllib.parse.urlencode(params)
)
template_vars = {"link": link}
@ -177,6 +179,35 @@ class Mailer(object):
template_vars,
)
@defer.inlineCallbacks
def send_add_threepid_mail(self, email_address, token, client_secret, sid):
"""Send an email with a validation link to a user for adding a 3pid to their account
Args:
email_address (str): Email address we're sending the validation link to
token (str): Unique token generated by the server to verify the email was received
client_secret (str): Unique token generated by the client to group together
multiple email sending attempts
sid (str): The generated session ID
"""
params = {"token": token, "client_secret": client_secret, "sid": sid}
link = (
self.hs.config.public_baseurl
+ "_matrix/client/unstable/add_threepid/email/submit_token?%s"
% urllib.parse.urlencode(params)
)
template_vars = {"link": link}
yield self.send_email(
email_address,
"[%s] Validate Your Email" % self.hs.config.server_name,
template_vars,
)
@defer.inlineCallbacks
def send_notification_mail(
self, app_id, user_id, email_address, push_actions, reason
@ -280,7 +311,7 @@ class Mailer(object):
multipart_msg.attach(text_part)
multipart_msg.attach(html_part)
logger.info("Sending email notification to %s" % email_address)
logger.info("Sending email to %s" % email_address)
yield make_deferred_yieldable(
self.sendmail(

9
synapse/res/templates/add_threepid.html Normal file
View File

@ -0,0 +1,9 @@
<html>
<body>
<p>A request to add an email address to your Matrix account has been received. If this was you, please click the link below to confirm adding this email:</p>
<a href="{{ link }}">{{ link }}</a>
<p>If this was not you, you can safely ignore this email. Thank you.</p>
</body>
</html>

6
synapse/res/templates/add_threepid.txt Normal file
View File

@ -0,0 +1,6 @@
A request to add an email address to your Matrix account has been received. If this was you,
please click the link below to confirm adding this email:
{{ link }}
If this was not you, you can safely ignore this email. Thank you.

8
synapse/res/templates/add_threepid_failure.html Normal file
View File

@ -0,0 +1,8 @@
<html>
<head></head>
<body>
<p>The request failed for the following reason: {{ failure_reason }}.</p>
<p>No changes have been made to your account.</p>
</body>
</html>

6
synapse/res/templates/add_threepid_success.html Normal file
View File

@ -0,0 +1,6 @@
<html>
<head></head>
<body>
<p>Your email has now been validated, please return to your client. You may now close this window.</p>
</body>
</html>

341
synapse/rest/client/v2_alpha/account.py
View File

@ -103,16 +103,9 @@ class EmailPasswordRequestTokenRestServlet(RestServlet):
raise SynapseError(400, "Email not found", Codes.THREEPID_NOT_FOUND)
if self.config.threepid_behaviour_email == ThreepidBehaviour.REMOTE:
# Have the configured identity server handle the request
if not self.hs.config.account_threepid_delegate_email:
logger.warn(
"No upstream email account_threepid_delegate configured on the server to "
"handle this request"
)
raise SynapseError(
400, "Password reset by email is not supported on this homeserver"
)
assert self.hs.config.account_threepid_delegate_email
# Have the configured identity server handle the request
ret = yield self.identity_handler.requestEmailToken(
self.hs.config.account_threepid_delegate_email,
email,
@ -200,7 +193,7 @@ class PasswordResetSubmitTokenServlet(RestServlet):
"""Handles 3PID validation token submission"""
PATTERNS = client_patterns(
"/password_reset/(?P<medium>[^/]*)/submit_token/*$", releases=(), unstable=True
"/password_reset/(?P<medium>[^/]*)/submit_token$", releases=(), unstable=True
)
def __init__(self, hs):
@ -214,6 +207,11 @@ class PasswordResetSubmitTokenServlet(RestServlet):
self.config = hs.config
self.clock = hs.get_clock()
self.store = hs.get_datastore()
if self.config.threepid_behaviour_email == ThreepidBehaviour.LOCAL:
self.failure_email_template, = load_jinja2_templates(
self.config.email_template_dir,
[self.config.email_password_reset_template_failure_html],
)
@defer.inlineCallbacks
def on_GET(self, request, medium):
@ -261,34 +259,12 @@ class PasswordResetSubmitTokenServlet(RestServlet):
request.setResponseCode(e.code)
# Show a failure page with a reason
html_template, = load_jinja2_templates(
self.config.email_template_dir,
[self.config.email_password_reset_template_failure_html],
)
template_vars = {"failure_reason": e.msg}
html = html_template.render(**template_vars)
html = self.failure_email_template.render(**template_vars)
request.write(html.encode("utf-8"))
finish_request(request)
@defer.inlineCallbacks
def on_POST(self, request, medium):
if medium != "email":
raise SynapseError(
400, "This medium is currently not supported for password resets"
)
body = parse_json_object_from_request(request)
assert_params_in_dict(body, ["sid", "client_secret", "token"])
valid, _ = yield self.store.validate_threepid_session(
body["sid"], body["client_secret"], body["token"], self.clock.time_msec()
)
response_code = 200 if valid else 400
return response_code, {"success": valid}
class PasswordRestServlet(RestServlet):
PATTERNS = client_patterns("/account/password$")
@ -416,13 +392,35 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
self.identity_handler = hs.get_handlers().identity_handler
self.store = self.hs.get_datastore()
if self.config.threepid_behaviour_email == ThreepidBehaviour.LOCAL:
template_html, template_text = load_jinja2_templates(
self.config.email_template_dir,
[
self.config.email_add_threepid_template_html,
self.config.email_add_threepid_template_text,
],
public_baseurl=self.config.public_baseurl,
)
self.mailer = Mailer(
hs=self.hs,
app_name=self.config.email_app_name,
template_html=template_html,
template_text=template_text,
)
@defer.inlineCallbacks
def on_POST(self, request):
body = parse_json_object_from_request(request)
assert_params_in_dict(
body, ["id_server", "client_secret", "email", "send_attempt"]
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
if self.config.local_threepid_handling_disabled_due_to_email_config:
logger.warn(
"Adding emails have been disabled due to lack of an email config"
)
id_server = "https://" + body["id_server"] # Assume https
raise SynapseError(
400, "Adding an email to your account is disabled on this server"
)
body = parse_json_object_from_request(request)
assert_params_in_dict(body, ["client_secret", "email", "send_attempt"])
client_secret = body["client_secret"]
email = body["email"]
send_attempt = body["send_attempt"]
@ -442,9 +440,30 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
if existing_user_id is not None:
raise SynapseError(400, "Email is already in use", Codes.THREEPID_IN_USE)
if self.config.threepid_behaviour_email == ThreepidBehaviour.REMOTE:
assert self.hs.config.account_threepid_delegate_email
# Have the configured identity server handle the request
ret = yield self.identity_handler.requestEmailToken(
id_server, email, client_secret, send_attempt, next_link
self.hs.config.account_threepid_delegate_email,
email,
client_secret,
send_attempt,
next_link,
)
else:
# Send threepid validation emails from Synapse
sid = yield self.identity_handler.send_threepid_validation(
email,
client_secret,
send_attempt,
self.mailer.send_add_threepid_mail,
next_link,
)
# Wrap the session id in a JSON object
ret = {"sid": sid}
return 200, ret
@ -461,10 +480,8 @@ class MsisdnThreepidRequestTokenRestServlet(RestServlet):
def on_POST(self, request):
body = parse_json_object_from_request(request)
assert_params_in_dict(
body,
["id_server", "client_secret", "country", "phone_number", "send_attempt"],
body, ["client_secret", "country", "phone_number", "send_attempt"]
)
id_server = "https://" + body["id_server"] # Assume https
client_secret = body["client_secret"]
country = body["country"]
phone_number = body["phone_number"]
@ -485,12 +502,146 @@ class MsisdnThreepidRequestTokenRestServlet(RestServlet):
if existing_user_id is not None:
raise SynapseError(400, "MSISDN is already in use", Codes.THREEPID_IN_USE)
ret = yield self.identity_handler.requestMsisdnToken(
id_server, country, phone_number, client_secret, send_attempt, next_link
if not self.hs.config.account_threepid_delegate_msisdn:
logger.warn(
"No upstream msisdn account_threepid_delegate configured on the server to "
"handle this request"
)
raise SynapseError(
400,
"Adding phone numbers to user account is not supported by this homeserver",
)
ret = yield self.identity_handler.requestMsisdnToken(
self.hs.config.account_threepid_delegate_msisdn,
country,
phone_number,
client_secret,
send_attempt,
next_link,
)
return 200, ret
class AddThreepidEmailSubmitTokenServlet(RestServlet):
"""Handles 3PID validation token submission for adding an email to a user's account"""
PATTERNS = client_patterns(
"/add_threepid/email/submit_token$", releases=(), unstable=True
)
def __init__(self, hs):
"""
Args:
hs (synapse.server.HomeServer): server
"""
super().__init__()
self.config = hs.config
self.clock = hs.get_clock()
self.store = hs.get_datastore()
if self.config.threepid_behaviour_email == ThreepidBehaviour.LOCAL:
self.failure_email_template, = load_jinja2_templates(
self.config.email_template_dir,
[self.config.email_add_threepid_template_failure_html],
)
@defer.inlineCallbacks
def on_GET(self, request):
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
if self.config.local_threepid_handling_disabled_due_to_email_config:
logger.warn(
"Adding emails have been disabled due to lack of an email config"
)
raise SynapseError(
400, "Adding an email to your account is disabled on this server"
)
elif self.config.threepid_behaviour_email == ThreepidBehaviour.REMOTE:
raise SynapseError(
400,
"This homeserver is not validating threepids. Use an identity server "
"instead.",
)
sid = parse_string(request, "sid", required=True)
client_secret = parse_string(request, "client_secret", required=True)
token = parse_string(request, "token", required=True)
# Attempt to validate a 3PID session
try:
# Mark the session as valid
next_link = yield self.store.validate_threepid_session(
sid, client_secret, token, self.clock.time_msec()
)
# Perform a 302 redirect if next_link is set
if next_link:
if next_link.startswith("file:///"):
logger.warn(
"Not redirecting to next_link as it is a local file: address"
)
else:
request.setResponseCode(302)
request.setHeader("Location", next_link)
finish_request(request)
return None
# Otherwise show the success template
html = self.config.email_add_threepid_template_success_html_content
request.setResponseCode(200)
except ThreepidValidationError as e:
request.setResponseCode(e.code)
# Show a failure page with a reason
template_vars = {"failure_reason": e.msg}
html = self.failure_email_template.render(**template_vars)
request.write(html.encode("utf-8"))
finish_request(request)
class AddThreepidMsisdnSubmitTokenServlet(RestServlet):
"""Handles 3PID validation token submission for adding a phone number to a user's
account
"""
PATTERNS = client_patterns(
"/add_threepid/msisdn/submit_token$", releases=(), unstable=True
)
def __init__(self, hs):
"""
Args:
hs (synapse.server.HomeServer): server
"""
super().__init__()
self.config = hs.config
self.clock = hs.get_clock()
self.store = hs.get_datastore()
self.identity_handler = hs.get_handlers().identity_handler
@defer.inlineCallbacks
def on_POST(self, request):
if not self.config.account_threepid_delegate_msisdn:
raise SynapseError(
400,
"This homeserver is not validating phone numbers. Use an identity server "
"instead.",
)
body = parse_json_object_from_request(request)
assert_params_in_dict(body, ["client_secret", "sid", "token"])
# Proxy submit_token request to msisdn threepid delegate
response = yield self.identity_handler.proxy_msisdn_submit_token(
self.config.account_threepid_delegate_msisdn,
body["client_secret"],
body["sid"],
body["token"],
)
return 200, response
class ThreepidRestServlet(RestServlet):
PATTERNS = client_patterns("/account/3pid$")
@ -512,6 +663,8 @@ class ThreepidRestServlet(RestServlet):
@defer.inlineCallbacks
def on_POST(self, request):
requester = yield self.auth.get_user_by_req(request)
user_id = requester.user.to_string()
body = parse_json_object_from_request(request)
threepid_creds = body.get("threePidCreds") or body.get("three_pid_creds")
@ -519,34 +672,96 @@ class ThreepidRestServlet(RestServlet):
raise SynapseError(
400, "Missing param three_pid_creds", Codes.MISSING_PARAM
)
assert_params_in_dict(threepid_creds, ["client_secret", "sid"])
client_secret = threepid_creds["client_secret"]
sid = threepid_creds["sid"]
validation_session = yield self.identity_handler.validate_threepid_session(
client_secret, sid
)
if validation_session:
yield self.auth_handler.add_threepid(
user_id,
validation_session["medium"],
validation_session["address"],
validation_session["validated_at"],
)
return 200, {}
raise SynapseError(
400, "No validated 3pid session found", Codes.THREEPID_AUTH_FAILED
)
class ThreepidAddRestServlet(RestServlet):
PATTERNS = client_patterns("/account/3pid/add$", releases=(), unstable=True)
def __init__(self, hs):
super(ThreepidAddRestServlet, self).__init__()
self.hs = hs
self.identity_handler = hs.get_handlers().identity_handler
self.auth = hs.get_auth()
self.auth_handler = hs.get_auth_handler()
@defer.inlineCallbacks
def on_POST(self, request):
requester = yield self.auth.get_user_by_req(request)
user_id = requester.user.to_string()
body = parse_json_object_from_request(request)
assert_params_in_dict(body, ["client_secret", "sid"])
client_secret = body["client_secret"]
sid = body["sid"]
validation_session = yield self.identity_handler.validate_threepid_session(
client_secret, sid
)
if validation_session:
yield self.auth_handler.add_threepid(
user_id,
validation_session["medium"],
validation_session["address"],
validation_session["validated_at"],
)
return 200, {}
raise SynapseError(
400, "No validated 3pid session found", Codes.THREEPID_AUTH_FAILED
)
class ThreepidBindRestServlet(RestServlet):
PATTERNS = client_patterns("/account/3pid/bind$", releases=(), unstable=True)
def __init__(self, hs):
super(ThreepidBindRestServlet, self).__init__()
self.hs = hs
self.identity_handler = hs.get_handlers().identity_handler
self.auth = hs.get_auth()
@defer.inlineCallbacks
def on_POST(self, request):
body = parse_json_object_from_request(request)
assert_params_in_dict(body, ["id_server", "sid", "client_secret"])
id_server = body["id_server"]
sid = body["sid"]
client_secret = body["client_secret"]
id_access_token = body.get("id_access_token") # optional
requester = yield self.auth.get_user_by_req(request)
user_id = requester.user.to_string()
# Specify None as the identity server to retrieve it from the request body instead
threepid = yield self.identity_handler.threepid_from_creds(None, threepid_creds)
if not threepid:
raise SynapseError(400, "Failed to auth 3pid", Codes.THREEPID_AUTH_FAILED)
for reqd in ["medium", "address", "validated_at"]:
if reqd not in threepid:
logger.warn("Couldn't add 3pid: invalid response from ID server")
raise SynapseError(500, "Invalid response from ID Server")
yield self.auth_handler.add_threepid(
user_id, threepid["medium"], threepid["address"], threepid["validated_at"]
yield self.identity_handler.bind_threepid(
client_secret, sid, user_id, id_server, id_access_token
)
if "bind" in body and body["bind"]:
logger.debug("Binding threepid %s to %s", threepid, user_id)
yield self.identity_handler.bind_threepid(threepid_creds, user_id)
return 200, {}
class ThreepidUnbindRestServlet(RestServlet):
PATTERNS = client_patterns("/account/3pid/unbind$")
PATTERNS = client_patterns("/account/3pid/unbind$", releases=(), unstable=True)
def __init__(self, hs):
super(ThreepidUnbindRestServlet, self).__init__()
@ -634,7 +849,11 @@ def register_servlets(hs, http_server):
DeactivateAccountRestServlet(hs).register(http_server)
EmailThreepidRequestTokenRestServlet(hs).register(http_server)
MsisdnThreepidRequestTokenRestServlet(hs).register(http_server)
AddThreepidEmailSubmitTokenServlet(hs).register(http_server)
AddThreepidMsisdnSubmitTokenServlet(hs).register(http_server)
ThreepidRestServlet(hs).register(http_server)
ThreepidAddRestServlet(hs).register(http_server)
ThreepidBindRestServlet(hs).register(http_server)
ThreepidUnbindRestServlet(hs).register(http_server)
ThreepidDeleteRestServlet(hs).register(http_server)
WhoamiRestServlet(hs).register(http_server)

35
synapse/rest/client/v2_alpha/register.py
View File

@ -131,15 +131,9 @@ class EmailRegisterRequestTokenRestServlet(RestServlet):
raise SynapseError(400, "Email is already in use", Codes.THREEPID_IN_USE)
if self.config.threepid_behaviour_email == ThreepidBehaviour.REMOTE:
if not self.hs.config.account_threepid_delegate_email:
logger.warn(
"No upstream email account_threepid_delegate configured on the server to "
"handle this request"
)
raise SynapseError(
400, "Registration by email is not supported on this homeserver"
)
assert self.hs.config.account_threepid_delegate_email
# Have the configured identity server handle the request
ret = yield self.identity_handler.requestEmailToken(
self.hs.config.account_threepid_delegate_email,
email,
@ -246,6 +240,18 @@ class RegistrationSubmitTokenServlet(RestServlet):
self.clock = hs.get_clock()
self.store = hs.get_datastore()
if self.config.threepid_behaviour_email == ThreepidBehaviour.LOCAL:
self.failure_email_template, = load_jinja2_templates(
self.config.email_template_dir,
[self.config.email_registration_template_failure_html],
)
if self.config.threepid_behaviour_email == ThreepidBehaviour.LOCAL:
self.failure_email_template, = load_jinja2_templates(
self.config.email_template_dir,
[self.config.email_registration_template_failure_html],
)
@defer.inlineCallbacks
def on_GET(self, request, medium):
if medium != "email":
@ -289,17 +295,11 @@ class RegistrationSubmitTokenServlet(RestServlet):
request.setResponseCode(200)
except ThreepidValidationError as e:
# Show a failure page with a reason
request.setResponseCode(e.code)
# Show a failure page with a reason
html_template, = load_jinja2_templates(
self.config.email_template_dir,
[self.config.email_registration_template_failure_html],
)
template_vars = {"failure_reason": e.msg}
html = html_template.render(**template_vars)
html = self.failure_email_template.render(**template_vars)
request.write(html.encode("utf-8"))
finish_request(request)
@ -334,6 +334,11 @@ class UsernameAvailabilityRestServlet(RestServlet):
@defer.inlineCallbacks
def on_GET(self, request):
if not self.hs.config.enable_registration:
raise SynapseError(
403, "Registration has been disabled", errcode=Codes.FORBIDDEN
)
ip = self.hs.get_ip_from_request(request)
with self.ratelimiter.ratelimit(ip) as wait_deferred:
yield wait_deferred

19
synapse/rest/client/versions.py
View File

@ -48,7 +48,24 @@ class VersionsRestServlet(RestServlet):
"r0.5.0",
],
# as per MSC1497:
"unstable_features": {"m.lazy_load_members": True},
"unstable_features": {
"m.lazy_load_members": True,
# as per MSC2190, as amended by MSC2264
# to be removed in r0.6.0
"m.id_access_token": True,
# Advertise to clients that they need not include an `id_server`
# parameter during registration or password reset, as Synapse now decides
# itself which identity server to use (or none at all).
#
# This is also used by a client when they wish to bind a 3PID to their
# account, but not bind it to an identity server, the endpoint for which
# also requires `id_server`. If the homeserver is handling 3PID
# verification itself, there is no need to ask the user for `id_server` to
# be supplied.
"m.require_identity_server": False,
# as per MSC2290
"m.separate_add_and_bind": True,
},
},
)

53
synapse/storage/registration.py
View File

@ -25,7 +25,7 @@ from twisted.internet import defer
from twisted.internet.defer import Deferred
from synapse.api.constants import UserTypes
from synapse.api.errors import Codes, StoreError, ThreepidValidationError
from synapse.api.errors import Codes, StoreError, SynapseError, ThreepidValidationError
from synapse.metrics.background_process_metrics import run_as_background_process
from synapse.storage import background_updates
from synapse.storage._base import SQLBaseStore
@ -607,6 +607,26 @@ class RegistrationWorkerStore(SQLBaseStore):
desc="add_user_bound_threepid",
)
def user_get_bound_threepids(self, user_id):
"""Get the threepids that a user has bound to an identity server through the homeserver
The homeserver remembers where binds to an identity server occurred. Using this
method can retrieve those threepids.
Args:
user_id (str): The ID of the user to retrieve threepids for
Returns:
Deferred[list[dict]]: List of dictionaries containing the following:
medium (str): The medium of the threepid (e.g "email")
address (str): The address of the threepid (e.g "bob@example.com")
"""
return self._simple_select_list(
table="user_threepid_id_server",
keyvalues={"user_id": user_id},
retcols=["medium", "address"],
desc="user_get_bound_threepids",
)
def remove_user_bound_threepid(self, user_id, medium, address, id_server):
"""The server proxied an unbind request to the given identity server on
behalf of the given user, so we remove the mapping of threepid to
@ -676,24 +696,37 @@ class RegistrationWorkerStore(SQLBaseStore):
self, medium, client_secret, address=None, sid=None, validated=True
):
"""Gets a session_id and last_send_attempt (if available) for a
client_secret/medium/(address|session_id) combo
combination of validation metadata
Args:
medium (str|None): The medium of the 3PID
address (str|None): The address of the 3PID
sid (str|None): The ID of the validation session
client_secret (str|None): A unique string provided by the client to
help identify this validation attempt
client_secret (str): A unique string provided by the client to help identify this
validation attempt
validated (bool|None): Whether sessions should be filtered by
whether they have been validated already or not. None to
perform no filtering
Returns:
deferred {str, int}|None: A dict containing the
latest session_id and send_attempt count for this 3PID.
Otherwise None if there hasn't been a previous attempt
Deferred[dict|None]: A dict containing the following:
* address - address of the 3pid
* medium - medium of the 3pid
* client_secret - a secret provided by the client for this validation session
* session_id - ID of the validation session
* send_attempt - a number serving to dedupe send attempts for this session
* validated_at - timestamp of when this session was validated if so
Otherwise None if a validation session is not found
"""
keyvalues = {"medium": medium, "client_secret": client_secret}
if not client_secret:
raise SynapseError(
400, "Missing parameter: client_secret", errcode=Codes.MISSING_PARAM
)
keyvalues = {"client_secret": client_secret}
if medium:
keyvalues["medium"] = medium
if address:
keyvalues["address"] = address
if sid:
@ -1250,6 +1283,10 @@ class RegistrationStore(
current_ts (int): The current unix time in milliseconds. Used for
checking token expiry status
Raises:
ThreepidValidationError: if a matching validation token was not found or has
expired
Returns:
deferred str|None: A str representing a link to redirect the user
to if there is one.

20
synapse/storage/schema/delta/56/drop_unused_event_tables.sql
View File

@ -1,20 +0,0 @@
/* Copyright 2019 The Matrix.org Foundation C.I.C.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-- these tables are never used.
DROP TABLE IF EXISTS room_names;
DROP TABLE IF EXISTS topics;
DROP TABLE IF EXISTS history_visibility;
DROP TABLE IF EXISTS guest_access;

43
synctl
View File

@ -71,7 +71,20 @@ def abort(message, colour=RED, stream=sys.stderr):
sys.exit(1)
def start(configfile, daemonize=True):
def start(configfile: str, daemonize: bool = True) -> bool:
"""Attempts to start synapse.
Args:
configfile: path to a yaml synapse config file
daemonize: whether to daemonize synapse or keep it attached to the current
session
Returns:
True if the process started successfully
False if there was an error starting the process
If deamonize is False it will only return once synapse exits.
"""
write("Starting ...")
args = SYNAPSE
@ -83,25 +96,40 @@ def start(configfile, daemonize=True):
try:
subprocess.check_call(args)
write("started synapse.app.homeserver(%r)" % (configfile,), colour=GREEN)
return True
except subprocess.CalledProcessError as e:
write(
"error starting (exit code: %d); see above for logs" % e.returncode,
colour=RED,
)
return False
def start_worker(app, configfile, worker_configfile):
def start_worker(app: str, configfile: str, worker_configfile: str) -> bool:
"""Attempts to start a synapse worker.
Args:
app: name of the worker's appservice
configfile: path to a yaml synapse config file
worker_configfile: path to worker specific yaml synapse file
Returns:
True if the process started successfully
False if there was an error starting the process
"""
args = [sys.executable, "-B", "-m", app, "-c", configfile, "-c", worker_configfile]
try:
subprocess.check_call(args)
write("started %s(%r)" % (app, worker_configfile), colour=GREEN)
return True
except subprocess.CalledProcessError as e:
write(
"error starting %s(%r) (exit code: %d); see above for logs"
% (app, worker_configfile, e.returncode),
colour=RED,
)
return False
def stop(pidfile, app):
@ -292,11 +320,14 @@ def main():
write("All processes exited; now restarting...")
if action == "start" or action == "restart":
error = False
if start_stop_synapse:
# Check if synapse is already running
if os.path.exists(pidfile) and pid_running(int(open(pidfile).read())):
abort("synapse.app.homeserver already running")
start(configfile, bool(options.daemonize))
if not start(configfile, bool(options.daemonize)):
error = True
for worker in workers:
env = os.environ.copy()
@ -307,12 +338,16 @@ def main():
for cache_name, factor in iteritems(worker.cache_factors):
os.environ["SYNAPSE_CACHE_FACTOR_" + cache_name.upper()] = str(factor)
start_worker(worker.app, configfile, worker.configfile)
if not start_worker(worker.app, configfile, worker.configfile):
error = True
# Reset env back to the original
os.environ.clear()
os.environ.update(env)
if error:
exit(1)
if __name__ == "__main__":
main()