Merge branch 'release-v0.99.4' into dinsic

This commit is contained in:
Brendan Abolivier 2019-05-14 11:43:03 +01:00
commit f608ddbe5c
No known key found for this signature in database
GPG Key ID: 1E015C145F1916CD
185 changed files with 3357 additions and 1597 deletions

View File

@ -1,3 +1,105 @@
Synapse 0.99.4rc1 (2019-05-13)
==============================
Features
--------
- Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with `pip install matrix-synapse[systemd]`. ([\#4339](https://github.com/matrix-org/synapse/issues/4339))
- Add a default .m.rule.tombstone push rule. ([\#4867](https://github.com/matrix-org/synapse/issues/4867))
- Add ability for password provider modules to bind email addresses to users upon registration. ([\#4947](https://github.com/matrix-org/synapse/issues/4947))
- Implementation of [MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711) including config options for requiring valid TLS certificates for federation traffic, the ability to disable TLS validation for specific domains, and the ability to specify your own list of CA certificates. ([\#4967](https://github.com/matrix-org/synapse/issues/4967))
- Remove presence list support as per MSC 1819. ([\#4989](https://github.com/matrix-org/synapse/issues/4989))
- Reduce CPU usage starting pushers during start up. ([\#4991](https://github.com/matrix-org/synapse/issues/4991))
- Add a delete group admin API. ([\#5002](https://github.com/matrix-org/synapse/issues/5002))
- Add config option to block users from looking up 3PIDs. ([\#5010](https://github.com/matrix-org/synapse/issues/5010))
- Add context to phonehome stats. ([\#5020](https://github.com/matrix-org/synapse/issues/5020))
- Configure the example systemd units to have a log identifier of `matrix-synapse`
instead of the executable name, `python`.
Contributed by Christoph Müller. ([\#5023](https://github.com/matrix-org/synapse/issues/5023))
- Add time-based account expiration. ([\#5027](https://github.com/matrix-org/synapse/issues/5027), [\#5047](https://github.com/matrix-org/synapse/issues/5047), [\#5073](https://github.com/matrix-org/synapse/issues/5073), [\#5116](https://github.com/matrix-org/synapse/issues/5116))
- Add support for handling /verions, /voip and /push_rules client endpoints to client_reader worker. ([\#5063](https://github.com/matrix-org/synapse/issues/5063), [\#5065](https://github.com/matrix-org/synapse/issues/5065), [\#5070](https://github.com/matrix-org/synapse/issues/5070))
- Add an configuration option to require authentication on /publicRooms and /profile endpoints. ([\#5083](https://github.com/matrix-org/synapse/issues/5083))
- Move admin APIs to `/_synapse/admin/v1`. (The old paths are retained for backwards-compatibility, for now). ([\#5119](https://github.com/matrix-org/synapse/issues/5119))
- Implement an admin API for sending server notices. Many thanks to @krombel who provided a foundation for this work. ([\#5121](https://github.com/matrix-org/synapse/issues/5121), [\#5142](https://github.com/matrix-org/synapse/issues/5142))
Bugfixes
--------
- Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in [#4220](https://github.com/matrix-org/synapse/pull/4220). Contributed by Marcel Fabian Krüger ("[zaugin](https://github.com/zauguin)"). ([\#4555](https://github.com/matrix-org/synapse/issues/4555))
- Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server. ([\#4942](https://github.com/matrix-org/synapse/issues/4942), [\#5103](https://github.com/matrix-org/synapse/issues/5103))
- Fix sync bug which made accepting invites unreliable in worker-mode synapses. ([\#4955](https://github.com/matrix-org/synapse/issues/4955), [\#4956](https://github.com/matrix-org/synapse/issues/4956))
- start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. ([\#4981](https://github.com/matrix-org/synapse/issues/4981))
- Transfer related groups on room upgrade. ([\#4990](https://github.com/matrix-org/synapse/issues/4990))
- Prevent the ability to kick users from a room they aren't in. ([\#4999](https://github.com/matrix-org/synapse/issues/4999))
- Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud <anders@jensenwaud.com>. ([\#5003](https://github.com/matrix-org/synapse/issues/5003))
- Clients timing out/disappearing while downloading from the media repository will now no longer log a spurious "Producer was not unregistered" message. ([\#5009](https://github.com/matrix-org/synapse/issues/5009))
- Fix "cannot import name execute_batch" error with postgres. ([\#5032](https://github.com/matrix-org/synapse/issues/5032))
- Fix disappearing exceptions in manhole. ([\#5035](https://github.com/matrix-org/synapse/issues/5035))
- Workaround bug in twisted where attempting too many concurrent DNS requests could cause it to hang due to running out of file descriptors. ([\#5037](https://github.com/matrix-org/synapse/issues/5037))
- Make sure we're not registering the same 3pid twice on registration. ([\#5071](https://github.com/matrix-org/synapse/issues/5071))
- Don't crash on lack of expiry templates. ([\#5077](https://github.com/matrix-org/synapse/issues/5077))
- Fix the ratelimting on third party invites. ([\#5104](https://github.com/matrix-org/synapse/issues/5104))
- Add some missing limitations to room alias creation. ([\#5124](https://github.com/matrix-org/synapse/issues/5124), [\#5128](https://github.com/matrix-org/synapse/issues/5128))
- Limit the number of EDUs in transactions to 100 as expected by synapse. Thanks to @superboum for this work! ([\#5138](https://github.com/matrix-org/synapse/issues/5138))
- Fix bogus imports in unit tests. ([\#5154](https://github.com/matrix-org/synapse/issues/5154))
Internal Changes
----------------
- Add test to verify threepid auth check added in #4435. ([\#4474](https://github.com/matrix-org/synapse/issues/4474))
- Fix/improve some docstrings in the replication code. ([\#4949](https://github.com/matrix-org/synapse/issues/4949))
- Split synapse.replication.tcp.streams into smaller files. ([\#4953](https://github.com/matrix-org/synapse/issues/4953))
- Refactor replication row generation/parsing. ([\#4954](https://github.com/matrix-org/synapse/issues/4954))
- Run `black` to clean up formatting on `synapse/storage/roommember.py` and `synapse/storage/events.py`. ([\#4959](https://github.com/matrix-org/synapse/issues/4959))
- Remove log line for password via the admin API. ([\#4965](https://github.com/matrix-org/synapse/issues/4965))
- Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor. ([\#4968](https://github.com/matrix-org/synapse/issues/4968))
- Refactor room version definitions. ([\#4969](https://github.com/matrix-org/synapse/issues/4969))
- Reduce log level of .well-known/matrix/client responses. ([\#4972](https://github.com/matrix-org/synapse/issues/4972))
- Add `config.signing_key_path` that can be read by `synapse.config` utility. ([\#4974](https://github.com/matrix-org/synapse/issues/4974))
- Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915. ([\#4982](https://github.com/matrix-org/synapse/issues/4982))
- Rewrite KeyringTestCase as a HomeserverTestCase. ([\#4985](https://github.com/matrix-org/synapse/issues/4985))
- README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section. ([\#4987](https://github.com/matrix-org/synapse/issues/4987))
- Remove a number of unused tables from the database schema. ([\#4992](https://github.com/matrix-org/synapse/issues/4992), [\#5028](https://github.com/matrix-org/synapse/issues/5028), [\#5033](https://github.com/matrix-org/synapse/issues/5033))
- Run `black` on the remainder of `synapse/storage/`. ([\#4996](https://github.com/matrix-org/synapse/issues/4996))
- Fix grammar in get_current_users_in_room and give it a docstring. ([\#4998](https://github.com/matrix-org/synapse/issues/4998))
- Clean up some code in the server-key Keyring. ([\#5001](https://github.com/matrix-org/synapse/issues/5001))
- Convert SYNAPSE_NO_TLS Docker variable to boolean for user friendliness. Contributed by Gabriel Eckerson. ([\#5005](https://github.com/matrix-org/synapse/issues/5005))
- Refactor synapse.storage._base._simple_select_list_paginate. ([\#5007](https://github.com/matrix-org/synapse/issues/5007))
- Store the notary server name correctly in server_keys_json. ([\#5024](https://github.com/matrix-org/synapse/issues/5024))
- Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions. ([\#5030](https://github.com/matrix-org/synapse/issues/5030))
- Remove extraneous period from copyright headers. ([\#5046](https://github.com/matrix-org/synapse/issues/5046))
- Update documentation for where to get Synapse packages. ([\#5067](https://github.com/matrix-org/synapse/issues/5067))
- Add workarounds for pep-517 install errors. ([\#5098](https://github.com/matrix-org/synapse/issues/5098))
- Improve logging when event-signature checks fail. ([\#5100](https://github.com/matrix-org/synapse/issues/5100))
- Factor out an "assert_requester_is_admin" function. ([\#5120](https://github.com/matrix-org/synapse/issues/5120))
- Remove the requirement to authenticate for /admin/server_version. ([\#5122](https://github.com/matrix-org/synapse/issues/5122))
- Prevent an exception from being raised in a IResolutionReceiver and use a more generic error message for blacklisted URL previews. ([\#5155](https://github.com/matrix-org/synapse/issues/5155))
- Run `black` on the tests directory. ([\#5170](https://github.com/matrix-org/synapse/issues/5170))
- Fix CI after new release of isort. ([\#5179](https://github.com/matrix-org/synapse/issues/5179))
Synapse 0.99.3.2 (2019-05-03)
=============================
Internal Changes
----------------
- Ensure that we have `urllib3` <1.25, to resolve incompatibility with `requests`. ([\#5135](https://github.com/matrix-org/synapse/issues/5135))
Synapse 0.99.3.1 (2019-05-03)
=============================
Security update
---------------
This release includes two security fixes:
- Switch to using a cryptographically-secure random number generator for token strings, ensuring they cannot be predicted by an attacker. Thanks to @opnsec for identifying and responsibly disclosing this issue! ([\#5133](https://github.com/matrix-org/synapse/issues/5133))
- Blacklist 0.0.0.0 and :: by default for URL previews. Thanks to @opnsec for identifying and responsibly disclosing this issue too! ([\#5134](https://github.com/matrix-org/synapse/issues/5134))
Synapse 0.99.3 (2019-04-01)
===========================

View File

@ -257,18 +257,29 @@ https://github.com/spantaleev/matrix-docker-ansible-deploy
#### Matrix.org packages
Matrix.org provides Debian/Ubuntu packages of the latest stable version of
Synapse via https://matrix.org/packages/debian/. To use them:
Synapse via https://packages.matrix.org/debian/. They are available for Debian
9 (Stretch), Ubuntu 16.04 (Xenial), and later. To use them:
```
sudo apt install -y lsb-release curl apt-transport-https
echo "deb https://matrix.org/packages/debian `lsb_release -cs` main" |
sudo apt install -y lsb-release wget apt-transport-https
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
sudo tee /etc/apt/sources.list.d/matrix-org.list
curl "https://matrix.org/packages/debian/repo-key.asc" |
sudo apt-key add -
sudo apt update
sudo apt install matrix-synapse-py3
```
**Note**: if you followed a previous version of these instructions which
recommended using `apt-key add` to add an old key from
`https://matrix.org/packages/debian/`, you should note that this key has been
revoked. You should remove the old key with `sudo apt-key remove
C35EB17E1EAE708E6603A9B3AD0592FE47F0DF61`, and follow the above instructions to
update your configuration.
The fingerprint of the repository signing key (as shown by `gpg
/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
`AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058`.
#### Downstream Debian/Ubuntu packages
For `buster` and `sid`, Synapse is available in the Debian repositories and

View File

@ -1 +0,0 @@
Add test to verify threepid auth check added in #4435.

View File

@ -1 +0,0 @@
Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in [#4220](https://github.com/matrix-org/synapse/pull/4220). Contributed by Marcel Fabian Krüger ("[zaugin](https://github.com/zauguin)").

View File

@ -1 +0,0 @@
Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server.

View File

@ -1 +0,0 @@
Add ability for password provider modules to bind email addresses to users upon registration.

View File

@ -1 +0,0 @@
Fix/improve some docstrings in the replication code.

View File

@ -1,2 +0,0 @@
Split synapse.replication.tcp.streams into smaller files.

View File

@ -1 +0,0 @@
Refactor replication row generation/parsing.

View File

@ -1 +0,0 @@
Fix sync bug which made accepting invites unreliable in worker-mode synapses.

View File

@ -1 +0,0 @@
Fix sync bug which made accepting invites unreliable in worker-mode synapses.

View File

@ -1 +0,0 @@
Run `black` to clean up formatting on `synapse/storage/roommember.py` and `synapse/storage/events.py`.

View File

@ -1 +0,0 @@
Remove log line for password via the admin API.

View File

@ -1 +0,0 @@
Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor.

View File

@ -1,2 +0,0 @@
Refactor room version definitions.

View File

@ -1 +0,0 @@
Add `config.signing_key_path` that can be read by `synapse.config` utility.

View File

@ -1 +0,0 @@
start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too.

View File

@ -1 +0,0 @@
Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915.

View File

@ -1 +0,0 @@
Rewrite KeyringTestCase as a HomeserverTestCase.

View File

@ -1 +0,0 @@
README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section.

View File

@ -1 +0,0 @@
Remove presence list support as per MSC 1819.

View File

@ -1 +0,0 @@
Transfer related groups on room upgrade.

View File

@ -1 +0,0 @@
Reduce CPU usage starting pushers during start up.

View File

@ -1 +0,0 @@
Remove a number of unused tables from the database schema.

View File

@ -1 +0,0 @@
Run `black` on the remainder of `synapse/storage/`.

View File

@ -1 +0,0 @@
Fix grammar in get_current_users_in_room and give it a docstring.

View File

@ -1 +0,0 @@
Prevent the ability to kick users from a room they aren't in.

View File

@ -1 +0,0 @@
Clean up some code in the server-key Keyring.

View File

@ -1 +0,0 @@
Add a delete group admin API.

View File

@ -1 +0,0 @@
Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud <anders@jensenwaud.com>.

View File

@ -1 +0,0 @@
Refactor synapse.storage._base._simple_select_list_paginate.

View File

@ -1 +0,0 @@
Add config option to block users from looking up 3PIDs.

View File

@ -1 +0,0 @@
Add context to phonehome stats.

View File

@ -1 +0,0 @@
Store the notary server name correctly in server_keys_json.

View File

@ -1 +0,0 @@
Add time-based account expiration.

View File

@ -1 +0,0 @@
Remove a number of unused tables from the database schema.

View File

@ -1 +0,0 @@
Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions.

View File

@ -1 +0,0 @@
Fix "cannot import name execute_batch" error with postgres.

View File

@ -1 +0,0 @@
Remove a number of unused tables from the database schema.

View File

@ -1 +0,0 @@
Fix disappearing exceptions in manhole.

View File

@ -1 +0,0 @@
Remove extraneous period from copyright headers.

View File

@ -12,6 +12,7 @@ ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.%i --config-path=/
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=3
SyslogIdentifier=matrix-synapse-%i
[Install]
WantedBy=matrix-synapse.service

View File

@ -11,6 +11,7 @@ ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --confi
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=3
SyslogIdentifier=matrix-synapse
[Install]
WantedBy=matrix.target

View File

@ -22,10 +22,10 @@ Group=nogroup
WorkingDirectory=/opt/synapse
ExecStart=/opt/synapse/env/bin/python -m synapse.app.homeserver --config-path=/opt/synapse/homeserver.yaml
SyslogIdentifier=matrix-synapse
# adjust the cache factor if necessary
# Environment=SYNAPSE_CACHE_FACTOR=2.0
[Install]
WantedBy=multi-user.target

19
debian/changelog vendored
View File

@ -1,3 +1,22 @@
matrix-synapse-py3 (0.99.3.2+nmu1) UNRELEASED; urgency=medium
[ Christoph Müller ]
* Configure the systemd units to have a log identifier of `matrix-synapse`
-- Christoph Müller <iblzm@hotmail.de> Wed, 17 Apr 2019 16:17:32 +0200
matrix-synapse-py3 (0.99.3.2) stable; urgency=medium
* New synapse release 0.99.3.2.
-- Synapse Packaging team <packages@matrix.org> Fri, 03 May 2019 18:56:20 +0100
matrix-synapse-py3 (0.99.3.1) stable; urgency=medium
* New synapse release 0.99.3.1.
-- Synapse Packaging team <packages@matrix.org> Fri, 03 May 2019 16:02:43 +0100
matrix-synapse-py3 (0.99.3) stable; urgency=medium
[ Richard van der Hoff ]

View File

@ -11,6 +11,7 @@ ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --confi
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=3
SyslogIdentifier=matrix-synapse
[Install]
WantedBy=multi-user.target

View File

@ -50,12 +50,15 @@ RUN apt-get update -qq -o Acquire::Languages=none \
debhelper \
devscripts \
dh-systemd \
libsystemd-dev \
lsb-release \
pkg-config \
python3-dev \
python3-pip \
python3-setuptools \
python3-venv \
sqlite3
sqlite3 \
libpq-dev
COPY --from=builder /dh-virtualenv_1.1-1_all.deb /

View File

@ -102,8 +102,9 @@ when ``SYNAPSE_CONFIG_PATH`` is not set.
* ``SYNAPSE_SERVER_NAME`` (mandatory), the server public hostname.
* ``SYNAPSE_REPORT_STATS``, (mandatory, ``yes`` or ``no``), enable anonymous
statistics reporting back to the Matrix project which helps us to get funding.
* ``SYNAPSE_NO_TLS``, set this variable to disable TLS in Synapse (use this if
you run your own TLS-capable reverse proxy).
* `SYNAPSE_NO_TLS`, (accepts `true`, `false`, `on`, `off`, `1`, `0`, `yes`, `no`]): disable
TLS in Synapse (use this if you run your own TLS-capable reverse proxy). Defaults
to `false` (ie, TLS is enabled by default).
* ``SYNAPSE_ENABLE_REGISTRATION``, set this variable to enable registration on
the Synapse instance.
* ``SYNAPSE_ALLOW_GUEST``, set this variable to allow guest joining this server.

View File

@ -59,6 +59,18 @@ else:
if not os.path.exists("/compiled"): os.mkdir("/compiled")
config_path = "/compiled/homeserver.yaml"
# Convert SYNAPSE_NO_TLS to boolean if exists
if "SYNAPSE_NO_TLS" in environ:
tlsanswerstring = str.lower(environ["SYNAPSE_NO_TLS"])
if tlsanswerstring in ("true", "on", "1", "yes"):
environ["SYNAPSE_NO_TLS"] = True
else:
if tlsanswerstring in ("false", "off", "0", "no"):
environ["SYNAPSE_NO_TLS"] = False
else:
print("Environment variable \"SYNAPSE_NO_TLS\" found but value \"" + tlsanswerstring + "\" unrecognized; exiting.")
sys.exit(2)
convert("/conf/homeserver.yaml", config_path, environ)
convert("/conf/log.config", "/compiled/log.config", environ)

View File

@ -177,7 +177,6 @@ You can do this with a `.well-known` file as follows:
on `customer.example.net:8000` it correctly handles HTTP requests with
Host header set to `customer.example.net:8000`.
## FAQ
### Synapse 0.99.0 has just been released, what do I need to do right now?

View File

@ -0,0 +1,42 @@
Account validity API
====================
This API allows a server administrator to manage the validity of an account. To
use it, you must enable the account validity feature (under
``account_validity``) in Synapse's configuration.
Renew account
-------------
This API extends the validity of an account by as much time as configured in the
``period`` parameter from the ``account_validity`` configuration.
The API is::
POST /_synapse/admin/v1/account_validity/validity
with the following body:
.. code:: json
{
"user_id": "<user ID for the account to renew>",
"expiration_ts": 0,
"enable_renewal_emails": true
}
``expiration_ts`` is an optional parameter and overrides the expiration date,
which otherwise defaults to now + validity period.
``enable_renewal_emails`` is also an optional parameter and enables/disables
sending renewal emails to the user. Defaults to true.
The API returns with the new expiration date for this account, as a timestamp in
milliseconds since epoch:
.. code:: json
{
"expiration_ts": 0
}

View File

@ -8,7 +8,7 @@ being deleted.
The API is:
```
POST /_matrix/client/r0/admin/delete_group/<group_id>
POST /_synapse/admin/v1/delete_group/<group_id>
```
including an `access_token` of a server admin.

View File

@ -4,7 +4,7 @@ This API gets a list of known media in a room.
The API is:
```
GET /_matrix/client/r0/admin/room/<room_id>/media
GET /_synapse/admin/v1/room/<room_id>/media
```
including an `access_token` of a server admin.

View File

@ -10,7 +10,7 @@ paginate further back in the room from the point being purged from.
The API is:
``POST /_matrix/client/r0/admin/purge_history/<room_id>[/<event_id>]``
``POST /_synapse/admin/v1/purge_history/<room_id>[/<event_id>]``
including an ``access_token`` of a server admin.
@ -49,7 +49,7 @@ Purge status query
It is possible to poll for updates on recent purges with a second API;
``GET /_matrix/client/r0/admin/purge_history_status/<purge_id>``
``GET /_synapse/admin/v1/purge_history_status/<purge_id>``
(again, with a suitable ``access_token``). This API returns a JSON body like
the following:

View File

@ -6,7 +6,7 @@ media.
The API is::
POST /_matrix/client/r0/admin/purge_media_cache?before_ts=<unix_timestamp_in_ms>&access_token=<access_token>
POST /_synapse/admin/v1/purge_media_cache?before_ts=<unix_timestamp_in_ms>&access_token=<access_token>
{}

View File

@ -12,7 +12,7 @@ is not enabled.
To fetch the nonce, you need to request one from the API::
> GET /_matrix/client/r0/admin/register
> GET /_synapse/admin/v1/register
< {"nonce": "thisisanonce"}
@ -22,7 +22,7 @@ body containing the nonce, username, password, whether they are an admin
As an example::
> POST /_matrix/client/r0/admin/register
> POST /_synapse/admin/v1/register
> {
"nonce": "thisisanonce",
"username": "pepper_roni",

View File

@ -0,0 +1,48 @@
# Server Notices
The API to send notices is as follows:
```
POST /_synapse/admin/v1/send_server_notice
```
or:
```
PUT /_synapse/admin/v1/send_server_notice/{txnId}
```
You will need to authenticate with an access token for an admin user.
When using the `PUT` form, retransmissions with the same transaction ID will be
ignored in the same way as with `PUT
/_matrix/client/r0/rooms/{roomId}/send/{eventType}/{txnId}`.
The request body should look something like the following:
```json
{
"user_id": "@target_user:server_name",
"content": {
"msgtype": "m.text",
"body": "This is my message"
}
}
```
You can optionally include the following additional parameters:
* `type`: the type of event. Defaults to `m.room.message`.
* `state_key`: Setting this will result in a state event being sent.
Once the notice has been sent, the API will return the following response:
```json
{
"event_id": "<event_id>"
}
```
Note that server notices must be enabled in `homeserver.yaml` before this API
can be used. See [server_notices.md](../server_notices.md) for more information.

View File

@ -5,7 +5,7 @@ This API returns information about a specific user account.
The api is::
GET /_matrix/client/r0/admin/whois/<user_id>
GET /_synapse/admin/v1/whois/<user_id>
including an ``access_token`` of a server admin.
@ -50,7 +50,7 @@ references to it).
The api is::
POST /_matrix/client/r0/admin/deactivate/<user_id>
POST /_synapse/admin/v1/deactivate/<user_id>
with a body of:
@ -73,7 +73,7 @@ Changes the password of another user.
The api is::
POST /_matrix/client/r0/admin/reset_password/<user_id>
POST /_synapse/admin/v1/reset_password/<user_id>
with a body of:

View File

@ -8,9 +8,7 @@ contains Synapse version information).
The api is::
GET /_matrix/client/r0/admin/server_version
including an ``access_token`` of a server admin.
GET /_synapse/admin/v1/server_version
It returns a JSON body like the following:

View File

@ -48,7 +48,10 @@ How to monitor Synapse metrics using Prometheus
- job_name: "synapse"
metrics_path: "/_synapse/metrics"
static_configs:
- targets: ["my.server.here:9092"]
- targets: ["my.server.here:port"]
where ``my.server.here`` is the IP address of Synapse, and ``port`` is the listener port
configured with the ``metrics`` resource.
If your prometheus is older than 1.5.2, you will need to replace
``static_configs`` in the above with ``target_groups``.

View File

@ -69,6 +69,7 @@ Let's assume that we expect clients to connect to our server at
SSLEngine on
ServerName matrix.example.com;
AllowEncodedSlashes NoDecode
ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix
</VirtualHost>
@ -77,6 +78,7 @@ Let's assume that we expect clients to connect to our server at
SSLEngine on
ServerName example.com;
AllowEncodedSlashes NoDecode
ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix
</VirtualHost>

View File

@ -69,6 +69,20 @@ pid_file: DATADIR/homeserver.pid
#
#use_presence: false
# Whether to require authentication to retrieve profile data (avatars,
# display names) of other users through the client API. Defaults to
# 'false'. Note that profile data is also available via the federation
# API, so this setting is of limited value if federation is enabled on
# the server.
#
#require_auth_for_profile_requests: true
# If set to 'true', requires authentication to access the server's
# public rooms directory through the client API, and forbids any other
# homeserver to fetch it via federation. Defaults to 'false'.
#
#restrict_public_rooms_to_local_users: true
# The GC threshold parameters to pass to `gc.set_threshold`, if defined
#
#gc_thresholds: [700, 10, 10]
@ -136,8 +150,8 @@ pid_file: DATADIR/homeserver.pid
#
# Valid resource names are:
#
# client: the client-server API (/_matrix/client). Also implies 'media' and
# 'static'.
# client: the client-server API (/_matrix/client), and the synapse admin
# API (/_synapse/admin). Also implies 'media' and 'static'.
#
# consent: user consent forms (/_matrix/consent). See
# docs/consent_tracking.md.
@ -239,6 +253,11 @@ listeners:
# Used by phonehome stats to group together related servers.
#server_context: context
# Whether to require a user to be in the room to add an alias to it.
# Defaults to 'true'.
#
#require_membership_for_aliases: false
## TLS ##
@ -260,6 +279,40 @@ listeners:
#
#tls_private_key_path: "CONFDIR/SERVERNAME.tls.key"
# Whether to verify TLS certificates when sending federation traffic.
#
# This currently defaults to `false`, however this will change in
# Synapse 1.0 when valid federation certificates will be required.
#
#federation_verify_certificates: true
# Skip federation certificate verification on the following whitelist
# of domains.
#
# This setting should only be used in very specific cases, such as
# federation over Tor hidden services and similar. For private networks
# of homeservers, you likely want to use a private CA instead.
#
# Only effective if federation_verify_certicates is `true`.
#
#federation_certificate_verification_whitelist:
# - lon.example.com
# - *.domain.com
# - *.onion
# List of custom certificate authorities for federation traffic.
#
# This setting should only normally be used within a private network of
# homeservers.
#
# Note that this list will replace those that are provided by your
# operating environment. Certificates must be in PEM format.
#
#federation_custom_ca_list:
# - myCA1.pem
# - myCA2.pem
# - myCA3.pem
# ACME support: This will configure Synapse to request a valid TLS certificate
# for your configured `server_name` via Let's Encrypt.
#
@ -509,11 +562,12 @@ uploads_path: "DATADIR/uploads"
# height: 600
# method: scale
# Is the preview URL API enabled? If enabled, you *must* specify
# an explicit url_preview_ip_range_blacklist of IPs that the spider is
# denied from accessing.
# Is the preview URL API enabled?
#
#url_preview_enabled: false
# 'false' by default: uncomment the following to enable it (and specify a
# url_preview_ip_range_blacklist blacklist).
#
#url_preview_enabled: true
# List of IP address CIDR ranges that the URL preview spider is denied
# from accessing. There are no defaults: you must explicitly
@ -523,6 +577,12 @@ uploads_path: "DATADIR/uploads"
# synapse to issue arbitrary GET requests to your internal services,
# causing serious security issues.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
# This must be specified if url_preview_enabled is set. It is recommended that
# you uncomment the following list as a starting point.
#
#url_preview_ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@ -533,7 +593,7 @@ uploads_path: "DATADIR/uploads"
# - '::1/128'
# - 'fe80::/64'
# - 'fc00::/7'
#
# List of IP address CIDR ranges that the URL preview spider is allowed
# to access even if they are specified in url_preview_ip_range_blacklist.
# This is useful for specifying exceptions to wide-ranging blacklisted
@ -646,11 +706,31 @@ uploads_path: "DATADIR/uploads"
#
#enable_registration: false
# Optional account validity parameter. This allows for, e.g., accounts to
# be denied any request after a given period.
# Optional account validity configuration. This allows for accounts to be denied
# any request after a given period.
#
# ``enabled`` defines whether the account validity feature is enabled. Defaults
# to False.
#
# ``period`` allows setting the period after which an account is valid
# after its registration. When renewing the account, its validity period
# will be extended by this amount of time. This parameter is required when using
# the account validity feature.
#
# ``renew_at`` is the amount of time before an account's expiry date at which
# Synapse will send an email to the account's email address with a renewal link.
# This needs the ``email`` and ``public_baseurl`` configuration sections to be
# filled.
#
# ``renew_email_subject`` is the subject of the email sent out with the renewal
# link. ``%(app)s`` can be used as a placeholder for the ``app_name`` parameter
# from the ``email`` section.
#
#account_validity:
# enabled: True
# period: 6w
# renew_at: 1w
# renew_email_subject: "Renew your %(app)s account"
# The user must provide all of the below types of 3PID when registering.
#
@ -949,7 +1029,7 @@ password_config:
# Enable sending emails for notification events
# Enable sending emails for notification events or expiry notices
# Defining a custom URL for Riot is only needed if email notifications
# should contain links to a self-hosted installation of Riot; when set
# the "app_name" setting is ignored.
@ -971,6 +1051,9 @@ password_config:
# #template_dir: res/templates
# notif_template_html: notif_mail.html
# notif_template_text: notif_mail.txt
# # Templates for account expiry notices.
# expiry_template_html: notice_expiry.html
# expiry_template_text: notice_expiry.txt
# notif_for_new_users: True
# riot_base_url: "http://localhost/riot"

View File

@ -1,5 +1,4 @@
Server Notices
==============
# Server Notices
'Server Notices' are a new feature introduced in Synapse 0.30. They provide a
channel whereby server administrators can send messages to users on the server.
@ -11,8 +10,7 @@ they may also find a use for features such as "Message of the day".
This is a feature specific to Synapse, but it uses standard Matrix
communication mechanisms, so should work with any Matrix client.
User experience
---------------
## User experience
When the user is first sent a server notice, they will get an invitation to a
room (typically called 'Server Notices', though this is configurable in
@ -29,8 +27,7 @@ levels.
Having joined the room, the user can leave the room if they want. Subsequent
server notices will then cause a new room to be created.
Synapse configuration
---------------------
## Synapse configuration
Server notices come from a specific user id on the server. Server
administrators are free to choose the user id - something like `server` is
@ -58,17 +55,7 @@ room which will be created.
`system_mxid_display_name` and `system_mxid_avatar_url` can be used to set the
displayname and avatar of the Server Notices user.
Sending notices
---------------
## Sending notices
As of the current version of synapse, there is no convenient interface for
sending notices (other than the automated ones sent as part of consent
tracking).
In the meantime, it is possible to test this feature using the manhole. Having
gone into the manhole as described in [manhole.md](manhole.md), a notice can be
sent with something like:
```
>>> hs.get_server_notices_manager().send_notice('@user:server.com', {'msgtype':'m.text', 'body':'foo'})
```
To send server notices to users you can use the
[admin_api](admin_api/server_notices.md).

View File

@ -227,6 +227,12 @@ following regular expressions::
^/_matrix/client/(api/v1|r0|unstable)/account/3pid$
^/_matrix/client/(api/v1|r0|unstable)/keys/query$
^/_matrix/client/(api/v1|r0|unstable)/keys/changes$
^/_matrix/client/versions$
^/_matrix/client/(api/v1|r0|unstable)/voip/turnServer$
Additionally, the following REST endpoints can be handled for GET requests::
^/_matrix/client/(api/v1|r0|unstable)/pushrules/.*$
Additionally, the following REST endpoints can be handled, but all requests must
be routed to the same instance::

View File

@ -24,6 +24,7 @@ DISTS = (
"ubuntu:xenial",
"ubuntu:bionic",
"ubuntu:cosmic",
"ubuntu:disco",
)
DESC = '''\

View File

@ -86,13 +86,9 @@ long_description = read_file(("README.rst",))
REQUIREMENTS = dependencies['REQUIREMENTS']
CONDITIONAL_REQUIREMENTS = dependencies['CONDITIONAL_REQUIREMENTS']
ALL_OPTIONAL_REQUIREMENTS = dependencies['ALL_OPTIONAL_REQUIREMENTS']
# Make `pip install matrix-synapse[all]` install all the optional dependencies.
ALL_OPTIONAL_REQUIREMENTS = set()
for optional_deps in CONDITIONAL_REQUIREMENTS.values():
ALL_OPTIONAL_REQUIREMENTS = set(optional_deps) | ALL_OPTIONAL_REQUIREMENTS
CONDITIONAL_REQUIREMENTS["all"] = list(ALL_OPTIONAL_REQUIREMENTS)

View File

@ -27,4 +27,4 @@ try:
except ImportError:
pass
__version__ = "0.99.3"
__version__ = "0.99.4rc1"

View File

@ -231,8 +231,9 @@ class Auth(object):
# Deny the request if the user account has expired.
if self._account_validity.enabled:
expiration_ts = yield self.store.get_expiration_ts_for_user(user)
if self.clock.time_msec() >= expiration_ts:
user_id = user.to_string()
expiration_ts = yield self.store.get_expiration_ts_for_user(user_id)
if expiration_ts is not None and self.clock.time_msec() >= expiration_ts:
raise AuthError(
403,
"User account has expired",
@ -548,7 +549,7 @@ class Auth(object):
""" Check if the given user is a local server admin.
Args:
user (str): mxid of user to check
user (UserID): user to check
Returns:
bool: True if the user is an admin

View File

@ -20,6 +20,9 @@
# the "depth" field on events is limited to 2**63 - 1
MAX_DEPTH = 2**63 - 1
# the maximum length for a room alias is 255 characters
MAX_ALIAS_LENGTH = 255
class Membership(object):

View File

@ -22,13 +22,14 @@ import traceback
import psutil
from daemonize import Daemonize
from twisted.internet import error, reactor
from twisted.internet import defer, error, reactor
from twisted.protocols.tls import TLSMemoryBIOFactory
import synapse
from synapse.app import check_bind_error
from synapse.crypto import context_factory
from synapse.util import PreserveLoggingContext
from synapse.util.async_helpers import Linearizer
from synapse.util.rlimit import change_resource_limit
from synapse.util.versionstring import get_version_string
@ -99,6 +100,8 @@ def start_reactor(
logger (logging.Logger): logger instance to pass to Daemonize
"""
install_dns_limiter(reactor)
def run():
# make sure that we run the reactor with the sentinel log context,
# otherwise other PreserveLoggingContext instances will get confused
@ -312,3 +315,81 @@ def setup_sentry(hs):
name = hs.config.worker_name if hs.config.worker_name else "master"
scope.set_tag("worker_app", app)
scope.set_tag("worker_name", name)
def install_dns_limiter(reactor, max_dns_requests_in_flight=100):
"""Replaces the resolver with one that limits the number of in flight DNS
requests.
This is to workaround https://twistedmatrix.com/trac/ticket/9620, where we
can run out of file descriptors and infinite loop if we attempt to do too
many DNS queries at once
"""
new_resolver = _LimitedHostnameResolver(
reactor.nameResolver, max_dns_requests_in_flight,
)
reactor.installNameResolver(new_resolver)
class _LimitedHostnameResolver(object):
"""Wraps a IHostnameResolver, limiting the number of in-flight DNS lookups.
"""
def __init__(self, resolver, max_dns_requests_in_flight):
self._resolver = resolver
self._limiter = Linearizer(
name="dns_client_limiter", max_count=max_dns_requests_in_flight,
)
def resolveHostName(self, resolutionReceiver, hostName, portNumber=0,
addressTypes=None, transportSemantics='TCP'):
# Note this is happening deep within the reactor, so we don't need to
# worry about log contexts.
# We need this function to return `resolutionReceiver` so we do all the
# actual logic involving deferreds in a separate function.
self._resolve(
resolutionReceiver, hostName, portNumber,
addressTypes, transportSemantics,
)
return resolutionReceiver
@defer.inlineCallbacks
def _resolve(self, resolutionReceiver, hostName, portNumber=0,
addressTypes=None, transportSemantics='TCP'):
with (yield self._limiter.queue(())):
# resolveHostName doesn't return a Deferred, so we need to hook into
# the receiver interface to get told when resolution has finished.
deferred = defer.Deferred()
receiver = _DeferredResolutionReceiver(resolutionReceiver, deferred)
self._resolver.resolveHostName(
receiver, hostName, portNumber,
addressTypes, transportSemantics,
)
yield deferred
class _DeferredResolutionReceiver(object):
"""Wraps a IResolutionReceiver and simply resolves the given deferred when
resolution is complete
"""
def __init__(self, receiver, deferred):
self._receiver = receiver
self._deferred = deferred
def resolutionBegan(self, resolutionInProgress):
self._receiver.resolutionBegan(resolutionInProgress)
def addressResolved(self, address):
self._receiver.addressResolved(address)
def resolutionComplete(self):
self._deferred.callback(())
self._receiver.resolutionComplete()

View File

@ -45,6 +45,7 @@ from synapse.replication.slave.storage.room import RoomStore
from synapse.replication.slave.storage.transactions import SlavedTransactionStore
from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.rest.client.v1.login import LoginRestServlet
from synapse.rest.client.v1.push_rule import PushRuleRestServlet
from synapse.rest.client.v1.room import (
JoinedRoomMemberListRestServlet,
PublicRoomListRestServlet,
@ -52,9 +53,11 @@ from synapse.rest.client.v1.room import (
RoomMemberListRestServlet,
RoomStateRestServlet,
)
from synapse.rest.client.v1.voip import VoipRestServlet
from synapse.rest.client.v2_alpha.account import ThreepidRestServlet
from synapse.rest.client.v2_alpha.keys import KeyChangesServlet, KeyQueryServlet
from synapse.rest.client.v2_alpha.register import RegisterRestServlet
from synapse.rest.client.versions import VersionsRestServlet
from synapse.server import HomeServer
from synapse.storage.engines import create_engine
from synapse.util.httpresourcetree import create_resource_tree
@ -109,12 +112,12 @@ class ClientReaderServer(HomeServer):
ThreepidRestServlet(self).register(resource)
KeyQueryServlet(self).register(resource)
KeyChangesServlet(self).register(resource)
VoipRestServlet(self).register(resource)
PushRuleRestServlet(self).register(resource)
VersionsRestServlet().register(resource)
resources.update({
"/_matrix/client/r0": resource,
"/_matrix/client/unstable": resource,
"/_matrix/client/v2_alpha": resource,
"/_matrix/client/api/v1": resource,
"/_matrix/client": resource,
})
root_resource = create_resource_tree(resources, NoResource())

View File

@ -62,6 +62,7 @@ from synapse.python_dependencies import check_requirements
from synapse.replication.http import REPLICATION_PREFIX, ReplicationRestResource
from synapse.replication.tcp.resource import ReplicationStreamProtocolFactory
from synapse.rest import ClientRestResource
from synapse.rest.admin import AdminRestResource
from synapse.rest.key.v2 import KeyApiV2Resource
from synapse.rest.media.v0.content_repository import ContentRepoResource
from synapse.rest.well_known import WellKnownResource
@ -180,6 +181,7 @@ class SynapseHomeServer(HomeServer):
"/_matrix/client/v2_alpha": client_resource,
"/_matrix/client/versions": client_resource,
"/.well-known/matrix/client": WellKnownResource(self),
"/_synapse/admin": AdminRestResource(self),
})
if self.get_config().saml2_enabled:

View File

@ -71,6 +71,12 @@ class EmailConfig(Config):
self.email_notif_from = email_config["notif_from"]
self.email_notif_template_html = email_config["notif_template_html"]
self.email_notif_template_text = email_config["notif_template_text"]
self.email_expiry_template_html = email_config.get(
"expiry_template_html", "notice_expiry.html",
)
self.email_expiry_template_text = email_config.get(
"expiry_template_text", "notice_expiry.txt",
)
template_dir = email_config.get("template_dir")
# we need an absolute path, because we change directory after starting (and
@ -120,7 +126,7 @@ class EmailConfig(Config):
def default_config(self, config_dir_path, server_name, **kwargs):
return """
# Enable sending emails for notification events
# Enable sending emails for notification events or expiry notices
# Defining a custom URL for Riot is only needed if email notifications
# should contain links to a self-hosted installation of Riot; when set
# the "app_name" setting is ignored.
@ -142,6 +148,9 @@ class EmailConfig(Config):
# #template_dir: res/templates
# notif_template_html: notif_mail.html
# notif_template_text: notif_mail.txt
# # Templates for account expiry notices.
# expiry_template_html: notice_expiry.html
# expiry_template_text: notice_expiry.txt
# notif_for_new_users: True
# riot_base_url: "http://localhost/riot"
"""

View File

@ -21,12 +21,26 @@ from synapse.util.stringutils import random_string_with_symbols
class AccountValidityConfig(Config):
def __init__(self, config):
self.enabled = (len(config) > 0)
def __init__(self, config, synapse_config):
self.enabled = config.get("enabled", False)
self.renew_by_email_enabled = ("renew_at" in config)
period = config.get("period", None)
if period:
self.period = self.parse_duration(period)
if self.enabled:
if "period" in config:
self.period = self.parse_duration(config["period"])
else:
raise ConfigError("'period' is required when using account validity")
if "renew_at" in config:
self.renew_at = self.parse_duration(config["renew_at"])
if "renew_email_subject" in config:
self.renew_email_subject = config["renew_email_subject"]
else:
self.renew_email_subject = "Renew your %(app)s account"
if self.renew_by_email_enabled and "public_baseurl" not in synapse_config:
raise ConfigError("Can't send renewal emails without 'public_baseurl'")
class RegistrationConfig(Config):
@ -40,7 +54,9 @@ class RegistrationConfig(Config):
strtobool(str(config["disable_registration"]))
)
self.account_validity = AccountValidityConfig(config.get("account_validity", {}))
self.account_validity = AccountValidityConfig(
config.get("account_validity", {}), config,
)
self.registrations_require_3pid = config.get("registrations_require_3pid", [])
self.allowed_local_3pids = config.get("allowed_local_3pids", [])
@ -108,11 +124,31 @@ class RegistrationConfig(Config):
#
#enable_registration: false
# Optional account validity parameter. This allows for, e.g., accounts to
# be denied any request after a given period.
# Optional account validity configuration. This allows for accounts to be denied
# any request after a given period.
#
# ``enabled`` defines whether the account validity feature is enabled. Defaults
# to False.
#
# ``period`` allows setting the period after which an account is valid
# after its registration. When renewing the account, its validity period
# will be extended by this amount of time. This parameter is required when using
# the account validity feature.
#
# ``renew_at`` is the amount of time before an account's expiry date at which
# Synapse will send an email to the account's email address with a renewal link.
# This needs the ``email`` and ``public_baseurl`` configuration sections to be
# filled.
#
# ``renew_email_subject`` is the subject of the email sent out with the renewal
# link. ``%%(app)s`` can be used as a placeholder for the ``app_name`` parameter
# from the ``email`` section.
#
#account_validity:
# enabled: True
# period: 6w
# renew_at: 1w
# renew_email_subject: "Renew your %%(app)s account"
# The user must provide all of the below types of 3PID when registering.
#

View File

@ -186,17 +186,21 @@ class ContentRepositoryConfig(Config):
except ImportError:
raise ConfigError(MISSING_NETADDR)
if "url_preview_ip_range_blacklist" in config:
self.url_preview_ip_range_blacklist = IPSet(
config["url_preview_ip_range_blacklist"]
)
else:
if "url_preview_ip_range_blacklist" not in config:
raise ConfigError(
"For security, you must specify an explicit target IP address "
"blacklist in url_preview_ip_range_blacklist for url previewing "
"to work"
)
self.url_preview_ip_range_blacklist = IPSet(
config["url_preview_ip_range_blacklist"]
)
# we always blacklist '0.0.0.0' and '::', which are supposed to be
# unroutable addresses.
self.url_preview_ip_range_blacklist.update(['0.0.0.0', '::'])
self.url_preview_ip_range_whitelist = IPSet(
config.get("url_preview_ip_range_whitelist", ())
)
@ -260,11 +264,12 @@ class ContentRepositoryConfig(Config):
#thumbnail_sizes:
%(formatted_thumbnail_sizes)s
# Is the preview URL API enabled? If enabled, you *must* specify
# an explicit url_preview_ip_range_blacklist of IPs that the spider is
# denied from accessing.
# Is the preview URL API enabled?
#
#url_preview_enabled: false
# 'false' by default: uncomment the following to enable it (and specify a
# url_preview_ip_range_blacklist blacklist).
#
#url_preview_enabled: true
# List of IP address CIDR ranges that the URL preview spider is denied
# from accessing. There are no defaults: you must explicitly
@ -274,6 +279,12 @@ class ContentRepositoryConfig(Config):
# synapse to issue arbitrary GET requests to your internal services,
# causing serious security issues.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
# This must be specified if url_preview_enabled is set. It is recommended that
# you uncomment the following list as a starting point.
#
#url_preview_ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@ -284,7 +295,7 @@ class ContentRepositoryConfig(Config):
# - '::1/128'
# - 'fe80::/64'
# - 'fc00::/7'
#
# List of IP address CIDR ranges that the URL preview spider is allowed
# to access even if they are specified in url_preview_ip_range_blacklist.
# This is useful for specifying exceptions to wide-ranging blacklisted

View File

@ -72,6 +72,19 @@ class ServerConfig(Config):
# master, potentially causing inconsistency.
self.enable_media_repo = config.get("enable_media_repo", True)
# Whether to require authentication to retrieve profile data (avatars,
# display names) of other users through the client API.
self.require_auth_for_profile_requests = config.get(
"require_auth_for_profile_requests", False,
)
# If set to 'True', requires authentication to access the server's
# public rooms directory through the client API, and forbids any other
# homeserver to fetch it via federation.
self.restrict_public_rooms_to_local_users = config.get(
"restrict_public_rooms_to_local_users", False,
)
# whether to enable search. If disabled, new entries will not be inserted
# into the search tables and they will not be indexed. Users will receive
# errors when attempting to search for messages.
@ -114,11 +127,13 @@ class ServerConfig(Config):
# FIXME: federation_domain_whitelist needs sytests
self.federation_domain_whitelist = None
federation_domain_whitelist = config.get(
"federation_domain_whitelist", None
"federation_domain_whitelist", None,
)
# turn the whitelist into a hash for speed of lookup
if federation_domain_whitelist is not None:
# turn the whitelist into a hash for speed of lookup
self.federation_domain_whitelist = {}
for domain in federation_domain_whitelist:
self.federation_domain_whitelist[domain] = True
@ -132,6 +147,12 @@ class ServerConfig(Config):
# sending out any replication updates.
self.replication_torture_level = config.get("replication_torture_level")
# Whether to require a user to be in the room to add an alias to it.
# Defaults to True.
self.require_membership_for_aliases = config.get(
"require_membership_for_aliases", True,
)
self.listeners = []
for listener in config.get("listeners", []):
if not isinstance(listener.get("port", None), int):
@ -319,6 +340,20 @@ class ServerConfig(Config):
#
#use_presence: false
# Whether to require authentication to retrieve profile data (avatars,
# display names) of other users through the client API. Defaults to
# 'false'. Note that profile data is also available via the federation
# API, so this setting is of limited value if federation is enabled on
# the server.
#
#require_auth_for_profile_requests: true
# If set to 'true', requires authentication to access the server's
# public rooms directory through the client API, and forbids any other
# homeserver to fetch it via federation. Defaults to 'false'.
#
#restrict_public_rooms_to_local_users: true
# The GC threshold parameters to pass to `gc.set_threshold`, if defined
#
#gc_thresholds: [700, 10, 10]
@ -386,8 +421,8 @@ class ServerConfig(Config):
#
# Valid resource names are:
#
# client: the client-server API (/_matrix/client). Also implies 'media' and
# 'static'.
# client: the client-server API (/_matrix/client), and the synapse admin
# API (/_synapse/admin). Also implies 'media' and 'static'.
#
# consent: user consent forms (/_matrix/consent). See
# docs/consent_tracking.md.
@ -488,6 +523,11 @@ class ServerConfig(Config):
# Used by phonehome stats to group together related servers.
#server_context: context
# Whether to require a user to be in the room to add an alias to it.
# Defaults to 'true'.
#
#require_membership_for_aliases: false
""" % locals()
def read_arguments(self, args):

View File

@ -24,8 +24,10 @@ import six
from unpaddedbase64 import encode_base64
from OpenSSL import crypto
from twisted.internet._sslverify import Certificate, trustRootFromCertificates
from synapse.config._base import Config, ConfigError
from synapse.util import glob_to_regex
logger = logging.getLogger(__name__)
@ -70,6 +72,53 @@ class TlsConfig(Config):
self.tls_fingerprints = list(self._original_tls_fingerprints)
# Whether to verify certificates on outbound federation traffic
self.federation_verify_certificates = config.get(
"federation_verify_certificates", False,
)
# Whitelist of domains to not verify certificates for
fed_whitelist_entries = config.get(
"federation_certificate_verification_whitelist", [],
)
# Support globs (*) in whitelist values
self.federation_certificate_verification_whitelist = []
for entry in fed_whitelist_entries:
# Convert globs to regex
entry_regex = glob_to_regex(entry)
self.federation_certificate_verification_whitelist.append(entry_regex)
# List of custom certificate authorities for federation traffic validation
custom_ca_list = config.get(
"federation_custom_ca_list", None,
)
# Read in and parse custom CA certificates
self.federation_ca_trust_root = None
if custom_ca_list is not None:
if len(custom_ca_list) == 0:
# A trustroot cannot be generated without any CA certificates.
# Raise an error if this option has been specified without any
# corresponding certificates.
raise ConfigError("federation_custom_ca_list specified without "
"any certificate files")
certs = []
for ca_file in custom_ca_list:
logger.debug("Reading custom CA certificate file: %s", ca_file)
content = self.read_file(ca_file)
# Parse the CA certificates
try:
cert_base = Certificate.loadPEM(content)
certs.append(cert_base)
except Exception as e:
raise ConfigError("Error parsing custom CA certificate file %s: %s"
% (ca_file, e))
self.federation_ca_trust_root = trustRootFromCertificates(certs)
# This config option applies to non-federation HTTP clients
# (e.g. for talking to recaptcha, identity servers, and such)
# It should never be used in production, and is intended for
@ -99,15 +148,15 @@ class TlsConfig(Config):
try:
with open(self.tls_certificate_file, 'rb') as f:
cert_pem = f.read()
except Exception:
logger.exception("Failed to read existing certificate off disk!")
raise
except Exception as e:
raise ConfigError("Failed to read existing certificate file %s: %s"
% (self.tls_certificate_file, e))
try:
tls_certificate = crypto.load_certificate(crypto.FILETYPE_PEM, cert_pem)
except Exception:
logger.exception("Failed to parse existing certificate off disk!")
raise
except Exception as e:
raise ConfigError("Failed to parse existing certificate file %s: %s"
% (self.tls_certificate_file, e))
if not allow_self_signed:
if tls_certificate.get_subject() == tls_certificate.get_issuer():
@ -192,6 +241,40 @@ class TlsConfig(Config):
#
#tls_private_key_path: "%(tls_private_key_path)s"
# Whether to verify TLS certificates when sending federation traffic.
#
# This currently defaults to `false`, however this will change in
# Synapse 1.0 when valid federation certificates will be required.
#
#federation_verify_certificates: true
# Skip federation certificate verification on the following whitelist
# of domains.
#
# This setting should only be used in very specific cases, such as
# federation over Tor hidden services and similar. For private networks
# of homeservers, you likely want to use a private CA instead.
#
# Only effective if federation_verify_certicates is `true`.
#
#federation_certificate_verification_whitelist:
# - lon.example.com
# - *.domain.com
# - *.onion
# List of custom certificate authorities for federation traffic.
#
# This setting should only normally be used within a private network of
# homeservers.
#
# Note that this list will replace those that are provided by your
# operating environment. Certificates must be in PEM format.
#
#federation_custom_ca_list:
# - myCA1.pem
# - myCA2.pem
# - myCA3.pem
# ACME support: This will configure Synapse to request a valid TLS certificate
# for your configured `server_name` via Let's Encrypt.
#

View File

@ -18,10 +18,10 @@ import logging
from zope.interface import implementer
from OpenSSL import SSL, crypto
from twisted.internet._sslverify import _defaultCurveName
from twisted.internet._sslverify import ClientTLSOptions, _defaultCurveName
from twisted.internet.abstract import isIPAddress, isIPv6Address
from twisted.internet.interfaces import IOpenSSLClientConnectionCreator
from twisted.internet.ssl import CertificateOptions, ContextFactory
from twisted.internet.ssl import CertificateOptions, ContextFactory, platformTrust
from twisted.python.failure import Failure
logger = logging.getLogger(__name__)
@ -90,7 +90,7 @@ def _tolerateErrors(wrapped):
@implementer(IOpenSSLClientConnectionCreator)
class ClientTLSOptions(object):
class ClientTLSOptionsNoVerify(object):
"""
Client creator for TLS without certificate identity verification. This is a
copy of twisted.internet._sslverify.ClientTLSOptions with the identity
@ -127,9 +127,30 @@ class ClientTLSOptionsFactory(object):
to remote servers for federation."""
def __init__(self, config):
# We don't use config options yet
self._options = CertificateOptions(verify=False)
self._config = config
self._options_noverify = CertificateOptions()
# Check if we're using a custom list of a CA certificates
trust_root = config.federation_ca_trust_root
if trust_root is None:
# Use CA root certs provided by OpenSSL
trust_root = platformTrust()
self._options_verify = CertificateOptions(trustRoot=trust_root)
def get_options(self, host):
# Use _makeContext so that we get a fresh OpenSSL CTX each time.
return ClientTLSOptions(host, self._options._makeContext())
# Check if certificate verification has been enabled
should_verify = self._config.federation_verify_certificates
# Check if we've disabled certificate verification for this host
if should_verify:
for regex in self._config.federation_certificate_verification_whitelist:
if regex.match(host):
should_verify = False
break
if should_verify:
return ClientTLSOptions(host, self._options_verify._makeContext())
return ClientTLSOptionsNoVerify(host, self._options_noverify._makeContext())

View File

@ -114,40 +114,54 @@ class Keyring(object):
server_name. The deferreds run their callbacks in the sentinel
logcontext.
"""
# a list of VerifyKeyRequests
verify_requests = []
handle = preserve_fn(_handle_key_deferred)
for server_name, json_object in server_and_json:
def process(server_name, json_object):
"""Process an entry in the request list
Given a (server_name, json_object) pair from the request list,
adds a key request to verify_requests, and returns a deferred which will
complete or fail (in the sentinel context) when verification completes.
"""
key_ids = signature_ids(json_object, server_name)
if not key_ids:
logger.warn("Request from %s: no supported signature keys",
server_name)
deferred = defer.fail(SynapseError(
400,
"Not signed with a supported algorithm",
Codes.UNAUTHORIZED,
))
else:
deferred = defer.Deferred()
return defer.fail(
SynapseError(
400,
"Not signed by %s" % (server_name,),
Codes.UNAUTHORIZED,
)
)
logger.debug("Verifying for %s with key_ids %s",
server_name, key_ids)
# add the key request to the queue, but don't start it off yet.
verify_request = VerifyKeyRequest(
server_name, key_ids, json_object, deferred
server_name, key_ids, json_object, defer.Deferred(),
)
verify_requests.append(verify_request)
run_in_background(self._start_key_lookups, verify_requests)
# now run _handle_key_deferred, which will wait for the key request
# to complete and then do the verification.
#
# We want _handle_key_request to log to the right context, so we
# wrap it with preserve_fn (aka run_in_background)
return handle(verify_request)
# Pass those keys to handle_key_deferred so that the json object
# signatures can be verified
handle = preserve_fn(_handle_key_deferred)
return [
handle(rq) for rq in verify_requests
results = [
process(server_name, json_object)
for server_name, json_object in server_and_json
]
if verify_requests:
run_in_background(self._start_key_lookups, verify_requests)
return results
@defer.inlineCallbacks
def _start_key_lookups(self, verify_requests):
"""Sets off the key fetches for each verify request

View File

@ -187,7 +187,9 @@ class EventContext(object):
Returns:
Deferred[dict[(str, str), str]|None]: Returns None if state_group
is None, which happens when the associated event is an outlier.
is None, which happens when the associated event is an outlier.
Maps a (type, state_key) to the event ID of the state event matching
this tuple.
"""
if not self._fetching_state_deferred:
@ -205,7 +207,9 @@ class EventContext(object):
Returns:
Deferred[dict[(str, str), str]|None]: Returns None if state_group
is None, which happens when the associated event is an outlier.
is None, which happens when the associated event is an outlier.
Maps a (type, state_key) to the event ID of the state event matching
this tuple.
"""
if not self._fetching_state_deferred:

View File

@ -15,8 +15,8 @@
from six import string_types
from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import SynapseError
from synapse.api.constants import MAX_ALIAS_LENGTH, EventTypes, Membership
from synapse.api.errors import Codes, SynapseError
from synapse.api.room_versions import EventFormatVersions
from synapse.types import EventID, RoomID, UserID
@ -56,6 +56,17 @@ class EventValidator(object):
if not isinstance(getattr(event, s), string_types):
raise SynapseError(400, "'%s' not a string type" % (s,))
if event.type == EventTypes.Aliases:
if "aliases" in event.content:
for alias in event.content["aliases"]:
if len(alias) > MAX_ALIAS_LENGTH:
raise SynapseError(
400,
("Can't create aliases longer than"
" %d characters" % (MAX_ALIAS_LENGTH,)),
Codes.INVALID_PARAM,
)
def validate_builder(self, event):
"""Validates that the builder/event has roughly the right format. Only
checks values that we expect a proto event to have, rather than all the

View File

@ -269,7 +269,18 @@ def _check_sigs_on_pdus(keyring, room_version, pdus):
for p in pdus_to_check_sender
])
def sender_err(e, pdu_to_check):
errmsg = "event id %s: unable to verify signature for sender %s: %s" % (
pdu_to_check.pdu.event_id,
pdu_to_check.sender_domain,
e.getErrorMessage(),
)
# XX not really sure if these are the right codes, but they are what
# we've done for ages
raise SynapseError(400, errmsg, Codes.UNAUTHORIZED)
for p, d in zip(pdus_to_check_sender, more_deferreds):
d.addErrback(sender_err, p)
p.deferreds.append(d)
# now let's look for events where the sender's domain is different to the
@ -291,7 +302,18 @@ def _check_sigs_on_pdus(keyring, room_version, pdus):
for p in pdus_to_check_event_id
])
def event_err(e, pdu_to_check):
errmsg = (
"event id %s: unable to verify signature for event id domain: %s" % (
pdu_to_check.pdu.event_id,
e.getErrorMessage(),
)
)
# XX as above: not really sure if these are the right codes
raise SynapseError(400, errmsg, Codes.UNAUTHORIZED)
for p, d in zip(pdus_to_check_event_id, more_deferreds):
d.addErrback(event_err, p)
p.deferreds.append(d)
# replace lists of deferreds with single Deferreds

View File

@ -33,12 +33,14 @@ from synapse.metrics.background_process_metrics import run_as_background_process
from synapse.storage import UserPresenceState
from synapse.util.retryutils import NotRetryingDestination, get_retry_limiter
# This is defined in the Matrix spec and enforced by the receiver.
MAX_EDUS_PER_TRANSACTION = 100
logger = logging.getLogger(__name__)
sent_edus_counter = Counter(
"synapse_federation_client_sent_edus",
"Total number of EDUs successfully sent",
"synapse_federation_client_sent_edus", "Total number of EDUs successfully sent"
)
sent_edus_by_type = Counter(
@ -58,6 +60,7 @@ class PerDestinationQueue(object):
destination (str): the server_name of the destination that we are managing
transmission for.
"""
def __init__(self, hs, transaction_manager, destination):
self._server_name = hs.hostname
self._clock = hs.get_clock()
@ -68,17 +71,17 @@ class PerDestinationQueue(object):
self.transmission_loop_running = False
# a list of tuples of (pending pdu, order)
self._pending_pdus = [] # type: list[tuple[EventBase, int]]
self._pending_edus = [] # type: list[Edu]
self._pending_pdus = [] # type: list[tuple[EventBase, int]]
self._pending_edus = [] # type: list[Edu]
# Pending EDUs by their "key". Keyed EDUs are EDUs that get clobbered
# based on their key (e.g. typing events by room_id)
# Map of (edu_type, key) -> Edu
self._pending_edus_keyed = {} # type: dict[tuple[str, str], Edu]
self._pending_edus_keyed = {} # type: dict[tuple[str, str], Edu]
# Map of user_id -> UserPresenceState of pending presence to be sent to this
# destination
self._pending_presence = {} # type: dict[str, UserPresenceState]
self._pending_presence = {} # type: dict[str, UserPresenceState]
# room_id -> receipt_type -> user_id -> receipt_dict
self._pending_rrs = {}
@ -120,9 +123,7 @@ class PerDestinationQueue(object):
Args:
states (iterable[UserPresenceState]): presence to send
"""
self._pending_presence.update({
state.user_id: state for state in states
})
self._pending_presence.update({state.user_id: state for state in states})
self.attempt_new_transaction()
def queue_read_receipt(self, receipt):
@ -132,14 +133,9 @@ class PerDestinationQueue(object):
Args:
receipt (synapse.api.receipt_info.ReceiptInfo): receipt to be queued
"""
self._pending_rrs.setdefault(
receipt.room_id, {},
).setdefault(
self._pending_rrs.setdefault(receipt.room_id, {}).setdefault(
receipt.receipt_type, {}
)[receipt.user_id] = {
"event_ids": receipt.event_ids,
"data": receipt.data,
}
)[receipt.user_id] = {"event_ids": receipt.event_ids, "data": receipt.data}
def flush_read_receipts_for_room(self, room_id):
# if we don't have any read-receipts for this room, it may be that we've already
@ -170,10 +166,7 @@ class PerDestinationQueue(object):
# request at which point pending_pdus just keeps growing.
# we need application-layer timeouts of some flavour of these
# requests
logger.debug(
"TX [%s] Transaction already in progress",
self._destination
)
logger.debug("TX [%s] Transaction already in progress", self._destination)
return
logger.debug("TX [%s] Starting transaction loop", self._destination)
@ -197,7 +190,8 @@ class PerDestinationQueue(object):
pending_pdus = []
while True:
device_message_edus, device_stream_id, dev_list_id = (
yield self._get_new_device_messages()
# We have to keep 2 free slots for presence and rr_edus
yield self._get_new_device_messages(MAX_EDUS_PER_TRANSACTION - 2)
)
# BEGIN CRITICAL SECTION
@ -216,19 +210,9 @@ class PerDestinationQueue(object):
pending_edus = []
pending_edus.extend(self._get_rr_edus(force_flush=False))
# We can only include at most 100 EDUs per transactions
pending_edus.extend(self._pop_pending_edus(100 - len(pending_edus)))
pending_edus.extend(
self._pending_edus_keyed.values()
)
self._pending_edus_keyed = {}
pending_edus.extend(device_message_edus)
# rr_edus and pending_presence take at most one slot each
pending_edus.extend(self._get_rr_edus(force_flush=False))
pending_presence = self._pending_presence
self._pending_presence = {}
if pending_presence:
@ -248,9 +232,23 @@ class PerDestinationQueue(object):
)
)
pending_edus.extend(device_message_edus)
pending_edus.extend(
self._pop_pending_edus(MAX_EDUS_PER_TRANSACTION - len(pending_edus))
)
while (
len(pending_edus) < MAX_EDUS_PER_TRANSACTION
and self._pending_edus_keyed
):
_, val = self._pending_edus_keyed.popitem()
pending_edus.append(val)
if pending_pdus:
logger.debug("TX [%s] len(pending_pdus_by_dest[dest]) = %d",
self._destination, len(pending_pdus))
logger.debug(
"TX [%s] len(pending_pdus_by_dest[dest]) = %d",
self._destination,
len(pending_pdus),
)
if not pending_pdus and not pending_edus:
logger.debug("TX [%s] Nothing to send", self._destination)
@ -259,7 +257,7 @@ class PerDestinationQueue(object):
# if we've decided to send a transaction anyway, and we have room, we
# may as well send any pending RRs
if len(pending_edus) < 100:
if len(pending_edus) < MAX_EDUS_PER_TRANSACTION:
pending_edus.extend(self._get_rr_edus(force_flush=True))
# END CRITICAL SECTION
@ -303,22 +301,25 @@ class PerDestinationQueue(object):
except HttpResponseException as e:
logger.warning(
"TX [%s] Received %d response to transaction: %s",
self._destination, e.code, e,
self._destination,
e.code,
e,
)
except RequestSendFailed as e:
logger.warning("TX [%s] Failed to send transaction: %s", self._destination, e)
logger.warning(
"TX [%s] Failed to send transaction: %s", self._destination, e
)
for p, _ in pending_pdus:
logger.info("Failed to send event %s to %s", p.event_id,
self._destination)
logger.info(
"Failed to send event %s to %s", p.event_id, self._destination
)
except Exception:
logger.exception(
"TX [%s] Failed to send transaction",
self._destination,
)
logger.exception("TX [%s] Failed to send transaction", self._destination)
for p, _ in pending_pdus:
logger.info("Failed to send event %s to %s", p.event_id,
self._destination)
logger.info(
"Failed to send event %s to %s", p.event_id, self._destination
)
finally:
# We want to be *very* sure we clear this after we stop processing
self.transmission_loop_running = False
@ -346,27 +347,13 @@ class PerDestinationQueue(object):
return pending_edus
@defer.inlineCallbacks
def _get_new_device_messages(self):
last_device_stream_id = self._last_device_stream_id
to_device_stream_id = self._store.get_to_device_stream_token()
contents, stream_id = yield self._store.get_new_device_msgs_for_remote(
self._destination, last_device_stream_id, to_device_stream_id
)
edus = [
Edu(
origin=self._server_name,
destination=self._destination,
edu_type="m.direct_to_device",
content=content,
)
for content in contents
]
def _get_new_device_messages(self, limit):
last_device_list = self._last_device_list_stream_id
# Will return at most 20 entries
now_stream_id, results = yield self._store.get_devices_by_remote(
self._destination, last_device_list
)
edus.extend(
edus = [
Edu(
origin=self._server_name,
destination=self._destination,
@ -374,5 +361,26 @@ class PerDestinationQueue(object):
content=content,
)
for content in results
]
assert len(edus) <= limit, "get_devices_by_remote returned too many EDUs"
last_device_stream_id = self._last_device_stream_id
to_device_stream_id = self._store.get_to_device_stream_token()
contents, stream_id = yield self._store.get_new_device_msgs_for_remote(
self._destination,
last_device_stream_id,
to_device_stream_id,
limit - len(edus),
)
edus.extend(
Edu(
origin=self._server_name,
destination=self._destination,
edu_type="m.direct_to_device",
content=content,
)
for content in contents
)
defer.returnValue((edus, stream_id, now_stream_id))

View File

@ -716,8 +716,17 @@ class PublicRoomList(BaseFederationServlet):
PATH = "/publicRooms"
def __init__(self, handler, authenticator, ratelimiter, server_name, deny_access):
super(PublicRoomList, self).__init__(
handler, authenticator, ratelimiter, server_name,
)
self.deny_access = deny_access
@defer.inlineCallbacks
def on_GET(self, origin, content, query):
if self.deny_access:
raise FederationDeniedError(origin)
limit = parse_integer_from_args(query, "limit", 0)
since_token = parse_string_from_args(query, "since", None)
include_all_networks = parse_boolean_from_args(
@ -1417,6 +1426,7 @@ def register_servlets(hs, resource, authenticator, ratelimiter, servlet_groups=N
authenticator=authenticator,
ratelimiter=ratelimiter,
server_name=hs.hostname,
deny_access=hs.config.restrict_public_rooms_to_local_users,
).register(resource)
if "group_server" in servlet_groups:

View File

@ -0,0 +1,253 @@
# -*- coding: utf-8 -*-
# Copyright 2019 New Vector Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import email.mime.multipart
import email.utils
import logging
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from twisted.internet import defer
from synapse.api.errors import StoreError
from synapse.types import UserID
from synapse.util import stringutils
from synapse.util.logcontext import make_deferred_yieldable
try:
from synapse.push.mailer import load_jinja2_templates
except ImportError:
load_jinja2_templates = None
logger = logging.getLogger(__name__)
class AccountValidityHandler(object):
def __init__(self, hs):
self.hs = hs
self.store = self.hs.get_datastore()
self.sendmail = self.hs.get_sendmail()
self.clock = self.hs.get_clock()
self._account_validity = self.hs.config.account_validity
if self._account_validity.renew_by_email_enabled and load_jinja2_templates:
# Don't do email-specific configuration if renewal by email is disabled.
try:
app_name = self.hs.config.email_app_name
self._subject = self._account_validity.renew_email_subject % {
"app": app_name,
}
self._from_string = self.hs.config.email_notif_from % {
"app": app_name,
}
except Exception:
# If substitution failed, fall back to the bare strings.
self._subject = self._account_validity.renew_email_subject
self._from_string = self.hs.config.email_notif_from
self._raw_from = email.utils.parseaddr(self._from_string)[1]
self._template_html, self._template_text = load_jinja2_templates(
config=self.hs.config,
template_html_name=self.hs.config.email_expiry_template_html,
template_text_name=self.hs.config.email_expiry_template_text,
)
# Check the renewal emails to send and send them every 30min.
self.clock.looping_call(
self.send_renewal_emails,
30 * 60 * 1000,
)
@defer.inlineCallbacks
def send_renewal_emails(self):
"""Gets the list of users whose account is expiring in the amount of time
configured in the ``renew_at`` parameter from the ``account_validity``
configuration, and sends renewal emails to all of these users as long as they
have an email 3PID attached to their account.
"""
expiring_users = yield self.store.get_users_expiring_soon()
if expiring_users:
for user in expiring_users:
yield self._send_renewal_email(
user_id=user["user_id"],
expiration_ts=user["expiration_ts_ms"],
)
@defer.inlineCallbacks
def send_renewal_email_to_user(self, user_id):
expiration_ts = yield self.store.get_expiration_ts_for_user(user_id)
yield self._send_renewal_email(user_id, expiration_ts)
@defer.inlineCallbacks
def _send_renewal_email(self, user_id, expiration_ts):
"""Sends out a renewal email to every email address attached to the given user
with a unique link allowing them to renew their account.
Args:
user_id (str): ID of the user to send email(s) to.
expiration_ts (int): Timestamp in milliseconds for the expiration date of
this user's account (used in the email templates).
"""
addresses = yield self._get_email_addresses_for_user(user_id)
# Stop right here if the user doesn't have at least one email address.
# In this case, they will have to ask their server admin to renew their
# account manually.
if not addresses:
return
try:
user_display_name = yield self.store.get_profile_displayname(
UserID.from_string(user_id).localpart
)
if user_display_name is None:
user_display_name = user_id
except StoreError:
user_display_name = user_id
renewal_token = yield self._get_renewal_token(user_id)
url = "%s_matrix/client/unstable/account_validity/renew?token=%s" % (
self.hs.config.public_baseurl,
renewal_token,
)
template_vars = {
"display_name": user_display_name,
"expiration_ts": expiration_ts,
"url": url,
}
html_text = self._template_html.render(**template_vars)
html_part = MIMEText(html_text, "html", "utf8")
plain_text = self._template_text.render(**template_vars)
text_part = MIMEText(plain_text, "plain", "utf8")
for address in addresses:
raw_to = email.utils.parseaddr(address)[1]
multipart_msg = MIMEMultipart('alternative')
multipart_msg['Subject'] = self._subject
multipart_msg['From'] = self._from_string
multipart_msg['To'] = address
multipart_msg['Date'] = email.utils.formatdate()
multipart_msg['Message-ID'] = email.utils.make_msgid()
multipart_msg.attach(text_part)
multipart_msg.attach(html_part)
logger.info("Sending renewal email to %s", address)
yield make_deferred_yieldable(self.sendmail(
self.hs.config.email_smtp_host,
self._raw_from, raw_to, multipart_msg.as_string().encode('utf8'),
reactor=self.hs.get_reactor(),
port=self.hs.config.email_smtp_port,
requireAuthentication=self.hs.config.email_smtp_user is not None,
username=self.hs.config.email_smtp_user,
password=self.hs.config.email_smtp_pass,
requireTransportSecurity=self.hs.config.require_transport_security
))
yield self.store.set_renewal_mail_status(
user_id=user_id,
email_sent=True,
)
@defer.inlineCallbacks
def _get_email_addresses_for_user(self, user_id):
"""Retrieve the list of email addresses attached to a user's account.
Args:
user_id (str): ID of the user to lookup email addresses for.
Returns:
defer.Deferred[list[str]]: Email addresses for this account.
"""
threepids = yield self.store.user_get_threepids(user_id)
addresses = []
for threepid in threepids:
if threepid["medium"] == "email":
addresses.append(threepid["address"])
defer.returnValue(addresses)
@defer.inlineCallbacks
def _get_renewal_token(self, user_id):
"""Generates a 32-byte long random string that will be inserted into the
user's renewal email's unique link, then saves it into the database.
Args:
user_id (str): ID of the user to generate a string for.
Returns:
defer.Deferred[str]: The generated string.
Raises:
StoreError(500): Couldn't generate a unique string after 5 attempts.
"""
attempts = 0
while attempts < 5:
try:
renewal_token = stringutils.random_string(32)
yield self.store.set_renewal_token_for_user(user_id, renewal_token)
defer.returnValue(renewal_token)
except StoreError:
attempts += 1
raise StoreError(500, "Couldn't generate a unique string as refresh string.")
@defer.inlineCallbacks
def renew_account(self, renewal_token):
"""Renews the account attached to a given renewal token by pushing back the
expiration date by the current validity period in the server's configuration.
Args:
renewal_token (str): Token sent with the renewal request.
"""
user_id = yield self.store.get_user_from_renewal_token(renewal_token)
logger.debug("Renewing an account for user %s", user_id)
yield self.renew_account_for_user(user_id)
@defer.inlineCallbacks
def renew_account_for_user(self, user_id, expiration_ts=None, email_sent=False):
"""Renews the account attached to a given user by pushing back the
expiration date by the current validity period in the server's
configuration.
Args:
renewal_token (str): Token sent with the renewal request.
expiration_ts (int): New expiration date. Defaults to now + validity period.
email_sent (bool): Whether an email has been sent for this validity period.
Defaults to False.
Returns:
defer.Deferred[int]: New expiration date for this account, as a timestamp
in milliseconds since epoch.
"""
if expiration_ts is None:
expiration_ts = self.clock.time_msec() + self._account_validity.period
yield self.store.set_account_validity_for_user(
user_id=user_id,
expiration_ts=expiration_ts,
email_sent=email_sent,
)
defer.returnValue(expiration_ts)

View File

@ -19,7 +19,7 @@ import string
from twisted.internet import defer
from synapse.api.constants import EventTypes
from synapse.api.constants import MAX_ALIAS_LENGTH, EventTypes
from synapse.api.errors import (
AuthError,
CodeMessageException,
@ -43,8 +43,10 @@ class DirectoryHandler(BaseHandler):
self.state = hs.get_state_handler()
self.appservice_handler = hs.get_application_service_handler()
self.event_creation_handler = hs.get_event_creation_handler()
self.store = hs.get_datastore()
self.config = hs.config
self.enable_room_list_search = hs.config.enable_room_list_search
self.require_membership = hs.config.require_membership_for_aliases
self.federation = hs.get_federation_client()
hs.get_federation_registry().register_query_handler(
@ -83,7 +85,7 @@ class DirectoryHandler(BaseHandler):
@defer.inlineCallbacks
def create_association(self, requester, room_alias, room_id, servers=None,
send_event=True):
send_event=True, check_membership=True):
"""Attempt to create a new alias
Args:
@ -93,6 +95,8 @@ class DirectoryHandler(BaseHandler):
servers (list[str]|None): List of servers that others servers
should try and join via
send_event (bool): Whether to send an updated m.room.aliases event
check_membership (bool): Whether to check if the user is in the room
before the alias can be set (if the server's config requires it).
Returns:
Deferred
@ -100,6 +104,13 @@ class DirectoryHandler(BaseHandler):
user_id = requester.user.to_string()
if len(room_alias.to_string()) > MAX_ALIAS_LENGTH:
raise SynapseError(
400,
"Can't create aliases longer than %s characters" % MAX_ALIAS_LENGTH,
Codes.INVALID_PARAM,
)
service = requester.app_service
if service:
if not service.is_interested_in_alias(room_alias.to_string()):
@ -108,6 +119,14 @@ class DirectoryHandler(BaseHandler):
" this kind of alias.", errcode=Codes.EXCLUSIVE
)
else:
if self.require_membership and check_membership:
rooms_for_user = yield self.store.get_rooms_for_user(user_id)
if room_id not in rooms_for_user:
raise AuthError(
403,
"You must be in the room to create an alias for it",
)
if not self.spam_checker.user_may_create_room_alias(user_id, room_alias):
raise AuthError(
403, "This user is not permitted to create this alias",

View File

@ -228,6 +228,7 @@ class EventCreationHandler(object):
self.ratelimiter = hs.get_ratelimiter()
self.notifier = hs.get_notifier()
self.config = hs.config
self.require_membership_for_aliases = hs.config.require_membership_for_aliases
self.send_event_to_master = ReplicationSendEventRestServlet.make_client(hs)
@ -336,6 +337,35 @@ class EventCreationHandler(object):
prev_events_and_hashes=prev_events_and_hashes,
)
# In an ideal world we wouldn't need the second part of this condition. However,
# this behaviour isn't spec'd yet, meaning we should be able to deactivate this
# behaviour. Another reason is that this code is also evaluated each time a new
# m.room.aliases event is created, which includes hitting a /directory route.
# Therefore not including this condition here would render the similar one in
# synapse.handlers.directory pointless.
if builder.type == EventTypes.Aliases and self.require_membership_for_aliases:
# Ideally we'd do the membership check in event_auth.check(), which
# describes a spec'd algorithm for authenticating events received over
# federation as well as those created locally. As of room v3, aliases events
# can be created by users that are not in the room, therefore we have to
# tolerate them in event_auth.check().
prev_state_ids = yield context.get_prev_state_ids(self.store)
prev_event_id = prev_state_ids.get((EventTypes.Member, event.sender))
prev_event = yield self.store.get_event(prev_event_id, allow_none=True)
if not prev_event or prev_event.membership != Membership.JOIN:
logger.warning(
("Attempt to send `m.room.aliases` in room %s by user %s but"
" membership is %s"),
event.room_id,
event.sender,
prev_event.membership if prev_event else None,
)
raise AuthError(
403,
"You must be in the room to create an alias for it",
)
self.validator.validate_new(event)
defer.returnValue((event, context))

View File

@ -828,6 +828,11 @@ class PresenceHandler(object):
if typ != EventTypes.Member:
continue
if event_id is None:
# state has been deleted, so this is not a join. We only care about
# joins.
continue
event = yield self.store.get_event(event_id)
if event.content.get("membership") != Membership.JOIN:
# We only care about joins

View File

@ -139,6 +139,7 @@ class BaseProfileHandler(BaseHandler):
@defer.inlineCallbacks
def get_profile(self, user_id):
target_user = UserID.from_string(user_id)
if self.hs.is_mine(target_user):
try:
displayname = yield self.store.get_profile_displayname(
@ -424,6 +425,48 @@ class BaseProfileHandler(BaseHandler):
room_id, str(e)
)
@defer.inlineCallbacks
def check_profile_query_allowed(self, target_user, requester=None):
"""Checks whether a profile query is allowed. If the
'require_auth_for_profile_requests' config flag is set to True and a
'requester' is provided, the query is only allowed if the two users
share a room.
Args:
target_user (UserID): The owner of the queried profile.
requester (None|UserID): The user querying for the profile.
Raises:
SynapseError(403): The two users share no room, or ne user couldn't
be found to be in any room the server is in, and therefore the query
is denied.
"""
# Implementation of MSC1301: don't allow looking up profiles if the
# requester isn't in the same room as the target. We expect requester to
# be None when this function is called outside of a profile query, e.g.
# when building a membership event. In this case, we must allow the
# lookup.
if not self.hs.config.require_auth_for_profile_requests or not requester:
return
try:
requester_rooms = yield self.store.get_rooms_for_user(
requester.to_string()
)
target_user_rooms = yield self.store.get_rooms_for_user(
target_user.to_string(),
)
# Check if the room lists have no elements in common.
if requester_rooms.isdisjoint(target_user_rooms):
raise SynapseError(403, "Profile isn't available", Codes.FORBIDDEN)
except StoreError as e:
if e.code == 404:
# This likely means that one of the users doesn't exist,
# so we act as if we couldn't find the profile.
raise SynapseError(403, "Profile isn't available", Codes.FORBIDDEN)
raise
class MasterProfileHandler(BaseProfileHandler):
PROFILE_UPDATE_MS = 60 * 1000

View File

@ -421,7 +421,7 @@ class RoomCreationHandler(BaseHandler):
yield directory_handler.create_association(
requester, RoomAlias.from_string(alias),
new_room_id, servers=(self.hs.hostname, ),
send_event=False,
send_event=False, check_membership=False,
)
logger.info("Moved alias %s to new room", alias)
except SynapseError as e:
@ -571,6 +571,7 @@ class RoomCreationHandler(BaseHandler):
room_alias=room_alias,
servers=[self.hs.hostname],
send_event=False,
check_membership=False,
)
preset_config = config.get(

View File

@ -29,6 +29,8 @@ from synapse.types import RoomID, UserID
from synapse.util.async_helpers import Linearizer
from synapse.util.distributor import user_joined_room, user_left_room
from ._base import BaseHandler
logger = logging.getLogger(__name__)
id_server_scheme = "https://"
@ -69,6 +71,11 @@ class RoomMemberHandler(object):
self._server_notices_mxid = self.config.server_notices_mxid
self.rewrite_identity_server_urls = self.config.rewrite_identity_server_urls
# This is only used to get at ratelimit function, and
# maybe_kick_guest_users. It's fine there are multiple of these as
# it doesn't store state.
self.base_handler = BaseHandler(hs)
@abc.abstractmethod
def _remote_join(self, requester, remote_room_hosts, room_id, user, content):
"""Try and join a room that this server is not in
@ -753,6 +760,10 @@ class RoomMemberHandler(object):
Codes.FORBIDDEN,
)
# We need to rate limit *before* we send out any 3PID invites, so we
# can't just rely on the standard ratelimiting of events.
yield self.base_handler.ratelimit(requester)
invitee = yield self._lookup_3pid(
id_server, medium, address
)

View File

@ -90,9 +90,32 @@ class IPBlacklistingResolver(object):
def resolveHostName(self, recv, hostname, portNumber=0):
r = recv()
d = defer.Deferred()
addresses = []
def _callback():
r.resolutionBegan(None)
has_bad_ip = False
for i in addresses:
ip_address = IPAddress(i.host)
if check_against_blacklist(
ip_address, self._ip_whitelist, self._ip_blacklist
):
logger.info(
"Dropped %s from DNS resolution to %s due to blacklist" %
(ip_address, hostname)
)
has_bad_ip = True
# if we have a blacklisted IP, we'd like to raise an error to block the
# request, but all we can really do from here is claim that there were no
# valid results.
if not has_bad_ip:
for i in addresses:
r.addressResolved(i)
r.resolutionComplete()
@provider(IResolutionReceiver)
class EndpointReceiver(object):
@staticmethod
@ -101,34 +124,16 @@ class IPBlacklistingResolver(object):
@staticmethod
def addressResolved(address):
ip_address = IPAddress(address.host)
if check_against_blacklist(
ip_address, self._ip_whitelist, self._ip_blacklist
):
logger.info(
"Dropped %s from DNS resolution to %s" % (ip_address, hostname)
)
raise SynapseError(403, "IP address blocked by IP blacklist entry")
addresses.append(address)
@staticmethod
def resolutionComplete():
d.callback(addresses)
_callback()
self._reactor.nameResolver.resolveHostName(
EndpointReceiver, hostname, portNumber=portNumber
)
def _callback(addrs):
r.resolutionBegan(None)
for i in addrs:
r.addressResolved(i)
r.resolutionComplete()
d.addCallback(_callback)
return r

View File

@ -149,7 +149,7 @@ class MatrixFederationAgent(object):
tls_options = None
else:
tls_options = self._tls_client_options_factory.get_options(
res.tls_server_name.decode("ascii")
res.tls_server_name.decode("ascii"),
)
# make sure that the Host header is set correctly

View File

@ -261,6 +261,23 @@ BASE_APPEND_OVERRIDE_RULES = [
'value': True,
}
]
},
{
'rule_id': 'global/override/.m.rule.tombstone',
'conditions': [
{
'kind': 'event_match',
'key': 'type',
'pattern': 'm.room.tombstone',
'_id': '_tombstone',
}
],
'actions': [
'notify', {
'set_tweak': 'highlight',
'value': True,
}
]
}
]

View File

@ -521,11 +521,11 @@ def format_ts_filter(value, format):
return time.strftime(format, time.localtime(value / 1000))
def load_jinja2_templates(config):
def load_jinja2_templates(config, template_html_name, template_text_name):
"""Load the jinja2 email templates from disk
Returns:
(notif_template_html, notif_template_text)
(template_html, template_text)
"""
logger.info("loading email templates from '%s'", config.email_template_dir)
loader = jinja2.FileSystemLoader(config.email_template_dir)
@ -533,14 +533,10 @@ def load_jinja2_templates(config):
env.filters["format_ts"] = format_ts_filter
env.filters["mxc_to_http"] = _create_mxc_to_http_filter(config)
notif_template_html = env.get_template(
config.email_notif_template_html
)
notif_template_text = env.get_template(
config.email_notif_template_text
)
template_html = env.get_template(template_html_name)
template_text = env.get_template(template_text_name)
return notif_template_html, notif_template_text
return template_html, template_text
def _create_mxc_to_http_filter(config):

View File

@ -44,7 +44,11 @@ class PusherFactory(object):
if hs.config.email_enable_notifs:
self.mailers = {} # app_name -> Mailer
templates = load_jinja2_templates(hs.config)
templates = load_jinja2_templates(
config=hs.config,
template_html_name=hs.config.email_notif_template_html,
template_text_name=hs.config.email_notif_template_text,
)
self.notif_template_html, self.notif_template_text = templates
self.pusher_types["email"] = self._create_email_pusher

View File

@ -69,6 +69,14 @@ REQUIREMENTS = [
"attrs>=17.4.0",
"netaddr>=0.7.18",
# requests is a transitive dep of treq, and urlib3 is a transitive dep
# of requests, as well as of sentry-sdk.
#
# As of requests 2.21, requests does not yet support urllib3 1.25.
# (If we do not pin it here, pip will give us the latest urllib3
# due to the dep via sentry-sdk.)
"urllib3<1.25",
]
CONDITIONAL_REQUIREMENTS = {
@ -86,18 +94,22 @@ CONDITIONAL_REQUIREMENTS = {
"acme": ["txacme>=0.9.2"],
"saml2": ["pysaml2>=4.5.0"],
"systemd": ["systemd-python>=231"],
"url_preview": ["lxml>=3.5.0"],
"test": ["mock>=2.0", "parameterized"],
"sentry": ["sentry-sdk>=0.7.2"],
}
ALL_OPTIONAL_REQUIREMENTS = set()
for name, optional_deps in CONDITIONAL_REQUIREMENTS.items():
# Exclude systemd as it's a system-based requirement.
if name not in ["systemd"]:
ALL_OPTIONAL_REQUIREMENTS = set(optional_deps) | ALL_OPTIONAL_REQUIREMENTS
def list_requirements():
deps = set(REQUIREMENTS)
for opt in CONDITIONAL_REQUIREMENTS.values():
deps = set(opt) | deps
return list(deps)
return list(set(REQUIREMENTS) | ALL_OPTIONAL_REQUIREMENTS)
class DependencyException(Exception):

View File

@ -0,0 +1,4 @@
.noticetext {
margin-top: 10px;
margin-bottom: 10px;
}

View File

@ -0,0 +1,43 @@
<!doctype html>
<html lang="en">
<head>
<style type="text/css">
{% include 'mail.css' without context %}
{% include "mail-%s.css" % app_name ignore missing without context %}
{% include 'mail-expiry.css' without context %}
</style>
</head>
<body>
<table id="page">
<tr>
<td> </td>
<td id="inner">
<table class="header">
<tr>
<td>
<div class="salutation">Hi {{ display_name }},</div>
</td>
<td class="logo">
{% if app_name == "Riot" %}
<img src="http://riot.im/img/external/riot-logo-email.png" width="83" height="83" alt="[Riot]"/>
{% elif app_name == "Vector" %}
<img src="http://matrix.org/img/vector-logo-email.png" width="64" height="83" alt="[Vector]"/>
{% else %}
<img src="http://matrix.org/img/matrix-120x51.png" width="120" height="51" alt="[matrix]"/>
{% endif %}
</td>
</tr>
<tr>
<td colspan="2">
<div class="noticetext">Your account will expire on {{ expiration_ts|format_ts("%d-%m-%Y") }}. This means that you will lose access to your account after this date.</div>
<div class="noticetext">To extend the validity of your account, please click on the link bellow (or copy and paste it into a new browser tab):</div>
<div class="noticetext"><a href="{{ url }}">{{ url }}</a></div>
</td>
</tr>
</table>
</td>
<td> </td>
</tr>
</table>
</body>
</html>

View File

@ -0,0 +1,7 @@
Hi {{ display_name }},
Your account will expire on {{ expiration_ts|format_ts("%d-%m-%Y") }}. This means that you will lose access to your account after this date.
To extend the validity of your account, please click on the link bellow (or copy and paste it to a new browser tab):
{{ url }}

Some files were not shown because too many files have changed in this diff Show More