Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration

This commit is contained in:
Erik Johnston 2019-04-17 19:44:40 +01:00
commit ca90336a69
216 changed files with 5057 additions and 10038 deletions

View File

@ -257,13 +257,13 @@ https://github.com/spantaleev/matrix-docker-ansible-deploy
#### Matrix.org packages #### Matrix.org packages
Matrix.org provides Debian/Ubuntu packages of the latest stable version of Matrix.org provides Debian/Ubuntu packages of the latest stable version of
Synapse via https://matrix.org/packages/debian/. To use them: Synapse via https://packages.matrix.org/debian/. To use them:
``` ```
sudo apt install -y lsb-release curl apt-transport-https sudo apt install -y lsb-release curl apt-transport-https
echo "deb https://matrix.org/packages/debian `lsb_release -cs` main" | echo "deb https://packages.matrix.org/debian `lsb_release -cs` main" |
sudo tee /etc/apt/sources.list.d/matrix-org.list sudo tee /etc/apt/sources.list.d/matrix-org.list
curl "https://matrix.org/packages/debian/repo-key.asc" | curl "https://packages.matrix.org/debian/repo-key.asc" |
sudo apt-key add - sudo apt-key add -
sudo apt update sudo apt update
sudo apt install matrix-synapse-py3 sudo apt install matrix-synapse-py3

1
changelog.d/4339.feature Normal file
View File

@ -0,0 +1 @@
Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with `pip install matrix-synapse[systemd]`.

1
changelog.d/4474.misc Normal file
View File

@ -0,0 +1 @@
Add test to verify threepid auth check added in #4435.

1
changelog.d/4555.bugfix Normal file
View File

@ -0,0 +1 @@
Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in [#4220](https://github.com/matrix-org/synapse/pull/4220). Contributed by Marcel Fabian Krüger ("[zaugin](https://github.com/zauguin)").

1
changelog.d/4942.bugfix Normal file
View File

@ -0,0 +1 @@
Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server.

1
changelog.d/4947.feature Normal file
View File

@ -0,0 +1 @@
Add ability for password provider modules to bind email addresses to users upon registration.

1
changelog.d/4949.misc Normal file
View File

@ -0,0 +1 @@
Fix/improve some docstrings in the replication code.

2
changelog.d/4953.misc Normal file
View File

@ -0,0 +1,2 @@
Split synapse.replication.tcp.streams into smaller files.

1
changelog.d/4954.misc Normal file
View File

@ -0,0 +1 @@
Refactor replication row generation/parsing.

1
changelog.d/4955.bugfix Normal file
View File

@ -0,0 +1 @@
Fix sync bug which made accepting invites unreliable in worker-mode synapses.

1
changelog.d/4956.bugfix Normal file
View File

@ -0,0 +1 @@
Fix sync bug which made accepting invites unreliable in worker-mode synapses.

1
changelog.d/4959.misc Normal file
View File

@ -0,0 +1 @@
Run `black` to clean up formatting on `synapse/storage/roommember.py` and `synapse/storage/events.py`.

1
changelog.d/4965.misc Normal file
View File

@ -0,0 +1 @@
Remove log line for password via the admin API.

1
changelog.d/4968.misc Normal file
View File

@ -0,0 +1 @@
Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor.

2
changelog.d/4969.misc Normal file
View File

@ -0,0 +1,2 @@
Refactor room version definitions.

1
changelog.d/4974.misc Normal file
View File

@ -0,0 +1 @@
Add `config.signing_key_path` that can be read by `synapse.config` utility.

1
changelog.d/4981.bugfix Normal file
View File

@ -0,0 +1 @@
start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too.

1
changelog.d/4982.misc Normal file
View File

@ -0,0 +1 @@
Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915.

1
changelog.d/4985.misc Normal file
View File

@ -0,0 +1 @@
Rewrite KeyringTestCase as a HomeserverTestCase.

1
changelog.d/4987.misc Normal file
View File

@ -0,0 +1 @@
README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section.

1
changelog.d/4989.feature Normal file
View File

@ -0,0 +1 @@
Remove presence list support as per MSC 1819.

1
changelog.d/4990.bugfix Normal file
View File

@ -0,0 +1 @@
Transfer related groups on room upgrade.

1
changelog.d/4991.feature Normal file
View File

@ -0,0 +1 @@
Reduce CPU usage starting pushers during start up.

1
changelog.d/4992.misc Normal file
View File

@ -0,0 +1 @@
Remove a number of unused tables from the database schema.

1
changelog.d/4996.misc Normal file
View File

@ -0,0 +1 @@
Run `black` on the remainder of `synapse/storage/`.

1
changelog.d/4998.misc Normal file
View File

@ -0,0 +1 @@
Fix grammar in get_current_users_in_room and give it a docstring.

1
changelog.d/4999.bugfix Normal file
View File

@ -0,0 +1 @@
Prevent the ability to kick users from a room they aren't in.

1
changelog.d/5001.misc Normal file
View File

@ -0,0 +1 @@
Clean up some code in the server-key Keyring.

1
changelog.d/5002.feature Normal file
View File

@ -0,0 +1 @@
Add a delete group admin API.

1
changelog.d/5003.bugfix Normal file
View File

@ -0,0 +1 @@
Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud <anders@jensenwaud.com>.

1
changelog.d/5007.misc Normal file
View File

@ -0,0 +1 @@
Refactor synapse.storage._base._simple_select_list_paginate.

1
changelog.d/5010.feature Normal file
View File

@ -0,0 +1 @@
Add config option to block users from looking up 3PIDs.

1
changelog.d/5020.feature Normal file
View File

@ -0,0 +1 @@
Add context to phonehome stats.

1
changelog.d/5024.misc Normal file
View File

@ -0,0 +1 @@
Store the notary server name correctly in server_keys_json.

1
changelog.d/5028.misc Normal file
View File

@ -0,0 +1 @@
Remove a number of unused tables from the database schema.

1
changelog.d/5030.misc Normal file
View File

@ -0,0 +1 @@
Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions.

1
changelog.d/5032.bugfix Normal file
View File

@ -0,0 +1 @@
Fix "cannot import name execute_batch" error with postgres.

1
changelog.d/5033.misc Normal file
View File

@ -0,0 +1 @@
Remove a number of unused tables from the database schema.

1
changelog.d/5035.bugfix Normal file
View File

@ -0,0 +1 @@
Fix disappearing exceptions in manhole.

1
changelog.d/5046.misc Normal file
View File

@ -0,0 +1 @@
Remove extraneous period from copyright headers.

1
changelog.d/5063.feature Normal file
View File

@ -0,0 +1 @@
Add support for handling /verions, /voip and /push_rules client endpoints to client_reader worker.

1
changelog.d/5065.feature Normal file
View File

@ -0,0 +1 @@
Add support for handling /verions, /voip and /push_rules client endpoints to client_reader worker.

1
changelog.d/5067.misc Normal file
View File

@ -0,0 +1 @@
Update documentation for where to get Synapse packages.

1
changelog.d/5070.feature Normal file
View File

@ -0,0 +1 @@
Add support for handling /verions, /voip and /push_rules client endpoints to client_reader worker.

1
changelog.d/5071.bugfix Normal file
View File

@ -0,0 +1 @@
Make sure we're not registering the same 3pid twice on registration.

File diff suppressed because one or more lines are too long

View File

@ -27,16 +27,26 @@ for port in 8080 8081 8082; do
--config-path "$DIR/etc/$port.config" \ --config-path "$DIR/etc/$port.config" \
--report-stats no --report-stats no
printf '\n\n# Customisation made by demo/start.sh\n' >> $DIR/etc/$port.config
echo 'enable_registration: true' >> $DIR/etc/$port.config
# Check script parameters # Check script parameters
if [ $# -eq 1 ]; then if [ $# -eq 1 ]; then
if [ $1 = "--no-rate-limit" ]; then if [ $1 = "--no-rate-limit" ]; then
# Set high limits in config file to disable rate limiting # messages rate limit
perl -p -i -e 's/rc_messages_per_second.*/rc_messages_per_second: 1000/g' $DIR/etc/$port.config echo 'rc_messages_per_second: 1000' >> $DIR/etc/$port.config
perl -p -i -e 's/rc_message_burst_count.*/rc_message_burst_count: 1000/g' $DIR/etc/$port.config echo 'rc_message_burst_count: 1000' >> $DIR/etc/$port.config
fi
fi
perl -p -i -e 's/^enable_registration:.*/enable_registration: true/g' $DIR/etc/$port.config # registration rate limit
printf 'rc_registration:\n per_second: 1000\n burst_count: 1000\n' >> $DIR/etc/$port.config
# login rate limit
echo 'rc_login:' >> $DIR/etc/$port.config
printf ' address:\n per_second: 1000\n burst_count: 1000\n' >> $DIR/etc/$port.config
printf ' account:\n per_second: 1000\n burst_count: 1000\n' >> $DIR/etc/$port.config
printf ' failed_attempts:\n per_second: 1000\n burst_count: 1000\n' >> $DIR/etc/$port.config
fi
fi
if ! grep -F "full_twisted_stacktraces" -q $DIR/etc/$port.config; then if ! grep -F "full_twisted_stacktraces" -q $DIR/etc/$port.config; then
echo "full_twisted_stacktraces: true" >> $DIR/etc/$port.config echo "full_twisted_stacktraces: true" >> $DIR/etc/$port.config

View File

@ -50,7 +50,9 @@ RUN apt-get update -qq -o Acquire::Languages=none \
debhelper \ debhelper \
devscripts \ devscripts \
dh-systemd \ dh-systemd \
libsystemd-dev \
lsb-release \ lsb-release \
pkg-config \
python3-dev \ python3-dev \
python3-pip \ python3-pip \
python3-setuptools \ python3-setuptools \

View File

@ -31,6 +31,7 @@ docker run \
--mount type=volume,src=synapse-data,dst=/data \ --mount type=volume,src=synapse-data,dst=/data \
-e SYNAPSE_SERVER_NAME=my.matrix.host \ -e SYNAPSE_SERVER_NAME=my.matrix.host \
-e SYNAPSE_REPORT_STATS=yes \ -e SYNAPSE_REPORT_STATS=yes \
-p 8448:8448 \
matrixdotorg/synapse:latest matrixdotorg/synapse:latest
``` ```
@ -57,9 +58,10 @@ configuration file there. Multiple application services are supported.
Synapse requires a valid TLS certificate. You can do one of the following: Synapse requires a valid TLS certificate. You can do one of the following:
* Provide your own certificate and key (as * Provide your own certificate and key (as
`${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt` and `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.tls.crt` and
`${DATA_PATH}/${SYNAPSE_SERVER_NAME}.key`, or elsewhere by providing an `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.tls.key`, or elsewhere by providing an
entire config as `${SYNAPSE_CONFIG_PATH}`). entire config as `${SYNAPSE_CONFIG_PATH}`). In this case, you should forward
traffic to port 8448 in the container, for example with `-p 443:8448`.
* Use a reverse proxy to terminate incoming TLS, and forward the plain http * Use a reverse proxy to terminate incoming TLS, and forward the plain http
traffic to port 8008 in the container. In this case you should set `-e traffic to port 8008 in the container. In this case you should set `-e
@ -137,7 +139,7 @@ Database specific values (will use SQLite if not set):
**NOTE**: You are highly encouraged to use postgresql! Please use the compose **NOTE**: You are highly encouraged to use postgresql! Please use the compose
file to make it easier to deploy. file to make it easier to deploy.
* `POSTGRES_USER` - The user for the synapse postgres database. [default: * `POSTGRES_USER` - The user for the synapse postgres database. [default:
`matrix`] `synapse`]
Mail server specific values (will not send emails if not set): Mail server specific values (will not send emails if not set):

View File

@ -0,0 +1,14 @@
# Delete a local group
This API lets a server admin delete a local group. Doing so will kick all
users out of the group so that their clients will correctly handle the group
being deleted.
The API is:
```
POST /_matrix/client/r0/admin/delete_group/<group_id>
```
including an `access_token` of a server admin.

View File

@ -236,6 +236,9 @@ listeners:
# - medium: 'email' # - medium: 'email'
# address: 'reserved_user@example.com' # address: 'reserved_user@example.com'
# Used by phonehome stats to group together related servers.
#server_context: context
## TLS ## ## TLS ##
@ -691,6 +694,10 @@ uploads_path: "DATADIR/uploads"
# - medium: msisdn # - medium: msisdn
# pattern: '\+44' # pattern: '\+44'
# Enable 3PIDs lookup requests to identity servers from this server.
#
#enable_3pid_lookup: true
# If set, allows registration of standard or admin accounts by anyone who # If set, allows registration of standard or admin accounts by anyone who
# has the shared secret, even if registration is otherwise disabled. # has the shared secret, even if registration is otherwise disabled.
# #

View File

@ -227,6 +227,12 @@ following regular expressions::
^/_matrix/client/(api/v1|r0|unstable)/account/3pid$ ^/_matrix/client/(api/v1|r0|unstable)/account/3pid$
^/_matrix/client/(api/v1|r0|unstable)/keys/query$ ^/_matrix/client/(api/v1|r0|unstable)/keys/query$
^/_matrix/client/(api/v1|r0|unstable)/keys/changes$ ^/_matrix/client/(api/v1|r0|unstable)/keys/changes$
^/_matrix/client/versions$
^/_matrix/client/(api/v1|r0|unstable)/voip/turnServer$
Additionally, the following REST endpoints can be handled for GET requests::
^/_matrix/client/(api/v1|r0|unstable)/pushrules/.*$
Additionally, the following REST endpoints can be handled, but all requests must Additionally, the following REST endpoints can be handled, but all requests must
be routed to the same instance:: be routed to the same instance::

View File

@ -58,15 +58,11 @@ BOOLEAN_COLUMNS = {
APPEND_ONLY_TABLES = [ APPEND_ONLY_TABLES = [
"event_content_hashes",
"event_reference_hashes", "event_reference_hashes",
"event_signatures",
"event_edge_hashes",
"events", "events",
"event_json", "event_json",
"state_events", "state_events",
"room_memberships", "room_memberships",
"feedback",
"topics", "topics",
"room_names", "room_names",
"rooms", "rooms",
@ -88,7 +84,6 @@ APPEND_ONLY_TABLES = [
"event_search", "event_search",
"presence_stream", "presence_stream",
"push_rules_stream", "push_rules_stream",
"current_state_resets",
"ex_outlier_stream", "ex_outlier_stream",
"cache_invalidation_stream", "cache_invalidation_stream",
"public_room_list_stream", "public_room_list_stream",
@ -811,7 +806,7 @@ class CursesProgress(Progress):
middle_space = 1 middle_space = 1
items = self.tables.items() items = self.tables.items()
items.sort(key=lambda i: (i[1]["perc"], i[0])) items = sorted(items, key=lambda i: (i[1]["perc"], i[0]))
for i, (table, data) in enumerate(items): for i, (table, data) in enumerate(items):
if i + 2 >= rows: if i + 2 >= rows:

View File

@ -86,13 +86,9 @@ long_description = read_file(("README.rst",))
REQUIREMENTS = dependencies['REQUIREMENTS'] REQUIREMENTS = dependencies['REQUIREMENTS']
CONDITIONAL_REQUIREMENTS = dependencies['CONDITIONAL_REQUIREMENTS'] CONDITIONAL_REQUIREMENTS = dependencies['CONDITIONAL_REQUIREMENTS']
ALL_OPTIONAL_REQUIREMENTS = dependencies['ALL_OPTIONAL_REQUIREMENTS']
# Make `pip install matrix-synapse[all]` install all the optional dependencies. # Make `pip install matrix-synapse[all]` install all the optional dependencies.
ALL_OPTIONAL_REQUIREMENTS = set()
for optional_deps in CONDITIONAL_REQUIREMENTS.values():
ALL_OPTIONAL_REQUIREMENTS = set(optional_deps) | ALL_OPTIONAL_REQUIREMENTS
CONDITIONAL_REQUIREMENTS["all"] = list(ALL_OPTIONAL_REQUIREMENTS) CONDITIONAL_REQUIREMENTS["all"] = list(ALL_OPTIONAL_REQUIREMENTS)

View File

@ -1,7 +1,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2014-2016 OpenMarket Ltd # Copyright 2014-2016 OpenMarket Ltd
# Copyright 2017 Vector Creations Ltd # Copyright 2017 Vector Creations Ltd
# Copyright 2018 New Vector Ltd. # Copyright 2018 New Vector Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
@ -69,6 +69,7 @@ class EventTypes(object):
Redaction = "m.room.redaction" Redaction = "m.room.redaction"
ThirdPartyInvite = "m.room.third_party_invite" ThirdPartyInvite = "m.room.third_party_invite"
Encryption = "m.room.encryption" Encryption = "m.room.encryption"
RelatedGroups = "m.room.related_groups"
RoomHistoryVisibility = "m.room.history_visibility" RoomHistoryVisibility = "m.room.history_visibility"
CanonicalAlias = "m.room.canonical_alias" CanonicalAlias = "m.room.canonical_alias"
@ -102,46 +103,6 @@ class ThirdPartyEntityKind(object):
LOCATION = "location" LOCATION = "location"
class RoomVersions(object):
V1 = "1"
V2 = "2"
V3 = "3"
STATE_V2_TEST = "state-v2-test"
class RoomDisposition(object):
STABLE = "stable"
UNSTABLE = "unstable"
# the version we will give rooms which are created on this server
DEFAULT_ROOM_VERSION = RoomVersions.V1
# vdh-test-version is a placeholder to get room versioning support working and tested
# until we have a working v2.
KNOWN_ROOM_VERSIONS = {
RoomVersions.V1,
RoomVersions.V2,
RoomVersions.V3,
RoomVersions.STATE_V2_TEST,
RoomVersions.V3,
}
class EventFormatVersions(object):
"""This is an internal enum for tracking the version of the event format,
independently from the room version.
"""
V1 = 1
V2 = 2
KNOWN_EVENT_FORMAT_VERSIONS = {
EventFormatVersions.V1,
EventFormatVersions.V2,
}
ServerNoticeMsgType = "m.server_notice" ServerNoticeMsgType = "m.server_notice"
ServerNoticeLimitReached = "m.server_notice.usage_limit_reached" ServerNoticeLimitReached = "m.server_notice.usage_limit_reached"

View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2014-2016 OpenMarket Ltd # Copyright 2014-2016 OpenMarket Ltd
# Copyright 2018 New Vector Ltd. # Copyright 2018 New Vector Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.

View File

@ -0,0 +1,91 @@
# -*- coding: utf-8 -*-
# Copyright 2019 New Vector Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import attr
class EventFormatVersions(object):
"""This is an internal enum for tracking the version of the event format,
independently from the room version.
"""
V1 = 1 # $id:server format
V2 = 2 # MSC1659-style $hash format: introduced for room v3
KNOWN_EVENT_FORMAT_VERSIONS = {
EventFormatVersions.V1,
EventFormatVersions.V2,
}
class StateResolutionVersions(object):
"""Enum to identify the state resolution algorithms"""
V1 = 1 # room v1 state res
V2 = 2 # MSC1442 state res: room v2 and later
class RoomDisposition(object):
STABLE = "stable"
UNSTABLE = "unstable"
@attr.s(slots=True, frozen=True)
class RoomVersion(object):
"""An object which describes the unique attributes of a room version."""
identifier = attr.ib() # str; the identifier for this version
disposition = attr.ib() # str; one of the RoomDispositions
event_format = attr.ib() # int; one of the EventFormatVersions
state_res = attr.ib() # int; one of the StateResolutionVersions
class RoomVersions(object):
V1 = RoomVersion(
"1",
RoomDisposition.STABLE,
EventFormatVersions.V1,
StateResolutionVersions.V1,
)
STATE_V2_TEST = RoomVersion(
"state-v2-test",
RoomDisposition.UNSTABLE,
EventFormatVersions.V1,
StateResolutionVersions.V2,
)
V2 = RoomVersion(
"2",
RoomDisposition.STABLE,
EventFormatVersions.V1,
StateResolutionVersions.V2,
)
V3 = RoomVersion(
"3",
RoomDisposition.STABLE,
EventFormatVersions.V2,
StateResolutionVersions.V2,
)
# the version we will give rooms which are created on this server
DEFAULT_ROOM_VERSION = RoomVersions.V1
KNOWN_ROOM_VERSIONS = {
v.identifier: v for v in (
RoomVersions.V1,
RoomVersions.V2,
RoomVersions.V3,
RoomVersions.STATE_V2_TEST,
)
} # type: dict[str, RoomVersion]

View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2014-2016 OpenMarket Ltd # Copyright 2014-2016 OpenMarket Ltd
# Copyright 2018 New Vector Ltd. # Copyright 2018 New Vector Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.

View File

@ -45,6 +45,7 @@ from synapse.replication.slave.storage.room import RoomStore
from synapse.replication.slave.storage.transactions import SlavedTransactionStore from synapse.replication.slave.storage.transactions import SlavedTransactionStore
from synapse.replication.tcp.client import ReplicationClientHandler from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.rest.client.v1.login import LoginRestServlet from synapse.rest.client.v1.login import LoginRestServlet
from synapse.rest.client.v1.push_rule import PushRuleRestServlet
from synapse.rest.client.v1.room import ( from synapse.rest.client.v1.room import (
JoinedRoomMemberListRestServlet, JoinedRoomMemberListRestServlet,
PublicRoomListRestServlet, PublicRoomListRestServlet,
@ -52,9 +53,11 @@ from synapse.rest.client.v1.room import (
RoomMemberListRestServlet, RoomMemberListRestServlet,
RoomStateRestServlet, RoomStateRestServlet,
) )
from synapse.rest.client.v1.voip import VoipRestServlet
from synapse.rest.client.v2_alpha.account import ThreepidRestServlet from synapse.rest.client.v2_alpha.account import ThreepidRestServlet
from synapse.rest.client.v2_alpha.keys import KeyChangesServlet, KeyQueryServlet from synapse.rest.client.v2_alpha.keys import KeyChangesServlet, KeyQueryServlet
from synapse.rest.client.v2_alpha.register import RegisterRestServlet from synapse.rest.client.v2_alpha.register import RegisterRestServlet
from synapse.rest.client.versions import VersionsRestServlet
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.util.httpresourcetree import create_resource_tree from synapse.util.httpresourcetree import create_resource_tree
@ -109,12 +112,12 @@ class ClientReaderServer(HomeServer):
ThreepidRestServlet(self).register(resource) ThreepidRestServlet(self).register(resource)
KeyQueryServlet(self).register(resource) KeyQueryServlet(self).register(resource)
KeyChangesServlet(self).register(resource) KeyChangesServlet(self).register(resource)
VoipRestServlet(self).register(resource)
PushRuleRestServlet(self).register(resource)
VersionsRestServlet().register(resource)
resources.update({ resources.update({
"/_matrix/client/r0": resource, "/_matrix/client": resource,
"/_matrix/client/unstable": resource,
"/_matrix/client/v2_alpha": resource,
"/_matrix/client/api/v1": resource,
}) })
root_resource = create_resource_tree(resources, NoResource()) root_resource = create_resource_tree(resources, NoResource())

View File

@ -38,7 +38,7 @@ from synapse.replication.slave.storage.receipts import SlavedReceiptsStore
from synapse.replication.slave.storage.registration import SlavedRegistrationStore from synapse.replication.slave.storage.registration import SlavedRegistrationStore
from synapse.replication.slave.storage.transactions import SlavedTransactionStore from synapse.replication.slave.storage.transactions import SlavedTransactionStore
from synapse.replication.tcp.client import ReplicationClientHandler from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.replication.tcp.streams import ReceiptsStream from synapse.replication.tcp.streams._base import ReceiptsStream
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.types import ReadReceipt from synapse.types import ReadReceipt

View File

@ -518,6 +518,7 @@ def run(hs):
uptime = 0 uptime = 0
stats["homeserver"] = hs.config.server_name stats["homeserver"] = hs.config.server_name
stats["server_context"] = hs.config.server_context
stats["timestamp"] = now stats["timestamp"] = now
stats["uptime_seconds"] = uptime stats["uptime_seconds"] = uptime
version = sys.version_info version = sys.version_info
@ -558,7 +559,6 @@ def run(hs):
stats["database_engine"] = hs.get_datastore().database_engine_name stats["database_engine"] = hs.get_datastore().database_engine_name
stats["database_server_version"] = hs.get_datastore().get_server_version() stats["database_server_version"] = hs.get_datastore().get_server_version()
logger.info("Reporting stats to matrix.org: %s" % (stats,)) logger.info("Reporting stats to matrix.org: %s" % (stats,))
try: try:
yield hs.get_simple_http_client().put_json( yield hs.get_simple_http_client().put_json(

View File

@ -48,6 +48,7 @@ from synapse.replication.slave.storage.receipts import SlavedReceiptsStore
from synapse.replication.slave.storage.registration import SlavedRegistrationStore from synapse.replication.slave.storage.registration import SlavedRegistrationStore
from synapse.replication.slave.storage.room import RoomStore from synapse.replication.slave.storage.room import RoomStore
from synapse.replication.tcp.client import ReplicationClientHandler from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.replication.tcp.streams.events import EventsStreamEventRow
from synapse.rest.client.v1 import events from synapse.rest.client.v1 import events
from synapse.rest.client.v1.initial_sync import InitialSyncRestServlet from synapse.rest.client.v1.initial_sync import InitialSyncRestServlet
from synapse.rest.client.v1.room import RoomInitialSyncRestServlet from synapse.rest.client.v1.room import RoomInitialSyncRestServlet
@ -369,7 +370,9 @@ class SyncReplicationHandler(ReplicationClientHandler):
# We shouldn't get multiple rows per token for events stream, so # We shouldn't get multiple rows per token for events stream, so
# we don't need to optimise this for multiple rows. # we don't need to optimise this for multiple rows.
for row in rows: for row in rows:
event = yield self.store.get_event(row.event_id) if row.type != EventsStreamEventRow.TypeId:
continue
event = yield self.store.get_event(row.data.event_id)
extra_users = () extra_users = ()
if event.type == EventTypes.Member: if event.type == EventTypes.Member:
extra_users = (event.state_key,) extra_users = (event.state_key,)

View File

@ -36,6 +36,10 @@ from synapse.replication.slave.storage.client_ips import SlavedClientIpStore
from synapse.replication.slave.storage.events import SlavedEventStore from synapse.replication.slave.storage.events import SlavedEventStore
from synapse.replication.slave.storage.registration import SlavedRegistrationStore from synapse.replication.slave.storage.registration import SlavedRegistrationStore
from synapse.replication.tcp.client import ReplicationClientHandler from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.replication.tcp.streams.events import (
EventsStream,
EventsStreamCurrentStateRow,
)
from synapse.rest.client.v2_alpha import user_directory from synapse.rest.client.v2_alpha import user_directory
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
@ -73,19 +77,18 @@ class UserDirectorySlaveStore(
prefilled_cache=curr_state_delta_prefill, prefilled_cache=curr_state_delta_prefill,
) )
self._current_state_delta_pos = events_max
def stream_positions(self): def stream_positions(self):
result = super(UserDirectorySlaveStore, self).stream_positions() result = super(UserDirectorySlaveStore, self).stream_positions()
result["current_state_deltas"] = self._current_state_delta_pos
return result return result
def process_replication_rows(self, stream_name, token, rows): def process_replication_rows(self, stream_name, token, rows):
if stream_name == "current_state_deltas": if stream_name == EventsStream.NAME:
self._current_state_delta_pos = token self._stream_id_gen.advance(token)
for row in rows: for row in rows:
if row.type != EventsStreamCurrentStateRow.TypeId:
continue
self._curr_state_delta_stream_cache.entity_has_changed( self._curr_state_delta_stream_cache.entity_has_changed(
row.room_id, token row.data.room_id, token
) )
return super(UserDirectorySlaveStore, self).process_replication_rows( return super(UserDirectorySlaveStore, self).process_replication_rows(
stream_name, token, rows stream_name, token, rows
@ -170,7 +173,7 @@ class UserDirectoryReplicationHandler(ReplicationClientHandler):
yield super(UserDirectoryReplicationHandler, self).on_rdata( yield super(UserDirectoryReplicationHandler, self).on_rdata(
stream_name, token, rows stream_name, token, rows
) )
if stream_name == "current_state_deltas": if stream_name == EventsStream.NAME:
run_in_background(self._notify_directory) run_in_background(self._notify_directory)
@defer.inlineCallbacks @defer.inlineCallbacks

View File

@ -42,7 +42,8 @@ class KeyConfig(Config):
if "signing_key" in config: if "signing_key" in config:
self.signing_key = read_signing_keys([config["signing_key"]]) self.signing_key = read_signing_keys([config["signing_key"]])
else: else:
self.signing_key = self.read_signing_key(config["signing_key_path"]) self.signing_key_path = config["signing_key_path"]
self.signing_key = self.read_signing_key(self.signing_key_path)
self.old_signing_keys = self.read_old_signing_keys( self.old_signing_keys = self.read_old_signing_keys(
config.get("old_signing_keys", {}) config.get("old_signing_keys", {})

View File

@ -60,6 +60,7 @@ class RegistrationConfig(Config):
self.registrations_require_3pid = config.get("registrations_require_3pid", []) self.registrations_require_3pid = config.get("registrations_require_3pid", [])
self.allowed_local_3pids = config.get("allowed_local_3pids", []) self.allowed_local_3pids = config.get("allowed_local_3pids", [])
self.enable_3pid_lookup = config.get("enable_3pid_lookup", True)
self.registration_shared_secret = config.get("registration_shared_secret") self.registration_shared_secret = config.get("registration_shared_secret")
self.bcrypt_rounds = config.get("bcrypt_rounds", 12) self.bcrypt_rounds = config.get("bcrypt_rounds", 12)
@ -150,6 +151,10 @@ class RegistrationConfig(Config):
# - medium: msisdn # - medium: msisdn
# pattern: '\\+44' # pattern: '\\+44'
# Enable 3PIDs lookup requests to identity servers from this server.
#
#enable_3pid_lookup: true
# If set, allows registration of standard or admin accounts by anyone who # If set, allows registration of standard or admin accounts by anyone who
# has the shared secret, even if registration is otherwise disabled. # has the shared secret, even if registration is otherwise disabled.
# #

View File

@ -1,5 +1,5 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2018 New Vector Ltd. # Copyright 2018 New Vector Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.

View File

@ -37,6 +37,7 @@ class ServerConfig(Config):
def read_config(self, config): def read_config(self, config):
self.server_name = config["server_name"] self.server_name = config["server_name"]
self.server_context = config.get("server_context", None)
try: try:
parse_and_validate_server_name(self.server_name) parse_and_validate_server_name(self.server_name)
@ -484,6 +485,9 @@ class ServerConfig(Config):
#mau_limit_reserved_threepids: #mau_limit_reserved_threepids:
# - medium: 'email' # - medium: 'email'
# address: 'reserved_user@example.com' # address: 'reserved_user@example.com'
# Used by phonehome stats to group together related servers.
#server_context: context
""" % locals() """ % locals()
def read_arguments(self, args): def read_arguments(self, args):

View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2014-2016 OpenMarket Ltd # Copyright 2014-2016 OpenMarket Ltd
# Copyright 2017, 2018 New Vector Ltd. # Copyright 2017, 2018 New Vector Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
@ -20,6 +20,7 @@ from collections import namedtuple
from six import raise_from from six import raise_from
from six.moves import urllib from six.moves import urllib
import nacl.signing
from signedjson.key import ( from signedjson.key import (
decode_verify_key_bytes, decode_verify_key_bytes,
encode_verify_key_base64, encode_verify_key_base64,
@ -274,10 +275,6 @@ class Keyring(object):
@defer.inlineCallbacks @defer.inlineCallbacks
def do_iterations(): def do_iterations():
with Measure(self.clock, "get_server_verify_keys"): with Measure(self.clock, "get_server_verify_keys"):
# dict[str, dict[str, VerifyKey]]: results so far.
# map server_name -> key_id -> VerifyKey
merged_results = {}
# dict[str, set(str)]: keys to fetch for each server # dict[str, set(str)]: keys to fetch for each server
missing_keys = {} missing_keys = {}
for verify_request in verify_requests: for verify_request in verify_requests:
@ -287,29 +284,29 @@ class Keyring(object):
for fn in key_fetch_fns: for fn in key_fetch_fns:
results = yield fn(missing_keys.items()) results = yield fn(missing_keys.items())
merged_results.update(results)
# We now need to figure out which verify requests we have keys # We now need to figure out which verify requests we have keys
# for and which we don't # for and which we don't
missing_keys = {} missing_keys = {}
requests_missing_keys = [] requests_missing_keys = []
for verify_request in verify_requests: for verify_request in verify_requests:
server_name = verify_request.server_name
result_keys = merged_results[server_name]
if verify_request.deferred.called: if verify_request.deferred.called:
# We've already called this deferred, which probably # We've already called this deferred, which probably
# means that we've already found a key for it. # means that we've already found a key for it.
continue continue
server_name = verify_request.server_name
# see if any of the keys we got this time are sufficient to
# complete this VerifyKeyRequest.
result_keys = results.get(server_name, {})
for key_id in verify_request.key_ids: for key_id in verify_request.key_ids:
if key_id in result_keys: key = result_keys.get(key_id)
if key:
with PreserveLoggingContext(): with PreserveLoggingContext():
verify_request.deferred.callback(( verify_request.deferred.callback(
server_name, (server_name, key_id, key)
key_id, )
result_keys[key_id],
))
break break
else: else:
# The else block is only reached if the loop above # The else block is only reached if the loop above
@ -343,27 +340,24 @@ class Keyring(object):
@defer.inlineCallbacks @defer.inlineCallbacks
def get_keys_from_store(self, server_name_and_key_ids): def get_keys_from_store(self, server_name_and_key_ids):
""" """
Args: Args:
server_name_and_key_ids (list[(str, iterable[str])]): server_name_and_key_ids (iterable(Tuple[str, iterable[str]]):
list of (server_name, iterable[key_id]) tuples to fetch keys for list of (server_name, iterable[key_id]) tuples to fetch keys for
Returns: Returns:
Deferred: resolves to dict[str, dict[str, VerifyKey]]: map from Deferred: resolves to dict[str, dict[str, VerifyKey|None]]: map from
server_name -> key_id -> VerifyKey server_name -> key_id -> VerifyKey
""" """
res = yield logcontext.make_deferred_yieldable(defer.gatherResults( keys_to_fetch = (
[ (server_name, key_id)
run_in_background(
self.store.get_server_verify_keys,
server_name, key_ids,
).addCallback(lambda ks, server: (server, ks), server_name)
for server_name, key_ids in server_name_and_key_ids for server_name, key_ids in server_name_and_key_ids
], for key_id in key_ids
consumeErrors=True, )
).addErrback(unwrapFirstError)) res = yield self.store.get_server_verify_keys(keys_to_fetch)
keys = {}
defer.returnValue(dict(res)) for (server_name, key_id), key in res.items():
keys.setdefault(server_name, {})[key_id] = key
defer.returnValue(keys)
@defer.inlineCallbacks @defer.inlineCallbacks
def get_keys_from_perspectives(self, server_name_and_key_ids): def get_keys_from_perspectives(self, server_name_and_key_ids):
@ -494,11 +488,11 @@ class Keyring(object):
) )
processed_response = yield self.process_v2_response( processed_response = yield self.process_v2_response(
perspective_name, response, only_from_server=False perspective_name, response
) )
server_name = response["server_name"]
for server_name, response_keys in processed_response.items(): keys.setdefault(server_name, {}).update(processed_response)
keys.setdefault(server_name, {}).update(response_keys)
yield logcontext.make_deferred_yieldable(defer.gatherResults( yield logcontext.make_deferred_yieldable(defer.gatherResults(
[ [
@ -517,7 +511,7 @@ class Keyring(object):
@defer.inlineCallbacks @defer.inlineCallbacks
def get_server_verify_key_v2_direct(self, server_name, key_ids): def get_server_verify_key_v2_direct(self, server_name, key_ids):
keys = {} keys = {} # type: dict[str, nacl.signing.VerifyKey]
for requested_key_id in key_ids: for requested_key_id in key_ids:
if requested_key_id in keys: if requested_key_id in keys:
@ -542,6 +536,11 @@ class Keyring(object):
or server_name not in response[u"signatures"]): or server_name not in response[u"signatures"]):
raise KeyLookupError("Key response not signed by remote server") raise KeyLookupError("Key response not signed by remote server")
if response["server_name"] != server_name:
raise KeyLookupError("Expected a response for server %r not %r" % (
server_name, response["server_name"]
))
response_keys = yield self.process_v2_response( response_keys = yield self.process_v2_response(
from_server=server_name, from_server=server_name,
requested_ids=[requested_key_id], requested_ids=[requested_key_id],
@ -550,24 +549,45 @@ class Keyring(object):
keys.update(response_keys) keys.update(response_keys)
yield logcontext.make_deferred_yieldable(defer.gatherResults( yield self.store_keys(
[ server_name=server_name,
run_in_background(
self.store_keys,
server_name=key_server_name,
from_server=server_name, from_server=server_name,
verify_keys=verify_keys, verify_keys=keys,
) )
for key_server_name, verify_keys in keys.items() defer.returnValue({server_name: keys})
],
consumeErrors=True
).addErrback(unwrapFirstError))
defer.returnValue(keys)
@defer.inlineCallbacks @defer.inlineCallbacks
def process_v2_response(self, from_server, response_json, def process_v2_response(
requested_ids=[], only_from_server=True): self, from_server, response_json, requested_ids=[],
):
"""Parse a 'Server Keys' structure from the result of a /key request
This is used to parse either the entirety of the response from
GET /_matrix/key/v2/server, or a single entry from the list returned by
POST /_matrix/key/v2/query.
Checks that each signature in the response that claims to come from the origin
server is valid. (Does not check that there actually is such a signature, for
some reason.)
Stores the json in server_keys_json so that it can be used for future responses
to /_matrix/key/v2/query.
Args:
from_server (str): the name of the server producing this result: either
the origin server for a /_matrix/key/v2/server request, or the notary
for a /_matrix/key/v2/query.
response_json (dict): the json-decoded Server Keys response object
requested_ids (iterable[str]): a list of the key IDs that were requested.
We will store the json for these key ids as well as any that are
actually in the response
Returns:
Deferred[dict[str, nacl.signing.VerifyKey]]:
map from key_id to key object
"""
time_now_ms = self.clock.time_msec() time_now_ms = self.clock.time_msec()
response_keys = {} response_keys = {}
verify_keys = {} verify_keys = {}
@ -589,15 +609,7 @@ class Keyring(object):
verify_key.time_added = time_now_ms verify_key.time_added = time_now_ms
old_verify_keys[key_id] = verify_key old_verify_keys[key_id] = verify_key
results = {}
server_name = response_json["server_name"] server_name = response_json["server_name"]
if only_from_server:
if server_name != from_server:
raise KeyLookupError(
"Expected a response for server %r not %r" % (
from_server, server_name
)
)
for key_id in response_json["signatures"].get(server_name, {}): for key_id in response_json["signatures"].get(server_name, {}):
if key_id not in response_json["verify_keys"]: if key_id not in response_json["verify_keys"]:
raise KeyLookupError( raise KeyLookupError(
@ -633,7 +645,7 @@ class Keyring(object):
self.store.store_server_keys_json, self.store.store_server_keys_json,
server_name=server_name, server_name=server_name,
key_id=key_id, key_id=key_id,
from_server=server_name, from_server=from_server,
ts_now_ms=time_now_ms, ts_now_ms=time_now_ms,
ts_expires_ms=ts_valid_until_ms, ts_expires_ms=ts_valid_until_ms,
key_json_bytes=signed_key_json_bytes, key_json_bytes=signed_key_json_bytes,
@ -643,9 +655,7 @@ class Keyring(object):
consumeErrors=True, consumeErrors=True,
).addErrback(unwrapFirstError)) ).addErrback(unwrapFirstError))
results[server_name] = response_keys defer.returnValue(response_keys)
defer.returnValue(results)
def store_keys(self, server_name, from_server, verify_keys): def store_keys(self, server_name, from_server, verify_keys):
"""Store a collection of verify keys for a given server """Store a collection of verify keys for a given server

View File

@ -20,15 +20,9 @@ from signedjson.key import decode_verify_key_bytes
from signedjson.sign import SignatureVerifyException, verify_signed_json from signedjson.sign import SignatureVerifyException, verify_signed_json
from unpaddedbase64 import decode_base64 from unpaddedbase64 import decode_base64
from synapse.api.constants import ( from synapse.api.constants import EventTypes, JoinRules, Membership
KNOWN_ROOM_VERSIONS,
EventFormatVersions,
EventTypes,
JoinRules,
Membership,
RoomVersions,
)
from synapse.api.errors import AuthError, EventSizeError, SynapseError from synapse.api.errors import AuthError, EventSizeError, SynapseError
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, EventFormatVersions
from synapse.types import UserID, get_domain_from_id from synapse.types import UserID, get_domain_from_id
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -452,16 +446,18 @@ def check_redaction(room_version, event, auth_events):
if user_level >= redact_level: if user_level >= redact_level:
return False return False
if room_version in (RoomVersions.V1, RoomVersions.V2,): v = KNOWN_ROOM_VERSIONS.get(room_version)
if not v:
raise RuntimeError("Unrecognized room version %r" % (room_version,))
if v.event_format == EventFormatVersions.V1:
redacter_domain = get_domain_from_id(event.event_id) redacter_domain = get_domain_from_id(event.event_id)
redactee_domain = get_domain_from_id(event.redacts) redactee_domain = get_domain_from_id(event.redacts)
if redacter_domain == redactee_domain: if redacter_domain == redactee_domain:
return True return True
elif room_version == RoomVersions.V3: else:
event.internal_metadata.recheck_redaction = True event.internal_metadata.recheck_redaction = True
return True return True
else:
raise RuntimeError("Unrecognized room version %r" % (room_version,))
raise AuthError( raise AuthError(
403, 403,

View File

@ -21,7 +21,7 @@ import six
from unpaddedbase64 import encode_base64 from unpaddedbase64 import encode_base64
from synapse.api.constants import KNOWN_ROOM_VERSIONS, EventFormatVersions, RoomVersions from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, EventFormatVersions
from synapse.util.caches import intern_dict from synapse.util.caches import intern_dict
from synapse.util.frozenutils import freeze from synapse.util.frozenutils import freeze
@ -351,18 +351,13 @@ def room_version_to_event_format(room_version):
Returns: Returns:
int int
""" """
if room_version not in KNOWN_ROOM_VERSIONS: v = KNOWN_ROOM_VERSIONS.get(room_version)
if not v:
# We should have already checked version, so this should not happen # We should have already checked version, so this should not happen
raise RuntimeError("Unrecognized room version %s" % (room_version,)) raise RuntimeError("Unrecognized room version %s" % (room_version,))
if room_version in ( return v.event_format
RoomVersions.V1, RoomVersions.V2, RoomVersions.STATE_V2_TEST,
):
return EventFormatVersions.V1
elif room_version in (RoomVersions.V3,):
return EventFormatVersions.V2
else:
raise RuntimeError("Unrecognized room version %s" % (room_version,))
def event_type_from_format_version(format_version): def event_type_from_format_version(format_version):

View File

@ -17,21 +17,17 @@ import attr
from twisted.internet import defer from twisted.internet import defer
from synapse.api.constants import ( from synapse.api.constants import MAX_DEPTH
from synapse.api.room_versions import (
KNOWN_EVENT_FORMAT_VERSIONS, KNOWN_EVENT_FORMAT_VERSIONS,
KNOWN_ROOM_VERSIONS, KNOWN_ROOM_VERSIONS,
MAX_DEPTH,
EventFormatVersions, EventFormatVersions,
) )
from synapse.crypto.event_signing import add_hashes_and_signatures from synapse.crypto.event_signing import add_hashes_and_signatures
from synapse.types import EventID from synapse.types import EventID
from synapse.util.stringutils import random_string from synapse.util.stringutils import random_string
from . import ( from . import _EventInternalMetadata, event_type_from_format_version
_EventInternalMetadata,
event_type_from_format_version,
room_version_to_event_format,
)
@attr.s(slots=True, cmp=False, frozen=True) @attr.s(slots=True, cmp=False, frozen=True)
@ -170,21 +166,34 @@ class EventBuilderFactory(object):
def new(self, room_version, key_values): def new(self, room_version, key_values):
"""Generate an event builder appropriate for the given room version """Generate an event builder appropriate for the given room version
Deprecated: use for_room_version with a RoomVersion object instead
Args: Args:
room_version (str): Version of the room that we're creating an room_version (str): Version of the room that we're creating an event builder
event builder for for
key_values (dict): Fields used as the basis of the new event key_values (dict): Fields used as the basis of the new event
Returns: Returns:
EventBuilder EventBuilder
""" """
v = KNOWN_ROOM_VERSIONS.get(room_version)
# There's currently only the one event version defined if not v:
if room_version not in KNOWN_ROOM_VERSIONS:
raise Exception( raise Exception(
"No event format defined for version %r" % (room_version,) "No event format defined for version %r" % (room_version,)
) )
return self.for_room_version(v, key_values)
def for_room_version(self, room_version, key_values):
"""Generate an event builder appropriate for the given room version
Args:
room_version (synapse.api.room_versions.RoomVersion):
Version of the room that we're creating an event builder for
key_values (dict): Fields used as the basis of the new event
Returns:
EventBuilder
"""
return EventBuilder( return EventBuilder(
store=self.store, store=self.store,
state=self.state, state=self.state,
@ -192,7 +201,7 @@ class EventBuilderFactory(object):
clock=self.clock, clock=self.clock,
hostname=self.hostname, hostname=self.hostname,
signing_key=self.signing_key, signing_key=self.signing_key,
format_version=room_version_to_event_format(room_version), format_version=room_version.event_format,
type=key_values["type"], type=key_values["type"],
state_key=key_values.get("state_key"), state_key=key_values.get("state_key"),
room_id=key_values["room_id"], room_id=key_values["room_id"],
@ -222,7 +231,6 @@ def create_local_event_from_event_dict(clock, hostname, signing_key,
FrozenEvent FrozenEvent
""" """
# There's currently only the one event version defined
if format_version not in KNOWN_EVENT_FORMAT_VERSIONS: if format_version not in KNOWN_EVENT_FORMAT_VERSIONS:
raise Exception( raise Exception(
"No event format defined for version %r" % (format_version,) "No event format defined for version %r" % (format_version,)

View File

@ -1,5 +1,5 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2017 New Vector Ltd. # Copyright 2017 New Vector Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.

View File

@ -15,8 +15,9 @@
from six import string_types from six import string_types
from synapse.api.constants import EventFormatVersions, EventTypes, Membership from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import SynapseError from synapse.api.errors import SynapseError
from synapse.api.room_versions import EventFormatVersions
from synapse.types import EventID, RoomID, UserID from synapse.types import EventID, RoomID, UserID

View File

@ -20,8 +20,9 @@ import six
from twisted.internet import defer from twisted.internet import defer
from twisted.internet.defer import DeferredList from twisted.internet.defer import DeferredList
from synapse.api.constants import MAX_DEPTH, EventTypes, Membership, RoomVersions from synapse.api.constants import MAX_DEPTH, EventTypes, Membership
from synapse.api.errors import Codes, SynapseError from synapse.api.errors import Codes, SynapseError
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, EventFormatVersions
from synapse.crypto.event_signing import check_event_content_hash from synapse.crypto.event_signing import check_event_content_hash
from synapse.events import event_type_from_format_version from synapse.events import event_type_from_format_version
from synapse.events.utils import prune_event from synapse.events.utils import prune_event
@ -274,9 +275,12 @@ def _check_sigs_on_pdus(keyring, room_version, pdus):
# now let's look for events where the sender's domain is different to the # now let's look for events where the sender's domain is different to the
# event id's domain (normally only the case for joins/leaves), and add additional # event id's domain (normally only the case for joins/leaves), and add additional
# checks. Only do this if the room version has a concept of event ID domain # checks. Only do this if the room version has a concept of event ID domain
if room_version in ( # (ie, the room version uses old-style non-hash event IDs).
RoomVersions.V1, RoomVersions.V2, RoomVersions.STATE_V2_TEST, v = KNOWN_ROOM_VERSIONS.get(room_version)
): if not v:
raise RuntimeError("Unrecognized room version %s" % (room_version,))
if v.event_format == EventFormatVersions.V1:
pdus_to_check_event_id = [ pdus_to_check_event_id = [
p for p in pdus_to_check p for p in pdus_to_check
if p.sender_domain != get_domain_from_id(p.pdu.event_id) if p.sender_domain != get_domain_from_id(p.pdu.event_id)
@ -289,10 +293,6 @@ def _check_sigs_on_pdus(keyring, room_version, pdus):
for p, d in zip(pdus_to_check_event_id, more_deferreds): for p, d in zip(pdus_to_check_event_id, more_deferreds):
p.deferreds.append(d) p.deferreds.append(d)
elif room_version in (RoomVersions.V3,):
pass # No further checks needed, as event IDs are hashes here
else:
raise RuntimeError("Unrecognized room version %s" % (room_version,))
# replace lists of deferreds with single Deferreds # replace lists of deferreds with single Deferreds
return [_flatten_deferred_list(p.deferreds) for p in pdus_to_check] return [_flatten_deferred_list(p.deferreds) for p in pdus_to_check]

View File

@ -25,12 +25,7 @@ from prometheus_client import Counter
from twisted.internet import defer from twisted.internet import defer
from synapse.api.constants import ( from synapse.api.constants import EventTypes, Membership
KNOWN_ROOM_VERSIONS,
EventTypes,
Membership,
RoomVersions,
)
from synapse.api.errors import ( from synapse.api.errors import (
CodeMessageException, CodeMessageException,
Codes, Codes,
@ -38,6 +33,11 @@ from synapse.api.errors import (
HttpResponseException, HttpResponseException,
SynapseError, SynapseError,
) )
from synapse.api.room_versions import (
KNOWN_ROOM_VERSIONS,
EventFormatVersions,
RoomVersions,
)
from synapse.events import builder, room_version_to_event_format from synapse.events import builder, room_version_to_event_format
from synapse.federation.federation_base import FederationBase, event_from_pdu_json from synapse.federation.federation_base import FederationBase, event_from_pdu_json
from synapse.util import logcontext, unwrapFirstError from synapse.util import logcontext, unwrapFirstError
@ -570,7 +570,7 @@ class FederationClient(FederationBase):
Deferred[tuple[str, FrozenEvent, int]]: resolves to a tuple of Deferred[tuple[str, FrozenEvent, int]]: resolves to a tuple of
`(origin, event, event_format)` where origin is the remote `(origin, event, event_format)` where origin is the remote
homeserver which generated the event, and event_format is one of homeserver which generated the event, and event_format is one of
`synapse.api.constants.EventFormatVersions`. `synapse.api.room_versions.EventFormatVersions`.
Fails with a ``SynapseError`` if the chosen remote server Fails with a ``SynapseError`` if the chosen remote server
returns a 300/400 code. returns a 300/400 code.
@ -592,7 +592,7 @@ class FederationClient(FederationBase):
# Note: If not supplied, the room version may be either v1 or v2, # Note: If not supplied, the room version may be either v1 or v2,
# however either way the event format version will be v1. # however either way the event format version will be v1.
room_version = ret.get("room_version", RoomVersions.V1) room_version = ret.get("room_version", RoomVersions.V1.identifier)
event_format = room_version_to_event_format(room_version) event_format = room_version_to_event_format(room_version)
pdu_dict = ret.get("event", None) pdu_dict = ret.get("event", None)
@ -695,7 +695,9 @@ class FederationClient(FederationBase):
room_version = None room_version = None
for e in state: for e in state:
if (e.type, e.state_key) == (EventTypes.Create, ""): if (e.type, e.state_key) == (EventTypes.Create, ""):
room_version = e.content.get("room_version", RoomVersions.V1) room_version = e.content.get(
"room_version", RoomVersions.V1.identifier
)
break break
if room_version is None: if room_version is None:
@ -802,11 +804,10 @@ class FederationClient(FederationBase):
raise err raise err
# Otherwise, we assume that the remote server doesn't understand # Otherwise, we assume that the remote server doesn't understand
# the v2 invite API. # the v2 invite API. That's ok provided the room uses old-style event
# IDs.
if room_version in (RoomVersions.V1, RoomVersions.V2): v = KNOWN_ROOM_VERSIONS.get(room_version)
pass # We'll fall through if v.event_format != EventFormatVersions.V1:
else:
raise SynapseError( raise SynapseError(
400, 400,
"User's homeserver does not support this room version", "User's homeserver does not support this room version",

View File

@ -25,7 +25,7 @@ from twisted.internet import defer
from twisted.internet.abstract import isIPAddress from twisted.internet.abstract import isIPAddress
from twisted.python import failure from twisted.python import failure
from synapse.api.constants import KNOWN_ROOM_VERSIONS, EventTypes, Membership from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import ( from synapse.api.errors import (
AuthError, AuthError,
Codes, Codes,
@ -34,6 +34,7 @@ from synapse.api.errors import (
NotFoundError, NotFoundError,
SynapseError, SynapseError,
) )
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS
from synapse.crypto.event_signing import compute_event_signature from synapse.crypto.event_signing import compute_event_signature
from synapse.events import room_version_to_event_format from synapse.events import room_version_to_event_format
from synapse.federation.federation_base import FederationBase, event_from_pdu_json from synapse.federation.federation_base import FederationBase, event_from_pdu_json

View File

@ -55,7 +55,12 @@ class FederationRemoteSendQueue(object):
self.is_mine_id = hs.is_mine_id self.is_mine_id = hs.is_mine_id
self.presence_map = {} # Pending presence map user_id -> UserPresenceState self.presence_map = {} # Pending presence map user_id -> UserPresenceState
self.presence_changed = SortedDict() # Stream position -> user_id self.presence_changed = SortedDict() # Stream position -> list[user_id]
# Stores the destinations we need to explicitly send presence to about a
# given user.
# Stream position -> (user_id, destinations)
self.presence_destinations = SortedDict()
self.keyed_edu = {} # (destination, key) -> EDU self.keyed_edu = {} # (destination, key) -> EDU
self.keyed_edu_changed = SortedDict() # stream position -> (destination, key) self.keyed_edu_changed = SortedDict() # stream position -> (destination, key)
@ -77,7 +82,7 @@ class FederationRemoteSendQueue(object):
for queue_name in [ for queue_name in [
"presence_map", "presence_changed", "keyed_edu", "keyed_edu_changed", "presence_map", "presence_changed", "keyed_edu", "keyed_edu_changed",
"edus", "device_messages", "pos_time", "edus", "device_messages", "pos_time", "presence_destinations",
]: ]:
register(queue_name, getattr(self, queue_name)) register(queue_name, getattr(self, queue_name))
@ -121,6 +126,15 @@ class FederationRemoteSendQueue(object):
for user_id in uids for user_id in uids
) )
keys = self.presence_destinations.keys()
i = self.presence_destinations.bisect_left(position_to_delete)
for key in keys[:i]:
del self.presence_destinations[key]
user_ids.update(
user_id for user_id, _ in self.presence_destinations.values()
)
to_del = [ to_del = [
user_id for user_id in self.presence_map if user_id not in user_ids user_id for user_id in self.presence_map if user_id not in user_ids
] ]
@ -209,6 +223,20 @@ class FederationRemoteSendQueue(object):
self.notifier.on_new_replication_data() self.notifier.on_new_replication_data()
def send_presence_to_destinations(self, states, destinations):
"""As per FederationSender
Args:
states (list[UserPresenceState])
destinations (list[str])
"""
for state in states:
pos = self._next_pos()
self.presence_map.update({state.user_id: state for state in states})
self.presence_destinations[pos] = (state.user_id, destinations)
self.notifier.on_new_replication_data()
def send_device_messages(self, destination): def send_device_messages(self, destination):
"""As per FederationSender""" """As per FederationSender"""
pos = self._next_pos() pos = self._next_pos()
@ -261,6 +289,16 @@ class FederationRemoteSendQueue(object):
state=self.presence_map[user_id], state=self.presence_map[user_id],
))) )))
# Fetch presence to send to destinations
i = self.presence_destinations.bisect_right(from_token)
j = self.presence_destinations.bisect_right(to_token) + 1
for pos, (user_id, dests) in self.presence_destinations.items()[i:j]:
rows.append((pos, PresenceDestinationsRow(
state=self.presence_map[user_id],
destinations=list(dests),
)))
# Fetch changes keyed edus # Fetch changes keyed edus
i = self.keyed_edu_changed.bisect_right(from_token) i = self.keyed_edu_changed.bisect_right(from_token)
j = self.keyed_edu_changed.bisect_right(to_token) + 1 j = self.keyed_edu_changed.bisect_right(to_token) + 1
@ -357,6 +395,29 @@ class PresenceRow(BaseFederationRow, namedtuple("PresenceRow", (
buff.presence.append(self.state) buff.presence.append(self.state)
class PresenceDestinationsRow(BaseFederationRow, namedtuple("PresenceDestinationsRow", (
"state", # UserPresenceState
"destinations", # list[str]
))):
TypeId = "pd"
@staticmethod
def from_data(data):
return PresenceDestinationsRow(
state=UserPresenceState.from_dict(data["state"]),
destinations=data["dests"],
)
def to_data(self):
return {
"state": self.state.as_dict(),
"dests": self.destinations,
}
def add_to_buffer(self, buff):
buff.presence_destinations.append((self.state, self.destinations))
class KeyedEduRow(BaseFederationRow, namedtuple("KeyedEduRow", ( class KeyedEduRow(BaseFederationRow, namedtuple("KeyedEduRow", (
"key", # tuple(str) - the edu key passed to send_edu "key", # tuple(str) - the edu key passed to send_edu
"edu", # Edu "edu", # Edu
@ -428,6 +489,7 @@ TypeToRow = {
Row.TypeId: Row Row.TypeId: Row
for Row in ( for Row in (
PresenceRow, PresenceRow,
PresenceDestinationsRow,
KeyedEduRow, KeyedEduRow,
EduRow, EduRow,
DeviceRow, DeviceRow,
@ -437,6 +499,7 @@ TypeToRow = {
ParsedFederationStreamData = namedtuple("ParsedFederationStreamData", ( ParsedFederationStreamData = namedtuple("ParsedFederationStreamData", (
"presence", # list(UserPresenceState) "presence", # list(UserPresenceState)
"presence_destinations", # list of tuples of UserPresenceState and destinations
"keyed_edus", # dict of destination -> { key -> Edu } "keyed_edus", # dict of destination -> { key -> Edu }
"edus", # dict of destination -> [Edu] "edus", # dict of destination -> [Edu]
"device_destinations", # set of destinations "device_destinations", # set of destinations
@ -458,6 +521,7 @@ def process_rows_for_federation(transaction_queue, rows):
buff = ParsedFederationStreamData( buff = ParsedFederationStreamData(
presence=[], presence=[],
presence_destinations=[],
keyed_edus={}, keyed_edus={},
edus={}, edus={},
device_destinations=set(), device_destinations=set(),
@ -476,6 +540,11 @@ def process_rows_for_federation(transaction_queue, rows):
if buff.presence: if buff.presence:
transaction_queue.send_presence(buff.presence) transaction_queue.send_presence(buff.presence)
for state, destinations in buff.presence_destinations:
transaction_queue.send_presence_to_destinations(
states=[state], destinations=destinations,
)
for destination, edu_map in iteritems(buff.keyed_edus): for destination, edu_map in iteritems(buff.keyed_edus):
for key, edu in edu_map.items(): for key, edu in edu_map.items():
transaction_queue.send_edu(edu, key) transaction_queue.send_edu(edu, key)

View File

@ -371,7 +371,7 @@ class FederationSender(object):
return return
# First we queue up the new presence by user ID, so multiple presence # First we queue up the new presence by user ID, so multiple presence
# updates in quick successtion are correctly handled # updates in quick succession are correctly handled.
# We only want to send presence for our own users, so lets always just # We only want to send presence for our own users, so lets always just
# filter here just in case. # filter here just in case.
self.pending_presence.update({ self.pending_presence.update({
@ -402,6 +402,23 @@ class FederationSender(object):
finally: finally:
self._processing_pending_presence = False self._processing_pending_presence = False
def send_presence_to_destinations(self, states, destinations):
"""Send the given presence states to the given destinations.
Args:
states (list[UserPresenceState])
destinations (list[str])
"""
if not states or not self.hs.config.use_presence:
# No-op if presence is disabled.
return
for destination in destinations:
if destination == self.server_name:
continue
self._get_per_destination_queue(destination).send_presence(states)
@measure_func("txnqueue._process_presence") @measure_func("txnqueue._process_presence")
@defer.inlineCallbacks @defer.inlineCallbacks
def _process_presence_inner(self, states): def _process_presence_inner(self, states):

View File

@ -21,8 +21,8 @@ import re
from twisted.internet import defer from twisted.internet import defer
import synapse import synapse
from synapse.api.constants import RoomVersions
from synapse.api.errors import Codes, FederationDeniedError, SynapseError from synapse.api.errors import Codes, FederationDeniedError, SynapseError
from synapse.api.room_versions import RoomVersions
from synapse.api.urls import FEDERATION_V1_PREFIX, FEDERATION_V2_PREFIX from synapse.api.urls import FEDERATION_V1_PREFIX, FEDERATION_V2_PREFIX
from synapse.http.endpoint import parse_and_validate_server_name from synapse.http.endpoint import parse_and_validate_server_name
from synapse.http.server import JsonResource from synapse.http.server import JsonResource
@ -513,7 +513,7 @@ class FederationV1InviteServlet(BaseFederationServlet):
# state resolution algorithm, and we don't use that for processing # state resolution algorithm, and we don't use that for processing
# invites # invites
content = yield self.handler.on_invite_request( content = yield self.handler.on_invite_request(
origin, content, room_version=RoomVersions.V1, origin, content, room_version=RoomVersions.V1.identifier,
) )
# V1 federation API is defined to return a content of `[200, {...}]` # V1 federation API is defined to return a content of `[200, {...}]`

View File

@ -22,6 +22,7 @@ from twisted.internet import defer
from synapse.api.errors import SynapseError from synapse.api.errors import SynapseError
from synapse.types import GroupID, RoomID, UserID, get_domain_from_id from synapse.types import GroupID, RoomID, UserID, get_domain_from_id
from synapse.util.async_helpers import concurrently_execute
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -896,6 +897,78 @@ class GroupsServerHandler(object):
"group_id": group_id, "group_id": group_id,
}) })
@defer.inlineCallbacks
def delete_group(self, group_id, requester_user_id):
"""Deletes a group, kicking out all current members.
Only group admins or server admins can call this request
Args:
group_id (str)
request_user_id (str)
Returns:
Deferred
"""
yield self.check_group_is_ours(
group_id, requester_user_id,
and_exists=True,
)
# Only server admins or group admins can delete groups.
is_admin = yield self.store.is_user_admin_in_group(
group_id, requester_user_id
)
if not is_admin:
is_admin = yield self.auth.is_server_admin(
UserID.from_string(requester_user_id),
)
if not is_admin:
raise SynapseError(403, "User is not an admin")
# Before deleting the group lets kick everyone out of it
users = yield self.store.get_users_in_group(
group_id, include_private=True,
)
@defer.inlineCallbacks
def _kick_user_from_group(user_id):
if self.hs.is_mine_id(user_id):
groups_local = self.hs.get_groups_local_handler()
yield groups_local.user_removed_from_group(group_id, user_id, {})
else:
yield self.transport_client.remove_user_from_group_notification(
get_domain_from_id(user_id), group_id, user_id, {}
)
yield self.store.maybe_delete_remote_profile_cache(user_id)
# We kick users out in the order of:
# 1. Non-admins
# 2. Other admins
# 3. The requester
#
# This is so that if the deletion fails for some reason other admins or
# the requester still has auth to retry.
non_admins = []
admins = []
for u in users:
if u["user_id"] == requester_user_id:
continue
if u["is_admin"]:
admins.append(u["user_id"])
else:
non_admins.append(u["user_id"])
yield concurrently_execute(_kick_user_from_group, non_admins, 10)
yield concurrently_execute(_kick_user_from_group, admins, 10)
yield _kick_user_from_group(requester_user_id)
yield self.store.delete_group(group_id)
def _parse_join_policy_from_contents(content): def _parse_join_policy_from_contents(content):
"""Given a content for a request, return the specified join policy or None """Given a content for a request, return the specified join policy or None

View File

@ -912,7 +912,7 @@ class AuthHandler(BaseHandler):
) )
@defer.inlineCallbacks @defer.inlineCallbacks
def delete_threepid(self, user_id, medium, address): def delete_threepid(self, user_id, medium, address, id_server=None):
"""Attempts to unbind the 3pid on the identity servers and deletes it """Attempts to unbind the 3pid on the identity servers and deletes it
from the local database. from the local database.
@ -920,6 +920,10 @@ class AuthHandler(BaseHandler):
user_id (str) user_id (str)
medium (str) medium (str)
address (str) address (str)
id_server (str|None): Use the given identity server when unbinding
any threepids. If None then will attempt to unbind using the
identity server specified when binding (if known).
Returns: Returns:
Deferred[bool]: Returns True if successfully unbound the 3pid on Deferred[bool]: Returns True if successfully unbound the 3pid on
@ -937,6 +941,7 @@ class AuthHandler(BaseHandler):
{ {
'medium': medium, 'medium': medium,
'address': address, 'address': address,
'id_server': id_server,
}, },
) )

View File

@ -43,12 +43,15 @@ class DeactivateAccountHandler(BaseHandler):
hs.get_reactor().callWhenRunning(self._start_user_parting) hs.get_reactor().callWhenRunning(self._start_user_parting)
@defer.inlineCallbacks @defer.inlineCallbacks
def deactivate_account(self, user_id, erase_data): def deactivate_account(self, user_id, erase_data, id_server=None):
"""Deactivate a user's account """Deactivate a user's account
Args: Args:
user_id (str): ID of user to be deactivated user_id (str): ID of user to be deactivated
erase_data (bool): whether to GDPR-erase the user's data erase_data (bool): whether to GDPR-erase the user's data
id_server (str|None): Use the given identity server when unbinding
any threepids. If None then will attempt to unbind using the
identity server specified when binding (if known).
Returns: Returns:
Deferred[bool]: True if identity server supports removing Deferred[bool]: True if identity server supports removing
@ -74,6 +77,7 @@ class DeactivateAccountHandler(BaseHandler):
{ {
'medium': threepid['medium'], 'medium': threepid['medium'],
'address': threepid['address'], 'address': threepid['address'],
'id_server': id_server,
}, },
) )
identity_server_supports_unbinding &= result identity_server_supports_unbinding &= result

View File

@ -68,7 +68,7 @@ class DirectoryHandler(BaseHandler):
# TODO(erikj): Add transactions. # TODO(erikj): Add transactions.
# TODO(erikj): Check if there is a current association. # TODO(erikj): Check if there is a current association.
if not servers: if not servers:
users = yield self.state.get_current_user_in_room(room_id) users = yield self.state.get_current_users_in_room(room_id)
servers = set(get_domain_from_id(u) for u in users) servers = set(get_domain_from_id(u) for u in users)
if not servers: if not servers:
@ -268,7 +268,7 @@ class DirectoryHandler(BaseHandler):
Codes.NOT_FOUND Codes.NOT_FOUND
) )
users = yield self.state.get_current_user_in_room(room_id) users = yield self.state.get_current_users_in_room(room_id)
extra_servers = set(get_domain_from_id(u) for u in users) extra_servers = set(get_domain_from_id(u) for u in users)
servers = set(extra_servers) | set(servers) servers = set(extra_servers) | set(servers)

View File

@ -102,7 +102,7 @@ class EventStreamHandler(BaseHandler):
# Send down presence. # Send down presence.
if event.state_key == auth_user_id: if event.state_key == auth_user_id:
# Send down presence for everyone in the room. # Send down presence for everyone in the room.
users = yield self.state.get_current_user_in_room(event.room_id) users = yield self.state.get_current_users_in_room(event.room_id)
states = yield presence_handler.get_states( states = yield presence_handler.get_states(
users, users,
as_event=True, as_event=True,

View File

@ -29,13 +29,7 @@ from unpaddedbase64 import decode_base64
from twisted.internet import defer from twisted.internet import defer
from synapse.api.constants import ( from synapse.api.constants import EventTypes, Membership, RejectedReason
KNOWN_ROOM_VERSIONS,
EventTypes,
Membership,
RejectedReason,
RoomVersions,
)
from synapse.api.errors import ( from synapse.api.errors import (
AuthError, AuthError,
CodeMessageException, CodeMessageException,
@ -44,6 +38,7 @@ from synapse.api.errors import (
StoreError, StoreError,
SynapseError, SynapseError,
) )
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersions
from synapse.crypto.event_signing import compute_event_signature from synapse.crypto.event_signing import compute_event_signature
from synapse.event_auth import auth_types_for_event from synapse.event_auth import auth_types_for_event
from synapse.events.validator import EventValidator from synapse.events.validator import EventValidator
@ -1733,7 +1728,9 @@ class FederationHandler(BaseHandler):
# invalid, and it would fail auth checks anyway. # invalid, and it would fail auth checks anyway.
raise SynapseError(400, "No create event in state") raise SynapseError(400, "No create event in state")
room_version = create_event.content.get("room_version", RoomVersions.V1) room_version = create_event.content.get(
"room_version", RoomVersions.V1.identifier,
)
missing_auth_events = set() missing_auth_events = set()
for e in itertools.chain(auth_events, state, [event]): for e in itertools.chain(auth_events, state, [event]):

View File

@ -132,6 +132,14 @@ class IdentityHandler(BaseHandler):
} }
) )
logger.debug("bound threepid %r to %s", creds, mxid) logger.debug("bound threepid %r to %s", creds, mxid)
# Remember where we bound the threepid
yield self.store.add_user_bound_threepid(
user_id=mxid,
medium=data["medium"],
address=data["address"],
id_server=id_server,
)
except CodeMessageException as e: except CodeMessageException as e:
data = json.loads(e.msg) # XXX WAT? data = json.loads(e.msg) # XXX WAT?
defer.returnValue(data) defer.returnValue(data)
@ -140,9 +148,48 @@ class IdentityHandler(BaseHandler):
def try_unbind_threepid(self, mxid, threepid): def try_unbind_threepid(self, mxid, threepid):
"""Removes a binding from an identity server """Removes a binding from an identity server
Args:
mxid (str): Matrix user ID of binding to be removed
threepid (dict): Dict with medium & address of binding to be
removed, and an optional id_server.
Raises:
SynapseError: If we failed to contact the identity server
Returns:
Deferred[bool]: True on success, otherwise False if the identity
server doesn't support unbinding (or no identity server found to
contact).
"""
if threepid.get("id_server"):
id_servers = [threepid["id_server"]]
else:
id_servers = yield self.store.get_id_servers_user_bound(
user_id=mxid,
medium=threepid["medium"],
address=threepid["address"],
)
# We don't know where to unbind, so we don't have a choice but to return
if not id_servers:
defer.returnValue(False)
changed = True
for id_server in id_servers:
changed &= yield self.try_unbind_threepid_with_id_server(
mxid, threepid, id_server,
)
defer.returnValue(changed)
@defer.inlineCallbacks
def try_unbind_threepid_with_id_server(self, mxid, threepid, id_server):
"""Removes a binding from an identity server
Args: Args:
mxid (str): Matrix user ID of binding to be removed mxid (str): Matrix user ID of binding to be removed
threepid (dict): Dict with medium & address of binding to be removed threepid (dict): Dict with medium & address of binding to be removed
id_server (str): Identity server to unbind from
Raises: Raises:
SynapseError: If we failed to contact the identity server SynapseError: If we failed to contact the identity server
@ -151,21 +198,13 @@ class IdentityHandler(BaseHandler):
Deferred[bool]: True on success, otherwise False if the identity Deferred[bool]: True on success, otherwise False if the identity
server doesn't support unbinding server doesn't support unbinding
""" """
logger.debug("unbinding threepid %r from %s", threepid, mxid)
if not self.trusted_id_servers:
logger.warn("Can't unbind threepid: no trusted ID servers set in config")
defer.returnValue(False)
# We don't track what ID server we added 3pids on (perhaps we ought to)
# but we assume that any of the servers in the trusted list are in the
# same ID server federation, so we can pick any one of them to send the
# deletion request to.
id_server = next(iter(self.trusted_id_servers))
url = "https://%s/_matrix/identity/api/v1/3pid/unbind" % (id_server,) url = "https://%s/_matrix/identity/api/v1/3pid/unbind" % (id_server,)
content = { content = {
"mxid": mxid, "mxid": mxid,
"threepid": threepid, "threepid": {
"medium": threepid["medium"],
"address": threepid["address"],
},
} }
# we abuse the federation http client to sign the request, but we have to send it # we abuse the federation http client to sign the request, but we have to send it
@ -188,16 +227,24 @@ class IdentityHandler(BaseHandler):
content, content,
headers, headers,
) )
changed = True
except HttpResponseException as e: except HttpResponseException as e:
changed = False
if e.code in (400, 404, 501,): if e.code in (400, 404, 501,):
# The remote server probably doesn't support unbinding (yet) # The remote server probably doesn't support unbinding (yet)
logger.warn("Received %d response while unbinding threepid", e.code) logger.warn("Received %d response while unbinding threepid", e.code)
defer.returnValue(False)
else: else:
logger.error("Failed to unbind threepid on identity server: %s", e) logger.error("Failed to unbind threepid on identity server: %s", e)
raise SynapseError(502, "Failed to contact identity server") raise SynapseError(502, "Failed to contact identity server")
defer.returnValue(True) yield self.store.remove_user_bound_threepid(
user_id=mxid,
medium=threepid["medium"],
address=threepid["address"],
id_server=id_server,
)
defer.returnValue(changed)
@defer.inlineCallbacks @defer.inlineCallbacks
def requestEmailToken(self, id_server, email, client_secret, send_attempt, **kwargs): def requestEmailToken(self, id_server, email, client_secret, send_attempt, **kwargs):

View File

@ -22,7 +22,7 @@ from canonicaljson import encode_canonical_json, json
from twisted.internet import defer from twisted.internet import defer
from twisted.internet.defer import succeed from twisted.internet.defer import succeed
from synapse.api.constants import EventTypes, Membership, RoomVersions from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import ( from synapse.api.errors import (
AuthError, AuthError,
Codes, Codes,
@ -30,6 +30,7 @@ from synapse.api.errors import (
NotFoundError, NotFoundError,
SynapseError, SynapseError,
) )
from synapse.api.room_versions import RoomVersions
from synapse.api.urls import ConsentURIBuilder from synapse.api.urls import ConsentURIBuilder
from synapse.events.utils import serialize_event from synapse.events.utils import serialize_event
from synapse.events.validator import EventValidator from synapse.events.validator import EventValidator
@ -191,7 +192,7 @@ class MessageHandler(object):
"Getting joined members after leaving is not implemented" "Getting joined members after leaving is not implemented"
) )
users_with_profile = yield self.state.get_current_user_in_room(room_id) users_with_profile = yield self.state.get_current_users_in_room(room_id)
# If this is an AS, double check that they are allowed to see the members. # If this is an AS, double check that they are allowed to see the members.
# This can either be because the AS user is in the room or because there # This can either be because the AS user is in the room or because there
@ -603,7 +604,9 @@ class EventCreationHandler(object):
""" """
if event.is_state() and (event.type, event.state_key) == (EventTypes.Create, ""): if event.is_state() and (event.type, event.state_key) == (EventTypes.Create, ""):
room_version = event.content.get("room_version", RoomVersions.V1) room_version = event.content.get(
"room_version", RoomVersions.V1.identifier
)
else: else:
room_version = yield self.store.get_room_version(event.room_id) room_version = yield self.store.get_room_version(event.room_id)

View File

@ -31,9 +31,11 @@ from prometheus_client import Counter
from twisted.internet import defer from twisted.internet import defer
from synapse.api.constants import PresenceState import synapse.metrics
from synapse.api.constants import EventTypes, Membership, PresenceState
from synapse.api.errors import SynapseError from synapse.api.errors import SynapseError
from synapse.metrics import LaterGauge from synapse.metrics import LaterGauge
from synapse.metrics.background_process_metrics import run_as_background_process
from synapse.storage.presence import UserPresenceState from synapse.storage.presence import UserPresenceState
from synapse.types import UserID, get_domain_from_id from synapse.types import UserID, get_domain_from_id
from synapse.util.async_helpers import Linearizer from synapse.util.async_helpers import Linearizer
@ -98,6 +100,7 @@ class PresenceHandler(object):
self.hs = hs self.hs = hs
self.is_mine = hs.is_mine self.is_mine = hs.is_mine
self.is_mine_id = hs.is_mine_id self.is_mine_id = hs.is_mine_id
self.server_name = hs.hostname
self.clock = hs.get_clock() self.clock = hs.get_clock()
self.store = hs.get_datastore() self.store = hs.get_datastore()
self.wheel_timer = WheelTimer() self.wheel_timer = WheelTimer()
@ -110,30 +113,6 @@ class PresenceHandler(object):
federation_registry.register_edu_handler( federation_registry.register_edu_handler(
"m.presence", self.incoming_presence "m.presence", self.incoming_presence
) )
federation_registry.register_edu_handler(
"m.presence_invite",
lambda origin, content: self.invite_presence(
observed_user=UserID.from_string(content["observed_user"]),
observer_user=UserID.from_string(content["observer_user"]),
)
)
federation_registry.register_edu_handler(
"m.presence_accept",
lambda origin, content: self.accept_presence(
observed_user=UserID.from_string(content["observed_user"]),
observer_user=UserID.from_string(content["observer_user"]),
)
)
federation_registry.register_edu_handler(
"m.presence_deny",
lambda origin, content: self.deny_presence(
observed_user=UserID.from_string(content["observed_user"]),
observer_user=UserID.from_string(content["observer_user"]),
)
)
distributor = hs.get_distributor()
distributor.observe("user_joined_room", self.user_joined_room)
active_presence = self.store.take_presence_startup_info() active_presence = self.store.take_presence_startup_info()
@ -220,6 +199,15 @@ class PresenceHandler(object):
LaterGauge("synapse_handlers_presence_wheel_timer_size", "", [], LaterGauge("synapse_handlers_presence_wheel_timer_size", "", [],
lambda: len(self.wheel_timer)) lambda: len(self.wheel_timer))
# Used to handle sending of presence to newly joined users/servers
if hs.config.use_presence:
self.notifier.add_replication_callback(self.notify_new_event)
# Presence is best effort and quickly heals itself, so lets just always
# stream from the current state when we restart.
self._event_pos = self.store.get_current_events_token()
self._event_processing = False
@defer.inlineCallbacks @defer.inlineCallbacks
def _on_shutdown(self): def _on_shutdown(self):
"""Gets called when shutting down. This lets us persist any updates that """Gets called when shutting down. This lets us persist any updates that
@ -750,162 +738,6 @@ class PresenceHandler(object):
yield self._update_states([prev_state.copy_and_replace(**new_fields)]) yield self._update_states([prev_state.copy_and_replace(**new_fields)])
@defer.inlineCallbacks
def user_joined_room(self, user, room_id):
"""Called (via the distributor) when a user joins a room. This funciton
sends presence updates to servers, either:
1. the joining user is a local user and we send their presence to
all servers in the room.
2. the joining user is a remote user and so we send presence for all
local users in the room.
"""
# We only need to send presence to servers that don't have it yet. We
# don't need to send to local clients here, as that is done as part
# of the event stream/sync.
# TODO: Only send to servers not already in the room.
if self.is_mine(user):
state = yield self.current_state_for_user(user.to_string())
self._push_to_remotes([state])
else:
user_ids = yield self.store.get_users_in_room(room_id)
user_ids = list(filter(self.is_mine_id, user_ids))
states = yield self.current_state_for_users(user_ids)
self._push_to_remotes(list(states.values()))
@defer.inlineCallbacks
def get_presence_list(self, observer_user, accepted=None):
"""Returns the presence for all users in their presence list.
"""
if not self.is_mine(observer_user):
raise SynapseError(400, "User is not hosted on this Home Server")
presence_list = yield self.store.get_presence_list(
observer_user.localpart, accepted=accepted
)
results = yield self.get_states(
target_user_ids=[row["observed_user_id"] for row in presence_list],
as_event=False,
)
now = self.clock.time_msec()
results[:] = [format_user_presence_state(r, now) for r in results]
is_accepted = {
row["observed_user_id"]: row["accepted"] for row in presence_list
}
for result in results:
result.update({
"accepted": is_accepted,
})
defer.returnValue(results)
@defer.inlineCallbacks
def send_presence_invite(self, observer_user, observed_user):
"""Sends a presence invite.
"""
yield self.store.add_presence_list_pending(
observer_user.localpart, observed_user.to_string()
)
if self.is_mine(observed_user):
yield self.invite_presence(observed_user, observer_user)
else:
yield self.federation.build_and_send_edu(
destination=observed_user.domain,
edu_type="m.presence_invite",
content={
"observed_user": observed_user.to_string(),
"observer_user": observer_user.to_string(),
}
)
@defer.inlineCallbacks
def invite_presence(self, observed_user, observer_user):
"""Handles new presence invites.
"""
if not self.is_mine(observed_user):
raise SynapseError(400, "User is not hosted on this Home Server")
# TODO: Don't auto accept
if self.is_mine(observer_user):
yield self.accept_presence(observed_user, observer_user)
else:
self.federation.build_and_send_edu(
destination=observer_user.domain,
edu_type="m.presence_accept",
content={
"observed_user": observed_user.to_string(),
"observer_user": observer_user.to_string(),
}
)
state_dict = yield self.get_state(observed_user, as_event=False)
state_dict = format_user_presence_state(state_dict, self.clock.time_msec())
self.federation.build_and_send_edu(
destination=observer_user.domain,
edu_type="m.presence",
content={
"push": [state_dict]
}
)
@defer.inlineCallbacks
def accept_presence(self, observed_user, observer_user):
"""Handles a m.presence_accept EDU. Mark a presence invite from a
local or remote user as accepted in a local user's presence list.
Starts polling for presence updates from the local or remote user.
Args:
observed_user(UserID): The user to update in the presence list.
observer_user(UserID): The owner of the presence list to update.
"""
yield self.store.set_presence_list_accepted(
observer_user.localpart, observed_user.to_string()
)
@defer.inlineCallbacks
def deny_presence(self, observed_user, observer_user):
"""Handle a m.presence_deny EDU. Removes a local or remote user from a
local user's presence list.
Args:
observed_user(UserID): The local or remote user to remove from the
list.
observer_user(UserID): The local owner of the presence list.
Returns:
A Deferred.
"""
yield self.store.del_presence_list(
observer_user.localpart, observed_user.to_string()
)
# TODO(paul): Inform the user somehow?
@defer.inlineCallbacks
def drop(self, observed_user, observer_user):
"""Remove a local or remote user from a local user's presence list and
unsubscribe the local user from updates that user.
Args:
observed_user(UserId): The local or remote user to remove from the
list.
observer_user(UserId): The local owner of the presence list.
Returns:
A Deferred.
"""
if not self.is_mine(observer_user):
raise SynapseError(400, "User is not hosted on this Home Server")
yield self.store.del_presence_list(
observer_user.localpart, observed_user.to_string()
)
# TODO: Inform the remote that we've dropped the presence list.
@defer.inlineCallbacks @defer.inlineCallbacks
def is_visible(self, observed_user, observer_user): def is_visible(self, observed_user, observer_user):
"""Returns whether a user can see another user's presence. """Returns whether a user can see another user's presence.
@ -920,11 +752,7 @@ class PresenceHandler(object):
if observer_room_ids & observed_room_ids: if observer_room_ids & observed_room_ids:
defer.returnValue(True) defer.returnValue(True)
accepted_observers = yield self.store.get_presence_list_observers_accepted( defer.returnValue(False)
observed_user.to_string()
)
defer.returnValue(observer_user.to_string() in accepted_observers)
@defer.inlineCallbacks @defer.inlineCallbacks
def get_all_presence_updates(self, last_id, current_id): def get_all_presence_updates(self, last_id, current_id):
@ -945,6 +773,140 @@ class PresenceHandler(object):
rows = yield self.store.get_all_presence_updates(last_id, current_id) rows = yield self.store.get_all_presence_updates(last_id, current_id)
defer.returnValue(rows) defer.returnValue(rows)
def notify_new_event(self):
"""Called when new events have happened. Handles users and servers
joining rooms and require being sent presence.
"""
if self._event_processing:
return
@defer.inlineCallbacks
def _process_presence():
assert not self._event_processing
self._event_processing = True
try:
yield self._unsafe_process()
finally:
self._event_processing = False
run_as_background_process("presence.notify_new_event", _process_presence)
@defer.inlineCallbacks
def _unsafe_process(self):
# Loop round handling deltas until we're up to date
while True:
with Measure(self.clock, "presence_delta"):
deltas = yield self.store.get_current_state_deltas(self._event_pos)
if not deltas:
return
yield self._handle_state_delta(deltas)
self._event_pos = deltas[-1]["stream_id"]
# Expose current event processing position to prometheus
synapse.metrics.event_processing_positions.labels("presence").set(
self._event_pos
)
@defer.inlineCallbacks
def _handle_state_delta(self, deltas):
"""Process current state deltas to find new joins that need to be
handled.
"""
for delta in deltas:
typ = delta["type"]
state_key = delta["state_key"]
room_id = delta["room_id"]
event_id = delta["event_id"]
prev_event_id = delta["prev_event_id"]
logger.debug("Handling: %r %r, %s", typ, state_key, event_id)
if typ != EventTypes.Member:
continue
event = yield self.store.get_event(event_id)
if event.content.get("membership") != Membership.JOIN:
# We only care about joins
continue
if prev_event_id:
prev_event = yield self.store.get_event(prev_event_id)
if prev_event.content.get("membership") == Membership.JOIN:
# Ignore changes to join events.
continue
yield self._on_user_joined_room(room_id, state_key)
@defer.inlineCallbacks
def _on_user_joined_room(self, room_id, user_id):
"""Called when we detect a user joining the room via the current state
delta stream.
Args:
room_id (str)
user_id (str)
Returns:
Deferred
"""
if self.is_mine_id(user_id):
# If this is a local user then we need to send their presence
# out to hosts in the room (who don't already have it)
# TODO: We should be able to filter the hosts down to those that
# haven't previously seen the user
state = yield self.current_state_for_user(user_id)
hosts = yield self.state.get_current_hosts_in_room(room_id)
# Filter out ourselves.
hosts = set(host for host in hosts if host != self.server_name)
self.federation.send_presence_to_destinations(
states=[state],
destinations=hosts,
)
else:
# A remote user has joined the room, so we need to:
# 1. Check if this is a new server in the room
# 2. If so send any presence they don't already have for
# local users in the room.
# TODO: We should be able to filter the users down to those that
# the server hasn't previously seen
# TODO: Check that this is actually a new server joining the
# room.
user_ids = yield self.state.get_current_users_in_room(room_id)
user_ids = list(filter(self.is_mine_id, user_ids))
states = yield self.current_state_for_users(user_ids)
# Filter out old presence, i.e. offline presence states where
# the user hasn't been active for a week. We can change this
# depending on what we want the UX to be, but at the least we
# should filter out offline presence where the state is just the
# default state.
now = self.clock.time_msec()
states = [
state for state in states.values()
if state.state != PresenceState.OFFLINE
or now - state.last_active_ts < 7 * 24 * 60 * 60 * 1000
or state.status_msg is not None
]
if states:
self.federation.send_presence_to_destinations(
states=states,
destinations=[get_domain_from_id(user_id)],
)
def should_notify(old_state, new_state): def should_notify(old_state, new_state):
"""Decides if a presence state change should be sent to interested parties. """Decides if a presence state change should be sent to interested parties.
@ -1086,10 +1048,7 @@ class PresenceEventSource(object):
updates for updates for
""" """
user_id = user.to_string() user_id = user.to_string()
plist = yield self.store.get_presence_list_accepted( users_interested_in = set()
user.localpart, on_invalidate=cache_context.invalidate,
)
users_interested_in = set(row["observed_user_id"] for row in plist)
users_interested_in.add(user_id) # So that we receive our own presence users_interested_in.add(user_id) # So that we receive our own presence
users_who_share_room = yield self.store.get_users_who_share_room_with_user( users_who_share_room = yield self.store.get_users_who_share_room_with_user(
@ -1294,10 +1253,6 @@ def get_interested_parties(store, states):
for room_id in room_ids: for room_id in room_ids:
room_ids_to_states.setdefault(room_id, []).append(state) room_ids_to_states.setdefault(room_id, []).append(state)
plist = yield store.get_presence_list_observers_accepted(state.user_id)
for u in plist:
users_to_states.setdefault(u, []).append(state)
# Always notify self # Always notify self
users_to_states.setdefault(state.user_id, []).append(state) users_to_states.setdefault(state.user_id, []).append(state)

View File

@ -153,6 +153,7 @@ class RegistrationHandler(BaseHandler):
user_type=None, user_type=None,
default_display_name=None, default_display_name=None,
address=None, address=None,
bind_emails=[],
): ):
"""Registers a new client on the server. """Registers a new client on the server.
@ -172,6 +173,7 @@ class RegistrationHandler(BaseHandler):
default_display_name (unicode|None): if set, the new user's displayname default_display_name (unicode|None): if set, the new user's displayname
will be set to this. Defaults to 'localpart'. will be set to this. Defaults to 'localpart'.
address (str|None): the IP address used to perform the registration. address (str|None): the IP address used to perform the registration.
bind_emails (List[str]): list of emails to bind to this account.
Returns: Returns:
A tuple of (user_id, access_token). A tuple of (user_id, access_token).
Raises: Raises:
@ -261,6 +263,21 @@ class RegistrationHandler(BaseHandler):
if not self.hs.config.user_consent_at_registration: if not self.hs.config.user_consent_at_registration:
yield self._auto_join_rooms(user_id) yield self._auto_join_rooms(user_id)
# Bind any specified emails to this account
current_time = self.hs.get_clock().time_msec()
for email in bind_emails:
# generate threepid dict
threepid_dict = {
"medium": "email",
"address": email,
"validated_at": current_time,
}
# Bind email to new account
yield self._register_email_threepid(
user_id, threepid_dict, None, False,
)
defer.returnValue((user_id, token)) defer.returnValue((user_id, token))
@defer.inlineCallbacks @defer.inlineCallbacks

View File

@ -25,14 +25,9 @@ from six import iteritems, string_types
from twisted.internet import defer from twisted.internet import defer
from synapse.api.constants import ( from synapse.api.constants import EventTypes, JoinRules, RoomCreationPreset
DEFAULT_ROOM_VERSION,
KNOWN_ROOM_VERSIONS,
EventTypes,
JoinRules,
RoomCreationPreset,
)
from synapse.api.errors import AuthError, Codes, NotFoundError, StoreError, SynapseError from synapse.api.errors import AuthError, Codes, NotFoundError, StoreError, SynapseError
from synapse.api.room_versions import DEFAULT_ROOM_VERSION, KNOWN_ROOM_VERSIONS
from synapse.storage.state import StateFilter from synapse.storage.state import StateFilter
from synapse.types import RoomAlias, RoomID, RoomStreamToken, StreamToken, UserID from synapse.types import RoomAlias, RoomID, RoomStreamToken, StreamToken, UserID
from synapse.util import stringutils from synapse.util import stringutils
@ -285,6 +280,7 @@ class RoomCreationHandler(BaseHandler):
(EventTypes.RoomAvatar, ""), (EventTypes.RoomAvatar, ""),
(EventTypes.Encryption, ""), (EventTypes.Encryption, ""),
(EventTypes.ServerACL, ""), (EventTypes.ServerACL, ""),
(EventTypes.RelatedGroups, ""),
) )
old_room_state_ids = yield self.store.get_filtered_current_state_ids( old_room_state_ids = yield self.store.get_filtered_current_state_ids(
@ -479,7 +475,7 @@ class RoomCreationHandler(BaseHandler):
if ratelimit: if ratelimit:
yield self.ratelimit(requester) yield self.ratelimit(requester)
room_version = config.get("room_version", DEFAULT_ROOM_VERSION) room_version = config.get("room_version", DEFAULT_ROOM_VERSION.identifier)
if not isinstance(room_version, string_types): if not isinstance(room_version, string_types):
raise SynapseError( raise SynapseError(
400, 400,

View File

@ -167,7 +167,7 @@ class RoomListHandler(BaseHandler):
if not latest_event_ids: if not latest_event_ids:
return return
joined_users = yield self.state_handler.get_current_user_in_room( joined_users = yield self.state_handler.get_current_users_in_room(
room_id, latest_event_ids, room_id, latest_event_ids,
) )

View File

@ -70,6 +70,7 @@ class RoomMemberHandler(object):
self.clock = hs.get_clock() self.clock = hs.get_clock()
self.spam_checker = hs.get_spam_checker() self.spam_checker = hs.get_spam_checker()
self._server_notices_mxid = self.config.server_notices_mxid self._server_notices_mxid = self.config.server_notices_mxid
self._enable_lookup = hs.config.enable_3pid_lookup
@abc.abstractmethod @abc.abstractmethod
def _remote_join(self, requester, remote_room_hosts, room_id, user, content): def _remote_join(self, requester, remote_room_hosts, room_id, user, content):
@ -421,6 +422,9 @@ class RoomMemberHandler(object):
room_id, latest_event_ids=latest_event_ids, room_id, latest_event_ids=latest_event_ids,
) )
# TODO: Refactor into dictionary of explicitly allowed transitions
# between old and new state, with specific error messages for some
# transitions and generic otherwise
old_state_id = current_state_ids.get((EventTypes.Member, target.to_string())) old_state_id = current_state_ids.get((EventTypes.Member, target.to_string()))
if old_state_id: if old_state_id:
old_state = yield self.store.get_event(old_state_id, allow_none=True) old_state = yield self.store.get_event(old_state_id, allow_none=True)
@ -446,6 +450,9 @@ class RoomMemberHandler(object):
if same_sender and same_membership and same_content: if same_sender and same_membership and same_content:
defer.returnValue(old_state) defer.returnValue(old_state)
if old_membership in ["ban", "leave"] and action == "kick":
raise AuthError(403, "The target user is not in the room")
# we don't allow people to reject invites to the server notice # we don't allow people to reject invites to the server notice
# room, but they can leave it once they are joined. # room, but they can leave it once they are joined.
if ( if (
@ -459,6 +466,9 @@ class RoomMemberHandler(object):
"You cannot reject this invite", "You cannot reject this invite",
errcode=Codes.CANNOT_LEAVE_SERVER_NOTICE_ROOM, errcode=Codes.CANNOT_LEAVE_SERVER_NOTICE_ROOM,
) )
else:
if action == "kick":
raise AuthError(403, "The target user is not in the room")
is_host_in_room = yield self._is_host_in_room(current_state_ids) is_host_in_room = yield self._is_host_in_room(current_state_ids)
@ -729,6 +739,10 @@ class RoomMemberHandler(object):
Returns: Returns:
str: the matrix ID of the 3pid, or None if it is not recognized. str: the matrix ID of the 3pid, or None if it is not recognized.
""" """
if not self._enable_lookup:
raise SynapseError(
403, "Looking up third-party identifiers is denied from this server",
)
try: try:
data = yield self.simple_http_client.get_json( data = yield self.simple_http_client.get_json(
"%s%s/_matrix/identity/api/v1/lookup" % (id_server_scheme, id_server,), "%s%s/_matrix/identity/api/v1/lookup" % (id_server_scheme, id_server,),

View File

@ -1049,11 +1049,11 @@ class SyncHandler(object):
# TODO: Be more clever than this, i.e. remove users who we already # TODO: Be more clever than this, i.e. remove users who we already
# share a room with? # share a room with?
for room_id in newly_joined_rooms: for room_id in newly_joined_rooms:
joined_users = yield self.state.get_current_user_in_room(room_id) joined_users = yield self.state.get_current_users_in_room(room_id)
newly_joined_users.update(joined_users) newly_joined_users.update(joined_users)
for room_id in newly_left_rooms: for room_id in newly_left_rooms:
left_users = yield self.state.get_current_user_in_room(room_id) left_users = yield self.state.get_current_users_in_room(room_id)
newly_left_users.update(left_users) newly_left_users.update(left_users)
# TODO: Check that these users are actually new, i.e. either they # TODO: Check that these users are actually new, i.e. either they
@ -1213,7 +1213,7 @@ class SyncHandler(object):
extra_users_ids = set(newly_joined_users) extra_users_ids = set(newly_joined_users)
for room_id in newly_joined_rooms: for room_id in newly_joined_rooms:
users = yield self.state.get_current_user_in_room(room_id) users = yield self.state.get_current_users_in_room(room_id)
extra_users_ids.update(users) extra_users_ids.update(users)
extra_users_ids.discard(user.to_string()) extra_users_ids.discard(user.to_string())
@ -1855,7 +1855,7 @@ class SyncHandler(object):
extrems = yield self.store.get_forward_extremeties_for_room( extrems = yield self.store.get_forward_extremeties_for_room(
room_id, stream_ordering, room_id, stream_ordering,
) )
users_in_room = yield self.state.get_current_user_in_room( users_in_room = yield self.state.get_current_users_in_room(
room_id, extrems, room_id, extrems,
) )
if user_id in users_in_room: if user_id in users_in_room:

View File

@ -218,7 +218,7 @@ class TypingHandler(object):
@defer.inlineCallbacks @defer.inlineCallbacks
def _push_remote(self, member, typing): def _push_remote(self, member, typing):
try: try:
users = yield self.state.get_current_user_in_room(member.room_id) users = yield self.state.get_current_users_in_room(member.room_id)
self._member_last_federation_poke[member] = self.clock.time_msec() self._member_last_federation_poke[member] = self.clock.time_msec()
now = self.clock.time_msec() now = self.clock.time_msec()
@ -261,7 +261,7 @@ class TypingHandler(object):
) )
return return
users = yield self.state.get_current_user_in_room(room_id) users = yield self.state.get_current_users_in_room(room_id)
domains = set(get_domain_from_id(u) for u in users) domains = set(get_domain_from_id(u) for u in users)
if self.server_name in domains: if self.server_name in domains:

View File

@ -276,7 +276,7 @@ class UserDirectoryHandler(StateDeltasHandler):
# ignore the change # ignore the change
return return
users_with_profile = yield self.state.get_current_user_in_room(room_id) users_with_profile = yield self.state.get_current_users_in_room(room_id)
# Remove every user from the sharing tables for that room. # Remove every user from the sharing tables for that room.
for user_id in iterkeys(users_with_profile): for user_id in iterkeys(users_with_profile):
@ -325,7 +325,7 @@ class UserDirectoryHandler(StateDeltasHandler):
room_id room_id
) )
# Now we update users who share rooms with users. # Now we update users who share rooms with users.
users_with_profile = yield self.state.get_current_user_in_room(room_id) users_with_profile = yield self.state.get_current_users_in_room(room_id)
if is_public: if is_public:
yield self.store.add_users_in_public_rooms(room_id, (user_id,)) yield self.store.add_users_in_public_rooms(room_id, (user_id,))

View File

@ -74,14 +74,14 @@ class ModuleApi(object):
return self._auth_handler.check_user_exists(user_id) return self._auth_handler.check_user_exists(user_id)
@defer.inlineCallbacks @defer.inlineCallbacks
def register(self, localpart, displayname=None): def register(self, localpart, displayname=None, emails=[]):
"""Registers a new user with given localpart and optional """Registers a new user with given localpart and optional
displayname. displayname, emails.
Args: Args:
localpart (str): The localpart of the new user. localpart (str): The localpart of the new user.
displayname (str|None): The displayname of the new user. If None, displayname (str|None): The displayname of the new user.
the user's displayname will default to `localpart`. emails (List[str]): Emails to bind to the new user.
Returns: Returns:
Deferred: a 2-tuple of (user_id, access_token) Deferred: a 2-tuple of (user_id, access_token)
@ -90,6 +90,7 @@ class ModuleApi(object):
reg = self.hs.get_registration_handler() reg = self.hs.get_registration_handler()
user_id, access_token = yield reg.register( user_id, access_token = yield reg.register(
localpart=localpart, default_display_name=displayname, localpart=localpart, default_display_name=displayname,
bind_emails=emails,
) )
defer.returnValue((user_id, access_token)) defer.returnValue((user_id, access_token))

View File

@ -72,8 +72,15 @@ class EmailPusher(object):
self._is_processing = False self._is_processing = False
def on_started(self): def on_started(self, should_check_for_notifs):
if self.mailer is not None: """Called when this pusher has been started.
Args:
should_check_for_notifs (bool): Whether we should immediately
check for push to send. Set to False only if it's known there
is nothing to send
"""
if should_check_for_notifs and self.mailer is not None:
self._start_processing() self._start_processing()
def on_stop(self): def on_stop(self):

View File

@ -112,7 +112,15 @@ class HttpPusher(object):
self.data_minus_url.update(self.data) self.data_minus_url.update(self.data)
del self.data_minus_url['url'] del self.data_minus_url['url']
def on_started(self): def on_started(self, should_check_for_notifs):
"""Called when this pusher has been started.
Args:
should_check_for_notifs (bool): Whether we should immediately
check for push to send. Set to False only if it's known there
is nothing to send
"""
if should_check_for_notifs:
self._start_processing() self._start_processing()
def on_new_notifications(self, min_stream_ordering, max_stream_ordering): def on_new_notifications(self, min_stream_ordering, max_stream_ordering):

View File

@ -21,6 +21,7 @@ from twisted.internet import defer
from synapse.metrics.background_process_metrics import run_as_background_process from synapse.metrics.background_process_metrics import run_as_background_process
from synapse.push import PusherConfigException from synapse.push import PusherConfigException
from synapse.push.pusher import PusherFactory from synapse.push.pusher import PusherFactory
from synapse.util.async_helpers import concurrently_execute
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -197,7 +198,7 @@ class PusherPool:
p = r p = r
if p: if p:
self._start_pusher(p) yield self._start_pusher(p)
@defer.inlineCallbacks @defer.inlineCallbacks
def _start_pushers(self): def _start_pushers(self):
@ -208,10 +209,14 @@ class PusherPool:
""" """
pushers = yield self.store.get_all_pushers() pushers = yield self.store.get_all_pushers()
logger.info("Starting %d pushers", len(pushers)) logger.info("Starting %d pushers", len(pushers))
for pusherdict in pushers:
self._start_pusher(pusherdict) # Stagger starting up the pushers so we don't completely drown the
# process on start up.
yield concurrently_execute(self._start_pusher, pushers, 10)
logger.info("Started pushers") logger.info("Started pushers")
@defer.inlineCallbacks
def _start_pusher(self, pusherdict): def _start_pusher(self, pusherdict):
"""Start the given pusher """Start the given pusher
@ -248,7 +253,22 @@ class PusherPool:
if appid_pushkey in byuser: if appid_pushkey in byuser:
byuser[appid_pushkey].on_stop() byuser[appid_pushkey].on_stop()
byuser[appid_pushkey] = p byuser[appid_pushkey] = p
p.on_started()
# Check if there *may* be push to process. We do this as this check is a
# lot cheaper to do than actually fetching the exact rows we need to
# push.
user_id = pusherdict["user_name"]
last_stream_ordering = pusherdict["last_stream_ordering"]
if last_stream_ordering:
have_notifs = yield self.store.get_if_maybe_push_in_range_for_user(
user_id, last_stream_ordering,
)
else:
# We always want to default to starting up the pusher rather than
# risk missing push.
have_notifs = True
p.on_started(have_notifs)
@defer.inlineCallbacks @defer.inlineCallbacks
def remove_pusher(self, app_id, pushkey, user_id): def remove_pusher(self, app_id, pushkey, user_id):

View File

@ -74,7 +74,9 @@ REQUIREMENTS = [
CONDITIONAL_REQUIREMENTS = { CONDITIONAL_REQUIREMENTS = {
"email.enable_notifs": ["Jinja2>=2.9", "bleach>=1.4.2"], "email.enable_notifs": ["Jinja2>=2.9", "bleach>=1.4.2"],
"matrix-synapse-ldap3": ["matrix-synapse-ldap3>=0.1"], "matrix-synapse-ldap3": ["matrix-synapse-ldap3>=0.1"],
"postgres": ["psycopg2>=2.6"],
# we use execute_batch, which arrived in psycopg 2.7.
"postgres": ["psycopg2>=2.7"],
# ConsentResource uses select_autoescape, which arrived in jinja 2.9 # ConsentResource uses select_autoescape, which arrived in jinja 2.9
"resources.consent": ["Jinja2>=2.9"], "resources.consent": ["Jinja2>=2.9"],
@ -84,18 +86,22 @@ CONDITIONAL_REQUIREMENTS = {
"acme": ["txacme>=0.9.2"], "acme": ["txacme>=0.9.2"],
"saml2": ["pysaml2>=4.5.0"], "saml2": ["pysaml2>=4.5.0"],
"systemd": ["systemd-python>=231"],
"url_preview": ["lxml>=3.5.0"], "url_preview": ["lxml>=3.5.0"],
"test": ["mock>=2.0", "parameterized"], "test": ["mock>=2.0", "parameterized"],
"sentry": ["sentry-sdk>=0.7.2"], "sentry": ["sentry-sdk>=0.7.2"],
} }
ALL_OPTIONAL_REQUIREMENTS = set()
for name, optional_deps in CONDITIONAL_REQUIREMENTS.items():
# Exclude systemd as it's a system-based requirement.
if name not in ["systemd"]:
ALL_OPTIONAL_REQUIREMENTS = set(optional_deps) | ALL_OPTIONAL_REQUIREMENTS
def list_requirements(): def list_requirements():
deps = set(REQUIREMENTS) return list(set(REQUIREMENTS) | ALL_OPTIONAL_REQUIREMENTS)
for opt in CONDITIONAL_REQUIREMENTS.values():
deps = set(opt) | deps
return list(deps)
class DependencyException(Exception): class DependencyException(Exception):

Some files were not shown because too many files have changed in this diff Show More