mirror of
https://git.anonymousland.org/anonymousland/synapse.git
synced 2024-10-01 11:49:51 -04:00
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers
This commit is contained in:
David Baker
commit
bca3b91c2d
11
.travis.yml
11
.travis.yml
@ -23,6 +23,9 @@ branches:
|
||||
- develop
|
||||
- /^release-v/
|
||||
|
||||
# When running the tox environments that call Twisted Trial, we can pass the -j
|
||||
# flag to run the tests concurrently. We set this to 2 for CPU bound tests
|
||||
# (SQLite) and 4 for I/O bound tests (PostgreSQL).
|
||||
matrix:
|
||||
fast_finish: true
|
||||
include:
|
||||
@ -33,10 +36,10 @@ matrix:
|
||||
env: TOX_ENV="pep8,check_isort"
|
||||
|
||||
- python: 2.7
|
||||
env: TOX_ENV=py27
|
||||
env: TOX_ENV=py27 TRIAL_FLAGS="-j 2"
|
||||
|
||||
- python: 2.7
|
||||
env: TOX_ENV=py27-old
|
||||
env: TOX_ENV=py27-old TRIAL_FLAGS="-j 2"
|
||||
|
||||
- python: 2.7
|
||||
env: TOX_ENV=py27-postgres TRIAL_FLAGS="-j 4"
|
||||
@ -44,10 +47,10 @@ matrix:
|
||||
- postgresql
|
||||
|
||||
- python: 3.5
|
||||
env: TOX_ENV=py35
|
||||
env: TOX_ENV=py35 TRIAL_FLAGS="-j 2"
|
||||
|
||||
- python: 3.6
|
||||
env: TOX_ENV=py36
|
||||
env: TOX_ENV=py36 TRIAL_FLAGS="-j 2"
|
||||
|
||||
- python: 3.6
|
||||
env: TOX_ENV=py36-postgres TRIAL_FLAGS="-j 4"
|
||||
|
||||
63
CHANGES.md
63
CHANGES.md
@ -1,3 +1,66 @@
|
||||
Synapse 0.33.8 (2018-11-01)
|
||||
===========================
|
||||
|
||||
No significant changes.
|
||||
|
||||
|
||||
Synapse 0.33.8rc2 (2018-10-31)
|
||||
==============================
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Searches that request profile info now no longer fail with a 500. Fixes
|
||||
a regression in 0.33.8rc1. ([\#4122](https://github.com/matrix-org/synapse/issues/4122))
|
||||
|
||||
|
||||
Synapse 0.33.8rc1 (2018-10-29)
|
||||
==============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Servers with auto-join rooms will now automatically create those rooms when the first user registers ([\#3975](https://github.com/matrix-org/synapse/issues/3975))
|
||||
- Add config option to control alias creation ([\#4051](https://github.com/matrix-org/synapse/issues/4051))
|
||||
- The register_new_matrix_user script is now ported to Python 3. ([\#4085](https://github.com/matrix-org/synapse/issues/4085))
|
||||
- Configure Docker image to listen on both ipv4 and ipv6. ([\#4089](https://github.com/matrix-org/synapse/issues/4089))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix HTTP error response codes for federated group requests. ([\#3969](https://github.com/matrix-org/synapse/issues/3969))
|
||||
- Fix issue where Python 3 users couldn't paginate /publicRooms ([\#4046](https://github.com/matrix-org/synapse/issues/4046))
|
||||
- Fix URL previewing to work in Python 3.7 ([\#4050](https://github.com/matrix-org/synapse/issues/4050))
|
||||
- synctl will use the right python executable to run worker processes ([\#4057](https://github.com/matrix-org/synapse/issues/4057))
|
||||
- Manhole now works again on Python 3, instead of failing with a "couldn't match all kex parts" when connecting. ([\#4060](https://github.com/matrix-org/synapse/issues/4060), [\#4067](https://github.com/matrix-org/synapse/issues/4067))
|
||||
- Fix some metrics being racy and causing exceptions when polled by Prometheus. ([\#4061](https://github.com/matrix-org/synapse/issues/4061))
|
||||
- Fix bug which prevented email notifications from being sent unless an absolute path was given for `email_templates`. ([\#4068](https://github.com/matrix-org/synapse/issues/4068))
|
||||
- Correctly account for cpu usage by background threads ([\#4074](https://github.com/matrix-org/synapse/issues/4074))
|
||||
- Fix race condition where config defined reserved users were not being added to
|
||||
the monthly active user list prior to the homeserver reactor firing up ([\#4081](https://github.com/matrix-org/synapse/issues/4081))
|
||||
- Fix bug which prevented backslashes being used in event field filters ([\#4083](https://github.com/matrix-org/synapse/issues/4083))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Add information about the [matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy) playbook ([\#3698](https://github.com/matrix-org/synapse/issues/3698))
|
||||
- Add initial implementation of new state resolution algorithm ([\#3786](https://github.com/matrix-org/synapse/issues/3786))
|
||||
- Reduce database load when fetching state groups ([\#4011](https://github.com/matrix-org/synapse/issues/4011))
|
||||
- Various cleanups in the federation client code ([\#4031](https://github.com/matrix-org/synapse/issues/4031))
|
||||
- Run the CircleCI builds in docker containers ([\#4041](https://github.com/matrix-org/synapse/issues/4041))
|
||||
- Only colourise synctl output when attached to tty ([\#4049](https://github.com/matrix-org/synapse/issues/4049))
|
||||
- Refactor room alias creation code ([\#4063](https://github.com/matrix-org/synapse/issues/4063))
|
||||
- Make the Python scripts in the top-level scripts folders meet pep8 and pass flake8. ([\#4068](https://github.com/matrix-org/synapse/issues/4068))
|
||||
- The README now contains example for the Caddy web server. Contributed by steamp0rt. ([\#4072](https://github.com/matrix-org/synapse/issues/4072))
|
||||
- Add psutil as an explicit dependency ([\#4073](https://github.com/matrix-org/synapse/issues/4073))
|
||||
- Clean up threading and logcontexts in pushers ([\#4075](https://github.com/matrix-org/synapse/issues/4075))
|
||||
- Correctly manage logcontexts during startup to fix some "Unexpected logging context" warnings ([\#4076](https://github.com/matrix-org/synapse/issues/4076))
|
||||
- Give some more things logcontexts ([\#4077](https://github.com/matrix-org/synapse/issues/4077))
|
||||
- Clean up some bits of code which were flagged by the linter ([\#4082](https://github.com/matrix-org/synapse/issues/4082))
|
||||
|
||||
|
||||
Synapse 0.33.7 (2018-10-18)
|
||||
===========================
|
||||
|
||||
|
||||
@ -1 +0,0 @@
|
||||
Add information about the [matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy) playbook
|
||||
1
changelog.d/3778.misc
Normal file
1
changelog.d/3778.misc
Normal file
@ -0,0 +1 @@
|
||||
Fix build of Docker image with docker-compose
|
||||
@ -1 +0,0 @@
|
||||
Add initial implementation of new state resolution algorithm
|
||||
@ -1 +0,0 @@
|
||||
Fix HTTP error response codes for federated group requests.
|
||||
@ -1 +0,0 @@
|
||||
Servers with auto-join rooms will now automatically create those rooms when the first user registers
|
||||
1
changelog.d/4004.feature
Normal file
1
changelog.d/4004.feature
Normal file
@ -0,0 +1 @@
|
||||
Include flags to optionally add `m.login.terms` to the registration flow when consent tracking is enabled.
|
||||
1
changelog.d/4006.misc
Normal file
1
changelog.d/4006.misc
Normal file
@ -0,0 +1 @@
|
||||
Delete unreferenced state groups during history purge
|
||||
@ -1 +0,0 @@
|
||||
Reduce database load when fetching state groups
|
||||
@ -1 +0,0 @@
|
||||
Various cleanups in the federation client code
|
||||
@ -1 +0,0 @@
|
||||
Run the CircleCI builds in docker containers
|
||||
@ -1 +0,0 @@
|
||||
Fix issue where Python 3 users couldn't paginate /publicRooms
|
||||
@ -1 +0,0 @@
|
||||
Only colourise synctl output when attached to tty
|
||||
@ -1 +0,0 @@
|
||||
Fix URL priewing to work in Python 3.7
|
||||
@ -1 +0,0 @@
|
||||
Add config option to control alias creation
|
||||
@ -1 +0,0 @@
|
||||
synctl will use the right python executable to run worker processes
|
||||
@ -1 +0,0 @@
|
||||
Manhole now works again on Python 3, instead of failing with a "couldn't match all kex parts" when connecting.
|
||||
@ -1 +0,0 @@
|
||||
Fix some metrics being racy and causing exceptions when polled by Prometheus.
|
||||
@ -1 +0,0 @@
|
||||
Refactor room alias creation code
|
||||
@ -1 +0,0 @@
|
||||
Manhole now works again on Python 3, instead of failing with a "couldn't match all kex parts" when connecting.
|
||||
@ -1 +0,0 @@
|
||||
Fix bug which prevented email notifications from being sent unless an absolute path was given for `email_templates`.
|
||||
@ -1 +0,0 @@
|
||||
Make the Python scripts in the top-level scripts folders meet pep8 and pass flake8.
|
||||
@ -1 +0,0 @@
|
||||
The README now contains example for the Caddy web server. Contributed by steamp0rt.
|
||||
@ -1 +0,0 @@
|
||||
Add psutil as an explicit dependency
|
||||
@ -1 +0,0 @@
|
||||
Correctly account for cpu usage by background threads
|
||||
@ -1 +0,0 @@
|
||||
Clean up threading and logcontexts in pushers
|
||||
@ -1 +0,0 @@
|
||||
Correctly manage logcontexts during startup to fix some "Unexpected logging context" warnings
|
||||
@ -1 +0,0 @@
|
||||
Give some more things logcontexts
|
||||
@ -1,2 +0,0 @@
|
||||
Fix race condition where config defined reserved users were not being added to
|
||||
the monthly active user list prior to the homeserver reactor firing up
|
||||
@ -1 +0,0 @@
|
||||
Clean up some bits of code which were flagged by the linter
|
||||
@ -1 +0,0 @@
|
||||
Fix bug which prevented backslashes being used in event field filters
|
||||
@ -1 +0,0 @@
|
||||
The register_new_matrix_user script is now ported to Python 3.
|
||||
@ -1 +0,0 @@
|
||||
Configure Docker image to listen on both ipv4 and ipv6.
|
||||
1
changelog.d/4095.bugfix
Normal file
1
changelog.d/4095.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Fix exceptions when using the email mailer on Python 3.
|
||||
1
changelog.d/4101.feature
Normal file
1
changelog.d/4101.feature
Normal file
@ -0,0 +1 @@
|
||||
Support for replacing rooms with new ones
|
||||
1
changelog.d/4118.removal
Normal file
1
changelog.d/4118.removal
Normal file
@ -0,0 +1 @@
|
||||
The obsolete and non-functional /pull federation endpoint has been removed.
|
||||
1
changelog.d/4119.removal
Normal file
1
changelog.d/4119.removal
Normal file
@ -0,0 +1 @@
|
||||
The deprecated v1 key exchange endpoints have been removed.
|
||||
1
changelog.d/4120.removal
Normal file
1
changelog.d/4120.removal
Normal file
@ -0,0 +1 @@
|
||||
Synapse will no longer fetch keys using the fallback deprecated v1 key exchange method and will now always use v2.
|
||||
1
changelog.d/4121.misc
Normal file
1
changelog.d/4121.misc
Normal file
@ -0,0 +1 @@
|
||||
Log some bits about room creation
|
||||
1
changelog.d/4122.bugfix
Normal file
1
changelog.d/4122.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Searches that request profile info now no longer fail with a 500.
|
||||
1
changelog.d/4123.bugfix
Normal file
1
changelog.d/4123.bugfix
Normal file
@ -0,0 +1 @@
|
||||
fix return code of empty key backups
|
||||
1
changelog.d/4124.misc
Normal file
1
changelog.d/4124.misc
Normal file
@ -0,0 +1 @@
|
||||
Fix `tox` failure on old systems
|
||||
1
changelog.d/4127.bugfix
Normal file
1
changelog.d/4127.bugfix
Normal file
@ -0,0 +1 @@
|
||||
If the typing stream ID goes backwards (as on a worker when the master restarts), the worker's typing handler will no longer erroneously report rooms containing new typing events.
|
||||
1
changelog.d/4128.misc
Normal file
1
changelog.d/4128.misc
Normal file
@ -0,0 +1 @@
|
||||
Add STATE_V2_TEST room version
|
||||
1
changelog.d/4132.bugfix
Normal file
1
changelog.d/4132.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Fix table lock of device_lists_remote_cache which could freeze the application
|
||||
1
changelog.d/4133.feature
Normal file
1
changelog.d/4133.feature
Normal file
@ -0,0 +1 @@
|
||||
Include flags to optionally add `m.login.terms` to the registration flow when consent tracking is enabled.
|
||||
1
changelog.d/4135.bugfix
Normal file
1
changelog.d/4135.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Fix exception when using state res v2 algorithm
|
||||
1
changelog.d/4137.misc
Normal file
1
changelog.d/4137.misc
Normal file
@ -0,0 +1 @@
|
||||
Clean up event accesses and tests
|
||||
1
changelog.d/4138.misc
Normal file
1
changelog.d/4138.misc
Normal file
@ -0,0 +1 @@
|
||||
The default logging config will now set an explicit log file encoding of UTF-8.
|
||||
1
changelog.d/4139.misc
Normal file
1
changelog.d/4139.misc
Normal file
@ -0,0 +1 @@
|
||||
Add helpers functions for getting prev and auth events of an event
|
||||
1
changelog.d/4140.bugfix
Normal file
1
changelog.d/4140.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Generating the user consent URI no longer fails on Python 3.
|
||||
1
changelog.d/4142.feature
Normal file
1
changelog.d/4142.feature
Normal file
@ -0,0 +1 @@
|
||||
Include flags to optionally add `m.login.terms` to the registration flow when consent tracking is enabled.
|
||||
1
changelog.d/4149.misc
Normal file
1
changelog.d/4149.misc
Normal file
@ -0,0 +1 @@
|
||||
Add some tests for the HTTP pusher.
|
||||
1
changelog.d/4155.misc
Normal file
1
changelog.d/4155.misc
Normal file
@ -0,0 +1 @@
|
||||
add purge_history.sh and purge_remote_media.sh scripts to contrib/
|
||||
1
changelog.d/4156.misc
Normal file
1
changelog.d/4156.misc
Normal file
@ -0,0 +1 @@
|
||||
HTTP tests have been refactored to contain less boilerplate.
|
||||
1
changelog.d/4157.bugfix
Normal file
1
changelog.d/4157.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Loading URL previews from the DB cache on Postgres will no longer cause Unicode type errors when responding to the request, and URL previews will no longer fail if the remote server returns a Content-Type header with the chartype in quotes.
|
||||
1
changelog.d/4161.bugfix
Normal file
1
changelog.d/4161.bugfix
Normal file
@ -0,0 +1 @@
|
||||
The hash_password script now works on Python 3.
|
||||
1
changelog.d/4163.bugfix
Normal file
1
changelog.d/4163.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Generating the user consent URI no longer fails on Python 3.
|
||||
1
changelog.d/4164.bugfix
Normal file
1
changelog.d/4164.bugfix
Normal file
@ -0,0 +1 @@
|
||||
Fix noop checks when updating device keys, reducing spurious device list update notifications.
|
||||
@ -6,9 +6,11 @@ version: '3'
|
||||
services:
|
||||
|
||||
synapse:
|
||||
build: ../..
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: docker/Dockerfile
|
||||
image: docker.io/matrixdotorg/synapse:latest
|
||||
# Since snyapse does not retry to connect to the database, restart upon
|
||||
# Since synapse does not retry to connect to the database, restart upon
|
||||
# failure
|
||||
restart: unless-stopped
|
||||
# See the readme for a full documentation of the environment settings
|
||||
@ -47,4 +49,4 @@ services:
|
||||
# You may store the database tables in a local folder..
|
||||
- ./schemas:/var/lib/postgresql/data
|
||||
# .. or store them on some high performance storage for better results
|
||||
# - /path/to/ssd/storage:/var/lib/postfesql/data
|
||||
# - /path/to/ssd/storage:/var/lib/postgresql/data
|
||||
|
||||
16
contrib/purge_api/README.md
Normal file
16
contrib/purge_api/README.md
Normal file
@ -0,0 +1,16 @@
|
||||
Purge history API examples
|
||||
==========================
|
||||
|
||||
# `purge_history.sh`
|
||||
|
||||
A bash file, that uses the [purge history API](/docs/admin_api/README.rst) to
|
||||
purge all messages in a list of rooms up to a certain event. You can select a
|
||||
timeframe or a number of messages that you want to keep in the room.
|
||||
|
||||
Just configure the variables DOMAIN, ADMIN, ROOMS_ARRAY and TIME at the top of
|
||||
the script.
|
||||
|
||||
# `purge_remote_media.sh`
|
||||
|
||||
A bash file, that uses the [purge history API](/docs/admin_api/README.rst) to
|
||||
purge all old cached remote media.
|
||||
141
contrib/purge_api/purge_history.sh
Normal file
141
contrib/purge_api/purge_history.sh
Normal file
@ -0,0 +1,141 @@
|
||||
#!/bin/bash
|
||||
|
||||
# this script will use the api:
|
||||
# https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.rst
|
||||
#
|
||||
# It will purge all messages in a list of rooms up to a cetrain event
|
||||
|
||||
###################################################################################################
|
||||
# define your domain and admin user
|
||||
###################################################################################################
|
||||
# add this user as admin in your home server:
|
||||
DOMAIN=yourserver.tld
|
||||
# add this user as admin in your home server:
|
||||
ADMIN="@you_admin_username:$DOMAIN"
|
||||
|
||||
API_URL="$DOMAIN:8008/_matrix/client/r0"
|
||||
|
||||
###################################################################################################
|
||||
#choose the rooms to prune old messages from (add a free comment at the end)
|
||||
###################################################################################################
|
||||
# the room_id's you can get e.g. from your Riot clients "View Source" button on each message
|
||||
ROOMS_ARRAY=(
|
||||
'!DgvjtOljKujDBrxyHk:matrix.org#riot:matrix.org'
|
||||
'!QtykxKocfZaZOUrTwp:matrix.org#Matrix HQ'
|
||||
)
|
||||
|
||||
# ALTERNATIVELY:
|
||||
# you can select all the rooms that are not encrypted and loop over the result:
|
||||
# SELECT room_id FROM rooms WHERE room_id NOT IN (SELECT DISTINCT room_id FROM events WHERE type ='m.room.encrypted')
|
||||
# or
|
||||
# select all rooms with at least 100 members:
|
||||
# SELECT q.room_id FROM (select count(*) as numberofusers, room_id FROM current_state_events WHERE type ='m.room.member'
|
||||
# GROUP BY room_id) AS q LEFT JOIN room_aliases a ON q.room_id=a.room_id WHERE q.numberofusers > 100 ORDER BY numberofusers desc
|
||||
|
||||
###################################################################################################
|
||||
# evaluate the EVENT_ID before which should be pruned
|
||||
###################################################################################################
|
||||
# choose a time before which the messages should be pruned:
|
||||
TIME='12 months ago'
|
||||
# ALTERNATIVELY:
|
||||
# a certain time:
|
||||
# TIME='2016-08-31 23:59:59'
|
||||
|
||||
# creates a timestamp from the given time string:
|
||||
UNIX_TIMESTAMP=$(date +%s%3N --date='TZ="UTC+2" '"$TIME")
|
||||
|
||||
# ALTERNATIVELY:
|
||||
# prune all messages that are older than 1000 messages ago:
|
||||
# LAST_MESSAGES=1000
|
||||
# SQL_GET_EVENT="SELECT event_id from events WHERE type='m.room.message' AND room_id ='$ROOM' ORDER BY received_ts DESC LIMIT 1 offset $(($LAST_MESSAGES - 1))"
|
||||
|
||||
# ALTERNATIVELY:
|
||||
# select the EVENT_ID manually:
|
||||
#EVENT_ID='$1471814088343495zpPNI:matrix.org' # an example event from 21st of Aug 2016 by Matthew
|
||||
|
||||
###################################################################################################
|
||||
# make the admin user a server admin in the database with
|
||||
###################################################################################################
|
||||
# psql -A -t --dbname=synapse -c "UPDATE users SET admin=1 WHERE name LIKE '$ADMIN'"
|
||||
|
||||
###################################################################################################
|
||||
# database function
|
||||
###################################################################################################
|
||||
sql (){
|
||||
# for sqlite3:
|
||||
#sqlite3 homeserver.db "pragma busy_timeout=20000;$1" | awk '{print $2}'
|
||||
# for postgres:
|
||||
psql -A -t --dbname=synapse -c "$1" | grep -v 'Pager'
|
||||
}
|
||||
|
||||
###################################################################################################
|
||||
# get an access token
|
||||
###################################################################################################
|
||||
# for example externally by watching Riot in your browser's network inspector
|
||||
# or internally on the server locally, use this:
|
||||
TOKEN=$(sql "SELECT token FROM access_tokens WHERE user_id='$ADMIN' ORDER BY id DESC LIMIT 1")
|
||||
AUTH="Authorization: Bearer $TOKEN"
|
||||
|
||||
###################################################################################################
|
||||
# check, if your TOKEN works. For example this works:
|
||||
###################################################################################################
|
||||
# $ curl --header "$AUTH" "$API_URL/rooms/$ROOM/state/m.room.power_levels"
|
||||
|
||||
###################################################################################################
|
||||
# finally start pruning the room:
|
||||
###################################################################################################
|
||||
POSTDATA='{"delete_local_events":"true"}' # this will really delete local events, so the messages in the room really disappear unless they are restored by remote federation
|
||||
|
||||
for ROOM in "${ROOMS_ARRAY[@]}"; do
|
||||
echo "########################################### $(date) ################# "
|
||||
echo "pruning room: $ROOM ..."
|
||||
ROOM=${ROOM%#*}
|
||||
#set -x
|
||||
echo "check for alias in db..."
|
||||
# for postgres:
|
||||
sql "SELECT * FROM room_aliases WHERE room_id='$ROOM'"
|
||||
echo "get event..."
|
||||
# for postgres:
|
||||
EVENT_ID=$(sql "SELECT event_id FROM events WHERE type='m.room.message' AND received_ts<'$UNIX_TIMESTAMP' AND room_id='$ROOM' ORDER BY received_ts DESC LIMIT 1;")
|
||||
if [ "$EVENT_ID" == "" ]; then
|
||||
echo "no event $TIME"
|
||||
else
|
||||
echo "event: $EVENT_ID"
|
||||
SLEEP=2
|
||||
set -x
|
||||
# call purge
|
||||
OUT=$(curl --header "$AUTH" -s -d $POSTDATA POST "$API_URL/admin/purge_history/$ROOM/$EVENT_ID")
|
||||
PURGE_ID=$(echo "$OUT" |grep purge_id|cut -d'"' -f4 )
|
||||
if [ "$PURGE_ID" == "" ]; then
|
||||
# probably the history purge is already in progress for $ROOM
|
||||
: "continuing with next room"
|
||||
else
|
||||
while : ; do
|
||||
# get status of purge and sleep longer each time if still active
|
||||
sleep $SLEEP
|
||||
STATUS=$(curl --header "$AUTH" -s GET "$API_URL/admin/purge_history_status/$PURGE_ID" |grep status|cut -d'"' -f4)
|
||||
: "$ROOM --> Status: $STATUS"
|
||||
[[ "$STATUS" == "active" ]] || break
|
||||
SLEEP=$((SLEEP + 1))
|
||||
done
|
||||
fi
|
||||
set +x
|
||||
sleep 1
|
||||
fi
|
||||
done
|
||||
|
||||
|
||||
###################################################################################################
|
||||
# additionally
|
||||
###################################################################################################
|
||||
# to benefit from pruning large amounts of data, you need to call VACUUM to free the unused space.
|
||||
# This can take a very long time (hours) and the client have to be stopped while you do so:
|
||||
# $ synctl stop
|
||||
# $ sqlite3 -line homeserver.db "vacuum;"
|
||||
# $ synctl start
|
||||
|
||||
# This could be set, so you don't need to prune every time after deleting some rows:
|
||||
# $ sqlite3 homeserver.db "PRAGMA auto_vacuum = FULL;"
|
||||
# be cautious, it could make the database somewhat slow if there are a lot of deletions
|
||||
|
||||
exit
|
||||
54
contrib/purge_api/purge_remote_media.sh
Normal file
54
contrib/purge_api/purge_remote_media.sh
Normal file
@ -0,0 +1,54 @@
|
||||
#!/bin/bash
|
||||
|
||||
DOMAIN=yourserver.tld
|
||||
# add this user as admin in your home server:
|
||||
ADMIN="@you_admin_username:$DOMAIN"
|
||||
|
||||
API_URL="$DOMAIN:8008/_matrix/client/r0"
|
||||
|
||||
# choose a time before which the messages should be pruned:
|
||||
# TIME='2016-08-31 23:59:59'
|
||||
TIME='12 months ago'
|
||||
|
||||
# creates a timestamp from the given time string:
|
||||
UNIX_TIMESTAMP=$(date +%s%3N --date='TZ="UTC+2" '"$TIME")
|
||||
|
||||
|
||||
###################################################################################################
|
||||
# database function
|
||||
###################################################################################################
|
||||
sql (){
|
||||
# for sqlite3:
|
||||
#sqlite3 homeserver.db "pragma busy_timeout=20000;$1" | awk '{print $2}'
|
||||
# for postgres:
|
||||
psql -A -t --dbname=synapse -c "$1" | grep -v 'Pager'
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# make the admin user a server admin in the database with
|
||||
###############################################################################
|
||||
# sql "UPDATE users SET admin=1 WHERE name LIKE '$ADMIN'"
|
||||
|
||||
###############################################################################
|
||||
# get an access token
|
||||
###############################################################################
|
||||
# for example externally by watching Riot in your browser's network inspector
|
||||
# or internally on the server locally, use this:
|
||||
TOKEN=$(sql "SELECT token FROM access_tokens WHERE user_id='$ADMIN' ORDER BY id DESC LIMIT 1")
|
||||
|
||||
###############################################################################
|
||||
# check, if your TOKEN works. For example this works:
|
||||
###############################################################################
|
||||
# curl --header "Authorization: Bearer $TOKEN" "$API_URL/rooms/$ROOM/state/m.room.power_levels"
|
||||
|
||||
###############################################################################
|
||||
# optional check size before
|
||||
###############################################################################
|
||||
# echo calculate used storage before ...
|
||||
# du -shc ../.synapse/media_store/*
|
||||
|
||||
###############################################################################
|
||||
# finally start pruning media:
|
||||
###############################################################################
|
||||
set -x # for debugging the generated string
|
||||
curl --header "Authorization: Bearer $TOKEN" -v POST "$API_URL/admin/purge_media_cache/?before_ts=$UNIX_TIMESTAMP"
|
||||
@ -31,7 +31,7 @@ Note that the templates must be stored under a name giving the language of the
|
||||
template - currently this must always be `en` (for "English");
|
||||
internationalisation support is intended for the future.
|
||||
|
||||
The template for the policy itself should be versioned and named according to
|
||||
The template for the policy itself should be versioned and named according to
|
||||
the version: for example `1.0.html`. The version of the policy which the user
|
||||
has agreed to is stored in the database.
|
||||
|
||||
@ -85,6 +85,37 @@ Once this is complete, and the server has been restarted, try visiting
|
||||
an error "Missing string query parameter 'u'". It is now possible to manually
|
||||
construct URIs where users can give their consent.
|
||||
|
||||
### Enabling consent tracking at registration
|
||||
|
||||
1. Add the following to your configuration:
|
||||
|
||||
```yaml
|
||||
user_consent:
|
||||
require_at_registration: true
|
||||
policy_name: "Privacy Policy" # or whatever you'd like to call the policy
|
||||
```
|
||||
|
||||
2. In your consent templates, make use of the `public_version` variable to
|
||||
see if an unauthenticated user is viewing the page. This is typically
|
||||
wrapped around the form that would be used to actually agree to the document:
|
||||
|
||||
```
|
||||
{% if not public_version %}
|
||||
<!-- The variables used here are only provided when the 'u' param is given to the homeserver -->
|
||||
<form method="post" action="consent">
|
||||
<input type="hidden" name="v" value="{{version}}"/>
|
||||
<input type="hidden" name="u" value="{{user}}"/>
|
||||
<input type="hidden" name="h" value="{{userhmac}}"/>
|
||||
<input type="submit" value="Sure thing!"/>
|
||||
</form>
|
||||
{% endif %}
|
||||
```
|
||||
|
||||
3. Restart Synapse to apply the changes.
|
||||
|
||||
Visiting `https://<server>/_matrix/consent` should now give you a view of the privacy
|
||||
document. This is what users will be able to see when registering for accounts.
|
||||
|
||||
### Constructing the consent URI
|
||||
|
||||
It may be useful to manually construct the "consent URI" for a given user - for
|
||||
@ -106,6 +137,12 @@ query parameters:
|
||||
`https://<server>/_matrix/consent?u=<user>&h=68a152465a4d...`.
|
||||
|
||||
|
||||
Note that not providing a `u` parameter will be interpreted as wanting to view
|
||||
the document from an unauthenticated perspective, such as prior to registration.
|
||||
Therefore, the `h` parameter is not required in this scenario. To enable this
|
||||
behaviour, set `require_at_registration` to `true` in your `user_consent` config.
|
||||
|
||||
|
||||
Sending users a server notice asking them to agree to the policy
|
||||
----------------------------------------------------------------
|
||||
|
||||
|
||||
@ -12,12 +12,15 @@
|
||||
<p>
|
||||
All your base are belong to us.
|
||||
</p>
|
||||
<form method="post" action="consent">
|
||||
<input type="hidden" name="v" value="{{version}}"/>
|
||||
<input type="hidden" name="u" value="{{user}}"/>
|
||||
<input type="hidden" name="h" value="{{userhmac}}"/>
|
||||
<input type="submit" value="Sure thing!"/>
|
||||
</form>
|
||||
{% if not public_version %}
|
||||
<!-- The variables used here are only provided when the 'u' param is given to the homeserver -->
|
||||
<form method="post" action="consent">
|
||||
<input type="hidden" name="v" value="{{version}}"/>
|
||||
<input type="hidden" name="u" value="{{user}}"/>
|
||||
<input type="hidden" name="h" value="{{userhmac}}"/>
|
||||
<input type="submit" value="Sure thing!"/>
|
||||
</form>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@ -14,22 +14,3 @@ fi
|
||||
|
||||
# set up the virtualenv
|
||||
tox -e py27 --notest -v
|
||||
|
||||
TOX_BIN=$TOX_DIR/py27/bin
|
||||
|
||||
# cryptography 2.2 requires setuptools >= 18.5.
|
||||
#
|
||||
# older versions of virtualenv (?) give us a virtualenv with the same version
|
||||
# of setuptools as is installed on the system python (and tox runs virtualenv
|
||||
# under python3, so we get the version of setuptools that is installed on that).
|
||||
#
|
||||
# anyway, make sure that we have a recent enough setuptools.
|
||||
$TOX_BIN/pip install 'setuptools>=18.5'
|
||||
|
||||
# we also need a semi-recent version of pip, because old ones fail to install
|
||||
# the "enum34" dependency of cryptography.
|
||||
$TOX_BIN/pip install 'pip>=10'
|
||||
|
||||
{ python synapse/python_dependencies.py
|
||||
echo lxml
|
||||
} | xargs $TOX_BIN/pip install
|
||||
|
||||
@ -154,10 +154,15 @@ def request_json(method, origin_name, origin_key, destination, path, content):
|
||||
s = requests.Session()
|
||||
s.mount("matrix://", MatrixConnectionAdapter())
|
||||
|
||||
headers = {"Host": destination, "Authorization": authorization_headers[0]}
|
||||
|
||||
if method == "POST":
|
||||
headers["Content-Type"] = "application/json"
|
||||
|
||||
result = s.request(
|
||||
method=method,
|
||||
url=dest,
|
||||
headers={"Host": destination, "Authorization": authorization_headers[0]},
|
||||
headers=headers,
|
||||
verify=False,
|
||||
data=content,
|
||||
)
|
||||
@ -203,7 +208,7 @@ def main():
|
||||
parser.add_argument(
|
||||
"-X",
|
||||
"--method",
|
||||
help="HTTP method to use for the request. Defaults to GET if --data is"
|
||||
help="HTTP method to use for the request. Defaults to GET if --body is"
|
||||
"unspecified, POST if it is.",
|
||||
)
|
||||
|
||||
|
||||
@ -3,13 +3,15 @@
|
||||
import argparse
|
||||
import getpass
|
||||
import sys
|
||||
import unicodedata
|
||||
|
||||
import bcrypt
|
||||
import yaml
|
||||
|
||||
bcrypt_rounds=12
|
||||
bcrypt_rounds = 12
|
||||
password_pepper = ""
|
||||
|
||||
|
||||
def prompt_for_pass():
|
||||
password = getpass.getpass("Password: ")
|
||||
|
||||
@ -23,19 +25,27 @@ def prompt_for_pass():
|
||||
|
||||
return password
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Calculate the hash of a new password, so that passwords"
|
||||
" can be reset")
|
||||
description=(
|
||||
"Calculate the hash of a new password, so that passwords can be reset"
|
||||
)
|
||||
)
|
||||
parser.add_argument(
|
||||
"-p", "--password",
|
||||
"-p",
|
||||
"--password",
|
||||
default=None,
|
||||
help="New password for user. Will prompt if omitted.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-c", "--config",
|
||||
"-c",
|
||||
"--config",
|
||||
type=argparse.FileType('r'),
|
||||
help="Path to server config file. Used to read in bcrypt_rounds and password_pepper.",
|
||||
help=(
|
||||
"Path to server config file. "
|
||||
"Used to read in bcrypt_rounds and password_pepper."
|
||||
),
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
@ -49,4 +59,21 @@ if __name__ == "__main__":
|
||||
if not password:
|
||||
password = prompt_for_pass()
|
||||
|
||||
print bcrypt.hashpw(password + password_pepper, bcrypt.gensalt(bcrypt_rounds))
|
||||
# On Python 2, make sure we decode it to Unicode before we normalise it
|
||||
if isinstance(password, bytes):
|
||||
try:
|
||||
password = password.decode(sys.stdin.encoding)
|
||||
except UnicodeDecodeError:
|
||||
print(
|
||||
"ERROR! Your password is not decodable using your terminal encoding (%s)."
|
||||
% (sys.stdin.encoding,)
|
||||
)
|
||||
|
||||
pw = unicodedata.normalize("NFKC", password)
|
||||
|
||||
hashed = bcrypt.hashpw(
|
||||
pw.encode('utf8') + password_pepper.encode("utf8"),
|
||||
bcrypt.gensalt(bcrypt_rounds),
|
||||
).decode('ascii')
|
||||
|
||||
print(hashed)
|
||||
|
||||
@ -27,4 +27,4 @@ try:
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "0.33.7"
|
||||
__version__ = "0.33.8"
|
||||
|
||||
@ -51,6 +51,7 @@ class LoginType(object):
|
||||
EMAIL_IDENTITY = u"m.login.email.identity"
|
||||
MSISDN = u"m.login.msisdn"
|
||||
RECAPTCHA = u"m.login.recaptcha"
|
||||
TERMS = u"m.login.terms"
|
||||
DUMMY = u"m.login.dummy"
|
||||
|
||||
# Only for C/S API v1
|
||||
@ -102,6 +103,7 @@ class ThirdPartyEntityKind(object):
|
||||
class RoomVersions(object):
|
||||
V1 = "1"
|
||||
VDH_TEST = "vdh-test-version"
|
||||
STATE_V2_TEST = "state-v2-test"
|
||||
|
||||
|
||||
# the version we will give rooms which are created on this server
|
||||
@ -109,7 +111,11 @@ DEFAULT_ROOM_VERSION = RoomVersions.V1
|
||||
|
||||
# vdh-test-version is a placeholder to get room versioning support working and tested
|
||||
# until we have a working v2.
|
||||
KNOWN_ROOM_VERSIONS = {RoomVersions.V1, RoomVersions.VDH_TEST}
|
||||
KNOWN_ROOM_VERSIONS = {
|
||||
RoomVersions.V1,
|
||||
RoomVersions.VDH_TEST,
|
||||
RoomVersions.STATE_V2_TEST,
|
||||
}
|
||||
|
||||
ServerNoticeMsgType = "m.server_notice"
|
||||
ServerNoticeLimitReached = "m.server_notice.usage_limit_reached"
|
||||
|
||||
@ -28,7 +28,6 @@ FEDERATION_PREFIX = "/_matrix/federation/v1"
|
||||
STATIC_PREFIX = "/_matrix/static"
|
||||
WEB_CLIENT_PREFIX = "/_matrix/client"
|
||||
CONTENT_REPO_PREFIX = "/_matrix/content"
|
||||
SERVER_KEY_PREFIX = "/_matrix/key/v1"
|
||||
SERVER_KEY_V2_PREFIX = "/_matrix/key/v2"
|
||||
MEDIA_PREFIX = "/_matrix/media/r0"
|
||||
LEGACY_MEDIA_PREFIX = "/_matrix/media/v1"
|
||||
|
||||
@ -37,7 +37,6 @@ from synapse.api.urls import (
|
||||
FEDERATION_PREFIX,
|
||||
LEGACY_MEDIA_PREFIX,
|
||||
MEDIA_PREFIX,
|
||||
SERVER_KEY_PREFIX,
|
||||
SERVER_KEY_V2_PREFIX,
|
||||
STATIC_PREFIX,
|
||||
WEB_CLIENT_PREFIX,
|
||||
@ -59,7 +58,6 @@ from synapse.python_dependencies import CONDITIONAL_REQUIREMENTS, check_requirem
|
||||
from synapse.replication.http import REPLICATION_PREFIX, ReplicationRestResource
|
||||
from synapse.replication.tcp.resource import ReplicationStreamProtocolFactory
|
||||
from synapse.rest import ClientRestResource
|
||||
from synapse.rest.key.v1.server_key_resource import LocalKey
|
||||
from synapse.rest.key.v2 import KeyApiV2Resource
|
||||
from synapse.rest.media.v0.content_repository import ContentRepoResource
|
||||
from synapse.server import HomeServer
|
||||
@ -236,10 +234,7 @@ class SynapseHomeServer(HomeServer):
|
||||
)
|
||||
|
||||
if name in ["keys", "federation"]:
|
||||
resources.update({
|
||||
SERVER_KEY_PREFIX: LocalKey(self),
|
||||
SERVER_KEY_V2_PREFIX: KeyApiV2Resource(self),
|
||||
})
|
||||
resources[SERVER_KEY_V2_PREFIX] = KeyApiV2Resource(self)
|
||||
|
||||
if name == "webclient":
|
||||
resources[WEB_CLIENT_PREFIX] = build_resource_for_web_client(self)
|
||||
|
||||
@ -226,7 +226,15 @@ class SynchrotronPresence(object):
|
||||
class SynchrotronTyping(object):
|
||||
def __init__(self, hs):
|
||||
self._latest_room_serial = 0
|
||||
self._reset()
|
||||
|
||||
def _reset(self):
|
||||
"""
|
||||
Reset the typing handler's data caches.
|
||||
"""
|
||||
# map room IDs to serial numbers
|
||||
self._room_serials = {}
|
||||
# map room IDs to sets of users currently typing
|
||||
self._room_typing = {}
|
||||
|
||||
def stream_positions(self):
|
||||
@ -236,6 +244,12 @@ class SynchrotronTyping(object):
|
||||
return {"typing": self._latest_room_serial}
|
||||
|
||||
def process_replication_rows(self, token, rows):
|
||||
if self._latest_room_serial > token:
|
||||
# The master has gone backwards. To prevent inconsistent data, just
|
||||
# clear everything.
|
||||
self._reset()
|
||||
|
||||
# Set the latest serial token to whatever the server gave us.
|
||||
self._latest_room_serial = token
|
||||
|
||||
for row in rows:
|
||||
|
||||
@ -42,6 +42,14 @@ DEFAULT_CONFIG = """\
|
||||
# until the user consents to the privacy policy. The value of the setting is
|
||||
# used as the text of the error.
|
||||
#
|
||||
# 'require_at_registration', if enabled, will add a step to the registration
|
||||
# process, similar to how captcha works. Users will be required to accept the
|
||||
# policy before their account is created.
|
||||
#
|
||||
# 'policy_name' is the display name of the policy users will see when registering
|
||||
# for an account. Has no effect unless `require_at_registration` is enabled.
|
||||
# Defaults to "Privacy Policy".
|
||||
#
|
||||
# user_consent:
|
||||
# template_dir: res/templates/privacy
|
||||
# version: 1.0
|
||||
@ -54,6 +62,8 @@ DEFAULT_CONFIG = """\
|
||||
# block_events_error: >-
|
||||
# To continue using this homeserver you must review and agree to the
|
||||
# terms and conditions at %(consent_uri)s
|
||||
# require_at_registration: False
|
||||
# policy_name: Privacy Policy
|
||||
#
|
||||
"""
|
||||
|
||||
@ -67,6 +77,8 @@ class ConsentConfig(Config):
|
||||
self.user_consent_server_notice_content = None
|
||||
self.user_consent_server_notice_to_guests = False
|
||||
self.block_events_without_consent_error = None
|
||||
self.user_consent_at_registration = False
|
||||
self.user_consent_policy_name = "Privacy Policy"
|
||||
|
||||
def read_config(self, config):
|
||||
consent_config = config.get("user_consent")
|
||||
@ -83,6 +95,12 @@ class ConsentConfig(Config):
|
||||
self.user_consent_server_notice_to_guests = bool(consent_config.get(
|
||||
"send_server_notice_to_guests", False,
|
||||
))
|
||||
self.user_consent_at_registration = bool(consent_config.get(
|
||||
"require_at_registration", False,
|
||||
))
|
||||
self.user_consent_policy_name = consent_config.get(
|
||||
"policy_name", "Privacy Policy",
|
||||
)
|
||||
|
||||
def default_config(self, **kwargs):
|
||||
return DEFAULT_CONFIG
|
||||
|
||||
@ -50,6 +50,7 @@ handlers:
|
||||
maxBytes: 104857600
|
||||
backupCount: 10
|
||||
filters: [context]
|
||||
encoding: utf8
|
||||
console:
|
||||
class: logging.StreamHandler
|
||||
formatter: precise
|
||||
|
||||
@ -15,6 +15,8 @@
|
||||
|
||||
import logging
|
||||
|
||||
from six.moves import urllib
|
||||
|
||||
from canonicaljson import json
|
||||
|
||||
from twisted.internet import defer, reactor
|
||||
@ -28,15 +30,15 @@ from synapse.util import logcontext
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
KEY_API_V1 = b"/_matrix/key/v1/"
|
||||
KEY_API_V2 = "/_matrix/key/v2/server/%s"
|
||||
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def fetch_server_key(server_name, tls_client_options_factory, path=KEY_API_V1):
|
||||
def fetch_server_key(server_name, tls_client_options_factory, key_id):
|
||||
"""Fetch the keys for a remote server."""
|
||||
|
||||
factory = SynapseKeyClientFactory()
|
||||
factory.path = path
|
||||
factory.path = KEY_API_V2 % (urllib.parse.quote(key_id), )
|
||||
factory.host = server_name
|
||||
endpoint = matrix_federation_endpoint(
|
||||
reactor, server_name, tls_client_options_factory, timeout=30
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# Copyright 2014-2016 OpenMarket Ltd
|
||||
# Copyright 2017 New Vector Ltd.
|
||||
# Copyright 2017, 2018 New Vector Ltd.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
@ -18,8 +18,6 @@ import hashlib
|
||||
import logging
|
||||
from collections import namedtuple
|
||||
|
||||
from six.moves import urllib
|
||||
|
||||
from signedjson.key import (
|
||||
decode_verify_key_bytes,
|
||||
encode_verify_key_base64,
|
||||
@ -395,32 +393,13 @@ class Keyring(object):
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def get_keys_from_server(self, server_name_and_key_ids):
|
||||
@defer.inlineCallbacks
|
||||
def get_key(server_name, key_ids):
|
||||
keys = None
|
||||
try:
|
||||
keys = yield self.get_server_verify_key_v2_direct(
|
||||
server_name, key_ids
|
||||
)
|
||||
except Exception as e:
|
||||
logger.info(
|
||||
"Unable to get key %r for %r directly: %s %s",
|
||||
key_ids, server_name,
|
||||
type(e).__name__, str(e),
|
||||
)
|
||||
|
||||
if not keys:
|
||||
keys = yield self.get_server_verify_key_v1_direct(
|
||||
server_name, key_ids
|
||||
)
|
||||
|
||||
keys = {server_name: keys}
|
||||
|
||||
defer.returnValue(keys)
|
||||
|
||||
results = yield logcontext.make_deferred_yieldable(defer.gatherResults(
|
||||
[
|
||||
run_in_background(get_key, server_name, key_ids)
|
||||
run_in_background(
|
||||
self.get_server_verify_key_v2_direct,
|
||||
server_name,
|
||||
key_ids,
|
||||
)
|
||||
for server_name, key_ids in server_name_and_key_ids
|
||||
],
|
||||
consumeErrors=True,
|
||||
@ -525,10 +504,7 @@ class Keyring(object):
|
||||
continue
|
||||
|
||||
(response, tls_certificate) = yield fetch_server_key(
|
||||
server_name, self.hs.tls_client_options_factory,
|
||||
path=("/_matrix/key/v2/server/%s" % (
|
||||
urllib.parse.quote(requested_key_id),
|
||||
)).encode("ascii"),
|
||||
server_name, self.hs.tls_client_options_factory, requested_key_id
|
||||
)
|
||||
|
||||
if (u"signatures" not in response
|
||||
@ -657,78 +633,6 @@ class Keyring(object):
|
||||
|
||||
defer.returnValue(results)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def get_server_verify_key_v1_direct(self, server_name, key_ids):
|
||||
"""Finds a verification key for the server with one of the key ids.
|
||||
Args:
|
||||
server_name (str): The name of the server to fetch a key for.
|
||||
keys_ids (list of str): The key_ids to check for.
|
||||
"""
|
||||
|
||||
# Try to fetch the key from the remote server.
|
||||
|
||||
(response, tls_certificate) = yield fetch_server_key(
|
||||
server_name, self.hs.tls_client_options_factory
|
||||
)
|
||||
|
||||
# Check the response.
|
||||
|
||||
x509_certificate_bytes = crypto.dump_certificate(
|
||||
crypto.FILETYPE_ASN1, tls_certificate
|
||||
)
|
||||
|
||||
if ("signatures" not in response
|
||||
or server_name not in response["signatures"]):
|
||||
raise KeyLookupError("Key response not signed by remote server")
|
||||
|
||||
if "tls_certificate" not in response:
|
||||
raise KeyLookupError("Key response missing TLS certificate")
|
||||
|
||||
tls_certificate_b64 = response["tls_certificate"]
|
||||
|
||||
if encode_base64(x509_certificate_bytes) != tls_certificate_b64:
|
||||
raise KeyLookupError("TLS certificate doesn't match")
|
||||
|
||||
# Cache the result in the datastore.
|
||||
|
||||
time_now_ms = self.clock.time_msec()
|
||||
|
||||
verify_keys = {}
|
||||
for key_id, key_base64 in response["verify_keys"].items():
|
||||
if is_signing_algorithm_supported(key_id):
|
||||
key_bytes = decode_base64(key_base64)
|
||||
verify_key = decode_verify_key_bytes(key_id, key_bytes)
|
||||
verify_key.time_added = time_now_ms
|
||||
verify_keys[key_id] = verify_key
|
||||
|
||||
for key_id in response["signatures"][server_name]:
|
||||
if key_id not in response["verify_keys"]:
|
||||
raise KeyLookupError(
|
||||
"Key response must include verification keys for all"
|
||||
" signatures"
|
||||
)
|
||||
if key_id in verify_keys:
|
||||
verify_signed_json(
|
||||
response,
|
||||
server_name,
|
||||
verify_keys[key_id]
|
||||
)
|
||||
|
||||
yield self.store.store_server_certificate(
|
||||
server_name,
|
||||
server_name,
|
||||
time_now_ms,
|
||||
tls_certificate,
|
||||
)
|
||||
|
||||
yield self.store_keys(
|
||||
server_name=server_name,
|
||||
from_server=server_name,
|
||||
verify_keys=verify_keys,
|
||||
)
|
||||
|
||||
defer.returnValue(verify_keys)
|
||||
|
||||
def store_keys(self, server_name, from_server, verify_keys):
|
||||
"""Store a collection of verify keys for a given server
|
||||
Args:
|
||||
|
||||
@ -200,11 +200,11 @@ def _is_membership_change_allowed(event, auth_events):
|
||||
membership = event.content["membership"]
|
||||
|
||||
# Check if this is the room creator joining:
|
||||
if len(event.prev_events) == 1 and Membership.JOIN == membership:
|
||||
if len(event.prev_event_ids()) == 1 and Membership.JOIN == membership:
|
||||
# Get room creation event:
|
||||
key = (EventTypes.Create, "", )
|
||||
create = auth_events.get(key)
|
||||
if create and event.prev_events[0][0] == create.event_id:
|
||||
if create and event.prev_event_ids()[0] == create.event_id:
|
||||
if create.content["creator"] == event.state_key:
|
||||
return
|
||||
|
||||
|
||||
@ -159,6 +159,24 @@ class EventBase(object):
|
||||
def keys(self):
|
||||
return six.iterkeys(self._event_dict)
|
||||
|
||||
def prev_event_ids(self):
|
||||
"""Returns the list of prev event IDs. The order matches the order
|
||||
specified in the event, though there is no meaning to it.
|
||||
|
||||
Returns:
|
||||
list[str]: The list of event IDs of this event's prev_events
|
||||
"""
|
||||
return [e for e, _ in self.prev_events]
|
||||
|
||||
def auth_event_ids(self):
|
||||
"""Returns the list of auth event IDs. The order matches the order
|
||||
specified in the event, though there is no meaning to it.
|
||||
|
||||
Returns:
|
||||
list[str]: The list of event IDs of this event's auth_events
|
||||
"""
|
||||
return [e for e, _ in self.auth_events]
|
||||
|
||||
|
||||
class FrozenEvent(EventBase):
|
||||
def __init__(self, event_dict, internal_metadata_dict={}, rejected_reason=None):
|
||||
|
||||
@ -323,11 +323,6 @@ class FederationServer(FederationBase):
|
||||
else:
|
||||
defer.returnValue((404, ""))
|
||||
|
||||
@defer.inlineCallbacks
|
||||
@log_function
|
||||
def on_pull_request(self, origin, versions):
|
||||
raise NotImplementedError("Pull transactions not implemented")
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def on_query_request(self, query_type, args):
|
||||
received_queries_counter.labels(query_type).inc()
|
||||
|
||||
@ -183,9 +183,7 @@ class TransactionQueue(object):
|
||||
# banned then it won't receive the event because it won't
|
||||
# be in the room after the ban.
|
||||
destinations = yield self.state.get_current_hosts_in_room(
|
||||
event.room_id, latest_event_ids=[
|
||||
prev_id for prev_id, _ in event.prev_events
|
||||
],
|
||||
event.room_id, latest_event_ids=event.prev_event_ids(),
|
||||
)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
|
||||
@ -362,14 +362,6 @@ class FederationSendServlet(BaseFederationServlet):
|
||||
defer.returnValue((code, response))
|
||||
|
||||
|
||||
class FederationPullServlet(BaseFederationServlet):
|
||||
PATH = "/pull/"
|
||||
|
||||
# This is for when someone asks us for everything since version X
|
||||
def on_GET(self, origin, content, query):
|
||||
return self.handler.on_pull_request(query["origin"][0], query["v"])
|
||||
|
||||
|
||||
class FederationEventServlet(BaseFederationServlet):
|
||||
PATH = "/event/(?P<event_id>[^/]*)/"
|
||||
|
||||
@ -1261,7 +1253,6 @@ class FederationGroupsSettingJoinPolicyServlet(BaseFederationServlet):
|
||||
|
||||
FEDERATION_SERVLET_CLASSES = (
|
||||
FederationSendServlet,
|
||||
FederationPullServlet,
|
||||
FederationEventServlet,
|
||||
FederationStateServlet,
|
||||
FederationStateIdsServlet,
|
||||
|
||||
@ -117,9 +117,6 @@ class Transaction(JsonEncodedObject):
|
||||
"Require 'transaction_id' to construct a Transaction"
|
||||
)
|
||||
|
||||
for p in pdus:
|
||||
p.transaction_id = kwargs["transaction_id"]
|
||||
|
||||
kwargs["pdus"] = [p.get_pdu_json() for p in pdus]
|
||||
|
||||
return Transaction(**kwargs)
|
||||
|
||||
@ -59,6 +59,7 @@ class AuthHandler(BaseHandler):
|
||||
LoginType.EMAIL_IDENTITY: self._check_email_identity,
|
||||
LoginType.MSISDN: self._check_msisdn,
|
||||
LoginType.DUMMY: self._check_dummy_auth,
|
||||
LoginType.TERMS: self._check_terms_auth,
|
||||
}
|
||||
self.bcrypt_rounds = hs.config.bcrypt_rounds
|
||||
|
||||
@ -431,6 +432,9 @@ class AuthHandler(BaseHandler):
|
||||
def _check_dummy_auth(self, authdict, _):
|
||||
return defer.succeed(True)
|
||||
|
||||
def _check_terms_auth(self, authdict, _):
|
||||
return defer.succeed(True)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def _check_threepid(self, medium, authdict):
|
||||
if 'threepid_creds' not in authdict:
|
||||
@ -462,6 +466,22 @@ class AuthHandler(BaseHandler):
|
||||
def _get_params_recaptcha(self):
|
||||
return {"public_key": self.hs.config.recaptcha_public_key}
|
||||
|
||||
def _get_params_terms(self):
|
||||
return {
|
||||
"policies": {
|
||||
"privacy_policy": {
|
||||
"version": self.hs.config.user_consent_version,
|
||||
"en": {
|
||||
"name": self.hs.config.user_consent_policy_name,
|
||||
"url": "%s/_matrix/consent?v=%s" % (
|
||||
self.hs.config.public_baseurl,
|
||||
self.hs.config.user_consent_version,
|
||||
),
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
def _auth_dict_for_flows(self, flows, session):
|
||||
public_flows = []
|
||||
for f in flows:
|
||||
@ -469,6 +489,7 @@ class AuthHandler(BaseHandler):
|
||||
|
||||
get_params = {
|
||||
LoginType.RECAPTCHA: self._get_params_recaptcha,
|
||||
LoginType.TERMS: self._get_params_terms,
|
||||
}
|
||||
|
||||
params = {}
|
||||
|
||||
@ -138,9 +138,30 @@ class DirectoryHandler(BaseHandler):
|
||||
)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def delete_association(self, requester, room_alias):
|
||||
# association deletion for human users
|
||||
def delete_association(self, requester, room_alias, send_event=True):
|
||||
"""Remove an alias from the directory
|
||||
|
||||
(this is only meant for human users; AS users should call
|
||||
delete_appservice_association)
|
||||
|
||||
Args:
|
||||
requester (Requester):
|
||||
room_alias (RoomAlias):
|
||||
send_event (bool): Whether to send an updated m.room.aliases event.
|
||||
Note that, if we delete the canonical alias, we will always attempt
|
||||
to send an m.room.canonical_alias event
|
||||
|
||||
Returns:
|
||||
Deferred[unicode]: room id that the alias used to point to
|
||||
|
||||
Raises:
|
||||
NotFoundError: if the alias doesn't exist
|
||||
|
||||
AuthError: if the user doesn't have perms to delete the alias (ie, the user
|
||||
is neither the creator of the alias, nor a server admin.
|
||||
|
||||
SynapseError: if the alias belongs to an AS
|
||||
"""
|
||||
user_id = requester.user.to_string()
|
||||
|
||||
try:
|
||||
@ -168,10 +189,11 @@ class DirectoryHandler(BaseHandler):
|
||||
room_id = yield self._delete_association(room_alias)
|
||||
|
||||
try:
|
||||
yield self.send_room_alias_update_event(
|
||||
requester,
|
||||
room_id
|
||||
)
|
||||
if send_event:
|
||||
yield self.send_room_alias_update_event(
|
||||
requester,
|
||||
room_id
|
||||
)
|
||||
|
||||
yield self._update_canonical_alias(
|
||||
requester,
|
||||
|
||||
@ -19,7 +19,7 @@ from six import iteritems
|
||||
|
||||
from twisted.internet import defer
|
||||
|
||||
from synapse.api.errors import RoomKeysVersionError, StoreError, SynapseError
|
||||
from synapse.api.errors import NotFoundError, RoomKeysVersionError, StoreError
|
||||
from synapse.util.async_helpers import Linearizer
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -55,6 +55,8 @@ class E2eRoomKeysHandler(object):
|
||||
room_id(string): room ID to get keys for, for None to get keys for all rooms
|
||||
session_id(string): session ID to get keys for, for None to get keys for all
|
||||
sessions
|
||||
Raises:
|
||||
NotFoundError: if the backup version does not exist
|
||||
Returns:
|
||||
A deferred list of dicts giving the session_data and message metadata for
|
||||
these room keys.
|
||||
@ -63,13 +65,19 @@ class E2eRoomKeysHandler(object):
|
||||
# we deliberately take the lock to get keys so that changing the version
|
||||
# works atomically
|
||||
with (yield self._upload_linearizer.queue(user_id)):
|
||||
# make sure the backup version exists
|
||||
try:
|
||||
yield self.store.get_e2e_room_keys_version_info(user_id, version)
|
||||
except StoreError as e:
|
||||
if e.code == 404:
|
||||
raise NotFoundError("Unknown backup version")
|
||||
else:
|
||||
raise
|
||||
|
||||
results = yield self.store.get_e2e_room_keys(
|
||||
user_id, version, room_id, session_id
|
||||
)
|
||||
|
||||
if results['rooms'] == {}:
|
||||
raise SynapseError(404, "No room_keys found")
|
||||
|
||||
defer.returnValue(results)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
@ -120,7 +128,7 @@ class E2eRoomKeysHandler(object):
|
||||
}
|
||||
|
||||
Raises:
|
||||
SynapseError: with code 404 if there are no versions defined
|
||||
NotFoundError: if there are no versions defined
|
||||
RoomKeysVersionError: if the uploaded version is not the current version
|
||||
"""
|
||||
|
||||
@ -134,7 +142,7 @@ class E2eRoomKeysHandler(object):
|
||||
version_info = yield self.store.get_e2e_room_keys_version_info(user_id)
|
||||
except StoreError as e:
|
||||
if e.code == 404:
|
||||
raise SynapseError(404, "Version '%s' not found" % (version,))
|
||||
raise NotFoundError("Version '%s' not found" % (version,))
|
||||
else:
|
||||
raise
|
||||
|
||||
@ -148,7 +156,7 @@ class E2eRoomKeysHandler(object):
|
||||
raise RoomKeysVersionError(current_version=version_info['version'])
|
||||
except StoreError as e:
|
||||
if e.code == 404:
|
||||
raise SynapseError(404, "Version '%s' not found" % (version,))
|
||||
raise NotFoundError("Version '%s' not found" % (version,))
|
||||
else:
|
||||
raise
|
||||
|
||||
|
||||
@ -239,7 +239,7 @@ class FederationHandler(BaseHandler):
|
||||
room_id, event_id, min_depth,
|
||||
)
|
||||
|
||||
prevs = {e_id for e_id, _ in pdu.prev_events}
|
||||
prevs = set(pdu.prev_event_ids())
|
||||
seen = yield self.store.have_seen_events(prevs)
|
||||
|
||||
if min_depth and pdu.depth < min_depth:
|
||||
@ -607,7 +607,7 @@ class FederationHandler(BaseHandler):
|
||||
if e.event_id in seen_ids:
|
||||
continue
|
||||
e.internal_metadata.outlier = True
|
||||
auth_ids = [e_id for e_id, _ in e.auth_events]
|
||||
auth_ids = e.auth_event_ids()
|
||||
auth = {
|
||||
(e.type, e.state_key): e for e in auth_chain
|
||||
if e.event_id in auth_ids or e.type == EventTypes.Create
|
||||
@ -726,7 +726,7 @@ class FederationHandler(BaseHandler):
|
||||
edges = [
|
||||
ev.event_id
|
||||
for ev in events
|
||||
if set(e_id for e_id, _ in ev.prev_events) - event_ids
|
||||
if set(ev.prev_event_ids()) - event_ids
|
||||
]
|
||||
|
||||
logger.info(
|
||||
@ -753,7 +753,7 @@ class FederationHandler(BaseHandler):
|
||||
required_auth = set(
|
||||
a_id
|
||||
for event in events + list(state_events.values()) + list(auth_events.values())
|
||||
for a_id, _ in event.auth_events
|
||||
for a_id in event.auth_event_ids()
|
||||
)
|
||||
auth_events.update({
|
||||
e_id: event_map[e_id] for e_id in required_auth if e_id in event_map
|
||||
@ -769,7 +769,7 @@ class FederationHandler(BaseHandler):
|
||||
auth_events.update(ret_events)
|
||||
|
||||
required_auth.update(
|
||||
a_id for event in ret_events.values() for a_id, _ in event.auth_events
|
||||
a_id for event in ret_events.values() for a_id in event.auth_event_ids()
|
||||
)
|
||||
missing_auth = required_auth - set(auth_events)
|
||||
|
||||
@ -796,7 +796,7 @@ class FederationHandler(BaseHandler):
|
||||
required_auth.update(
|
||||
a_id
|
||||
for event in results if event
|
||||
for a_id, _ in event.auth_events
|
||||
for a_id in event.auth_event_ids()
|
||||
)
|
||||
missing_auth = required_auth - set(auth_events)
|
||||
|
||||
@ -816,7 +816,7 @@ class FederationHandler(BaseHandler):
|
||||
"auth_events": {
|
||||
(auth_events[a_id].type, auth_events[a_id].state_key):
|
||||
auth_events[a_id]
|
||||
for a_id, _ in a.auth_events
|
||||
for a_id in a.auth_event_ids()
|
||||
if a_id in auth_events
|
||||
}
|
||||
})
|
||||
@ -828,7 +828,7 @@ class FederationHandler(BaseHandler):
|
||||
"auth_events": {
|
||||
(auth_events[a_id].type, auth_events[a_id].state_key):
|
||||
auth_events[a_id]
|
||||
for a_id, _ in event_map[e_id].auth_events
|
||||
for a_id in event_map[e_id].auth_event_ids()
|
||||
if a_id in auth_events
|
||||
}
|
||||
})
|
||||
@ -1041,17 +1041,17 @@ class FederationHandler(BaseHandler):
|
||||
Raises:
|
||||
SynapseError if the event does not pass muster
|
||||
"""
|
||||
if len(ev.prev_events) > 20:
|
||||
if len(ev.prev_event_ids()) > 20:
|
||||
logger.warn("Rejecting event %s which has %i prev_events",
|
||||
ev.event_id, len(ev.prev_events))
|
||||
ev.event_id, len(ev.prev_event_ids()))
|
||||
raise SynapseError(
|
||||
http_client.BAD_REQUEST,
|
||||
"Too many prev_events",
|
||||
)
|
||||
|
||||
if len(ev.auth_events) > 10:
|
||||
if len(ev.auth_event_ids()) > 10:
|
||||
logger.warn("Rejecting event %s which has %i auth_events",
|
||||
ev.event_id, len(ev.auth_events))
|
||||
ev.event_id, len(ev.auth_event_ids()))
|
||||
raise SynapseError(
|
||||
http_client.BAD_REQUEST,
|
||||
"Too many auth_events",
|
||||
@ -1076,7 +1076,7 @@ class FederationHandler(BaseHandler):
|
||||
def on_event_auth(self, event_id):
|
||||
event = yield self.store.get_event(event_id)
|
||||
auth = yield self.store.get_auth_chain(
|
||||
[auth_id for auth_id, _ in event.auth_events],
|
||||
[auth_id for auth_id in event.auth_event_ids()],
|
||||
include_given=True
|
||||
)
|
||||
defer.returnValue([e for e in auth])
|
||||
@ -1698,7 +1698,7 @@ class FederationHandler(BaseHandler):
|
||||
|
||||
missing_auth_events = set()
|
||||
for e in itertools.chain(auth_events, state, [event]):
|
||||
for e_id, _ in e.auth_events:
|
||||
for e_id in e.auth_event_ids():
|
||||
if e_id not in event_map:
|
||||
missing_auth_events.add(e_id)
|
||||
|
||||
@ -1717,7 +1717,7 @@ class FederationHandler(BaseHandler):
|
||||
for e in itertools.chain(auth_events, state, [event]):
|
||||
auth_for_e = {
|
||||
(event_map[e_id].type, event_map[e_id].state_key): event_map[e_id]
|
||||
for e_id, _ in e.auth_events
|
||||
for e_id in e.auth_event_ids()
|
||||
if e_id in event_map
|
||||
}
|
||||
if create_event:
|
||||
@ -1785,10 +1785,10 @@ class FederationHandler(BaseHandler):
|
||||
|
||||
# This is a hack to fix some old rooms where the initial join event
|
||||
# didn't reference the create event in its auth events.
|
||||
if event.type == EventTypes.Member and not event.auth_events:
|
||||
if len(event.prev_events) == 1 and event.depth < 5:
|
||||
if event.type == EventTypes.Member and not event.auth_event_ids():
|
||||
if len(event.prev_event_ids()) == 1 and event.depth < 5:
|
||||
c = yield self.store.get_event(
|
||||
event.prev_events[0][0],
|
||||
event.prev_event_ids()[0],
|
||||
allow_none=True,
|
||||
)
|
||||
if c and c.type == EventTypes.Create:
|
||||
@ -1835,7 +1835,7 @@ class FederationHandler(BaseHandler):
|
||||
|
||||
# Now get the current auth_chain for the event.
|
||||
local_auth_chain = yield self.store.get_auth_chain(
|
||||
[auth_id for auth_id, _ in event.auth_events],
|
||||
[auth_id for auth_id in event.auth_event_ids()],
|
||||
include_given=True
|
||||
)
|
||||
|
||||
@ -1891,7 +1891,7 @@ class FederationHandler(BaseHandler):
|
||||
"""
|
||||
# Check if we have all the auth events.
|
||||
current_state = set(e.event_id for e in auth_events.values())
|
||||
event_auth_events = set(e_id for e_id, _ in event.auth_events)
|
||||
event_auth_events = set(event.auth_event_ids())
|
||||
|
||||
if event.is_state():
|
||||
event_key = (event.type, event.state_key)
|
||||
@ -1935,7 +1935,7 @@ class FederationHandler(BaseHandler):
|
||||
continue
|
||||
|
||||
try:
|
||||
auth_ids = [e_id for e_id, _ in e.auth_events]
|
||||
auth_ids = e.auth_event_ids()
|
||||
auth = {
|
||||
(e.type, e.state_key): e for e in remote_auth_chain
|
||||
if e.event_id in auth_ids or e.type == EventTypes.Create
|
||||
@ -1956,7 +1956,7 @@ class FederationHandler(BaseHandler):
|
||||
pass
|
||||
|
||||
have_events = yield self.store.get_seen_events_with_rejections(
|
||||
[e_id for e_id, _ in event.auth_events]
|
||||
event.auth_event_ids()
|
||||
)
|
||||
seen_events = set(have_events.keys())
|
||||
except Exception:
|
||||
@ -2058,7 +2058,7 @@ class FederationHandler(BaseHandler):
|
||||
continue
|
||||
|
||||
try:
|
||||
auth_ids = [e_id for e_id, _ in ev.auth_events]
|
||||
auth_ids = ev.auth_event_ids()
|
||||
auth = {
|
||||
(e.type, e.state_key): e
|
||||
for e in result["auth_chain"]
|
||||
@ -2250,7 +2250,7 @@ class FederationHandler(BaseHandler):
|
||||
missing_remote_ids = [e.event_id for e in missing_remotes]
|
||||
base_remote_rejected = list(missing_remotes)
|
||||
for e in missing_remotes:
|
||||
for e_id, _ in e.auth_events:
|
||||
for e_id in e.auth_event_ids():
|
||||
if e_id in missing_remote_ids:
|
||||
try:
|
||||
base_remote_rejected.remove(e)
|
||||
|
||||
@ -427,6 +427,9 @@ class EventCreationHandler(object):
|
||||
|
||||
if event.is_state():
|
||||
prev_state = yield self.deduplicate_state_event(event, context)
|
||||
logger.info(
|
||||
"Not bothering to persist duplicate state event %s", event.event_id,
|
||||
)
|
||||
if prev_state is not None:
|
||||
defer.returnValue(prev_state)
|
||||
|
||||
|
||||
@ -104,6 +104,8 @@ class RoomCreationHandler(BaseHandler):
|
||||
creator_id=user_id, is_public=r["is_public"],
|
||||
)
|
||||
|
||||
logger.info("Creating new room %s to replace %s", new_room_id, old_room_id)
|
||||
|
||||
# we create and auth the tombstone event before properly creating the new
|
||||
# room, to check our user has perms in the old room.
|
||||
tombstone_event, tombstone_context = (
|
||||
@ -136,10 +138,15 @@ class RoomCreationHandler(BaseHandler):
|
||||
requester, tombstone_event, tombstone_context,
|
||||
)
|
||||
|
||||
# and finally, shut down the PLs in the old room, and update them in the new
|
||||
# room.
|
||||
old_room_state = yield tombstone_context.get_current_state_ids(self.store)
|
||||
|
||||
# update any aliases
|
||||
yield self._move_aliases_to_new_room(
|
||||
requester, old_room_id, new_room_id, old_room_state,
|
||||
)
|
||||
|
||||
# and finally, shut down the PLs in the old room, and update them in the new
|
||||
# room.
|
||||
yield self._update_upgraded_room_pls(
|
||||
requester, old_room_id, new_room_id, old_room_state,
|
||||
)
|
||||
@ -245,11 +252,6 @@ class RoomCreationHandler(BaseHandler):
|
||||
if not self.spam_checker.user_may_create_room(user_id):
|
||||
raise SynapseError(403, "You are not permitted to create rooms")
|
||||
|
||||
# XXX check alias is free
|
||||
# canonical_alias = None
|
||||
|
||||
# XXX create association in directory handler
|
||||
|
||||
creation_content = {
|
||||
"room_version": new_room_version,
|
||||
"predecessor": {
|
||||
@ -295,7 +297,111 @@ class RoomCreationHandler(BaseHandler):
|
||||
|
||||
# XXX invites/joins
|
||||
# XXX 3pid invites
|
||||
# XXX directory_handler.send_room_alias_update_event
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def _move_aliases_to_new_room(
|
||||
self, requester, old_room_id, new_room_id, old_room_state,
|
||||
):
|
||||
directory_handler = self.hs.get_handlers().directory_handler
|
||||
|
||||
aliases = yield self.store.get_aliases_for_room(old_room_id)
|
||||
|
||||
# check to see if we have a canonical alias.
|
||||
canonical_alias = None
|
||||
canonical_alias_event_id = old_room_state.get((EventTypes.CanonicalAlias, ""))
|
||||
if canonical_alias_event_id:
|
||||
canonical_alias_event = yield self.store.get_event(canonical_alias_event_id)
|
||||
if canonical_alias_event:
|
||||
canonical_alias = canonical_alias_event.content.get("alias", "")
|
||||
|
||||
# first we try to remove the aliases from the old room (we suppress sending
|
||||
# the room_aliases event until the end).
|
||||
#
|
||||
# Note that we'll only be able to remove aliases that (a) aren't owned by an AS,
|
||||
# and (b) unless the user is a server admin, which the user created.
|
||||
#
|
||||
# This is probably correct - given we don't allow such aliases to be deleted
|
||||
# normally, it would be odd to allow it in the case of doing a room upgrade -
|
||||
# but it makes the upgrade less effective, and you have to wonder why a room
|
||||
# admin can't remove aliases that point to that room anyway.
|
||||
# (cf https://github.com/matrix-org/synapse/issues/2360)
|
||||
#
|
||||
removed_aliases = []
|
||||
for alias_str in aliases:
|
||||
alias = RoomAlias.from_string(alias_str)
|
||||
try:
|
||||
yield directory_handler.delete_association(
|
||||
requester, alias, send_event=False,
|
||||
)
|
||||
removed_aliases.append(alias_str)
|
||||
except SynapseError as e:
|
||||
logger.warning(
|
||||
"Unable to remove alias %s from old room: %s",
|
||||
alias, e,
|
||||
)
|
||||
|
||||
# if we didn't find any aliases, or couldn't remove anyway, we can skip the rest
|
||||
# of this.
|
||||
if not removed_aliases:
|
||||
return
|
||||
|
||||
try:
|
||||
# this can fail if, for some reason, our user doesn't have perms to send
|
||||
# m.room.aliases events in the old room (note that we've already checked that
|
||||
# they have perms to send a tombstone event, so that's not terribly likely).
|
||||
#
|
||||
# If that happens, it's regrettable, but we should carry on: it's the same
|
||||
# as when you remove an alias from the directory normally - it just means that
|
||||
# the aliases event gets out of sync with the directory
|
||||
# (cf https://github.com/vector-im/riot-web/issues/2369)
|
||||
yield directory_handler.send_room_alias_update_event(
|
||||
requester, old_room_id,
|
||||
)
|
||||
except AuthError as e:
|
||||
logger.warning(
|
||||
"Failed to send updated alias event on old room: %s", e,
|
||||
)
|
||||
|
||||
# we can now add any aliases we successfully removed to the new room.
|
||||
for alias in removed_aliases:
|
||||
try:
|
||||
yield directory_handler.create_association(
|
||||
requester, RoomAlias.from_string(alias),
|
||||
new_room_id, servers=(self.hs.hostname, ),
|
||||
send_event=False,
|
||||
)
|
||||
logger.info("Moved alias %s to new room", alias)
|
||||
except SynapseError as e:
|
||||
# I'm not really expecting this to happen, but it could if the spam
|
||||
# checking module decides it shouldn't, or similar.
|
||||
logger.error(
|
||||
"Error adding alias %s to new room: %s",
|
||||
alias, e,
|
||||
)
|
||||
|
||||
try:
|
||||
if canonical_alias and (canonical_alias in removed_aliases):
|
||||
yield self.event_creation_handler.create_and_send_nonmember_event(
|
||||
requester,
|
||||
{
|
||||
"type": EventTypes.CanonicalAlias,
|
||||
"state_key": "",
|
||||
"room_id": new_room_id,
|
||||
"sender": requester.user.to_string(),
|
||||
"content": {"alias": canonical_alias, },
|
||||
},
|
||||
ratelimit=False
|
||||
)
|
||||
|
||||
yield directory_handler.send_room_alias_update_event(
|
||||
requester, new_room_id,
|
||||
)
|
||||
except SynapseError as e:
|
||||
# again I'm not really expecting this to fail, but if it does, I'd rather
|
||||
# we returned the new room to the client at this point.
|
||||
logger.error(
|
||||
"Unable to send updated alias events in new room: %s", e,
|
||||
)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def create_room(self, requester, config, ratelimit=True,
|
||||
@ -522,6 +628,7 @@ class RoomCreationHandler(BaseHandler):
|
||||
@defer.inlineCallbacks
|
||||
def send(etype, content, **kwargs):
|
||||
event = create(etype, content, **kwargs)
|
||||
logger.info("Sending %s in new room", etype)
|
||||
yield self.event_creation_handler.create_and_send_nonmember_event(
|
||||
creator,
|
||||
event,
|
||||
@ -544,6 +651,7 @@ class RoomCreationHandler(BaseHandler):
|
||||
content=creation_content,
|
||||
)
|
||||
|
||||
logger.info("Sending %s in new room", EventTypes.Member)
|
||||
yield self.room_member_handler.update_membership(
|
||||
creator,
|
||||
creator.user,
|
||||
|
||||
@ -24,6 +24,7 @@ from synapse.api.constants import EventTypes, Membership
|
||||
from synapse.api.errors import SynapseError
|
||||
from synapse.api.filtering import Filter
|
||||
from synapse.events.utils import serialize_event
|
||||
from synapse.storage.state import StateFilter
|
||||
from synapse.visibility import filter_events_for_client
|
||||
|
||||
from ._base import BaseHandler
|
||||
@ -324,9 +325,12 @@ class SearchHandler(BaseHandler):
|
||||
else:
|
||||
last_event_id = event.event_id
|
||||
|
||||
state_filter = StateFilter.from_types(
|
||||
[(EventTypes.Member, sender) for sender in senders]
|
||||
)
|
||||
|
||||
state = yield self.store.get_state_for_event(
|
||||
last_event_id,
|
||||
types=[(EventTypes.Member, sender) for sender in senders]
|
||||
last_event_id, state_filter
|
||||
)
|
||||
|
||||
res["profile_info"] = {
|
||||
|
||||
@ -63,11 +63,8 @@ class TypingHandler(object):
|
||||
self._member_typing_until = {} # clock time we expect to stop
|
||||
self._member_last_federation_poke = {}
|
||||
|
||||
# map room IDs to serial numbers
|
||||
self._room_serials = {}
|
||||
self._latest_room_serial = 0
|
||||
# map room IDs to sets of users currently typing
|
||||
self._room_typing = {}
|
||||
self._reset()
|
||||
|
||||
# caches which room_ids changed at which serials
|
||||
self._typing_stream_change_cache = StreamChangeCache(
|
||||
@ -79,6 +76,15 @@ class TypingHandler(object):
|
||||
5000,
|
||||
)
|
||||
|
||||
def _reset(self):
|
||||
"""
|
||||
Reset the typing handler's data caches.
|
||||
"""
|
||||
# map room IDs to serial numbers
|
||||
self._room_serials = {}
|
||||
# map room IDs to sets of users currently typing
|
||||
self._room_typing = {}
|
||||
|
||||
def _handle_timeouts(self):
|
||||
logger.info("Checking for typing timeouts")
|
||||
|
||||
|
||||
@ -468,13 +468,13 @@ def set_cors_headers(request):
|
||||
Args:
|
||||
request (twisted.web.http.Request): The http request to add CORs to.
|
||||
"""
|
||||
request.setHeader("Access-Control-Allow-Origin", "*")
|
||||
request.setHeader(b"Access-Control-Allow-Origin", b"*")
|
||||
request.setHeader(
|
||||
"Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS"
|
||||
b"Access-Control-Allow-Methods", b"GET, POST, PUT, DELETE, OPTIONS"
|
||||
)
|
||||
request.setHeader(
|
||||
"Access-Control-Allow-Headers",
|
||||
"Origin, X-Requested-With, Content-Type, Accept, Authorization"
|
||||
b"Access-Control-Allow-Headers",
|
||||
b"Origin, X-Requested-With, Content-Type, Accept, Authorization"
|
||||
)
|
||||
|
||||
|
||||
|
||||
@ -121,16 +121,15 @@ def parse_string(request, name, default=None, required=False,
|
||||
|
||||
Args:
|
||||
request: the twisted HTTP request.
|
||||
name (bytes/unicode): the name of the query parameter.
|
||||
default (bytes/unicode|None): value to use if the parameter is absent,
|
||||
name (bytes|unicode): the name of the query parameter.
|
||||
default (bytes|unicode|None): value to use if the parameter is absent,
|
||||
defaults to None. Must be bytes if encoding is None.
|
||||
required (bool): whether to raise a 400 SynapseError if the
|
||||
parameter is absent, defaults to False.
|
||||
allowed_values (list[bytes/unicode]): List of allowed values for the
|
||||
allowed_values (list[bytes|unicode]): List of allowed values for the
|
||||
string, or None if any value is allowed, defaults to None. Must be
|
||||
the same type as name, if given.
|
||||
encoding: The encoding to decode the name to, and decode the string
|
||||
content with.
|
||||
encoding (str|None): The encoding to decode the string content with.
|
||||
|
||||
Returns:
|
||||
bytes/unicode|None: A string value or the default. Unicode if encoding
|
||||
|
||||
@ -85,7 +85,10 @@ class EmailPusher(object):
|
||||
self.timed_call = None
|
||||
|
||||
def on_new_notifications(self, min_stream_ordering, max_stream_ordering):
|
||||
self.max_stream_ordering = max(max_stream_ordering, self.max_stream_ordering)
|
||||
if self.max_stream_ordering:
|
||||
self.max_stream_ordering = max(max_stream_ordering, self.max_stream_ordering)
|
||||
else:
|
||||
self.max_stream_ordering = max_stream_ordering
|
||||
self._start_processing()
|
||||
|
||||
def on_new_receipts(self, min_stream_id, max_stream_id):
|
||||
|
||||
@ -311,10 +311,10 @@ class HttpPusher(object):
|
||||
]
|
||||
}
|
||||
}
|
||||
if event.type == 'm.room.member':
|
||||
if event.type == 'm.room.member' and event.is_state():
|
||||
d['notification']['membership'] = event.content['membership']
|
||||
d['notification']['user_is_target'] = event.state_key == self.user_id
|
||||
if self.hs.config.push_include_content and 'content' in event:
|
||||
if self.hs.config.push_include_content and event.content:
|
||||
d['notification']['content'] = event.content
|
||||
|
||||
# We no longer send aliases separately, instead, we send the human
|
||||
|
||||
@ -26,7 +26,6 @@ import bleach
|
||||
import jinja2
|
||||
|
||||
from twisted.internet import defer
|
||||
from twisted.mail.smtp import sendmail
|
||||
|
||||
from synapse.api.constants import EventTypes
|
||||
from synapse.api.errors import StoreError
|
||||
@ -85,6 +84,7 @@ class Mailer(object):
|
||||
self.notif_template_html = notif_template_html
|
||||
self.notif_template_text = notif_template_text
|
||||
|
||||
self.sendmail = self.hs.get_sendmail()
|
||||
self.store = self.hs.get_datastore()
|
||||
self.macaroon_gen = self.hs.get_macaroon_generator()
|
||||
self.state_handler = self.hs.get_state_handler()
|
||||
@ -191,11 +191,11 @@ class Mailer(object):
|
||||
multipart_msg.attach(html_part)
|
||||
|
||||
logger.info("Sending email push notification to %s" % email_address)
|
||||
# logger.debug(html_text)
|
||||
|
||||
yield sendmail(
|
||||
yield self.sendmail(
|
||||
self.hs.config.email_smtp_host,
|
||||
raw_from, raw_to, multipart_msg.as_string(),
|
||||
raw_from, raw_to, multipart_msg.as_string().encode('utf8'),
|
||||
reactor=self.hs.get_reactor(),
|
||||
port=self.hs.config.email_smtp_port,
|
||||
requireAuthentication=self.hs.config.email_smtp_user is not None,
|
||||
username=self.hs.config.email_smtp_user,
|
||||
@ -333,7 +333,7 @@ class Mailer(object):
|
||||
notif_events, user_id, reason):
|
||||
if len(notifs_by_room) == 1:
|
||||
# Only one room has new stuff
|
||||
room_id = notifs_by_room.keys()[0]
|
||||
room_id = list(notifs_by_room.keys())[0]
|
||||
|
||||
# If the room has some kind of name, use it, but we don't
|
||||
# want the generated-from-names one here otherwise we'll
|
||||
|
||||
@ -124,7 +124,7 @@ class PushRuleEvaluatorForEvent(object):
|
||||
|
||||
# XXX: optimisation: cache our pattern regexps
|
||||
if condition['key'] == 'content.body':
|
||||
body = self._event["content"].get("body", None)
|
||||
body = self._event.content.get("body", None)
|
||||
if not body:
|
||||
return False
|
||||
|
||||
@ -140,7 +140,7 @@ class PushRuleEvaluatorForEvent(object):
|
||||
if not display_name:
|
||||
return False
|
||||
|
||||
body = self._event["content"].get("body", None)
|
||||
body = self._event.content.get("body", None)
|
||||
if not body:
|
||||
return False
|
||||
|
||||
|
||||
@ -68,6 +68,29 @@ function captchaDone() {
|
||||
</html>
|
||||
"""
|
||||
|
||||
TERMS_TEMPLATE = """
|
||||
<html>
|
||||
<head>
|
||||
<title>Authentication</title>
|
||||
<meta name='viewport' content='width=device-width, initial-scale=1,
|
||||
user-scalable=no, minimum-scale=1.0, maximum-scale=1.0'>
|
||||
<link rel="stylesheet" href="/_matrix/static/client/register/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<form id="registrationForm" method="post" action="%(myurl)s">
|
||||
<div>
|
||||
<p>
|
||||
Please click the button below if you agree to the
|
||||
<a href="%(terms_url)s">privacy policy of this homeserver.</a>
|
||||
</p>
|
||||
<input type="hidden" name="session" value="%(session)s" />
|
||||
<input type="submit" value="Agree" />
|
||||
</div>
|
||||
</form>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
SUCCESS_TEMPLATE = """
|
||||
<html>
|
||||
<head>
|
||||
@ -130,6 +153,27 @@ class AuthRestServlet(RestServlet):
|
||||
request.setHeader(b"Content-Type", b"text/html; charset=utf-8")
|
||||
request.setHeader(b"Content-Length", b"%d" % (len(html_bytes),))
|
||||
|
||||
request.write(html_bytes)
|
||||
finish_request(request)
|
||||
defer.returnValue(None)
|
||||
elif stagetype == LoginType.TERMS:
|
||||
session = request.args['session'][0]
|
||||
|
||||
html = TERMS_TEMPLATE % {
|
||||
'session': session,
|
||||
'terms_url': "%s/_matrix/consent?v=%s" % (
|
||||
self.hs.config.public_baseurl,
|
||||
self.hs.config.user_consent_version,
|
||||
),
|
||||
'myurl': "%s/auth/%s/fallback/web" % (
|
||||
CLIENT_V2_ALPHA_PREFIX, LoginType.TERMS
|
||||
),
|
||||
}
|
||||
html_bytes = html.encode("utf8")
|
||||
request.setResponseCode(200)
|
||||
request.setHeader(b"Content-Type", b"text/html; charset=utf-8")
|
||||
request.setHeader(b"Content-Length", b"%d" % (len(html_bytes),))
|
||||
|
||||
request.write(html_bytes)
|
||||
finish_request(request)
|
||||
defer.returnValue(None)
|
||||
@ -139,7 +183,7 @@ class AuthRestServlet(RestServlet):
|
||||
@defer.inlineCallbacks
|
||||
def on_POST(self, request, stagetype):
|
||||
yield
|
||||
if stagetype == "m.login.recaptcha":
|
||||
if stagetype == LoginType.RECAPTCHA:
|
||||
if ('g-recaptcha-response' not in request.args or
|
||||
len(request.args['g-recaptcha-response'])) == 0:
|
||||
raise SynapseError(400, "No captcha response supplied")
|
||||
@ -178,6 +222,41 @@ class AuthRestServlet(RestServlet):
|
||||
request.write(html_bytes)
|
||||
finish_request(request)
|
||||
|
||||
defer.returnValue(None)
|
||||
elif stagetype == LoginType.TERMS:
|
||||
if ('session' not in request.args or
|
||||
len(request.args['session'])) == 0:
|
||||
raise SynapseError(400, "No session supplied")
|
||||
|
||||
session = request.args['session'][0]
|
||||
authdict = {'session': session}
|
||||
|
||||
success = yield self.auth_handler.add_oob_auth(
|
||||
LoginType.TERMS,
|
||||
authdict,
|
||||
self.hs.get_ip_from_request(request)
|
||||
)
|
||||
|
||||
if success:
|
||||
html = SUCCESS_TEMPLATE
|
||||
else:
|
||||
html = TERMS_TEMPLATE % {
|
||||
'session': session,
|
||||
'terms_url': "%s/_matrix/consent?v=%s" % (
|
||||
self.hs.config.public_baseurl,
|
||||
self.hs.config.user_consent_version,
|
||||
),
|
||||
'myurl': "%s/auth/%s/fallback/web" % (
|
||||
CLIENT_V2_ALPHA_PREFIX, LoginType.TERMS
|
||||
),
|
||||
}
|
||||
html_bytes = html.encode("utf8")
|
||||
request.setResponseCode(200)
|
||||
request.setHeader(b"Content-Type", b"text/html; charset=utf-8")
|
||||
request.setHeader(b"Content-Length", b"%d" % (len(html_bytes),))
|
||||
|
||||
request.write(html_bytes)
|
||||
finish_request(request)
|
||||
defer.returnValue(None)
|
||||
else:
|
||||
raise SynapseError(404, "Unknown auth stage type")
|
||||
|
||||
@ -359,6 +359,13 @@ class RegisterRestServlet(RestServlet):
|
||||
[LoginType.MSISDN, LoginType.EMAIL_IDENTITY]
|
||||
])
|
||||
|
||||
# Append m.login.terms to all flows if we're requiring consent
|
||||
if self.hs.config.user_consent_at_registration:
|
||||
new_flows = []
|
||||
for flow in flows:
|
||||
flow.append(LoginType.TERMS)
|
||||
flows.extend(new_flows)
|
||||
|
||||
auth_result, params, session_id = yield self.auth_handler.check_auth(
|
||||
flows, body, self.hs.get_ip_from_request(request)
|
||||
)
|
||||
@ -445,6 +452,12 @@ class RegisterRestServlet(RestServlet):
|
||||
params.get("bind_msisdn")
|
||||
)
|
||||
|
||||
if auth_result and LoginType.TERMS in auth_result:
|
||||
logger.info("%s has consented to the privacy policy" % registered_user_id)
|
||||
yield self.store.user_set_consent_version(
|
||||
registered_user_id, self.hs.config.user_consent_version,
|
||||
)
|
||||
|
||||
defer.returnValue((200, return_dict))
|
||||
|
||||
def on_OPTIONS(self, _):
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user