mirror of
https://mau.dev/maunium/synapse.git
synced 2024-08-27 12:42:39 +00:00
Synapse 1.39.0rc3 (2021-07-28)
============================== Bugfixes -------- - Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477)) - Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485)) - Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486)) Internal Changes ---------------- - Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461)) - Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465)) -----BEGIN PGP SIGNATURE----- iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAmEBUJUQHGVyaWtAbWF0 cml4Lm9yZwAKCRClQuTtGw+sCWi6CACfA3B7rXfQIO893mctSZkNhtAG/e4o310M etvjAtovKmKeFqBvm89FPmFoXvGA2ypoz7Jz2fdkP56DNKIXvihnDc8EeQ3gGawf hQ8GBjICOL1e7R/4qzuOCWYdppBGoGNAbz7qyxAUxZ/XOzsPJ2VBGkgyXWZkyPe/ nJYsuMuMp117HUwTqPqs3oZuTN0MRTy6xgaDfbFbcX9UC/amLmGIWOhlme8iV15r HQAW0X7F2Un5h7eQwogWTDb9gBHKnJ4ApaSJLfZl9E72i0Sv7o9UFcLIdy2DRVSq gxlRBVzVU6ZndCQ4vQXFSjQ20VGNXc2vpKVRE1D6YXxmxVNbBQB4 =lb6H -----END PGP SIGNATURE----- Merge tag 'v1.39.0rc3' Synapse 1.39.0rc3 (2021-07-28) ============================== Bugfixes -------- - Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477)) - Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485)) - Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486)) Internal Changes ---------------- - Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461)) - Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465))
This commit is contained in:
commit
c36c277790
@ -1,78 +0,0 @@
|
|||||||
version: 2.1
|
|
||||||
jobs:
|
|
||||||
dockerhubuploadrelease:
|
|
||||||
docker:
|
|
||||||
- image: docker:git
|
|
||||||
steps:
|
|
||||||
- checkout
|
|
||||||
- docker_prepare
|
|
||||||
- run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
|
|
||||||
# for release builds, we want to get the amd64 image out asap, so first
|
|
||||||
# we do an amd64-only build, before following up with a multiarch build.
|
|
||||||
- docker_build:
|
|
||||||
tag: -t matrixdotorg/synapse:${CIRCLE_TAG}
|
|
||||||
platforms: linux/amd64
|
|
||||||
- docker_build:
|
|
||||||
tag: -t matrixdotorg/synapse:${CIRCLE_TAG}
|
|
||||||
platforms: linux/amd64,linux/arm64
|
|
||||||
|
|
||||||
dockerhubuploadlatest:
|
|
||||||
docker:
|
|
||||||
- image: docker:git
|
|
||||||
steps:
|
|
||||||
- checkout
|
|
||||||
- docker_prepare
|
|
||||||
- run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
|
|
||||||
# for `latest`, we don't want the arm images to disappear, so don't update the tag
|
|
||||||
# until all of the platforms are built.
|
|
||||||
- docker_build:
|
|
||||||
tag: -t matrixdotorg/synapse:latest
|
|
||||||
platforms: linux/amd64,linux/arm64
|
|
||||||
|
|
||||||
workflows:
|
|
||||||
build:
|
|
||||||
jobs:
|
|
||||||
- dockerhubuploadrelease:
|
|
||||||
filters:
|
|
||||||
tags:
|
|
||||||
only: /v[0-9].[0-9]+.[0-9]+.*/
|
|
||||||
branches:
|
|
||||||
ignore: /.*/
|
|
||||||
- dockerhubuploadlatest:
|
|
||||||
filters:
|
|
||||||
branches:
|
|
||||||
only: [ master, main ]
|
|
||||||
|
|
||||||
commands:
|
|
||||||
docker_prepare:
|
|
||||||
description: Sets up a remote docker server, downloads the buildx cli plugin, and enables multiarch images
|
|
||||||
parameters:
|
|
||||||
buildx_version:
|
|
||||||
type: string
|
|
||||||
default: "v0.4.1"
|
|
||||||
steps:
|
|
||||||
- setup_remote_docker:
|
|
||||||
# 19.03.13 was the most recent available on circleci at the time of
|
|
||||||
# writing.
|
|
||||||
version: 19.03.13
|
|
||||||
- run: apk add --no-cache curl
|
|
||||||
- run: mkdir -vp ~/.docker/cli-plugins/ ~/dockercache
|
|
||||||
- run: curl --silent -L "https://github.com/docker/buildx/releases/download/<< parameters.buildx_version >>/buildx-<< parameters.buildx_version >>.linux-amd64" > ~/.docker/cli-plugins/docker-buildx
|
|
||||||
- run: chmod a+x ~/.docker/cli-plugins/docker-buildx
|
|
||||||
# install qemu links in /proc/sys/fs/binfmt_misc on the docker instance running the circleci job
|
|
||||||
- run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
|
|
||||||
# create a context named `builder` for the builds
|
|
||||||
- run: docker context create builder
|
|
||||||
# create a buildx builder using the new context, and set it as the default
|
|
||||||
- run: docker buildx create builder --use
|
|
||||||
|
|
||||||
docker_build:
|
|
||||||
description: Builds and pushed images to dockerhub using buildx
|
|
||||||
parameters:
|
|
||||||
platforms:
|
|
||||||
type: string
|
|
||||||
default: linux/amd64
|
|
||||||
tag:
|
|
||||||
type: string
|
|
||||||
steps:
|
|
||||||
- run: docker buildx build -f docker/Dockerfile --push --platform << parameters.platforms >> --label gitsha1=${CIRCLE_SHA1} << parameters.tag >> --progress=plain .
|
|
72
.github/workflows/docker.yml
vendored
Normal file
72
.github/workflows/docker.yml
vendored
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
# GitHub actions workflow which builds and publishes the docker images.
|
||||||
|
|
||||||
|
name: Build docker images
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
tags: ["v*"]
|
||||||
|
branches: [ master, main ]
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
build:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Set up QEMU
|
||||||
|
id: qemu
|
||||||
|
uses: docker/setup-qemu-action@v1
|
||||||
|
with:
|
||||||
|
platforms: arm64
|
||||||
|
|
||||||
|
- name: Set up Docker Buildx
|
||||||
|
id: buildx
|
||||||
|
uses: docker/setup-buildx-action@v1
|
||||||
|
|
||||||
|
- name: Inspect builder
|
||||||
|
run: docker buildx inspect
|
||||||
|
|
||||||
|
- name: Log in to DockerHub
|
||||||
|
uses: docker/login-action@v1
|
||||||
|
with:
|
||||||
|
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||||
|
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||||
|
|
||||||
|
- name: Calculate docker image tag
|
||||||
|
id: set-tag
|
||||||
|
run: |
|
||||||
|
case "${GITHUB_REF}" in
|
||||||
|
refs/heads/master|refs/heads/main)
|
||||||
|
tag=latest
|
||||||
|
;;
|
||||||
|
refs/tags/*)
|
||||||
|
tag=${GITHUB_REF#refs/tags/}
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
tag=${GITHUB_SHA}
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
echo "::set-output name=tag::$tag"
|
||||||
|
|
||||||
|
# for release builds, we want to get the amd64 image out asap, so first
|
||||||
|
# we do an amd64-only build, before following up with a multiarch build.
|
||||||
|
- name: Build and push amd64
|
||||||
|
uses: docker/build-push-action@v2
|
||||||
|
if: "${{ startsWith(github.ref, 'refs/tags/v') }}"
|
||||||
|
with:
|
||||||
|
push: true
|
||||||
|
labels: "gitsha1=${{ github.sha }}"
|
||||||
|
tags: "matrixdotorg/synapse:${{ steps.set-tag.outputs.tag }}"
|
||||||
|
file: "docker/Dockerfile"
|
||||||
|
platforms: linux/amd64
|
||||||
|
|
||||||
|
- name: Build and push all platforms
|
||||||
|
uses: docker/build-push-action@v2
|
||||||
|
with:
|
||||||
|
push: true
|
||||||
|
labels: "gitsha1=${{ github.sha }}"
|
||||||
|
tags: "matrixdotorg/synapse:${{ steps.set-tag.outputs.tag }}"
|
||||||
|
file: "docker/Dockerfile"
|
||||||
|
platforms: linux/amd64,linux/arm64
|
21
.github/workflows/release-artifacts.yml
vendored
21
.github/workflows/release-artifacts.yml
vendored
@ -3,28 +3,33 @@
|
|||||||
name: Build release artifacts
|
name: Build release artifacts
|
||||||
|
|
||||||
on:
|
on:
|
||||||
|
# we build on PRs and develop to (hopefully) get early warning
|
||||||
|
# of things breaking (but only build one set of debs)
|
||||||
|
pull_request:
|
||||||
push:
|
push:
|
||||||
# we build on develop and release branches to (hopefully) get early warning
|
branches: ["develop"]
|
||||||
# of things breaking
|
|
||||||
branches: ["develop", "release-*"]
|
|
||||||
|
|
||||||
# we also rebuild on tags, so that we can be sure of picking the artifacts
|
# we do the full build on tags.
|
||||||
# from the right tag.
|
|
||||||
tags: ["v*"]
|
tags: ["v*"]
|
||||||
|
|
||||||
permissions:
|
permissions:
|
||||||
contents: write
|
contents: write
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
# first get the list of distros to build for.
|
|
||||||
get-distros:
|
get-distros:
|
||||||
|
name: "Calculate list of debian distros"
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v2
|
- uses: actions/checkout@v2
|
||||||
- uses: actions/setup-python@v2
|
- uses: actions/setup-python@v2
|
||||||
- id: set-distros
|
- id: set-distros
|
||||||
run: |
|
run: |
|
||||||
echo "::set-output name=distros::$(scripts-dev/build_debian_packages --show-dists-json)"
|
# if we're running from a tag, get the full list of distros; otherwise just use debian:sid
|
||||||
|
dists='["debian:sid"]'
|
||||||
|
if [[ $GITHUB_REF == refs/tags/* ]]; then
|
||||||
|
dists=$(scripts-dev/build_debian_packages --show-dists-json)
|
||||||
|
fi
|
||||||
|
echo "::set-output name=distros::$dists"
|
||||||
# map the step outputs to job outputs
|
# map the step outputs to job outputs
|
||||||
outputs:
|
outputs:
|
||||||
distros: ${{ steps.set-distros.outputs.distros }}
|
distros: ${{ steps.set-distros.outputs.distros }}
|
||||||
@ -66,7 +71,7 @@ jobs:
|
|||||||
# if it's a tag, create a release and attach the artifacts to it
|
# if it's a tag, create a release and attach the artifacts to it
|
||||||
attach-assets:
|
attach-assets:
|
||||||
name: "Attach assets to release"
|
name: "Attach assets to release"
|
||||||
if: startsWith(github.ref, 'refs/tags/')
|
if: ${{ !failure() && !cancelled() && startsWith(github.ref, 'refs/tags/') }}
|
||||||
needs:
|
needs:
|
||||||
- build-debs
|
- build-debs
|
||||||
- build-sdist
|
- build-sdist
|
||||||
|
26
.github/workflows/tests.yml
vendored
26
.github/workflows/tests.yml
vendored
@ -65,14 +65,14 @@ jobs:
|
|||||||
|
|
||||||
# Dummy step to gate other tests on without repeating the whole list
|
# Dummy step to gate other tests on without repeating the whole list
|
||||||
linting-done:
|
linting-done:
|
||||||
if: ${{ always() }} # Run this even if prior jobs were skipped
|
if: ${{ !cancelled() }} # Run this even if prior jobs were skipped
|
||||||
needs: [lint, lint-crlf, lint-newsfile, lint-sdist]
|
needs: [lint, lint-crlf, lint-newsfile, lint-sdist]
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- run: "true"
|
- run: "true"
|
||||||
|
|
||||||
trial:
|
trial:
|
||||||
if: ${{ !failure() }} # Allow previous steps to be skipped, but not fail
|
if: ${{ !cancelled() && !failure() }} # Allow previous steps to be skipped, but not fail
|
||||||
needs: linting-done
|
needs: linting-done
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
strategy:
|
||||||
@ -131,7 +131,7 @@ jobs:
|
|||||||
|| true
|
|| true
|
||||||
|
|
||||||
trial-olddeps:
|
trial-olddeps:
|
||||||
if: ${{ !failure() }} # Allow previous steps to be skipped, but not fail
|
if: ${{ !cancelled() && !failure() }} # Allow previous steps to be skipped, but not fail
|
||||||
needs: linting-done
|
needs: linting-done
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
@ -156,7 +156,7 @@ jobs:
|
|||||||
|
|
||||||
trial-pypy:
|
trial-pypy:
|
||||||
# Very slow; only run if the branch name includes 'pypy'
|
# Very slow; only run if the branch name includes 'pypy'
|
||||||
if: ${{ contains(github.ref, 'pypy') && !failure() }}
|
if: ${{ contains(github.ref, 'pypy') && !failure() && !cancelled() }}
|
||||||
needs: linting-done
|
needs: linting-done
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
strategy:
|
||||||
@ -185,7 +185,7 @@ jobs:
|
|||||||
|| true
|
|| true
|
||||||
|
|
||||||
sytest:
|
sytest:
|
||||||
if: ${{ !failure() }}
|
if: ${{ !failure() && !cancelled() }}
|
||||||
needs: linting-done
|
needs: linting-done
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
container:
|
container:
|
||||||
@ -245,7 +245,7 @@ jobs:
|
|||||||
/logs/**/*.log*
|
/logs/**/*.log*
|
||||||
|
|
||||||
portdb:
|
portdb:
|
||||||
if: ${{ !failure() }} # Allow previous steps to be skipped, but not fail
|
if: ${{ !failure() && !cancelled() }} # Allow previous steps to be skipped, but not fail
|
||||||
needs: linting-done
|
needs: linting-done
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
strategy:
|
||||||
@ -286,7 +286,7 @@ jobs:
|
|||||||
- run: .buildkite/scripts/test_synapse_port_db.sh
|
- run: .buildkite/scripts/test_synapse_port_db.sh
|
||||||
|
|
||||||
complement:
|
complement:
|
||||||
if: ${{ !failure() }}
|
if: ${{ !failure() && !cancelled() }}
|
||||||
needs: linting-done
|
needs: linting-done
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
container:
|
container:
|
||||||
@ -344,3 +344,15 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
COMPLEMENT_BASE_IMAGE: complement-synapse:latest
|
COMPLEMENT_BASE_IMAGE: complement-synapse:latest
|
||||||
working-directory: complement
|
working-directory: complement
|
||||||
|
|
||||||
|
# a job which marks all the other jobs as complete, thus allowing PRs to be merged.
|
||||||
|
tests-done:
|
||||||
|
needs:
|
||||||
|
- trial
|
||||||
|
- trial-olddeps
|
||||||
|
- sytest
|
||||||
|
- portdb
|
||||||
|
- complement
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- run: "true"
|
102
CHANGES.md
102
CHANGES.md
@ -1,3 +1,36 @@
|
|||||||
|
Synapse 1.39.0rc3 (2021-07-28)
|
||||||
|
==============================
|
||||||
|
|
||||||
|
Bugfixes
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477))
|
||||||
|
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485))
|
||||||
|
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486))
|
||||||
|
|
||||||
|
|
||||||
|
Internal Changes
|
||||||
|
----------------
|
||||||
|
|
||||||
|
- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461))
|
||||||
|
- Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465))
|
||||||
|
|
||||||
|
|
||||||
|
Synapse 1.39.0rc2 (2021-07-22)
|
||||||
|
==============================
|
||||||
|
|
||||||
|
Bugfixes
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Always include `device_one_time_keys_count` key in `/sync` response to work around a bug in Element Android that broke encryption for new devices. ([\#10457](https://github.com/matrix-org/synapse/issues/10457))
|
||||||
|
|
||||||
|
|
||||||
|
Internal Changes
|
||||||
|
----------------
|
||||||
|
|
||||||
|
- Move docker image build to Github Actions. ([\#10416](https://github.com/matrix-org/synapse/issues/10416))
|
||||||
|
|
||||||
|
|
||||||
Synapse 1.38.1 (2021-07-22)
|
Synapse 1.38.1 (2021-07-22)
|
||||||
===========================
|
===========================
|
||||||
|
|
||||||
@ -7,6 +40,75 @@ Bugfixes
|
|||||||
- Always include `device_one_time_keys_count` key in `/sync` response to work around a bug in Element Android that broke encryption for new devices. ([\#10457](https://github.com/matrix-org/synapse/issues/10457))
|
- Always include `device_one_time_keys_count` key in `/sync` response to work around a bug in Element Android that broke encryption for new devices. ([\#10457](https://github.com/matrix-org/synapse/issues/10457))
|
||||||
|
|
||||||
|
|
||||||
|
Synapse 1.39.0rc1 (2021-07-20)
|
||||||
|
==============================
|
||||||
|
|
||||||
|
The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.
|
||||||
|
|
||||||
|
Features
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Add the ability to override the account validity feature with a module. ([\#9884](https://github.com/matrix-org/synapse/issues/9884))
|
||||||
|
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](https://github.com/matrix-org/synapse/issues/10298), [\#10305](https://github.com/matrix-org/synapse/issues/10305))
|
||||||
|
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](https://github.com/matrix-org/synapse/issues/10332), [\#10427](https://github.com/matrix-org/synapse/issues/10427))
|
||||||
|
- Allow providing credentials to `http_proxy`. ([\#10360](https://github.com/matrix-org/synapse/issues/10360))
|
||||||
|
|
||||||
|
|
||||||
|
Bugfixes
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](https://github.com/matrix-org/synapse/issues/10433))
|
||||||
|
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](https://github.com/matrix-org/synapse/issues/10250))
|
||||||
|
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](https://github.com/matrix-org/synapse/issues/10276))
|
||||||
|
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](https://github.com/matrix-org/synapse/issues/10317))
|
||||||
|
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](https://github.com/matrix-org/synapse/issues/10343))
|
||||||
|
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](https://github.com/matrix-org/synapse/issues/10344))
|
||||||
|
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](https://github.com/matrix-org/synapse/issues/10355))
|
||||||
|
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](https://github.com/matrix-org/synapse/issues/10359))
|
||||||
|
- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](https://github.com/matrix-org/synapse/issues/10367))
|
||||||
|
- Fix a number of logged errors caused by remote servers being down. ([\#10400](https://github.com/matrix-org/synapse/issues/10400), [\#10414](https://github.com/matrix-org/synapse/issues/10414))
|
||||||
|
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](https://github.com/matrix-org/synapse/issues/10404))
|
||||||
|
|
||||||
|
|
||||||
|
Improved Documentation
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](https://github.com/matrix-org/synapse/issues/9971))
|
||||||
|
- Simplify structure of room admin API. ([\#10313](https://github.com/matrix-org/synapse/issues/10313))
|
||||||
|
- Refresh the logcontext dev documentation. ([\#10353](https://github.com/matrix-org/synapse/issues/10353)), ([\#10337](https://github.com/matrix-org/synapse/issues/10337))
|
||||||
|
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](https://github.com/matrix-org/synapse/issues/10368))
|
||||||
|
- Fix and clarify some links in `docs` and `contrib`. ([\#10370](https://github.com/matrix-org/synapse/issues/10370)), ([\#10322](https://github.com/matrix-org/synapse/issues/10322)), ([\#10399](https://github.com/matrix-org/synapse/issues/10399))
|
||||||
|
- Make deprecation notice of the spam checker doc more obvious. ([\#10395](https://github.com/matrix-org/synapse/issues/10395))
|
||||||
|
- Add instructions on installing Debian packages for release candidates. ([\#10396](https://github.com/matrix-org/synapse/issues/10396))
|
||||||
|
|
||||||
|
|
||||||
|
Deprecations and Removals
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](https://github.com/matrix-org/synapse/issues/9721))
|
||||||
|
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](https://github.com/matrix-org/synapse/issues/10386))
|
||||||
|
|
||||||
|
|
||||||
|
Internal Changes
|
||||||
|
----------------
|
||||||
|
|
||||||
|
- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](https://github.com/matrix-org/synapse/issues/10289))
|
||||||
|
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](https://github.com/matrix-org/synapse/issues/10315))
|
||||||
|
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](https://github.com/matrix-org/synapse/issues/10316))
|
||||||
|
- Minor change to the code that populates `user_daily_visits`. ([\#10324](https://github.com/matrix-org/synapse/issues/10324))
|
||||||
|
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](https://github.com/matrix-org/synapse/issues/10345), [\#10357](https://github.com/matrix-org/synapse/issues/10357))
|
||||||
|
- Run `pyupgrade` on the codebase. ([\#10347](https://github.com/matrix-org/synapse/issues/10347), [\#10348](https://github.com/matrix-org/synapse/issues/10348))
|
||||||
|
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](https://github.com/matrix-org/synapse/issues/10349))
|
||||||
|
- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](https://github.com/matrix-org/synapse/issues/10350), [\#10380](https://github.com/matrix-org/synapse/issues/10380), [\#10381](https://github.com/matrix-org/synapse/issues/10381), [\#10382](https://github.com/matrix-org/synapse/issues/10382), [\#10418](https://github.com/matrix-org/synapse/issues/10418))
|
||||||
|
- Make the Github Actions workflow configuration more efficient. ([\#10383](https://github.com/matrix-org/synapse/issues/10383))
|
||||||
|
- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](https://github.com/matrix-org/synapse/issues/10385))
|
||||||
|
- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](https://github.com/matrix-org/synapse/issues/10391))
|
||||||
|
- Add type hints and comments to event auth code. ([\#10393](https://github.com/matrix-org/synapse/issues/10393))
|
||||||
|
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](https://github.com/matrix-org/synapse/issues/10398))
|
||||||
|
- Remove unused `events_by_room` code (tech debt). ([\#10421](https://github.com/matrix-org/synapse/issues/10421))
|
||||||
|
- Add a github actions job which records success of other jobs. ([\#10430](https://github.com/matrix-org/synapse/issues/10430))
|
||||||
|
|
||||||
|
|
||||||
Synapse 1.38.0 (2021-07-13)
|
Synapse 1.38.0 (2021-07-13)
|
||||||
===========================
|
===========================
|
||||||
|
|
||||||
|
@ -1 +0,0 @@
|
|||||||
Make deprecation notice of the spam checker doc more obvious.
|
|
@ -56,7 +56,7 @@ services:
|
|||||||
- POSTGRES_USER=synapse
|
- POSTGRES_USER=synapse
|
||||||
- POSTGRES_PASSWORD=changeme
|
- POSTGRES_PASSWORD=changeme
|
||||||
# ensure the database gets created correctly
|
# ensure the database gets created correctly
|
||||||
# https://github.com/matrix-org/synapse/blob/master/docs/postgres.md#set-up-database
|
# https://matrix-org.github.io/synapse/latest/postgres.html#set-up-database
|
||||||
- POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C
|
- POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C
|
||||||
volumes:
|
volumes:
|
||||||
# You may store the database tables in a local folder..
|
# You may store the database tables in a local folder..
|
||||||
|
@ -1,6 +1,6 @@
|
|||||||
# Using the Synapse Grafana dashboard
|
# Using the Synapse Grafana dashboard
|
||||||
|
|
||||||
0. Set up Prometheus and Grafana. Out of scope for this readme. Useful documentation about using Grafana with Prometheus: http://docs.grafana.org/features/datasources/prometheus/
|
0. Set up Prometheus and Grafana. Out of scope for this readme. Useful documentation about using Grafana with Prometheus: http://docs.grafana.org/features/datasources/prometheus/
|
||||||
1. Have your Prometheus scrape your Synapse. https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md
|
1. Have your Prometheus scrape your Synapse. https://matrix-org.github.io/synapse/latest/metrics-howto.html
|
||||||
2. Import dashboard into Grafana. Download `synapse.json`. Import it to Grafana and select the correct Prometheus datasource. http://docs.grafana.org/reference/export_import/
|
2. Import dashboard into Grafana. Download `synapse.json`. Import it to Grafana and select the correct Prometheus datasource. http://docs.grafana.org/reference/export_import/
|
||||||
3. Set up required recording rules. https://github.com/matrix-org/synapse/tree/master/contrib/prometheus
|
3. Set up required recording rules. [contrib/prometheus](../prometheus)
|
||||||
|
@ -34,7 +34,7 @@ Add a new job to the main prometheus.yml file:
|
|||||||
```
|
```
|
||||||
|
|
||||||
An example of a Prometheus configuration with workers can be found in
|
An example of a Prometheus configuration with workers can be found in
|
||||||
[metrics-howto.md](https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md).
|
[metrics-howto.md](https://matrix-org.github.io/synapse/latest/metrics-howto.html).
|
||||||
|
|
||||||
To use `synapse.rules` add
|
To use `synapse.rules` add
|
||||||
|
|
||||||
|
@ -3,8 +3,9 @@ Purge history API examples
|
|||||||
|
|
||||||
# `purge_history.sh`
|
# `purge_history.sh`
|
||||||
|
|
||||||
A bash file, that uses the [purge history API](/docs/admin_api/purge_history_api.rst) to
|
A bash file, that uses the
|
||||||
purge all messages in a list of rooms up to a certain event. You can select a
|
[purge history API](https://matrix-org.github.io/synapse/latest/admin_api/purge_history_api.html)
|
||||||
|
to purge all messages in a list of rooms up to a certain event. You can select a
|
||||||
timeframe or a number of messages that you want to keep in the room.
|
timeframe or a number of messages that you want to keep in the room.
|
||||||
|
|
||||||
Just configure the variables DOMAIN, ADMIN, ROOMS_ARRAY and TIME at the top of
|
Just configure the variables DOMAIN, ADMIN, ROOMS_ARRAY and TIME at the top of
|
||||||
@ -12,5 +13,6 @@ the script.
|
|||||||
|
|
||||||
# `purge_remote_media.sh`
|
# `purge_remote_media.sh`
|
||||||
|
|
||||||
A bash file, that uses the [purge history API](/docs/admin_api/purge_history_api.rst) to
|
A bash file, that uses the
|
||||||
purge all old cached remote media.
|
[purge history API](https://matrix-org.github.io/synapse/latest/admin_api/purge_history_api.html)
|
||||||
|
to purge all old cached remote media.
|
||||||
|
@ -1,7 +1,7 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
# this script will use the api:
|
# this script will use the api:
|
||||||
# https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.rst
|
# https://matrix-org.github.io/synapse/latest/admin_api/purge_history_api.html
|
||||||
#
|
#
|
||||||
# It will purge all messages in a list of rooms up to a cetrain event
|
# It will purge all messages in a list of rooms up to a cetrain event
|
||||||
|
|
||||||
|
@ -1,2 +1,3 @@
|
|||||||
The documentation for using systemd to manage synapse workers is now part of
|
The documentation for using systemd to manage synapse workers is now part of
|
||||||
the main synapse distribution. See [docs/systemd-with-workers](../../docs/systemd-with-workers).
|
the main synapse distribution. See
|
||||||
|
[docs/systemd-with-workers](https://matrix-org.github.io/synapse/latest/systemd-with-workers/index.html).
|
||||||
|
12
debian/changelog
vendored
12
debian/changelog
vendored
@ -1,9 +1,21 @@
|
|||||||
|
matrix-synapse-py3 (1.39.0~rc3) stable; urgency=medium
|
||||||
|
|
||||||
|
* New synapse release 1.39.0~rc3.
|
||||||
|
|
||||||
|
-- Synapse Packaging team <packages@matrix.org> Wed, 28 Jul 2021 13:30:58 +0100
|
||||||
|
|
||||||
matrix-synapse-py3 (1.38.1) stable; urgency=medium
|
matrix-synapse-py3 (1.38.1) stable; urgency=medium
|
||||||
|
|
||||||
* New synapse release 1.38.1.
|
* New synapse release 1.38.1.
|
||||||
|
|
||||||
-- Synapse Packaging team <packages@matrix.org> Thu, 22 Jul 2021 15:37:06 +0100
|
-- Synapse Packaging team <packages@matrix.org> Thu, 22 Jul 2021 15:37:06 +0100
|
||||||
|
|
||||||
|
matrix-synapse-py3 (1.39.0~rc1) stable; urgency=medium
|
||||||
|
|
||||||
|
* New synapse release 1.39.0rc1.
|
||||||
|
|
||||||
|
-- Synapse Packaging team <packages@matrix.org> Tue, 20 Jul 2021 14:28:34 +0100
|
||||||
|
|
||||||
matrix-synapse-py3 (1.38.0) stable; urgency=medium
|
matrix-synapse-py3 (1.38.0) stable; urgency=medium
|
||||||
|
|
||||||
* New synapse release 1.38.0.
|
* New synapse release 1.38.0.
|
||||||
|
@ -15,6 +15,20 @@ cd /synapse/build
|
|||||||
dch -M -l "+$DIST" "build for $DIST"
|
dch -M -l "+$DIST" "build for $DIST"
|
||||||
dch -M -r "" --force-distribution --distribution "$DIST"
|
dch -M -r "" --force-distribution --distribution "$DIST"
|
||||||
|
|
||||||
|
# if this is a prerelease, set the Section accordingly.
|
||||||
|
#
|
||||||
|
# When the package is later added to the package repo, reprepro will use the
|
||||||
|
# Section to determine which "component" it should go into (see
|
||||||
|
# https://manpages.debian.org/stretch/reprepro/reprepro.1.en.html#GUESSING)
|
||||||
|
|
||||||
|
DEB_VERSION=`dpkg-parsechangelog -SVersion`
|
||||||
|
case $DEB_VERSION in
|
||||||
|
*rc*|*a*|*b*|*c*)
|
||||||
|
sed -ie '/^Section:/c\Section: prerelease' debian/control
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
|
||||||
dpkg-buildpackage -us -uc
|
dpkg-buildpackage -us -uc
|
||||||
|
|
||||||
ls -l ..
|
ls -l ..
|
||||||
|
@ -132,7 +132,7 @@ your domain, you can simply route all traffic through the reverse proxy by
|
|||||||
updating the SRV record appropriately (or removing it, if the proxy listens on
|
updating the SRV record appropriately (or removing it, if the proxy listens on
|
||||||
8448).
|
8448).
|
||||||
|
|
||||||
See [reverse_proxy.md](reverse_proxy.md) for information on setting up a
|
See [the reverse proxy documentation](reverse_proxy.md) for information on setting up a
|
||||||
reverse proxy.
|
reverse proxy.
|
||||||
|
|
||||||
#### Option 3: add a .well-known file to delegate your matrix traffic
|
#### Option 3: add a .well-known file to delegate your matrix traffic
|
||||||
@ -303,7 +303,7 @@ We no longer actively recommend against using a reverse proxy. Many admins will
|
|||||||
find it easier to direct federation traffic to a reverse proxy and manage their
|
find it easier to direct federation traffic to a reverse proxy and manage their
|
||||||
own TLS certificates, and this is a supported configuration.
|
own TLS certificates, and this is a supported configuration.
|
||||||
|
|
||||||
See [reverse_proxy.md](reverse_proxy.md) for information on setting up a
|
See [the reverse proxy documentation](reverse_proxy.md) for information on setting up a
|
||||||
reverse proxy.
|
reverse proxy.
|
||||||
|
|
||||||
### Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?
|
### Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?
|
||||||
|
@ -47,7 +47,7 @@ The API returns a JSON body like the following:
|
|||||||
## List all media uploaded by a user
|
## List all media uploaded by a user
|
||||||
|
|
||||||
Listing all media that has been uploaded by a local user can be achieved through
|
Listing all media that has been uploaded by a local user can be achieved through
|
||||||
the use of the [List media of a user](user_admin_api.rst#list-media-of-a-user)
|
the use of the [List media of a user](user_admin_api.md#list-media-of-a-user)
|
||||||
Admin API.
|
Admin API.
|
||||||
|
|
||||||
# Quarantine media
|
# Quarantine media
|
||||||
|
@ -1,13 +1,9 @@
|
|||||||
# Contents
|
# Contents
|
||||||
- [List Room API](#list-room-api)
|
- [List Room API](#list-room-api)
|
||||||
* [Parameters](#parameters)
|
|
||||||
* [Usage](#usage)
|
|
||||||
- [Room Details API](#room-details-api)
|
- [Room Details API](#room-details-api)
|
||||||
- [Room Members API](#room-members-api)
|
- [Room Members API](#room-members-api)
|
||||||
- [Room State API](#room-state-api)
|
- [Room State API](#room-state-api)
|
||||||
- [Delete Room API](#delete-room-api)
|
- [Delete Room API](#delete-room-api)
|
||||||
* [Parameters](#parameters-1)
|
|
||||||
* [Response](#response)
|
|
||||||
* [Undoing room shutdowns](#undoing-room-shutdowns)
|
* [Undoing room shutdowns](#undoing-room-shutdowns)
|
||||||
- [Make Room Admin API](#make-room-admin-api)
|
- [Make Room Admin API](#make-room-admin-api)
|
||||||
- [Forward Extremities Admin API](#forward-extremities-admin-api)
|
- [Forward Extremities Admin API](#forward-extremities-admin-api)
|
||||||
@ -19,7 +15,7 @@ The List Room admin API allows server admins to get a list of rooms on their
|
|||||||
server. There are various parameters available that allow for filtering and
|
server. There are various parameters available that allow for filtering and
|
||||||
sorting the returned list. This API supports pagination.
|
sorting the returned list. This API supports pagination.
|
||||||
|
|
||||||
## Parameters
|
**Parameters**
|
||||||
|
|
||||||
The following query parameters are available:
|
The following query parameters are available:
|
||||||
|
|
||||||
@ -46,6 +42,8 @@ The following query parameters are available:
|
|||||||
* `search_term` - Filter rooms by their room name. Search term can be contained in any
|
* `search_term` - Filter rooms by their room name. Search term can be contained in any
|
||||||
part of the room name. Defaults to no filtering.
|
part of the room name. Defaults to no filtering.
|
||||||
|
|
||||||
|
**Response**
|
||||||
|
|
||||||
The following fields are possible in the JSON response body:
|
The following fields are possible in the JSON response body:
|
||||||
|
|
||||||
* `rooms` - An array of objects, each containing information about a room.
|
* `rooms` - An array of objects, each containing information about a room.
|
||||||
@ -79,17 +77,15 @@ The following fields are possible in the JSON response body:
|
|||||||
Use `prev_batch` for the `from` value in the next request to
|
Use `prev_batch` for the `from` value in the next request to
|
||||||
get the "previous page" of results.
|
get the "previous page" of results.
|
||||||
|
|
||||||
## Usage
|
The API is:
|
||||||
|
|
||||||
A standard request with no filtering:
|
A standard request with no filtering:
|
||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms
|
GET /_synapse/admin/v1/rooms
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```jsonc
|
```jsonc
|
||||||
{
|
{
|
||||||
@ -137,11 +133,9 @@ Filtering by room name:
|
|||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms?search_term=TWIM
|
GET /_synapse/admin/v1/rooms?search_term=TWIM
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@ -172,11 +166,9 @@ Paginating through a list of rooms:
|
|||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms?order_by=size
|
GET /_synapse/admin/v1/rooms?order_by=size
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```jsonc
|
```jsonc
|
||||||
{
|
{
|
||||||
@ -228,11 +220,9 @@ parameter to the value of `next_token`.
|
|||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms?order_by=size&from=100
|
GET /_synapse/admin/v1/rooms?order_by=size&from=100
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```jsonc
|
```jsonc
|
||||||
{
|
{
|
||||||
@ -304,17 +294,13 @@ The following fields are possible in the JSON response body:
|
|||||||
* `history_visibility` - Who can see the room history. One of: ["invited", "joined", "shared", "world_readable"].
|
* `history_visibility` - Who can see the room history. One of: ["invited", "joined", "shared", "world_readable"].
|
||||||
* `state_events` - Total number of state_events of a room. Complexity of the room.
|
* `state_events` - Total number of state_events of a room. Complexity of the room.
|
||||||
|
|
||||||
## Usage
|
The API is:
|
||||||
|
|
||||||
A standard request:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms/<room_id>
|
GET /_synapse/admin/v1/rooms/<room_id>
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@ -347,17 +333,13 @@ The response includes the following fields:
|
|||||||
* `members` - A list of all the members that are present in the room, represented by their ids.
|
* `members` - A list of all the members that are present in the room, represented by their ids.
|
||||||
* `total` - Total number of members in the room.
|
* `total` - Total number of members in the room.
|
||||||
|
|
||||||
## Usage
|
The API is:
|
||||||
|
|
||||||
A standard request:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms/<room_id>/members
|
GET /_synapse/admin/v1/rooms/<room_id>/members
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@ -378,17 +360,13 @@ The response includes the following fields:
|
|||||||
|
|
||||||
* `state` - The current state of the room at the time of request.
|
* `state` - The current state of the room at the time of request.
|
||||||
|
|
||||||
## Usage
|
The API is:
|
||||||
|
|
||||||
A standard request:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms/<room_id>/state
|
GET /_synapse/admin/v1/rooms/<room_id>/state
|
||||||
|
|
||||||
{}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Response:
|
A response body like the following is returned:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@ -432,6 +410,7 @@ DELETE /_synapse/admin/v1/rooms/<room_id>
|
|||||||
```
|
```
|
||||||
|
|
||||||
with a body of:
|
with a body of:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"new_room_user_id": "@someuser:example.com",
|
"new_room_user_id": "@someuser:example.com",
|
||||||
@ -461,7 +440,7 @@ A response body like the following is returned:
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Parameters
|
**Parameters**
|
||||||
|
|
||||||
The following parameters should be set in the URL:
|
The following parameters should be set in the URL:
|
||||||
|
|
||||||
@ -491,7 +470,7 @@ The following JSON body parameters are available:
|
|||||||
|
|
||||||
The JSON body must not be empty. The body must be at least `{}`.
|
The JSON body must not be empty. The body must be at least `{}`.
|
||||||
|
|
||||||
## Response
|
**Response**
|
||||||
|
|
||||||
The following fields are returned in the JSON response body:
|
The following fields are returned in the JSON response body:
|
||||||
|
|
||||||
@ -548,10 +527,10 @@ By default the server admin (the caller) is granted power, but another user can
|
|||||||
optionally be specified, e.g.:
|
optionally be specified, e.g.:
|
||||||
|
|
||||||
```
|
```
|
||||||
POST /_synapse/admin/v1/rooms/<room_id_or_alias>/make_room_admin
|
POST /_synapse/admin/v1/rooms/<room_id_or_alias>/make_room_admin
|
||||||
{
|
{
|
||||||
"user_id": "@foo:example.com"
|
"user_id": "@foo:example.com"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
# Forward Extremities Admin API
|
# Forward Extremities Admin API
|
||||||
@ -565,7 +544,7 @@ extremities accumulate in a room, performance can become degraded. For details,
|
|||||||
To check the status of forward extremities for a room:
|
To check the status of forward extremities for a room:
|
||||||
|
|
||||||
```
|
```
|
||||||
GET /_synapse/admin/v1/rooms/<room_id_or_alias>/forward_extremities
|
GET /_synapse/admin/v1/rooms/<room_id_or_alias>/forward_extremities
|
||||||
```
|
```
|
||||||
|
|
||||||
A response as follows will be returned:
|
A response as follows will be returned:
|
||||||
@ -594,7 +573,7 @@ If a room has lots of forward extremities, the extra can be
|
|||||||
deleted as follows:
|
deleted as follows:
|
||||||
|
|
||||||
```
|
```
|
||||||
DELETE /_synapse/admin/v1/rooms/<room_id_or_alias>/forward_extremities
|
DELETE /_synapse/admin/v1/rooms/<room_id_or_alias>/forward_extremities
|
||||||
```
|
```
|
||||||
|
|
||||||
A response as follows will be returned, indicating the amount of forward extremities
|
A response as follows will be returned, indicating the amount of forward extremities
|
||||||
|
@ -45,4 +45,4 @@ Once the notice has been sent, the API will return the following response:
|
|||||||
```
|
```
|
||||||
|
|
||||||
Note that server notices must be enabled in `homeserver.yaml` before this API
|
Note that server notices must be enabled in `homeserver.yaml` before this API
|
||||||
can be used. See [server_notices.md](../server_notices.md) for more information.
|
can be used. See [the server notices documentation](../server_notices.md) for more information.
|
||||||
|
@ -152,7 +152,7 @@ version of the policy. To do so:
|
|||||||
|
|
||||||
* ensure that the consent resource is configured, as in the previous section
|
* ensure that the consent resource is configured, as in the previous section
|
||||||
|
|
||||||
* ensure that server notices are configured, as in [server_notices.md](server_notices.md).
|
* ensure that server notices are configured, as in [the server notice documentation](server_notices.md).
|
||||||
|
|
||||||
* Add `server_notice_content` under `user_consent` in `homeserver.yaml`. For
|
* Add `server_notice_content` under `user_consent` in `homeserver.yaml`. For
|
||||||
example:
|
example:
|
||||||
|
@ -74,7 +74,7 @@ We no longer actively recommend against using a reverse proxy. Many admins will
|
|||||||
find it easier to direct federation traffic to a reverse proxy and manage their
|
find it easier to direct federation traffic to a reverse proxy and manage their
|
||||||
own TLS certificates, and this is a supported configuration.
|
own TLS certificates, and this is a supported configuration.
|
||||||
|
|
||||||
See [reverse_proxy.md](reverse_proxy.md) for information on setting up a
|
See [the reverse proxy documentation](reverse_proxy.md) for information on setting up a
|
||||||
reverse proxy.
|
reverse proxy.
|
||||||
|
|
||||||
### Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?
|
### Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?
|
||||||
|
@ -14,7 +14,7 @@ you set the `server_name` to match your machine's public DNS hostname.
|
|||||||
|
|
||||||
For this default configuration to work, you will need to listen for TLS
|
For this default configuration to work, you will need to listen for TLS
|
||||||
connections on port 8448. The preferred way to do that is by using a
|
connections on port 8448. The preferred way to do that is by using a
|
||||||
reverse proxy: see [reverse_proxy.md](reverse_proxy.md) for instructions
|
reverse proxy: see [the reverse proxy documentation](reverse_proxy.md) for instructions
|
||||||
on how to correctly set one up.
|
on how to correctly set one up.
|
||||||
|
|
||||||
In some cases you might not want to run Synapse on the machine that has
|
In some cases you might not want to run Synapse on the machine that has
|
||||||
@ -23,7 +23,7 @@ traffic to use a different port than 8448. For example, you might want to
|
|||||||
have your user names look like `@user:example.com`, but you want to run
|
have your user names look like `@user:example.com`, but you want to run
|
||||||
Synapse on `synapse.example.com` on port 443. This can be done using
|
Synapse on `synapse.example.com` on port 443. This can be done using
|
||||||
delegation, which allows an admin to control where federation traffic should
|
delegation, which allows an admin to control where federation traffic should
|
||||||
be sent. See [delegate.md](delegate.md) for instructions on how to set this up.
|
be sent. See [the delegation documentation](delegate.md) for instructions on how to set this up.
|
||||||
|
|
||||||
Once federation has been configured, you should be able to join a room over
|
Once federation has been configured, you should be able to join a room over
|
||||||
federation. A good place to start is `#synapse:matrix.org` - a room for
|
federation. A good place to start is `#synapse:matrix.org` - a room for
|
||||||
@ -44,8 +44,8 @@ a complicated dance which requires connections in both directions).
|
|||||||
|
|
||||||
Another common problem is that people on other servers can't join rooms that
|
Another common problem is that people on other servers can't join rooms that
|
||||||
you invite them to. This can be caused by an incorrectly-configured reverse
|
you invite them to. This can be caused by an incorrectly-configured reverse
|
||||||
proxy: see [reverse_proxy.md](reverse_proxy.md) for instructions on how to correctly
|
proxy: see [the reverse proxy documentation](reverse_proxy.md) for instructions on how
|
||||||
configure a reverse proxy.
|
to correctly configure a reverse proxy.
|
||||||
|
|
||||||
### Known issues
|
### Known issues
|
||||||
|
|
||||||
|
@ -14,12 +14,16 @@ The `synapse.logging.context` module provides a facilities for managing
|
|||||||
the current log context (as well as providing the `LoggingContextFilter`
|
the current log context (as well as providing the `LoggingContextFilter`
|
||||||
class).
|
class).
|
||||||
|
|
||||||
Deferreds make the whole thing complicated, so this document describes
|
Asynchronous functions make the whole thing complicated, so this document describes
|
||||||
how it all works, and how to write code which follows the rules.
|
how it all works, and how to write code which follows the rules.
|
||||||
|
|
||||||
##Logcontexts without Deferreds
|
In this document, "awaitable" refers to any object which can be `await`ed. In the context of
|
||||||
|
Synapse, that normally means either a coroutine or a Twisted
|
||||||
|
[`Deferred`](https://twistedmatrix.com/documents/current/api/twisted.internet.defer.Deferred.html).
|
||||||
|
|
||||||
In the absence of any Deferred voodoo, things are simple enough. As with
|
## Logcontexts without asynchronous code
|
||||||
|
|
||||||
|
In the absence of any asynchronous voodoo, things are simple enough. As with
|
||||||
any code of this nature, the rule is that our function should leave
|
any code of this nature, the rule is that our function should leave
|
||||||
things as it found them:
|
things as it found them:
|
||||||
|
|
||||||
@ -55,126 +59,109 @@ def do_request_handling():
|
|||||||
logger.debug("phew")
|
logger.debug("phew")
|
||||||
```
|
```
|
||||||
|
|
||||||
## Using logcontexts with Deferreds
|
## Using logcontexts with awaitables
|
||||||
|
|
||||||
Deferreds --- and in particular, `defer.inlineCallbacks` --- break the
|
Awaitables break the linear flow of code so that there is no longer a single entry point
|
||||||
linear flow of code so that there is no longer a single entry point
|
where we should set the logcontext and a single exit point where we should remove it.
|
||||||
where we should set the logcontext and a single exit point where we
|
|
||||||
should remove it.
|
|
||||||
|
|
||||||
Consider the example above, where `do_request_handling` needs to do some
|
Consider the example above, where `do_request_handling` needs to do some
|
||||||
blocking operation, and returns a deferred:
|
blocking operation, and returns an awaitable:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def handle_request(request_id):
|
||||||
def handle_request(request_id):
|
|
||||||
with context.LoggingContext() as request_context:
|
with context.LoggingContext() as request_context:
|
||||||
request_context.request = request_id
|
request_context.request = request_id
|
||||||
yield do_request_handling()
|
await do_request_handling()
|
||||||
logger.debug("finished")
|
logger.debug("finished")
|
||||||
```
|
```
|
||||||
|
|
||||||
In the above flow:
|
In the above flow:
|
||||||
|
|
||||||
- The logcontext is set
|
- The logcontext is set
|
||||||
- `do_request_handling` is called, and returns a deferred
|
- `do_request_handling` is called, and returns an awaitable
|
||||||
- `handle_request` yields the deferred
|
- `handle_request` awaits the awaitable
|
||||||
- The `inlineCallbacks` wrapper of `handle_request` returns a deferred
|
- Execution of `handle_request` is suspended
|
||||||
|
|
||||||
So we have stopped processing the request (and will probably go on to
|
So we have stopped processing the request (and will probably go on to
|
||||||
start processing the next), without clearing the logcontext.
|
start processing the next), without clearing the logcontext.
|
||||||
|
|
||||||
To circumvent this problem, synapse code assumes that, wherever you have
|
To circumvent this problem, synapse code assumes that, wherever you have
|
||||||
a deferred, you will want to yield on it. To that end, whereever
|
an awaitable, you will want to `await` it. To that end, whereever
|
||||||
functions return a deferred, we adopt the following conventions:
|
functions return awaitables, we adopt the following conventions:
|
||||||
|
|
||||||
**Rules for functions returning deferreds:**
|
**Rules for functions returning awaitables:**
|
||||||
|
|
||||||
> - If the deferred is already complete, the function returns with the
|
> - If the awaitable is already complete, the function returns with the
|
||||||
> same logcontext it started with.
|
> same logcontext it started with.
|
||||||
> - If the deferred is incomplete, the function clears the logcontext
|
> - If the awaitable is incomplete, the function clears the logcontext
|
||||||
> before returning; when the deferred completes, it restores the
|
> before returning; when the awaitable completes, it restores the
|
||||||
> logcontext before running any callbacks.
|
> logcontext before running any callbacks.
|
||||||
|
|
||||||
That sounds complicated, but actually it means a lot of code (including
|
That sounds complicated, but actually it means a lot of code (including
|
||||||
the example above) "just works". There are two cases:
|
the example above) "just works". There are two cases:
|
||||||
|
|
||||||
- If `do_request_handling` returns a completed deferred, then the
|
- If `do_request_handling` returns a completed awaitable, then the
|
||||||
logcontext will still be in place. In this case, execution will
|
logcontext will still be in place. In this case, execution will
|
||||||
continue immediately after the `yield`; the "finished" line will
|
continue immediately after the `await`; the "finished" line will
|
||||||
be logged against the right context, and the `with` block restores
|
be logged against the right context, and the `with` block restores
|
||||||
the original context before we return to the caller.
|
the original context before we return to the caller.
|
||||||
- If the returned deferred is incomplete, `do_request_handling` clears
|
- If the returned awaitable is incomplete, `do_request_handling` clears
|
||||||
the logcontext before returning. The logcontext is therefore clear
|
the logcontext before returning. The logcontext is therefore clear
|
||||||
when `handle_request` yields the deferred. At that point, the
|
when `handle_request` `await`s the awaitable.
|
||||||
`inlineCallbacks` wrapper adds a callback to the deferred, and
|
|
||||||
returns another (incomplete) deferred to the caller, and it is safe
|
|
||||||
to begin processing the next request.
|
|
||||||
|
|
||||||
Once `do_request_handling`'s deferred completes, it will reinstate
|
Once `do_request_handling`'s awaitable completes, it will reinstate
|
||||||
the logcontext, before running the callback added by the
|
the logcontext, before running the second half of `handle_request`,
|
||||||
`inlineCallbacks` wrapper. That callback runs the second half of
|
so again the "finished" line will be logged against the right context,
|
||||||
`handle_request`, so again the "finished" line will be logged
|
and the `with` block restores the original context.
|
||||||
against the right context, and the `with` block restores the
|
|
||||||
original context.
|
|
||||||
|
|
||||||
As an aside, it's worth noting that `handle_request` follows our rules
|
As an aside, it's worth noting that `handle_request` follows our rules
|
||||||
-though that only matters if the caller has its own logcontext which it
|
- though that only matters if the caller has its own logcontext which it
|
||||||
cares about.
|
cares about.
|
||||||
|
|
||||||
The following sections describe pitfalls and helpful patterns when
|
The following sections describe pitfalls and helpful patterns when
|
||||||
implementing these rules.
|
implementing these rules.
|
||||||
|
|
||||||
Always yield your deferreds
|
Always await your awaitables
|
||||||
---------------------------
|
----------------------------
|
||||||
|
|
||||||
Whenever you get a deferred back from a function, you should `yield` on
|
Whenever you get an awaitable back from a function, you should `await` on
|
||||||
it as soon as possible. (Returning it directly to your caller is ok too,
|
it as soon as possible. Do not pass go; do not do any logging; do not
|
||||||
if you're not doing `inlineCallbacks`.) Do not pass go; do not do any
|
call any other functions.
|
||||||
logging; do not call any other functions.
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def fun():
|
||||||
def fun():
|
|
||||||
logger.debug("starting")
|
logger.debug("starting")
|
||||||
yield do_some_stuff() # just like this
|
await do_some_stuff() # just like this
|
||||||
|
|
||||||
d = more_stuff()
|
coro = more_stuff()
|
||||||
result = yield d # also fine, of course
|
result = await coro # also fine, of course
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def nonInlineCallbacksFun():
|
|
||||||
logger.debug("just a wrapper really")
|
|
||||||
return do_some_stuff() # this is ok too - the caller will yield on
|
|
||||||
# it anyway.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Provided this pattern is followed all the way back up to the callchain
|
Provided this pattern is followed all the way back up to the callchain
|
||||||
to where the logcontext was set, this will make things work out ok:
|
to where the logcontext was set, this will make things work out ok:
|
||||||
provided `do_some_stuff` and `more_stuff` follow the rules above, then
|
provided `do_some_stuff` and `more_stuff` follow the rules above, then
|
||||||
so will `fun` (as wrapped by `inlineCallbacks`) and
|
so will `fun`.
|
||||||
`nonInlineCallbacksFun`.
|
|
||||||
|
|
||||||
It's all too easy to forget to `yield`: for instance if we forgot that
|
It's all too easy to forget to `await`: for instance if we forgot that
|
||||||
`do_some_stuff` returned a deferred, we might plough on regardless. This
|
`do_some_stuff` returned an awaitable, we might plough on regardless. This
|
||||||
leads to a mess; it will probably work itself out eventually, but not
|
leads to a mess; it will probably work itself out eventually, but not
|
||||||
before a load of stuff has been logged against the wrong context.
|
before a load of stuff has been logged against the wrong context.
|
||||||
(Normally, other things will break, more obviously, if you forget to
|
(Normally, other things will break, more obviously, if you forget to
|
||||||
`yield`, so this tends not to be a major problem in practice.)
|
`await`, so this tends not to be a major problem in practice.)
|
||||||
|
|
||||||
Of course sometimes you need to do something a bit fancier with your
|
Of course sometimes you need to do something a bit fancier with your
|
||||||
Deferreds - not all code follows the linear A-then-B-then-C pattern.
|
awaitable - not all code follows the linear A-then-B-then-C pattern.
|
||||||
Notes on implementing more complex patterns are in later sections.
|
Notes on implementing more complex patterns are in later sections.
|
||||||
|
|
||||||
## Where you create a new Deferred, make it follow the rules
|
## Where you create a new awaitable, make it follow the rules
|
||||||
|
|
||||||
Most of the time, a Deferred comes from another synapse function.
|
Most of the time, an awaitable comes from another synapse function.
|
||||||
Sometimes, though, we need to make up a new Deferred, or we get a
|
Sometimes, though, we need to make up a new awaitable, or we get an awaitable
|
||||||
Deferred back from external code. We need to make it follow our rules.
|
back from external code. We need to make it follow our rules.
|
||||||
|
|
||||||
The easy way to do it is with a combination of `defer.inlineCallbacks`,
|
The easy way to do it is by using `context.make_deferred_yieldable`. Suppose we want to implement
|
||||||
and `context.PreserveLoggingContext`. Suppose we want to implement
|
|
||||||
`sleep`, which returns a deferred which will run its callbacks after a
|
`sleep`, which returns a deferred which will run its callbacks after a
|
||||||
given number of seconds. That might look like:
|
given number of seconds. That might look like:
|
||||||
|
|
||||||
@ -186,25 +173,12 @@ def get_sleep_deferred(seconds):
|
|||||||
return d
|
return d
|
||||||
```
|
```
|
||||||
|
|
||||||
That doesn't follow the rules, but we can fix it by wrapping it with
|
That doesn't follow the rules, but we can fix it by calling it through
|
||||||
`PreserveLoggingContext` and `yield` ing on it:
|
`context.make_deferred_yieldable`:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def sleep(seconds):
|
||||||
def sleep(seconds):
|
return await context.make_deferred_yieldable(get_sleep_deferred(seconds))
|
||||||
with PreserveLoggingContext():
|
|
||||||
yield get_sleep_deferred(seconds)
|
|
||||||
```
|
|
||||||
|
|
||||||
This technique works equally for external functions which return
|
|
||||||
deferreds, or deferreds we have made ourselves.
|
|
||||||
|
|
||||||
You can also use `context.make_deferred_yieldable`, which just does the
|
|
||||||
boilerplate for you, so the above could be written:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def sleep(seconds):
|
|
||||||
return context.make_deferred_yieldable(get_sleep_deferred(seconds))
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Fire-and-forget
|
## Fire-and-forget
|
||||||
@ -213,20 +187,18 @@ Sometimes you want to fire off a chain of execution, but not wait for
|
|||||||
its result. That might look a bit like this:
|
its result. That might look a bit like this:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def do_request_handling():
|
||||||
def do_request_handling():
|
await foreground_operation()
|
||||||
yield foreground_operation()
|
|
||||||
|
|
||||||
# *don't* do this
|
# *don't* do this
|
||||||
background_operation()
|
background_operation()
|
||||||
|
|
||||||
logger.debug("Request handling complete")
|
logger.debug("Request handling complete")
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def background_operation():
|
||||||
def background_operation():
|
await first_background_step()
|
||||||
yield first_background_step()
|
|
||||||
logger.debug("Completed first step")
|
logger.debug("Completed first step")
|
||||||
yield second_background_step()
|
await second_background_step()
|
||||||
logger.debug("Completed second step")
|
logger.debug("Completed second step")
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -235,13 +207,13 @@ The above code does a couple of steps in the background after
|
|||||||
against the `request_context` logcontext, which may or may not be
|
against the `request_context` logcontext, which may or may not be
|
||||||
desirable. There are two big problems with the above, however. The first
|
desirable. There are two big problems with the above, however. The first
|
||||||
problem is that, if `background_operation` returns an incomplete
|
problem is that, if `background_operation` returns an incomplete
|
||||||
Deferred, it will expect its caller to `yield` immediately, so will have
|
awaitable, it will expect its caller to `await` immediately, so will have
|
||||||
cleared the logcontext. In this example, that means that 'Request
|
cleared the logcontext. In this example, that means that 'Request
|
||||||
handling complete' will be logged without any context.
|
handling complete' will be logged without any context.
|
||||||
|
|
||||||
The second problem, which is potentially even worse, is that when the
|
The second problem, which is potentially even worse, is that when the
|
||||||
Deferred returned by `background_operation` completes, it will restore
|
awaitable returned by `background_operation` completes, it will restore
|
||||||
the original logcontext. There is nothing waiting on that Deferred, so
|
the original logcontext. There is nothing waiting on that awaitable, so
|
||||||
the logcontext will leak into the reactor and possibly get attached to
|
the logcontext will leak into the reactor and possibly get attached to
|
||||||
some arbitrary future operation.
|
some arbitrary future operation.
|
||||||
|
|
||||||
@ -254,9 +226,8 @@ deferred completes will be the empty logcontext), and will restore the
|
|||||||
current logcontext before continuing the foreground process:
|
current logcontext before continuing the foreground process:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def do_request_handling():
|
||||||
def do_request_handling():
|
await foreground_operation()
|
||||||
yield foreground_operation()
|
|
||||||
|
|
||||||
# start background_operation off in the empty logcontext, to
|
# start background_operation off in the empty logcontext, to
|
||||||
# avoid leaking the current context into the reactor.
|
# avoid leaking the current context into the reactor.
|
||||||
@ -274,16 +245,15 @@ Obviously that option means that the operations done in
|
|||||||
|
|
||||||
The second option is to use `context.run_in_background`, which wraps a
|
The second option is to use `context.run_in_background`, which wraps a
|
||||||
function so that it doesn't reset the logcontext even when it returns
|
function so that it doesn't reset the logcontext even when it returns
|
||||||
an incomplete deferred, and adds a callback to the returned deferred to
|
an incomplete awaitable, and adds a callback to the returned awaitable to
|
||||||
reset the logcontext. In other words, it turns a function that follows
|
reset the logcontext. In other words, it turns a function that follows
|
||||||
the Synapse rules about logcontexts and Deferreds into one which behaves
|
the Synapse rules about logcontexts and awaitables into one which behaves
|
||||||
more like an external function --- the opposite operation to that
|
more like an external function --- the opposite operation to that
|
||||||
described in the previous section. It can be used like this:
|
described in the previous section. It can be used like this:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def do_request_handling():
|
||||||
def do_request_handling():
|
await foreground_operation()
|
||||||
yield foreground_operation()
|
|
||||||
|
|
||||||
context.run_in_background(background_operation)
|
context.run_in_background(background_operation)
|
||||||
|
|
||||||
@ -294,152 +264,53 @@ def do_request_handling():
|
|||||||
## Passing synapse deferreds into third-party functions
|
## Passing synapse deferreds into third-party functions
|
||||||
|
|
||||||
A typical example of this is where we want to collect together two or
|
A typical example of this is where we want to collect together two or
|
||||||
more deferred via `defer.gatherResults`:
|
more awaitables via `defer.gatherResults`:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
d1 = operation1()
|
a1 = operation1()
|
||||||
d2 = operation2()
|
a2 = operation2()
|
||||||
d3 = defer.gatherResults([d1, d2])
|
a3 = defer.gatherResults([a1, a2])
|
||||||
```
|
```
|
||||||
|
|
||||||
This is really a variation of the fire-and-forget problem above, in that
|
This is really a variation of the fire-and-forget problem above, in that
|
||||||
we are firing off `d1` and `d2` without yielding on them. The difference
|
we are firing off `a1` and `a2` without awaiting on them. The difference
|
||||||
is that we now have third-party code attached to their callbacks. Anyway
|
is that we now have third-party code attached to their callbacks. Anyway
|
||||||
either technique given in the [Fire-and-forget](#fire-and-forget)
|
either technique given in the [Fire-and-forget](#fire-and-forget)
|
||||||
section will work.
|
section will work.
|
||||||
|
|
||||||
Of course, the new Deferred returned by `gatherResults` needs to be
|
Of course, the new awaitable returned by `gather` needs to be
|
||||||
wrapped in order to make it follow the logcontext rules before we can
|
wrapped in order to make it follow the logcontext rules before we can
|
||||||
yield it, as described in [Where you create a new Deferred, make it
|
yield it, as described in [Where you create a new awaitable, make it
|
||||||
follow the
|
follow the
|
||||||
rules](#where-you-create-a-new-deferred-make-it-follow-the-rules).
|
rules](#where-you-create-a-new-awaitable-make-it-follow-the-rules).
|
||||||
|
|
||||||
So, option one: reset the logcontext before starting the operations to
|
So, option one: reset the logcontext before starting the operations to
|
||||||
be gathered:
|
be gathered:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def do_request_handling():
|
||||||
def do_request_handling():
|
|
||||||
with PreserveLoggingContext():
|
with PreserveLoggingContext():
|
||||||
d1 = operation1()
|
a1 = operation1()
|
||||||
d2 = operation2()
|
a2 = operation2()
|
||||||
result = yield defer.gatherResults([d1, d2])
|
result = await defer.gatherResults([a1, a2])
|
||||||
```
|
```
|
||||||
|
|
||||||
In this case particularly, though, option two, of using
|
In this case particularly, though, option two, of using
|
||||||
`context.preserve_fn` almost certainly makes more sense, so that
|
`context.run_in_background` almost certainly makes more sense, so that
|
||||||
`operation1` and `operation2` are both logged against the original
|
`operation1` and `operation2` are both logged against the original
|
||||||
logcontext. This looks like:
|
logcontext. This looks like:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@defer.inlineCallbacks
|
async def do_request_handling():
|
||||||
def do_request_handling():
|
a1 = context.run_in_background(operation1)
|
||||||
d1 = context.preserve_fn(operation1)()
|
a2 = context.run_in_background(operation2)
|
||||||
d2 = context.preserve_fn(operation2)()
|
|
||||||
|
|
||||||
with PreserveLoggingContext():
|
result = await make_deferred_yieldable(defer.gatherResults([a1, a2]))
|
||||||
result = yield defer.gatherResults([d1, d2])
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Was all this really necessary?
|
## A note on garbage-collection of awaitable chains
|
||||||
|
|
||||||
The conventions used work fine for a linear flow where everything
|
It turns out that our logcontext rules do not play nicely with awaitable
|
||||||
happens in series via `defer.inlineCallbacks` and `yield`, but are
|
|
||||||
certainly tricky to follow for any more exotic flows. It's hard not to
|
|
||||||
wonder if we could have done something else.
|
|
||||||
|
|
||||||
We're not going to rewrite Synapse now, so the following is entirely of
|
|
||||||
academic interest, but I'd like to record some thoughts on an
|
|
||||||
alternative approach.
|
|
||||||
|
|
||||||
I briefly prototyped some code following an alternative set of rules. I
|
|
||||||
think it would work, but I certainly didn't get as far as thinking how
|
|
||||||
it would interact with concepts as complicated as the cache descriptors.
|
|
||||||
|
|
||||||
My alternative rules were:
|
|
||||||
|
|
||||||
- functions always preserve the logcontext of their caller, whether or
|
|
||||||
not they are returning a Deferred.
|
|
||||||
- Deferreds returned by synapse functions run their callbacks in the
|
|
||||||
same context as the function was orignally called in.
|
|
||||||
|
|
||||||
The main point of this scheme is that everywhere that sets the
|
|
||||||
logcontext is responsible for clearing it before returning control to
|
|
||||||
the reactor.
|
|
||||||
|
|
||||||
So, for example, if you were the function which started a
|
|
||||||
`with LoggingContext` block, you wouldn't `yield` within it --- instead
|
|
||||||
you'd start off the background process, and then leave the `with` block
|
|
||||||
to wait for it:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def handle_request(request_id):
|
|
||||||
with context.LoggingContext() as request_context:
|
|
||||||
request_context.request = request_id
|
|
||||||
d = do_request_handling()
|
|
||||||
|
|
||||||
def cb(r):
|
|
||||||
logger.debug("finished")
|
|
||||||
|
|
||||||
d.addCallback(cb)
|
|
||||||
return d
|
|
||||||
```
|
|
||||||
|
|
||||||
(in general, mixing `with LoggingContext` blocks and
|
|
||||||
`defer.inlineCallbacks` in the same function leads to slighly
|
|
||||||
counter-intuitive code, under this scheme).
|
|
||||||
|
|
||||||
Because we leave the original `with` block as soon as the Deferred is
|
|
||||||
returned (as opposed to waiting for it to be resolved, as we do today),
|
|
||||||
the logcontext is cleared before control passes back to the reactor; so
|
|
||||||
if there is some code within `do_request_handling` which needs to wait
|
|
||||||
for a Deferred to complete, there is no need for it to worry about
|
|
||||||
clearing the logcontext before doing so:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def handle_request():
|
|
||||||
r = do_some_stuff()
|
|
||||||
r.addCallback(do_some_more_stuff)
|
|
||||||
return r
|
|
||||||
```
|
|
||||||
|
|
||||||
--- and provided `do_some_stuff` follows the rules of returning a
|
|
||||||
Deferred which runs its callbacks in the original logcontext, all is
|
|
||||||
happy.
|
|
||||||
|
|
||||||
The business of a Deferred which runs its callbacks in the original
|
|
||||||
logcontext isn't hard to achieve --- we have it today, in the shape of
|
|
||||||
`context._PreservingContextDeferred`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def do_some_stuff():
|
|
||||||
deferred = do_some_io()
|
|
||||||
pcd = _PreservingContextDeferred(LoggingContext.current_context())
|
|
||||||
deferred.chainDeferred(pcd)
|
|
||||||
return pcd
|
|
||||||
```
|
|
||||||
|
|
||||||
It turns out that, thanks to the way that Deferreds chain together, we
|
|
||||||
automatically get the property of a context-preserving deferred with
|
|
||||||
`defer.inlineCallbacks`, provided the final Defered the function
|
|
||||||
`yields` on has that property. So we can just write:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@defer.inlineCallbacks
|
|
||||||
def handle_request():
|
|
||||||
yield do_some_stuff()
|
|
||||||
yield do_some_more_stuff()
|
|
||||||
```
|
|
||||||
|
|
||||||
To conclude: I think this scheme would have worked equally well, with
|
|
||||||
less danger of messing it up, and probably made some more esoteric code
|
|
||||||
easier to write. But again --- changing the conventions of the entire
|
|
||||||
Synapse codebase is not a sensible option for the marginal improvement
|
|
||||||
offered.
|
|
||||||
|
|
||||||
## A note on garbage-collection of Deferred chains
|
|
||||||
|
|
||||||
It turns out that our logcontext rules do not play nicely with Deferred
|
|
||||||
chains which get orphaned and garbage-collected.
|
chains which get orphaned and garbage-collected.
|
||||||
|
|
||||||
Imagine we have some code that looks like this:
|
Imagine we have some code that looks like this:
|
||||||
@ -451,13 +322,12 @@ def on_something_interesting():
|
|||||||
for d in listener_queue:
|
for d in listener_queue:
|
||||||
d.callback("foo")
|
d.callback("foo")
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def await_something_interesting():
|
||||||
def await_something_interesting():
|
new_awaitable = defer.Deferred()
|
||||||
new_deferred = defer.Deferred()
|
listener_queue.append(new_awaitable)
|
||||||
listener_queue.append(new_deferred)
|
|
||||||
|
|
||||||
with PreserveLoggingContext():
|
with PreserveLoggingContext():
|
||||||
yield new_deferred
|
await new_awaitable
|
||||||
```
|
```
|
||||||
|
|
||||||
Obviously, the idea here is that we have a bunch of things which are
|
Obviously, the idea here is that we have a bunch of things which are
|
||||||
@ -476,18 +346,19 @@ def reset_listener_queue():
|
|||||||
listener_queue.clear()
|
listener_queue.clear()
|
||||||
```
|
```
|
||||||
|
|
||||||
So, both ends of the deferred chain have now dropped their references,
|
So, both ends of the awaitable chain have now dropped their references,
|
||||||
and the deferred chain is now orphaned, and will be garbage-collected at
|
and the awaitable chain is now orphaned, and will be garbage-collected at
|
||||||
some point. Note that `await_something_interesting` is a generator
|
some point. Note that `await_something_interesting` is a coroutine,
|
||||||
function, and when Python garbage-collects generator functions, it gives
|
which Python implements as a generator function. When Python
|
||||||
them a chance to clean up by making the `yield` raise a `GeneratorExit`
|
garbage-collects generator functions, it gives them a chance to
|
||||||
|
clean up by making the `async` (or `yield`) raise a `GeneratorExit`
|
||||||
exception. In our case, that means that the `__exit__` handler of
|
exception. In our case, that means that the `__exit__` handler of
|
||||||
`PreserveLoggingContext` will carefully restore the request context, but
|
`PreserveLoggingContext` will carefully restore the request context, but
|
||||||
there is now nothing waiting for its return, so the request context is
|
there is now nothing waiting for its return, so the request context is
|
||||||
never cleared.
|
never cleared.
|
||||||
|
|
||||||
To reiterate, this problem only arises when *both* ends of a deferred
|
To reiterate, this problem only arises when *both* ends of a awaitable
|
||||||
chain are dropped. Dropping the the reference to a deferred you're
|
chain are dropped. Dropping the the reference to an awaitable you're
|
||||||
supposed to be calling is probably bad practice, so this doesn't
|
supposed to be awaiting is bad practice, so this doesn't
|
||||||
actually happen too much. Unfortunately, when it does happen, it will
|
actually happen too much. Unfortunately, when it does happen, it will
|
||||||
lead to leaked logcontexts which are incredibly hard to track down.
|
lead to leaked logcontexts which are incredibly hard to track down.
|
||||||
|
109
docs/modules.md
109
docs/modules.md
@ -63,7 +63,7 @@ Modules can register web resources onto Synapse's web server using the following
|
|||||||
API method:
|
API method:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
def ModuleApi.register_web_resource(path: str, resource: IResource)
|
def ModuleApi.register_web_resource(path: str, resource: IResource) -> None
|
||||||
```
|
```
|
||||||
|
|
||||||
The path is the full absolute path to register the resource at. For example, if you
|
The path is the full absolute path to register the resource at. For example, if you
|
||||||
@ -91,12 +91,17 @@ are split in categories. A single module may implement callbacks from multiple c
|
|||||||
and is under no obligation to implement all callbacks from the categories it registers
|
and is under no obligation to implement all callbacks from the categories it registers
|
||||||
callbacks for.
|
callbacks for.
|
||||||
|
|
||||||
|
Modules can register callbacks using one of the module API's `register_[...]_callbacks`
|
||||||
|
methods. The callback functions are passed to these methods as keyword arguments, with
|
||||||
|
the callback name as the argument name and the function as its value. This is demonstrated
|
||||||
|
in the example below. A `register_[...]_callbacks` method exists for each module type
|
||||||
|
documented in this section.
|
||||||
|
|
||||||
#### Spam checker callbacks
|
#### Spam checker callbacks
|
||||||
|
|
||||||
To register one of the callbacks described in this section, a module needs to use the
|
Spam checker callbacks allow module developers to implement spam mitigation actions for
|
||||||
module API's `register_spam_checker_callbacks` method. The callback functions are passed
|
Synapse instances. Spam checker callbacks can be registered using the module API's
|
||||||
to `register_spam_checker_callbacks` as keyword arguments, with the callback name as the
|
`register_spam_checker_callbacks` method.
|
||||||
argument name and the function as its value. This is demonstrated in the example below.
|
|
||||||
|
|
||||||
The available spam checker callbacks are:
|
The available spam checker callbacks are:
|
||||||
|
|
||||||
@ -115,7 +120,7 @@ async def user_may_invite(inviter: str, invitee: str, room_id: str) -> bool
|
|||||||
|
|
||||||
Called when processing an invitation. The module must return a `bool` indicating whether
|
Called when processing an invitation. The module must return a `bool` indicating whether
|
||||||
the inviter can invite the invitee to the given room. Both inviter and invitee are
|
the inviter can invite the invitee to the given room. Both inviter and invitee are
|
||||||
represented by their Matrix user ID (i.e. `@alice:example.com`).
|
represented by their Matrix user ID (e.g. `@alice:example.com`).
|
||||||
|
|
||||||
```python
|
```python
|
||||||
async def user_may_create_room(user: str) -> bool
|
async def user_may_create_room(user: str) -> bool
|
||||||
@ -181,13 +186,103 @@ The arguments passed to this callback are:
|
|||||||
```python
|
```python
|
||||||
async def check_media_file_for_spam(
|
async def check_media_file_for_spam(
|
||||||
file_wrapper: "synapse.rest.media.v1.media_storage.ReadableFileWrapper",
|
file_wrapper: "synapse.rest.media.v1.media_storage.ReadableFileWrapper",
|
||||||
file_info: "synapse.rest.media.v1._base.FileInfo"
|
file_info: "synapse.rest.media.v1._base.FileInfo",
|
||||||
) -> bool
|
) -> bool
|
||||||
```
|
```
|
||||||
|
|
||||||
Called when storing a local or remote file. The module must return a boolean indicating
|
Called when storing a local or remote file. The module must return a boolean indicating
|
||||||
whether the given file can be stored in the homeserver's media store.
|
whether the given file can be stored in the homeserver's media store.
|
||||||
|
|
||||||
|
#### Account validity callbacks
|
||||||
|
|
||||||
|
Account validity callbacks allow module developers to add extra steps to verify the
|
||||||
|
validity on an account, i.e. see if a user can be granted access to their account on the
|
||||||
|
Synapse instance. Account validity callbacks can be registered using the module API's
|
||||||
|
`register_account_validity_callbacks` method.
|
||||||
|
|
||||||
|
The available account validity callbacks are:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def is_user_expired(user: str) -> Optional[bool]
|
||||||
|
```
|
||||||
|
|
||||||
|
Called when processing any authenticated request (except for logout requests). The module
|
||||||
|
can return a `bool` to indicate whether the user has expired and should be locked out of
|
||||||
|
their account, or `None` if the module wasn't able to figure it out. The user is
|
||||||
|
represented by their Matrix user ID (e.g. `@alice:example.com`).
|
||||||
|
|
||||||
|
If the module returns `True`, the current request will be denied with the error code
|
||||||
|
`ORG_MATRIX_EXPIRED_ACCOUNT` and the HTTP status code 403. Note that this doesn't
|
||||||
|
invalidate the user's access token.
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def on_user_registration(user: str) -> None
|
||||||
|
```
|
||||||
|
|
||||||
|
Called after successfully registering a user, in case the module needs to perform extra
|
||||||
|
operations to keep track of them. (e.g. add them to a database table). The user is
|
||||||
|
represented by their Matrix user ID.
|
||||||
|
|
||||||
|
#### Third party rules callbacks
|
||||||
|
|
||||||
|
Third party rules callbacks allow module developers to add extra checks to verify the
|
||||||
|
validity of incoming events. Third party event rules callbacks can be registered using
|
||||||
|
the module API's `register_third_party_rules_callbacks` method.
|
||||||
|
|
||||||
|
The available third party rules callbacks are:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def check_event_allowed(
|
||||||
|
event: "synapse.events.EventBase",
|
||||||
|
state_events: "synapse.types.StateMap",
|
||||||
|
) -> Tuple[bool, Optional[dict]]
|
||||||
|
```
|
||||||
|
|
||||||
|
**<span style="color:red">
|
||||||
|
This callback is very experimental and can and will break without notice. Module developers
|
||||||
|
are encouraged to implement `check_event_for_spam` from the spam checker category instead.
|
||||||
|
</span>**
|
||||||
|
|
||||||
|
Called when processing any incoming event, with the event and a `StateMap`
|
||||||
|
representing the current state of the room the event is being sent into. A `StateMap` is
|
||||||
|
a dictionary that maps tuples containing an event type and a state key to the
|
||||||
|
corresponding state event. For example retrieving the room's `m.room.create` event from
|
||||||
|
the `state_events` argument would look like this: `state_events.get(("m.room.create", ""))`.
|
||||||
|
The module must return a boolean indicating whether the event can be allowed.
|
||||||
|
|
||||||
|
Note that this callback function processes incoming events coming via federation
|
||||||
|
traffic (on top of client traffic). This means denying an event might cause the local
|
||||||
|
copy of the room's history to diverge from that of remote servers. This may cause
|
||||||
|
federation issues in the room. It is strongly recommended to only deny events using this
|
||||||
|
callback function if the sender is a local user, or in a private federation in which all
|
||||||
|
servers are using the same module, with the same configuration.
|
||||||
|
|
||||||
|
If the boolean returned by the module is `True`, it may also tell Synapse to replace the
|
||||||
|
event with new data by returning the new event's data as a dictionary. In order to do
|
||||||
|
that, it is recommended the module calls `event.get_dict()` to get the current event as a
|
||||||
|
dictionary, and modify the returned dictionary accordingly.
|
||||||
|
|
||||||
|
Note that replacing the event only works for events sent by local users, not for events
|
||||||
|
received over federation.
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def on_create_room(
|
||||||
|
requester: "synapse.types.Requester",
|
||||||
|
request_content: dict,
|
||||||
|
is_requester_admin: bool,
|
||||||
|
) -> None
|
||||||
|
```
|
||||||
|
|
||||||
|
Called when processing a room creation request, with the `Requester` object for the user
|
||||||
|
performing the request, a dictionary representing the room creation request's JSON body
|
||||||
|
(see [the spec](https://matrix.org/docs/spec/client_server/latest#post-matrix-client-r0-createroom)
|
||||||
|
for a list of possible parameters), and a boolean indicating whether the user performing
|
||||||
|
the request is a server admin.
|
||||||
|
|
||||||
|
Modules can modify the `request_content` (by e.g. adding events to its `initial_state`),
|
||||||
|
or deny the room's creation by raising a `module_api.errors.SynapseError`.
|
||||||
|
|
||||||
|
|
||||||
### Porting an existing module that uses the old interface
|
### Porting an existing module that uses the old interface
|
||||||
|
|
||||||
In order to port a module that uses Synapse's old module interface, its author needs to:
|
In order to port a module that uses Synapse's old module interface, its author needs to:
|
||||||
|
@ -28,7 +28,7 @@ minimal.
|
|||||||
|
|
||||||
### The Replication Protocol
|
### The Replication Protocol
|
||||||
|
|
||||||
See [tcp_replication.md](tcp_replication.md)
|
See [the TCP replication documentation](tcp_replication.md).
|
||||||
|
|
||||||
### The Slaved DataStore
|
### The Slaved DataStore
|
||||||
|
|
||||||
|
@ -21,7 +21,7 @@ port 8448. Where these are different, we refer to the 'client port' and the
|
|||||||
'federation port'. See [the Matrix
|
'federation port'. See [the Matrix
|
||||||
specification](https://matrix.org/docs/spec/server_server/latest#resolving-server-names)
|
specification](https://matrix.org/docs/spec/server_server/latest#resolving-server-names)
|
||||||
for more details of the algorithm used for federation connections, and
|
for more details of the algorithm used for federation connections, and
|
||||||
[delegate.md](delegate.md) for instructions on setting up delegation.
|
[Delegation](delegate.md) for instructions on setting up delegation.
|
||||||
|
|
||||||
**NOTE**: Your reverse proxy must not `canonicalise` or `normalise`
|
**NOTE**: Your reverse proxy must not `canonicalise` or `normalise`
|
||||||
the requested URI in any way (for example, by decoding `%xx` escapes).
|
the requested URI in any way (for example, by decoding `%xx` escapes).
|
||||||
@ -98,6 +98,33 @@ example.com:8448 {
|
|||||||
reverse_proxy http://localhost:8008
|
reverse_proxy http://localhost:8008
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
[Delegation](delegate.md) example:
|
||||||
|
```
|
||||||
|
(matrix-well-known-header) {
|
||||||
|
# Headers
|
||||||
|
header Access-Control-Allow-Origin "*"
|
||||||
|
header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
|
||||||
|
header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept, Authorization"
|
||||||
|
header Content-Type "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
example.com {
|
||||||
|
handle /.well-known/matrix/server {
|
||||||
|
import matrix-well-known-header
|
||||||
|
respond `{"m.server":"matrix.example.com:443"}`
|
||||||
|
}
|
||||||
|
|
||||||
|
handle /.well-known/matrix/client {
|
||||||
|
import matrix-well-known-header
|
||||||
|
respond `{"m.homeserver":{"base_url":"https://matrix.example.com"},"m.identity_server":{"base_url":"https://identity.example.com"}}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
matrix.example.com {
|
||||||
|
reverse_proxy /_matrix/* http://localhost:8008
|
||||||
|
reverse_proxy /_synapse/client/* http://localhost:8008
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
### Apache
|
### Apache
|
||||||
|
|
||||||
|
@ -1,9 +1,9 @@
|
|||||||
Room and User Statistics
|
Room and User Statistics
|
||||||
========================
|
========================
|
||||||
|
|
||||||
Synapse maintains room and user statistics (as well as a cache of room state),
|
Synapse maintains room and user statistics in various tables. These can be used
|
||||||
in various tables. These can be used for administrative purposes but are also
|
for administrative purposes but are also used when generating the public room
|
||||||
used when generating the public room directory.
|
directory.
|
||||||
|
|
||||||
|
|
||||||
# Synapse Developer Documentation
|
# Synapse Developer Documentation
|
||||||
@ -15,48 +15,8 @@ used when generating the public room directory.
|
|||||||
* **subject**: Something we are tracking stats about – currently a room or user.
|
* **subject**: Something we are tracking stats about – currently a room or user.
|
||||||
* **current row**: An entry for a subject in the appropriate current statistics
|
* **current row**: An entry for a subject in the appropriate current statistics
|
||||||
table. Each subject can have only one.
|
table. Each subject can have only one.
|
||||||
* **historical row**: An entry for a subject in the appropriate historical
|
|
||||||
statistics table. Each subject can have any number of these.
|
|
||||||
|
|
||||||
### Overview
|
### Overview
|
||||||
|
|
||||||
Stats are maintained as time series. There are two kinds of column:
|
Stats correspond to the present values. Current rows contain the most up-to-date
|
||||||
|
statistics for a room. Each subject can only have one entry.
|
||||||
* absolute columns – where the value is correct for the time given by `end_ts`
|
|
||||||
in the stats row. (Imagine a line graph for these values)
|
|
||||||
* They can also be thought of as 'gauges' in Prometheus, if you are familiar.
|
|
||||||
* per-slice columns – where the value corresponds to how many of the occurrences
|
|
||||||
occurred within the time slice given by `(end_ts − bucket_size)…end_ts`
|
|
||||||
or `start_ts…end_ts`. (Imagine a histogram for these values)
|
|
||||||
|
|
||||||
Stats are maintained in two tables (for each type): current and historical.
|
|
||||||
|
|
||||||
Current stats correspond to the present values. Each subject can only have one
|
|
||||||
entry.
|
|
||||||
|
|
||||||
Historical stats correspond to values in the past. Subjects may have multiple
|
|
||||||
entries.
|
|
||||||
|
|
||||||
## Concepts around the management of stats
|
|
||||||
|
|
||||||
### Current rows
|
|
||||||
|
|
||||||
Current rows contain the most up-to-date statistics for a room.
|
|
||||||
They only contain absolute columns
|
|
||||||
|
|
||||||
### Historical rows
|
|
||||||
|
|
||||||
Historical rows can always be considered to be valid for the time slice and
|
|
||||||
end time specified.
|
|
||||||
|
|
||||||
* historical rows will not exist for every time slice – they will be omitted
|
|
||||||
if there were no changes. In this case, the following assumptions can be
|
|
||||||
made to interpolate/recreate missing rows:
|
|
||||||
- absolute fields have the same values as in the preceding row
|
|
||||||
- per-slice fields are zero (`0`)
|
|
||||||
* historical rows will not be retained forever – rows older than a configurable
|
|
||||||
time will be purged.
|
|
||||||
|
|
||||||
#### Purge
|
|
||||||
|
|
||||||
The purging of historical rows is not yet implemented.
|
|
||||||
|
@ -1310,91 +1310,6 @@ account_threepid_delegates:
|
|||||||
#auto_join_rooms_for_guests: false
|
#auto_join_rooms_for_guests: false
|
||||||
|
|
||||||
|
|
||||||
## Account Validity ##
|
|
||||||
|
|
||||||
# Optional account validity configuration. This allows for accounts to be denied
|
|
||||||
# any request after a given period.
|
|
||||||
#
|
|
||||||
# Once this feature is enabled, Synapse will look for registered users without an
|
|
||||||
# expiration date at startup and will add one to every account it found using the
|
|
||||||
# current settings at that time.
|
|
||||||
# This means that, if a validity period is set, and Synapse is restarted (it will
|
|
||||||
# then derive an expiration date from the current validity period), and some time
|
|
||||||
# after that the validity period changes and Synapse is restarted, the users'
|
|
||||||
# expiration dates won't be updated unless their account is manually renewed. This
|
|
||||||
# date will be randomly selected within a range [now + period - d ; now + period],
|
|
||||||
# where d is equal to 10% of the validity period.
|
|
||||||
#
|
|
||||||
account_validity:
|
|
||||||
# The account validity feature is disabled by default. Uncomment the
|
|
||||||
# following line to enable it.
|
|
||||||
#
|
|
||||||
#enabled: true
|
|
||||||
|
|
||||||
# The period after which an account is valid after its registration. When
|
|
||||||
# renewing the account, its validity period will be extended by this amount
|
|
||||||
# of time. This parameter is required when using the account validity
|
|
||||||
# feature.
|
|
||||||
#
|
|
||||||
#period: 6w
|
|
||||||
|
|
||||||
# The amount of time before an account's expiry date at which Synapse will
|
|
||||||
# send an email to the account's email address with a renewal link. By
|
|
||||||
# default, no such emails are sent.
|
|
||||||
#
|
|
||||||
# If you enable this setting, you will also need to fill out the 'email' and
|
|
||||||
# 'public_baseurl' configuration sections.
|
|
||||||
#
|
|
||||||
#renew_at: 1w
|
|
||||||
|
|
||||||
# The subject of the email sent out with the renewal link. '%(app)s' can be
|
|
||||||
# used as a placeholder for the 'app_name' parameter from the 'email'
|
|
||||||
# section.
|
|
||||||
#
|
|
||||||
# Note that the placeholder must be written '%(app)s', including the
|
|
||||||
# trailing 's'.
|
|
||||||
#
|
|
||||||
# If this is not set, a default value is used.
|
|
||||||
#
|
|
||||||
#renew_email_subject: "Renew your %(app)s account"
|
|
||||||
|
|
||||||
# Directory in which Synapse will try to find templates for the HTML files to
|
|
||||||
# serve to the user when trying to renew an account. If not set, default
|
|
||||||
# templates from within the Synapse package will be used.
|
|
||||||
#
|
|
||||||
# The currently available templates are:
|
|
||||||
#
|
|
||||||
# * account_renewed.html: Displayed to the user after they have successfully
|
|
||||||
# renewed their account.
|
|
||||||
#
|
|
||||||
# * account_previously_renewed.html: Displayed to the user if they attempt to
|
|
||||||
# renew their account with a token that is valid, but that has already
|
|
||||||
# been used. In this case the account is not renewed again.
|
|
||||||
#
|
|
||||||
# * invalid_token.html: Displayed to the user when they try to renew an account
|
|
||||||
# with an unknown or invalid renewal token.
|
|
||||||
#
|
|
||||||
# See https://github.com/matrix-org/synapse/tree/master/synapse/res/templates for
|
|
||||||
# default template contents.
|
|
||||||
#
|
|
||||||
# The file name of some of these templates can be configured below for legacy
|
|
||||||
# reasons.
|
|
||||||
#
|
|
||||||
#template_dir: "res/templates"
|
|
||||||
|
|
||||||
# A custom file name for the 'account_renewed.html' template.
|
|
||||||
#
|
|
||||||
# If not set, the file is assumed to be named "account_renewed.html".
|
|
||||||
#
|
|
||||||
#account_renewed_html_path: "account_renewed.html"
|
|
||||||
|
|
||||||
# A custom file name for the 'invalid_token.html' template.
|
|
||||||
#
|
|
||||||
# If not set, the file is assumed to be named "invalid_token.html".
|
|
||||||
#
|
|
||||||
#invalid_token_html_path: "invalid_token.html"
|
|
||||||
|
|
||||||
|
|
||||||
## Metrics ###
|
## Metrics ###
|
||||||
|
|
||||||
# Enable collection and rendering of performance metrics
|
# Enable collection and rendering of performance metrics
|
||||||
@ -2653,11 +2568,6 @@ stats:
|
|||||||
#
|
#
|
||||||
#enabled: false
|
#enabled: false
|
||||||
|
|
||||||
# The size of each timeslice in the room_stats_historical and
|
|
||||||
# user_stats_historical tables, as a time period. Defaults to "1d".
|
|
||||||
#
|
|
||||||
#bucket_size: 1h
|
|
||||||
|
|
||||||
|
|
||||||
# Server Notices room configuration
|
# Server Notices room configuration
|
||||||
#
|
#
|
||||||
@ -2744,19 +2654,6 @@ stats:
|
|||||||
# action: allow
|
# action: allow
|
||||||
|
|
||||||
|
|
||||||
# Server admins can define a Python module that implements extra rules for
|
|
||||||
# allowing or denying incoming events. In order to work, this module needs to
|
|
||||||
# override the methods defined in synapse/events/third_party_rules.py.
|
|
||||||
#
|
|
||||||
# This feature is designed to be used in closed federations only, where each
|
|
||||||
# participating server enforces the same rules.
|
|
||||||
#
|
|
||||||
#third_party_event_rules:
|
|
||||||
# module: "my_custom_project.SuperRulesSet"
|
|
||||||
# config:
|
|
||||||
# example_option: 'things'
|
|
||||||
|
|
||||||
|
|
||||||
## Opentracing ##
|
## Opentracing ##
|
||||||
|
|
||||||
# These settings enable opentracing, which implements distributed tracing.
|
# These settings enable opentracing, which implements distributed tracing.
|
||||||
|
@ -3,8 +3,8 @@
|
|||||||
'Server Notices' are a new feature introduced in Synapse 0.30. They provide a
|
'Server Notices' are a new feature introduced in Synapse 0.30. They provide a
|
||||||
channel whereby server administrators can send messages to users on the server.
|
channel whereby server administrators can send messages to users on the server.
|
||||||
|
|
||||||
They are used as part of communication of the server polices(see
|
They are used as part of communication of the server polices (see
|
||||||
[consent_tracking.md](consent_tracking.md)), however the intention is that
|
[Consent Tracking](consent_tracking.md)), however the intention is that
|
||||||
they may also find a use for features such as "Message of the day".
|
they may also find a use for features such as "Message of the day".
|
||||||
|
|
||||||
This is a feature specific to Synapse, but it uses standard Matrix
|
This is a feature specific to Synapse, but it uses standard Matrix
|
||||||
|
@ -166,13 +166,16 @@ sudo dnf groupinstall "Development Tools"
|
|||||||
|
|
||||||
Installing prerequisites on macOS:
|
Installing prerequisites on macOS:
|
||||||
|
|
||||||
|
You may need to install the latest Xcode developer tools:
|
||||||
```sh
|
```sh
|
||||||
xcode-select --install
|
xcode-select --install
|
||||||
sudo easy_install pip
|
|
||||||
sudo pip install virtualenv
|
|
||||||
brew install pkg-config libffi
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
On ARM-based Macs you may need to explicitly install libjpeg which is a pillow dependency. You can use Homebrew (https://brew.sh):
|
||||||
|
```sh
|
||||||
|
brew install jpeg
|
||||||
|
```
|
||||||
|
|
||||||
On macOS Catalina (10.15) you may need to explicitly install OpenSSL
|
On macOS Catalina (10.15) you may need to explicitly install OpenSSL
|
||||||
via brew and inform `pip` about it so that `psycopg2` builds:
|
via brew and inform `pip` about it so that `psycopg2` builds:
|
||||||
|
|
||||||
@ -268,9 +271,8 @@ For more details, see
|
|||||||
|
|
||||||
##### Matrix.org packages
|
##### Matrix.org packages
|
||||||
|
|
||||||
Matrix.org provides Debian/Ubuntu packages of the latest stable version of
|
Matrix.org provides Debian/Ubuntu packages of Synapse via
|
||||||
Synapse via <https://packages.matrix.org/debian/>. They are available for Debian
|
<https://packages.matrix.org/debian/>. To install the latest release:
|
||||||
9 (Stretch), Ubuntu 16.04 (Xenial), and later. To use them:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
sudo apt install -y lsb-release wget apt-transport-https
|
sudo apt install -y lsb-release wget apt-transport-https
|
||||||
@ -281,12 +283,16 @@ sudo apt update
|
|||||||
sudo apt install matrix-synapse-py3
|
sudo apt install matrix-synapse-py3
|
||||||
```
|
```
|
||||||
|
|
||||||
**Note**: if you followed a previous version of these instructions which
|
Packages are also published for release candidates. To enable the prerelease
|
||||||
recommended using `apt-key add` to add an old key from
|
channel, add `prerelease` to the `sources.list` line. For example:
|
||||||
`https://matrix.org/packages/debian/`, you should note that this key has been
|
|
||||||
revoked. You should remove the old key with `sudo apt-key remove
|
```sh
|
||||||
C35EB17E1EAE708E6603A9B3AD0592FE47F0DF61`, and follow the above instructions to
|
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
|
||||||
update your configuration.
|
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main prerelease" |
|
||||||
|
sudo tee /etc/apt/sources.list.d/matrix-org.list
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install matrix-synapse-py3
|
||||||
|
```
|
||||||
|
|
||||||
The fingerprint of the repository signing key (as shown by `gpg
|
The fingerprint of the repository signing key (as shown by `gpg
|
||||||
/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
|
/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
|
||||||
@ -409,7 +415,7 @@ instead. Advantages include:
|
|||||||
- allowing the DB to be run on separate hardware
|
- allowing the DB to be run on separate hardware
|
||||||
|
|
||||||
For information on how to install and use PostgreSQL in Synapse, please see
|
For information on how to install and use PostgreSQL in Synapse, please see
|
||||||
[docs/postgres.md](../postgres.md)
|
[Using Postgres](../postgres.md)
|
||||||
|
|
||||||
SQLite is only acceptable for testing purposes. SQLite should not be used in
|
SQLite is only acceptable for testing purposes. SQLite should not be used in
|
||||||
a production server. Synapse will perform poorly when using
|
a production server. Synapse will perform poorly when using
|
||||||
@ -424,7 +430,7 @@ over HTTPS.
|
|||||||
|
|
||||||
The recommended way to do so is to set up a reverse proxy on port
|
The recommended way to do so is to set up a reverse proxy on port
|
||||||
`8448`. You can find documentation on doing so in
|
`8448`. You can find documentation on doing so in
|
||||||
[docs/reverse_proxy.md](../reverse_proxy.md).
|
[the reverse proxy documentation](../reverse_proxy.md).
|
||||||
|
|
||||||
Alternatively, you can configure Synapse to expose an HTTPS port. To do
|
Alternatively, you can configure Synapse to expose an HTTPS port. To do
|
||||||
so, you will need to edit `homeserver.yaml`, as follows:
|
so, you will need to edit `homeserver.yaml`, as follows:
|
||||||
@ -451,7 +457,7 @@ so, you will need to edit `homeserver.yaml`, as follows:
|
|||||||
`cert.pem`).
|
`cert.pem`).
|
||||||
|
|
||||||
For a more detailed guide to configuring your server for federation, see
|
For a more detailed guide to configuring your server for federation, see
|
||||||
[federate.md](../federate.md).
|
[Federation](../federate.md).
|
||||||
|
|
||||||
### Client Well-Known URI
|
### Client Well-Known URI
|
||||||
|
|
||||||
@ -563,9 +569,7 @@ on your server even if `enable_registration` is `false`.
|
|||||||
### Setting up a TURN server
|
### Setting up a TURN server
|
||||||
|
|
||||||
For reliable VoIP calls to be routed via this homeserver, you MUST configure
|
For reliable VoIP calls to be routed via this homeserver, you MUST configure
|
||||||
a TURN server. See
|
a TURN server. See [TURN setup](../turn-howto.md) for details.
|
||||||
[docs/turn-howto.md](../turn-howto.md)
|
|
||||||
for details.
|
|
||||||
|
|
||||||
### URL previews
|
### URL previews
|
||||||
|
|
||||||
|
@ -14,10 +14,12 @@ contains an example configuration for the `federation_reader` worker.
|
|||||||
|
|
||||||
## Synapse configuration files
|
## Synapse configuration files
|
||||||
|
|
||||||
See [workers.md](../workers.md) for information on how to set up the
|
See [the worker documentation](../workers.md) for information on how to set up the
|
||||||
configuration files and reverse-proxy correctly. You can find an example worker
|
configuration files and reverse-proxy correctly.
|
||||||
config in the [workers](https://github.com/matrix-org/synapse/tree/develop/docs/systemd-with-workers/workers/)
|
Below is a sample `federation_reader` worker configuration file.
|
||||||
folder.
|
```yaml
|
||||||
|
{{#include workers/federation_reader.yaml}}
|
||||||
|
```
|
||||||
|
|
||||||
Systemd manages daemonization itself, so ensure that none of the configuration
|
Systemd manages daemonization itself, so ensure that none of the configuration
|
||||||
files set either `daemonize` or `worker_daemonize`.
|
files set either `daemonize` or `worker_daemonize`.
|
||||||
@ -72,12 +74,12 @@ systemctl restart matrix-synapse.target
|
|||||||
|
|
||||||
**Optional:** If further hardening is desired, the file
|
**Optional:** If further hardening is desired, the file
|
||||||
`override-hardened.conf` may be copied from
|
`override-hardened.conf` may be copied from
|
||||||
`contrib/systemd/override-hardened.conf` in this repository to the location
|
[contrib/systemd/override-hardened.conf](https://github.com/matrix-org/synapse/tree/develop/contrib/systemd/)
|
||||||
|
in this repository to the location
|
||||||
`/etc/systemd/system/matrix-synapse.service.d/override-hardened.conf` (the
|
`/etc/systemd/system/matrix-synapse.service.d/override-hardened.conf` (the
|
||||||
directory may have to be created). It enables certain sandboxing features in
|
directory may have to be created). It enables certain sandboxing features in
|
||||||
systemd to further secure the synapse service. You may read the comments to
|
systemd to further secure the synapse service. You may read the comments to
|
||||||
understand what the override file is doing. The same file will need to be copied
|
understand what the override file is doing. The same file will need to be copied to
|
||||||
to
|
|
||||||
`/etc/systemd/system/matrix-synapse-worker@.service.d/override-hardened-worker.conf`
|
`/etc/systemd/system/matrix-synapse-worker@.service.d/override-hardened-worker.conf`
|
||||||
(this directory may also have to be created) in order to apply the same
|
(this directory may also have to be created) in order to apply the same
|
||||||
hardening options to any worker processes.
|
hardening options to any worker processes.
|
||||||
|
@ -86,6 +86,19 @@ process, for example:
|
|||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
# Upgrading to v1.39.0
|
||||||
|
|
||||||
|
## Deprecation of the current third-party rules module interface
|
||||||
|
|
||||||
|
The current third-party rules module interface is deprecated in favour of the new generic
|
||||||
|
modules system introduced in Synapse v1.37.0. Authors of third-party rules modules can refer
|
||||||
|
to [this documentation](modules.md#porting-an-existing-module-that-uses-the-old-interface)
|
||||||
|
to update their modules. Synapse administrators can refer to [this documentation](modules.md#using-modules)
|
||||||
|
to update their configuration once the modules they are using have been updated.
|
||||||
|
|
||||||
|
We plan to remove support for the current third-party rules interface in September 2021.
|
||||||
|
|
||||||
|
|
||||||
# Upgrading to v1.38.0
|
# Upgrading to v1.38.0
|
||||||
|
|
||||||
## Re-indexing of `events` table on Postgres databases
|
## Re-indexing of `events` table on Postgres databases
|
||||||
|
@ -11,4 +11,4 @@ a fresh config using Synapse by following the instructions in
|
|||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
{{#include ../../sample_log_config.yaml}}
|
{{#include ../../sample_log_config.yaml}}
|
||||||
``__`
|
```
|
@ -73,7 +73,7 @@ https://hub.docker.com/r/matrixdotorg/synapse/.
|
|||||||
To make effective use of the workers, you will need to configure an HTTP
|
To make effective use of the workers, you will need to configure an HTTP
|
||||||
reverse-proxy such as nginx or haproxy, which will direct incoming requests to
|
reverse-proxy such as nginx or haproxy, which will direct incoming requests to
|
||||||
the correct worker, or to the main synapse instance. See
|
the correct worker, or to the main synapse instance. See
|
||||||
[reverse_proxy.md](reverse_proxy.md) for information on setting up a reverse
|
[the reverse proxy documentation](reverse_proxy.md) for information on setting up a reverse
|
||||||
proxy.
|
proxy.
|
||||||
|
|
||||||
When using workers, each worker process has its own configuration file which
|
When using workers, each worker process has its own configuration file which
|
||||||
@ -170,8 +170,8 @@ Finally, you need to start your worker processes. This can be done with either
|
|||||||
`synctl` or your distribution's preferred service manager such as `systemd`. We
|
`synctl` or your distribution's preferred service manager such as `systemd`. We
|
||||||
recommend the use of `systemd` where available: for information on setting up
|
recommend the use of `systemd` where available: for information on setting up
|
||||||
`systemd` to start synapse workers, see
|
`systemd` to start synapse workers, see
|
||||||
[systemd-with-workers](systemd-with-workers). To use `synctl`, see
|
[Systemd with Workers](systemd-with-workers). To use `synctl`, see
|
||||||
[synctl_workers.md](synctl_workers.md).
|
[Using synctl with Workers](synctl_workers.md).
|
||||||
|
|
||||||
|
|
||||||
## Available worker applications
|
## Available worker applications
|
||||||
|
1
mypy.ini
1
mypy.ini
@ -83,6 +83,7 @@ files =
|
|||||||
synapse/util/stringutils.py,
|
synapse/util/stringutils.py,
|
||||||
synapse/visibility.py,
|
synapse/visibility.py,
|
||||||
tests/replication,
|
tests/replication,
|
||||||
|
tests/test_event_auth.py,
|
||||||
tests/test_utils,
|
tests/test_utils,
|
||||||
tests/handlers/test_password_providers.py,
|
tests/handlers/test_password_providers.py,
|
||||||
tests/rest/client/v1/test_login.py,
|
tests/rest/client/v1/test_login.py,
|
||||||
|
@ -139,6 +139,11 @@ def run():
|
|||||||
|
|
||||||
# Switch to the release branch.
|
# Switch to the release branch.
|
||||||
parsed_new_version = version.parse(new_version)
|
parsed_new_version = version.parse(new_version)
|
||||||
|
|
||||||
|
# We assume for debian changelogs that we only do RCs or full releases.
|
||||||
|
assert not parsed_new_version.is_devrelease
|
||||||
|
assert not parsed_new_version.is_postrelease
|
||||||
|
|
||||||
release_branch_name = (
|
release_branch_name = (
|
||||||
f"release-v{parsed_new_version.major}.{parsed_new_version.minor}"
|
f"release-v{parsed_new_version.major}.{parsed_new_version.minor}"
|
||||||
)
|
)
|
||||||
@ -190,10 +195,19 @@ def run():
|
|||||||
# Generate changelogs
|
# Generate changelogs
|
||||||
subprocess.run("python3 -m towncrier", shell=True)
|
subprocess.run("python3 -m towncrier", shell=True)
|
||||||
|
|
||||||
# Generate debian changelogs if its not an RC.
|
# Generate debian changelogs
|
||||||
if not rc:
|
if parsed_new_version.pre is not None:
|
||||||
|
# If this is an RC then we need to coerce the version string to match
|
||||||
|
# Debian norms, e.g. 1.39.0rc2 gets converted to 1.39.0~rc2.
|
||||||
|
base_ver = parsed_new_version.base_version
|
||||||
|
pre_type, pre_num = parsed_new_version.pre
|
||||||
|
debian_version = f"{base_ver}~{pre_type}{pre_num}"
|
||||||
|
else:
|
||||||
|
debian_version = new_version
|
||||||
|
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
f'dch -M -v {new_version} "New synapse release {new_version}."', shell=True
|
f'dch -M -v {debian_version} "New synapse release {debian_version}."',
|
||||||
|
shell=True,
|
||||||
)
|
)
|
||||||
subprocess.run('dch -M -r -D stable ""', shell=True)
|
subprocess.run('dch -M -r -D stable ""', shell=True)
|
||||||
|
|
||||||
|
@ -47,7 +47,7 @@ try:
|
|||||||
except ImportError:
|
except ImportError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
__version__ = "1.38.1"
|
__version__ = "1.39.0rc3"
|
||||||
|
|
||||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||||
# We import here so that we don't have to install a bunch of deps when
|
# We import here so that we don't have to install a bunch of deps when
|
||||||
|
@ -62,16 +62,14 @@ class Auth:
|
|||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
self.store = hs.get_datastore()
|
self.store = hs.get_datastore()
|
||||||
self.state = hs.get_state_handler()
|
self.state = hs.get_state_handler()
|
||||||
|
self._account_validity_handler = hs.get_account_validity_handler()
|
||||||
|
|
||||||
self.token_cache = LruCache(
|
self.token_cache: LruCache[str, Tuple[str, bool]] = LruCache(
|
||||||
10000, "token_cache"
|
10000, "token_cache"
|
||||||
) # type: LruCache[str, Tuple[str, bool]]
|
)
|
||||||
|
|
||||||
self._auth_blocking = AuthBlocking(self.hs)
|
self._auth_blocking = AuthBlocking(self.hs)
|
||||||
|
|
||||||
self._account_validity_enabled = (
|
|
||||||
hs.config.account_validity.account_validity_enabled
|
|
||||||
)
|
|
||||||
self._track_appservice_user_ips = hs.config.track_appservice_user_ips
|
self._track_appservice_user_ips = hs.config.track_appservice_user_ips
|
||||||
self._macaroon_secret_key = hs.config.macaroon_secret_key
|
self._macaroon_secret_key = hs.config.macaroon_secret_key
|
||||||
self._force_tracing_for_users = hs.config.tracing.force_tracing_for_users
|
self._force_tracing_for_users = hs.config.tracing.force_tracing_for_users
|
||||||
@ -187,12 +185,17 @@ class Auth:
|
|||||||
shadow_banned = user_info.shadow_banned
|
shadow_banned = user_info.shadow_banned
|
||||||
|
|
||||||
# Deny the request if the user account has expired.
|
# Deny the request if the user account has expired.
|
||||||
if self._account_validity_enabled and not allow_expired:
|
if not allow_expired:
|
||||||
if await self.store.is_account_expired(
|
if await self._account_validity_handler.is_user_expired(
|
||||||
user_info.user_id, self.clock.time_msec()
|
user_info.user_id
|
||||||
):
|
):
|
||||||
|
# Raise the error if either an account validity module has determined
|
||||||
|
# the account has expired, or the legacy account validity
|
||||||
|
# implementation is enabled and determined the account has expired
|
||||||
raise AuthError(
|
raise AuthError(
|
||||||
403, "User account has expired", errcode=Codes.EXPIRED_ACCOUNT
|
403,
|
||||||
|
"User account has expired",
|
||||||
|
errcode=Codes.EXPIRED_ACCOUNT,
|
||||||
)
|
)
|
||||||
|
|
||||||
device_id = user_info.device_id
|
device_id = user_info.device_id
|
||||||
@ -240,6 +243,37 @@ class Auth:
|
|||||||
except KeyError:
|
except KeyError:
|
||||||
raise MissingClientTokenError()
|
raise MissingClientTokenError()
|
||||||
|
|
||||||
|
async def validate_appservice_can_control_user_id(
|
||||||
|
self, app_service: ApplicationService, user_id: str
|
||||||
|
):
|
||||||
|
"""Validates that the app service is allowed to control
|
||||||
|
the given user.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
app_service: The app service that controls the user
|
||||||
|
user_id: The author MXID that the app service is controlling
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
AuthError: If the application service is not allowed to control the user
|
||||||
|
(user namespace regex does not match, wrong homeserver, etc)
|
||||||
|
or if the user has not been registered yet.
|
||||||
|
"""
|
||||||
|
|
||||||
|
# It's ok if the app service is trying to use the sender from their registration
|
||||||
|
if app_service.sender == user_id:
|
||||||
|
pass
|
||||||
|
# Check to make sure the app service is allowed to control the user
|
||||||
|
elif not app_service.is_interested_in_user(user_id):
|
||||||
|
raise AuthError(
|
||||||
|
403,
|
||||||
|
"Application service cannot masquerade as this user (%s)." % user_id,
|
||||||
|
)
|
||||||
|
# Check to make sure the user is already registered on the homeserver
|
||||||
|
elif not (await self.store.get_user_by_id(user_id)):
|
||||||
|
raise AuthError(
|
||||||
|
403, "Application service has not registered this user (%s)" % user_id
|
||||||
|
)
|
||||||
|
|
||||||
async def _get_appservice_user_id(
|
async def _get_appservice_user_id(
|
||||||
self, request: Request
|
self, request: Request
|
||||||
) -> Tuple[Optional[str], Optional[ApplicationService]]:
|
) -> Tuple[Optional[str], Optional[ApplicationService]]:
|
||||||
@ -261,13 +295,11 @@ class Auth:
|
|||||||
return app_service.sender, app_service
|
return app_service.sender, app_service
|
||||||
|
|
||||||
user_id = request.args[b"user_id"][0].decode("utf8")
|
user_id = request.args[b"user_id"][0].decode("utf8")
|
||||||
|
await self.validate_appservice_can_control_user_id(app_service, user_id)
|
||||||
|
|
||||||
if app_service.sender == user_id:
|
if app_service.sender == user_id:
|
||||||
return app_service.sender, app_service
|
return app_service.sender, app_service
|
||||||
|
|
||||||
if not app_service.is_interested_in_user(user_id):
|
|
||||||
raise AuthError(403, "Application service cannot masquerade as this user.")
|
|
||||||
if not (await self.store.get_user_by_id(user_id)):
|
|
||||||
raise AuthError(403, "Application service has not registered this user")
|
|
||||||
return user_id, app_service
|
return user_id, app_service
|
||||||
|
|
||||||
async def get_user_by_access_token(
|
async def get_user_by_access_token(
|
||||||
|
@ -127,6 +127,14 @@ class ToDeviceEventTypes:
|
|||||||
RoomKeyRequest = "m.room_key_request"
|
RoomKeyRequest = "m.room_key_request"
|
||||||
|
|
||||||
|
|
||||||
|
class DeviceKeyAlgorithms:
|
||||||
|
"""Spec'd algorithms for the generation of per-device keys"""
|
||||||
|
|
||||||
|
ED25519 = "ed25519"
|
||||||
|
CURVE25519 = "curve25519"
|
||||||
|
SIGNED_CURVE25519 = "signed_curve25519"
|
||||||
|
|
||||||
|
|
||||||
class EduTypes:
|
class EduTypes:
|
||||||
Presence = "m.presence"
|
Presence = "m.presence"
|
||||||
|
|
||||||
|
@ -118,7 +118,7 @@ class RedirectException(CodeMessageException):
|
|||||||
super().__init__(code=http_code, msg=msg)
|
super().__init__(code=http_code, msg=msg)
|
||||||
self.location = location
|
self.location = location
|
||||||
|
|
||||||
self.cookies = [] # type: List[bytes]
|
self.cookies: List[bytes] = []
|
||||||
|
|
||||||
|
|
||||||
class SynapseError(CodeMessageException):
|
class SynapseError(CodeMessageException):
|
||||||
@ -160,7 +160,7 @@ class ProxiedRequestError(SynapseError):
|
|||||||
):
|
):
|
||||||
super().__init__(code, msg, errcode)
|
super().__init__(code, msg, errcode)
|
||||||
if additional_fields is None:
|
if additional_fields is None:
|
||||||
self._additional_fields = {} # type: Dict
|
self._additional_fields: Dict = {}
|
||||||
else:
|
else:
|
||||||
self._additional_fields = dict(additional_fields)
|
self._additional_fields = dict(additional_fields)
|
||||||
|
|
||||||
|
@ -289,7 +289,7 @@ class Filter:
|
|||||||
room_id = None
|
room_id = None
|
||||||
ev_type = "m.presence"
|
ev_type = "m.presence"
|
||||||
contains_url = False
|
contains_url = False
|
||||||
labels = [] # type: List[str]
|
labels: List[str] = []
|
||||||
else:
|
else:
|
||||||
sender = event.get("sender", None)
|
sender = event.get("sender", None)
|
||||||
if not sender:
|
if not sender:
|
||||||
|
@ -46,9 +46,7 @@ class Ratelimiter:
|
|||||||
# * How many times an action has occurred since a point in time
|
# * How many times an action has occurred since a point in time
|
||||||
# * The point in time
|
# * The point in time
|
||||||
# * The rate_hz of this particular entry. This can vary per request
|
# * The rate_hz of this particular entry. This can vary per request
|
||||||
self.actions = (
|
self.actions: OrderedDict[Hashable, Tuple[float, int, float]] = OrderedDict()
|
||||||
OrderedDict()
|
|
||||||
) # type: OrderedDict[Hashable, Tuple[float, int, float]]
|
|
||||||
|
|
||||||
async def can_do_action(
|
async def can_do_action(
|
||||||
self,
|
self,
|
||||||
|
@ -195,7 +195,7 @@ class RoomVersions:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
KNOWN_ROOM_VERSIONS = {
|
KNOWN_ROOM_VERSIONS: Dict[str, RoomVersion] = {
|
||||||
v.identifier: v
|
v.identifier: v
|
||||||
for v in (
|
for v in (
|
||||||
RoomVersions.V1,
|
RoomVersions.V1,
|
||||||
@ -209,4 +209,4 @@ KNOWN_ROOM_VERSIONS = {
|
|||||||
RoomVersions.V7,
|
RoomVersions.V7,
|
||||||
)
|
)
|
||||||
# Note that we do not include MSC2043 here unless it is enabled in the config.
|
# Note that we do not include MSC2043 here unless it is enabled in the config.
|
||||||
} # type: Dict[str, RoomVersion]
|
}
|
||||||
|
@ -38,6 +38,7 @@ from synapse.app.phone_stats_home import start_phone_stats_home
|
|||||||
from synapse.config.homeserver import HomeServerConfig
|
from synapse.config.homeserver import HomeServerConfig
|
||||||
from synapse.crypto import context_factory
|
from synapse.crypto import context_factory
|
||||||
from synapse.events.spamcheck import load_legacy_spam_checkers
|
from synapse.events.spamcheck import load_legacy_spam_checkers
|
||||||
|
from synapse.events.third_party_rules import load_legacy_third_party_event_rules
|
||||||
from synapse.logging.context import PreserveLoggingContext
|
from synapse.logging.context import PreserveLoggingContext
|
||||||
from synapse.metrics.background_process_metrics import wrap_as_background_process
|
from synapse.metrics.background_process_metrics import wrap_as_background_process
|
||||||
from synapse.metrics.jemalloc import setup_jemalloc_stats
|
from synapse.metrics.jemalloc import setup_jemalloc_stats
|
||||||
@ -368,6 +369,7 @@ async def start(hs: "HomeServer"):
|
|||||||
module(config=config, api=module_api)
|
module(config=config, api=module_api)
|
||||||
|
|
||||||
load_legacy_spam_checkers(hs)
|
load_legacy_spam_checkers(hs)
|
||||||
|
load_legacy_third_party_event_rules(hs)
|
||||||
|
|
||||||
# If we've configured an expiry time for caches, start the background job now.
|
# If we've configured an expiry time for caches, start the background job now.
|
||||||
setup_expire_lru_cache_entries(hs)
|
setup_expire_lru_cache_entries(hs)
|
||||||
|
@ -270,7 +270,7 @@ class GenericWorkerServer(HomeServer):
|
|||||||
site_tag = port
|
site_tag = port
|
||||||
|
|
||||||
# We always include a health resource.
|
# We always include a health resource.
|
||||||
resources = {"/health": HealthResource()} # type: Dict[str, IResource]
|
resources: Dict[str, IResource] = {"/health": HealthResource()}
|
||||||
|
|
||||||
for res in listener_config.http_options.resources:
|
for res in listener_config.http_options.resources:
|
||||||
for name in res.names:
|
for name in res.names:
|
||||||
@ -395,11 +395,9 @@ class GenericWorkerServer(HomeServer):
|
|||||||
elif listener.type == "metrics":
|
elif listener.type == "metrics":
|
||||||
if not self.config.enable_metrics:
|
if not self.config.enable_metrics:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
(
|
|
||||||
"Metrics listener configured, but "
|
"Metrics listener configured, but "
|
||||||
"enable_metrics is not True!"
|
"enable_metrics is not True!"
|
||||||
)
|
)
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
_base.listen_metrics(listener.bind_addresses, listener.port)
|
_base.listen_metrics(listener.bind_addresses, listener.port)
|
||||||
else:
|
else:
|
||||||
|
@ -305,11 +305,9 @@ class SynapseHomeServer(HomeServer):
|
|||||||
elif listener.type == "metrics":
|
elif listener.type == "metrics":
|
||||||
if not self.config.enable_metrics:
|
if not self.config.enable_metrics:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
(
|
|
||||||
"Metrics listener configured, but "
|
"Metrics listener configured, but "
|
||||||
"enable_metrics is not True!"
|
"enable_metrics is not True!"
|
||||||
)
|
)
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
_base.listen_metrics(listener.bind_addresses, listener.port)
|
_base.listen_metrics(listener.bind_addresses, listener.port)
|
||||||
else:
|
else:
|
||||||
|
@ -71,6 +71,8 @@ async def phone_stats_home(hs, stats, stats_process=_stats_process):
|
|||||||
# General statistics
|
# General statistics
|
||||||
#
|
#
|
||||||
|
|
||||||
|
store = hs.get_datastore()
|
||||||
|
|
||||||
stats["homeserver"] = hs.config.server_name
|
stats["homeserver"] = hs.config.server_name
|
||||||
stats["server_context"] = hs.config.server_context
|
stats["server_context"] = hs.config.server_context
|
||||||
stats["timestamp"] = now
|
stats["timestamp"] = now
|
||||||
@ -79,34 +81,38 @@ async def phone_stats_home(hs, stats, stats_process=_stats_process):
|
|||||||
stats["python_version"] = "{}.{}.{}".format(
|
stats["python_version"] = "{}.{}.{}".format(
|
||||||
version.major, version.minor, version.micro
|
version.major, version.minor, version.micro
|
||||||
)
|
)
|
||||||
stats["total_users"] = await hs.get_datastore().count_all_users()
|
stats["total_users"] = await store.count_all_users()
|
||||||
|
|
||||||
total_nonbridged_users = await hs.get_datastore().count_nonbridged_users()
|
total_nonbridged_users = await store.count_nonbridged_users()
|
||||||
stats["total_nonbridged_users"] = total_nonbridged_users
|
stats["total_nonbridged_users"] = total_nonbridged_users
|
||||||
|
|
||||||
daily_user_type_results = await hs.get_datastore().count_daily_user_type()
|
daily_user_type_results = await store.count_daily_user_type()
|
||||||
for name, count in daily_user_type_results.items():
|
for name, count in daily_user_type_results.items():
|
||||||
stats["daily_user_type_" + name] = count
|
stats["daily_user_type_" + name] = count
|
||||||
|
|
||||||
room_count = await hs.get_datastore().get_room_count()
|
room_count = await store.get_room_count()
|
||||||
stats["total_room_count"] = room_count
|
stats["total_room_count"] = room_count
|
||||||
|
|
||||||
stats["daily_active_users"] = await hs.get_datastore().count_daily_users()
|
stats["daily_active_users"] = await store.count_daily_users()
|
||||||
stats["monthly_active_users"] = await hs.get_datastore().count_monthly_users()
|
stats["monthly_active_users"] = await store.count_monthly_users()
|
||||||
daily_active_e2ee_rooms = await hs.get_datastore().count_daily_active_e2ee_rooms()
|
daily_active_e2ee_rooms = await store.count_daily_active_e2ee_rooms()
|
||||||
stats["daily_active_e2ee_rooms"] = daily_active_e2ee_rooms
|
stats["daily_active_e2ee_rooms"] = daily_active_e2ee_rooms
|
||||||
stats["daily_e2ee_messages"] = await hs.get_datastore().count_daily_e2ee_messages()
|
stats["daily_e2ee_messages"] = await store.count_daily_e2ee_messages()
|
||||||
daily_sent_e2ee_messages = await hs.get_datastore().count_daily_sent_e2ee_messages()
|
daily_sent_e2ee_messages = await store.count_daily_sent_e2ee_messages()
|
||||||
stats["daily_sent_e2ee_messages"] = daily_sent_e2ee_messages
|
stats["daily_sent_e2ee_messages"] = daily_sent_e2ee_messages
|
||||||
stats["daily_active_rooms"] = await hs.get_datastore().count_daily_active_rooms()
|
stats["daily_active_rooms"] = await store.count_daily_active_rooms()
|
||||||
stats["daily_messages"] = await hs.get_datastore().count_daily_messages()
|
stats["daily_messages"] = await store.count_daily_messages()
|
||||||
daily_sent_messages = await hs.get_datastore().count_daily_sent_messages()
|
daily_sent_messages = await store.count_daily_sent_messages()
|
||||||
stats["daily_sent_messages"] = daily_sent_messages
|
stats["daily_sent_messages"] = daily_sent_messages
|
||||||
|
|
||||||
r30_results = await hs.get_datastore().count_r30_users()
|
r30_results = await store.count_r30_users()
|
||||||
for name, count in r30_results.items():
|
for name, count in r30_results.items():
|
||||||
stats["r30_users_" + name] = count
|
stats["r30_users_" + name] = count
|
||||||
|
|
||||||
|
r30v2_results = await store.count_r30v2_users()
|
||||||
|
for name, count in r30v2_results.items():
|
||||||
|
stats["r30v2_users_" + name] = count
|
||||||
|
|
||||||
stats["cache_factor"] = hs.config.caches.global_factor
|
stats["cache_factor"] = hs.config.caches.global_factor
|
||||||
stats["event_cache_size"] = hs.config.caches.event_cache_size
|
stats["event_cache_size"] = hs.config.caches.event_cache_size
|
||||||
|
|
||||||
@ -115,8 +121,8 @@ async def phone_stats_home(hs, stats, stats_process=_stats_process):
|
|||||||
#
|
#
|
||||||
|
|
||||||
# This only reports info about the *main* database.
|
# This only reports info about the *main* database.
|
||||||
stats["database_engine"] = hs.get_datastore().db_pool.engine.module.__name__
|
stats["database_engine"] = store.db_pool.engine.module.__name__
|
||||||
stats["database_server_version"] = hs.get_datastore().db_pool.engine.server_version
|
stats["database_server_version"] = store.db_pool.engine.server_version
|
||||||
|
|
||||||
#
|
#
|
||||||
# Logging configuration
|
# Logging configuration
|
||||||
|
@ -88,9 +88,9 @@ class ApplicationServiceApi(SimpleHttpClient):
|
|||||||
super().__init__(hs)
|
super().__init__(hs)
|
||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
|
|
||||||
self.protocol_meta_cache = ResponseCache(
|
self.protocol_meta_cache: ResponseCache[Tuple[str, str]] = ResponseCache(
|
||||||
hs.get_clock(), "as_protocol_meta", timeout_ms=HOUR_IN_MS
|
hs.get_clock(), "as_protocol_meta", timeout_ms=HOUR_IN_MS
|
||||||
) # type: ResponseCache[Tuple[str, str]]
|
)
|
||||||
|
|
||||||
async def query_user(self, service, user_id):
|
async def query_user(self, service, user_id):
|
||||||
if service.url is None:
|
if service.url is None:
|
||||||
|
@ -18,6 +18,21 @@ class AccountValidityConfig(Config):
|
|||||||
section = "account_validity"
|
section = "account_validity"
|
||||||
|
|
||||||
def read_config(self, config, **kwargs):
|
def read_config(self, config, **kwargs):
|
||||||
|
"""Parses the old account validity config. The config format looks like this:
|
||||||
|
|
||||||
|
account_validity:
|
||||||
|
enabled: true
|
||||||
|
period: 6w
|
||||||
|
renew_at: 1w
|
||||||
|
renew_email_subject: "Renew your %(app)s account"
|
||||||
|
template_dir: "res/templates"
|
||||||
|
account_renewed_html_path: "account_renewed.html"
|
||||||
|
invalid_token_html_path: "invalid_token.html"
|
||||||
|
|
||||||
|
We expect admins to use modules for this feature (which is why it doesn't appear
|
||||||
|
in the sample config file), but we want to keep support for it around for a bit
|
||||||
|
for backwards compatibility.
|
||||||
|
"""
|
||||||
account_validity_config = config.get("account_validity") or {}
|
account_validity_config = config.get("account_validity") or {}
|
||||||
self.account_validity_enabled = account_validity_config.get("enabled", False)
|
self.account_validity_enabled = account_validity_config.get("enabled", False)
|
||||||
self.account_validity_renew_by_email_enabled = (
|
self.account_validity_renew_by_email_enabled = (
|
||||||
@ -75,90 +90,3 @@ class AccountValidityConfig(Config):
|
|||||||
],
|
],
|
||||||
account_validity_template_dir,
|
account_validity_template_dir,
|
||||||
)
|
)
|
||||||
|
|
||||||
def generate_config_section(self, **kwargs):
|
|
||||||
return """\
|
|
||||||
## Account Validity ##
|
|
||||||
|
|
||||||
# Optional account validity configuration. This allows for accounts to be denied
|
|
||||||
# any request after a given period.
|
|
||||||
#
|
|
||||||
# Once this feature is enabled, Synapse will look for registered users without an
|
|
||||||
# expiration date at startup and will add one to every account it found using the
|
|
||||||
# current settings at that time.
|
|
||||||
# This means that, if a validity period is set, and Synapse is restarted (it will
|
|
||||||
# then derive an expiration date from the current validity period), and some time
|
|
||||||
# after that the validity period changes and Synapse is restarted, the users'
|
|
||||||
# expiration dates won't be updated unless their account is manually renewed. This
|
|
||||||
# date will be randomly selected within a range [now + period - d ; now + period],
|
|
||||||
# where d is equal to 10% of the validity period.
|
|
||||||
#
|
|
||||||
account_validity:
|
|
||||||
# The account validity feature is disabled by default. Uncomment the
|
|
||||||
# following line to enable it.
|
|
||||||
#
|
|
||||||
#enabled: true
|
|
||||||
|
|
||||||
# The period after which an account is valid after its registration. When
|
|
||||||
# renewing the account, its validity period will be extended by this amount
|
|
||||||
# of time. This parameter is required when using the account validity
|
|
||||||
# feature.
|
|
||||||
#
|
|
||||||
#period: 6w
|
|
||||||
|
|
||||||
# The amount of time before an account's expiry date at which Synapse will
|
|
||||||
# send an email to the account's email address with a renewal link. By
|
|
||||||
# default, no such emails are sent.
|
|
||||||
#
|
|
||||||
# If you enable this setting, you will also need to fill out the 'email' and
|
|
||||||
# 'public_baseurl' configuration sections.
|
|
||||||
#
|
|
||||||
#renew_at: 1w
|
|
||||||
|
|
||||||
# The subject of the email sent out with the renewal link. '%(app)s' can be
|
|
||||||
# used as a placeholder for the 'app_name' parameter from the 'email'
|
|
||||||
# section.
|
|
||||||
#
|
|
||||||
# Note that the placeholder must be written '%(app)s', including the
|
|
||||||
# trailing 's'.
|
|
||||||
#
|
|
||||||
# If this is not set, a default value is used.
|
|
||||||
#
|
|
||||||
#renew_email_subject: "Renew your %(app)s account"
|
|
||||||
|
|
||||||
# Directory in which Synapse will try to find templates for the HTML files to
|
|
||||||
# serve to the user when trying to renew an account. If not set, default
|
|
||||||
# templates from within the Synapse package will be used.
|
|
||||||
#
|
|
||||||
# The currently available templates are:
|
|
||||||
#
|
|
||||||
# * account_renewed.html: Displayed to the user after they have successfully
|
|
||||||
# renewed their account.
|
|
||||||
#
|
|
||||||
# * account_previously_renewed.html: Displayed to the user if they attempt to
|
|
||||||
# renew their account with a token that is valid, but that has already
|
|
||||||
# been used. In this case the account is not renewed again.
|
|
||||||
#
|
|
||||||
# * invalid_token.html: Displayed to the user when they try to renew an account
|
|
||||||
# with an unknown or invalid renewal token.
|
|
||||||
#
|
|
||||||
# See https://github.com/matrix-org/synapse/tree/master/synapse/res/templates for
|
|
||||||
# default template contents.
|
|
||||||
#
|
|
||||||
# The file name of some of these templates can be configured below for legacy
|
|
||||||
# reasons.
|
|
||||||
#
|
|
||||||
#template_dir: "res/templates"
|
|
||||||
|
|
||||||
# A custom file name for the 'account_renewed.html' template.
|
|
||||||
#
|
|
||||||
# If not set, the file is assumed to be named "account_renewed.html".
|
|
||||||
#
|
|
||||||
#account_renewed_html_path: "account_renewed.html"
|
|
||||||
|
|
||||||
# A custom file name for the 'invalid_token.html' template.
|
|
||||||
#
|
|
||||||
# If not set, the file is assumed to be named "invalid_token.html".
|
|
||||||
#
|
|
||||||
#invalid_token_html_path: "invalid_token.html"
|
|
||||||
"""
|
|
||||||
|
@ -57,14 +57,14 @@ def load_appservices(hostname, config_files):
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
# Dicts of value -> filename
|
# Dicts of value -> filename
|
||||||
seen_as_tokens = {} # type: Dict[str, str]
|
seen_as_tokens: Dict[str, str] = {}
|
||||||
seen_ids = {} # type: Dict[str, str]
|
seen_ids: Dict[str, str] = {}
|
||||||
|
|
||||||
appservices = []
|
appservices = []
|
||||||
|
|
||||||
for config_file in config_files:
|
for config_file in config_files:
|
||||||
try:
|
try:
|
||||||
with open(config_file, "r") as f:
|
with open(config_file) as f:
|
||||||
appservice = _load_appservice(hostname, yaml.safe_load(f), config_file)
|
appservice = _load_appservice(hostname, yaml.safe_load(f), config_file)
|
||||||
if appservice.id in seen_ids:
|
if appservice.id in seen_ids:
|
||||||
raise ConfigError(
|
raise ConfigError(
|
||||||
|
@ -25,7 +25,7 @@ from ._base import Config, ConfigError
|
|||||||
_CACHE_PREFIX = "SYNAPSE_CACHE_FACTOR"
|
_CACHE_PREFIX = "SYNAPSE_CACHE_FACTOR"
|
||||||
|
|
||||||
# Map from canonicalised cache name to cache.
|
# Map from canonicalised cache name to cache.
|
||||||
_CACHES = {} # type: Dict[str, Callable[[float], None]]
|
_CACHES: Dict[str, Callable[[float], None]] = {}
|
||||||
|
|
||||||
# a lock on the contents of _CACHES
|
# a lock on the contents of _CACHES
|
||||||
_CACHES_LOCK = threading.Lock()
|
_CACHES_LOCK = threading.Lock()
|
||||||
@ -157,7 +157,7 @@ class CacheConfig(Config):
|
|||||||
self.event_cache_size = self.parse_size(
|
self.event_cache_size = self.parse_size(
|
||||||
config.get("event_cache_size", _DEFAULT_EVENT_CACHE_SIZE)
|
config.get("event_cache_size", _DEFAULT_EVENT_CACHE_SIZE)
|
||||||
)
|
)
|
||||||
self.cache_factors = {} # type: Dict[str, float]
|
self.cache_factors: Dict[str, float] = {}
|
||||||
|
|
||||||
cache_config = config.get("caches") or {}
|
cache_config = config.get("caches") or {}
|
||||||
self.global_factor = cache_config.get(
|
self.global_factor = cache_config.get(
|
||||||
|
@ -134,9 +134,9 @@ class EmailConfig(Config):
|
|||||||
|
|
||||||
# trusted_third_party_id_servers does not contain a scheme whereas
|
# trusted_third_party_id_servers does not contain a scheme whereas
|
||||||
# account_threepid_delegate_email is expected to. Presume https
|
# account_threepid_delegate_email is expected to. Presume https
|
||||||
self.account_threepid_delegate_email = (
|
self.account_threepid_delegate_email: Optional[str] = (
|
||||||
"https://" + first_trusted_identity_server
|
"https://" + first_trusted_identity_server
|
||||||
) # type: Optional[str]
|
)
|
||||||
self.using_identity_server_from_trusted_list = True
|
self.using_identity_server_from_trusted_list = True
|
||||||
else:
|
else:
|
||||||
raise ConfigError(
|
raise ConfigError(
|
||||||
|
@ -25,10 +25,10 @@ class ExperimentalConfig(Config):
|
|||||||
experimental = config.get("experimental_features") or {}
|
experimental = config.get("experimental_features") or {}
|
||||||
|
|
||||||
# MSC2858 (multiple SSO identity providers)
|
# MSC2858 (multiple SSO identity providers)
|
||||||
self.msc2858_enabled = experimental.get("msc2858_enabled", False) # type: bool
|
self.msc2858_enabled: bool = experimental.get("msc2858_enabled", False)
|
||||||
|
|
||||||
# MSC3026 (busy presence state)
|
# MSC3026 (busy presence state)
|
||||||
self.msc3026_enabled = experimental.get("msc3026_enabled", False) # type: bool
|
self.msc3026_enabled: bool = experimental.get("msc3026_enabled", False)
|
||||||
|
|
||||||
# MSC2716 (backfill existing history)
|
# MSC2716 (backfill existing history)
|
||||||
self.msc2716_enabled = experimental.get("msc2716_enabled", False) # type: bool
|
self.msc2716_enabled: bool = experimental.get("msc2716_enabled", False)
|
||||||
|
@ -22,7 +22,7 @@ class FederationConfig(Config):
|
|||||||
|
|
||||||
def read_config(self, config, **kwargs):
|
def read_config(self, config, **kwargs):
|
||||||
# FIXME: federation_domain_whitelist needs sytests
|
# FIXME: federation_domain_whitelist needs sytests
|
||||||
self.federation_domain_whitelist = None # type: Optional[dict]
|
self.federation_domain_whitelist: Optional[dict] = None
|
||||||
federation_domain_whitelist = config.get("federation_domain_whitelist", None)
|
federation_domain_whitelist = config.get("federation_domain_whitelist", None)
|
||||||
|
|
||||||
if federation_domain_whitelist is not None:
|
if federation_domain_whitelist is not None:
|
||||||
|
@ -460,7 +460,7 @@ def _parse_oidc_config_dict(
|
|||||||
) from e
|
) from e
|
||||||
|
|
||||||
client_secret_jwt_key_config = oidc_config.get("client_secret_jwt_key")
|
client_secret_jwt_key_config = oidc_config.get("client_secret_jwt_key")
|
||||||
client_secret_jwt_key = None # type: Optional[OidcProviderClientSecretJwtKey]
|
client_secret_jwt_key: Optional[OidcProviderClientSecretJwtKey] = None
|
||||||
if client_secret_jwt_key_config is not None:
|
if client_secret_jwt_key_config is not None:
|
||||||
keyfile = client_secret_jwt_key_config.get("key_file")
|
keyfile = client_secret_jwt_key_config.get("key_file")
|
||||||
if keyfile:
|
if keyfile:
|
||||||
|
@ -25,7 +25,7 @@ class PasswordAuthProviderConfig(Config):
|
|||||||
section = "authproviders"
|
section = "authproviders"
|
||||||
|
|
||||||
def read_config(self, config, **kwargs):
|
def read_config(self, config, **kwargs):
|
||||||
self.password_providers = [] # type: List[Any]
|
self.password_providers: List[Any] = []
|
||||||
providers = []
|
providers = []
|
||||||
|
|
||||||
# We want to be backwards compatible with the old `ldap_config`
|
# We want to be backwards compatible with the old `ldap_config`
|
||||||
|
@ -62,7 +62,7 @@ def parse_thumbnail_requirements(thumbnail_sizes):
|
|||||||
Dictionary mapping from media type string to list of
|
Dictionary mapping from media type string to list of
|
||||||
ThumbnailRequirement tuples.
|
ThumbnailRequirement tuples.
|
||||||
"""
|
"""
|
||||||
requirements = {} # type: Dict[str, List]
|
requirements: Dict[str, List] = {}
|
||||||
for size in thumbnail_sizes:
|
for size in thumbnail_sizes:
|
||||||
width = size["width"]
|
width = size["width"]
|
||||||
height = size["height"]
|
height = size["height"]
|
||||||
@ -141,7 +141,7 @@ class ContentRepositoryConfig(Config):
|
|||||||
#
|
#
|
||||||
# We don't create the storage providers here as not all workers need
|
# We don't create the storage providers here as not all workers need
|
||||||
# them to be started.
|
# them to be started.
|
||||||
self.media_storage_providers = [] # type: List[tuple]
|
self.media_storage_providers: List[tuple] = []
|
||||||
|
|
||||||
for i, provider_config in enumerate(storage_providers):
|
for i, provider_config in enumerate(storage_providers):
|
||||||
# We special case the module "file_system" so as not to need to
|
# We special case the module "file_system" so as not to need to
|
||||||
|
@ -505,7 +505,7 @@ class ServerConfig(Config):
|
|||||||
" greater than 'allowed_lifetime_max'"
|
" greater than 'allowed_lifetime_max'"
|
||||||
)
|
)
|
||||||
|
|
||||||
self.retention_purge_jobs = [] # type: List[Dict[str, Optional[int]]]
|
self.retention_purge_jobs: List[Dict[str, Optional[int]]] = []
|
||||||
for purge_job_config in retention_config.get("purge_jobs", []):
|
for purge_job_config in retention_config.get("purge_jobs", []):
|
||||||
interval_config = purge_job_config.get("interval")
|
interval_config = purge_job_config.get("interval")
|
||||||
|
|
||||||
@ -688,23 +688,21 @@ class ServerConfig(Config):
|
|||||||
# not included in the sample configuration file on purpose as it's a temporary
|
# not included in the sample configuration file on purpose as it's a temporary
|
||||||
# hack, so that some users can trial the new defaults without impacting every
|
# hack, so that some users can trial the new defaults without impacting every
|
||||||
# user on the homeserver.
|
# user on the homeserver.
|
||||||
users_new_default_push_rules = (
|
users_new_default_push_rules: list = (
|
||||||
config.get("users_new_default_push_rules") or []
|
config.get("users_new_default_push_rules") or []
|
||||||
) # type: list
|
)
|
||||||
if not isinstance(users_new_default_push_rules, list):
|
if not isinstance(users_new_default_push_rules, list):
|
||||||
raise ConfigError("'users_new_default_push_rules' must be a list")
|
raise ConfigError("'users_new_default_push_rules' must be a list")
|
||||||
|
|
||||||
# Turn the list into a set to improve lookup speed.
|
# Turn the list into a set to improve lookup speed.
|
||||||
self.users_new_default_push_rules = set(
|
self.users_new_default_push_rules: set = set(users_new_default_push_rules)
|
||||||
users_new_default_push_rules
|
|
||||||
) # type: set
|
|
||||||
|
|
||||||
# Whitelist of domain names that given next_link parameters must have
|
# Whitelist of domain names that given next_link parameters must have
|
||||||
next_link_domain_whitelist = config.get(
|
next_link_domain_whitelist: Optional[List[str]] = config.get(
|
||||||
"next_link_domain_whitelist"
|
"next_link_domain_whitelist"
|
||||||
) # type: Optional[List[str]]
|
)
|
||||||
|
|
||||||
self.next_link_domain_whitelist = None # type: Optional[Set[str]]
|
self.next_link_domain_whitelist: Optional[Set[str]] = None
|
||||||
if next_link_domain_whitelist is not None:
|
if next_link_domain_whitelist is not None:
|
||||||
if not isinstance(next_link_domain_whitelist, list):
|
if not isinstance(next_link_domain_whitelist, list):
|
||||||
raise ConfigError("'next_link_domain_whitelist' must be a list")
|
raise ConfigError("'next_link_domain_whitelist' must be a list")
|
||||||
|
@ -34,7 +34,7 @@ class SpamCheckerConfig(Config):
|
|||||||
section = "spamchecker"
|
section = "spamchecker"
|
||||||
|
|
||||||
def read_config(self, config, **kwargs):
|
def read_config(self, config, **kwargs):
|
||||||
self.spam_checkers = [] # type: List[Tuple[Any, Dict]]
|
self.spam_checkers: List[Tuple[Any, Dict]] = []
|
||||||
|
|
||||||
spam_checkers = config.get("spam_checker") or []
|
spam_checkers = config.get("spam_checker") or []
|
||||||
if isinstance(spam_checkers, dict):
|
if isinstance(spam_checkers, dict):
|
||||||
|
@ -39,7 +39,7 @@ class SSOConfig(Config):
|
|||||||
section = "sso"
|
section = "sso"
|
||||||
|
|
||||||
def read_config(self, config, **kwargs):
|
def read_config(self, config, **kwargs):
|
||||||
sso_config = config.get("sso") or {} # type: Dict[str, Any]
|
sso_config: Dict[str, Any] = config.get("sso") or {}
|
||||||
|
|
||||||
# The sso-specific template_dir
|
# The sso-specific template_dir
|
||||||
self.sso_template_dir = sso_config.get("template_dir")
|
self.sso_template_dir = sso_config.get("template_dir")
|
||||||
|
@ -38,13 +38,9 @@ class StatsConfig(Config):
|
|||||||
|
|
||||||
def read_config(self, config, **kwargs):
|
def read_config(self, config, **kwargs):
|
||||||
self.stats_enabled = True
|
self.stats_enabled = True
|
||||||
self.stats_bucket_size = 86400 * 1000
|
|
||||||
stats_config = config.get("stats", None)
|
stats_config = config.get("stats", None)
|
||||||
if stats_config:
|
if stats_config:
|
||||||
self.stats_enabled = stats_config.get("enabled", self.stats_enabled)
|
self.stats_enabled = stats_config.get("enabled", self.stats_enabled)
|
||||||
self.stats_bucket_size = self.parse_duration(
|
|
||||||
stats_config.get("bucket_size", "1d")
|
|
||||||
)
|
|
||||||
if not self.stats_enabled:
|
if not self.stats_enabled:
|
||||||
logger.warning(ROOM_STATS_DISABLED_WARN)
|
logger.warning(ROOM_STATS_DISABLED_WARN)
|
||||||
|
|
||||||
@ -59,9 +55,4 @@ class StatsConfig(Config):
|
|||||||
# correctly.
|
# correctly.
|
||||||
#
|
#
|
||||||
#enabled: false
|
#enabled: false
|
||||||
|
|
||||||
# The size of each timeslice in the room_stats_historical and
|
|
||||||
# user_stats_historical tables, as a time period. Defaults to "1d".
|
|
||||||
#
|
|
||||||
#bucket_size: 1h
|
|
||||||
"""
|
"""
|
||||||
|
@ -28,18 +28,3 @@ class ThirdPartyRulesConfig(Config):
|
|||||||
self.third_party_event_rules = load_module(
|
self.third_party_event_rules = load_module(
|
||||||
provider, ("third_party_event_rules",)
|
provider, ("third_party_event_rules",)
|
||||||
)
|
)
|
||||||
|
|
||||||
def generate_config_section(self, **kwargs):
|
|
||||||
return """\
|
|
||||||
# Server admins can define a Python module that implements extra rules for
|
|
||||||
# allowing or denying incoming events. In order to work, this module needs to
|
|
||||||
# override the methods defined in synapse/events/third_party_rules.py.
|
|
||||||
#
|
|
||||||
# This feature is designed to be used in closed federations only, where each
|
|
||||||
# participating server enforces the same rules.
|
|
||||||
#
|
|
||||||
#third_party_event_rules:
|
|
||||||
# module: "my_custom_project.SuperRulesSet"
|
|
||||||
# config:
|
|
||||||
# example_option: 'things'
|
|
||||||
"""
|
|
||||||
|
@ -66,11 +66,9 @@ class TlsConfig(Config):
|
|||||||
if self.federation_client_minimum_tls_version == "1.3":
|
if self.federation_client_minimum_tls_version == "1.3":
|
||||||
if getattr(SSL, "OP_NO_TLSv1_3", None) is None:
|
if getattr(SSL, "OP_NO_TLSv1_3", None) is None:
|
||||||
raise ConfigError(
|
raise ConfigError(
|
||||||
(
|
|
||||||
"federation_client_minimum_tls_version cannot be 1.3, "
|
"federation_client_minimum_tls_version cannot be 1.3, "
|
||||||
"your OpenSSL does not support it"
|
"your OpenSSL does not support it"
|
||||||
)
|
)
|
||||||
)
|
|
||||||
|
|
||||||
# Whitelist of domains to not verify certificates for
|
# Whitelist of domains to not verify certificates for
|
||||||
fed_whitelist_entries = config.get(
|
fed_whitelist_entries = config.get(
|
||||||
@ -80,7 +78,7 @@ class TlsConfig(Config):
|
|||||||
fed_whitelist_entries = []
|
fed_whitelist_entries = []
|
||||||
|
|
||||||
# Support globs (*) in whitelist values
|
# Support globs (*) in whitelist values
|
||||||
self.federation_certificate_verification_whitelist = [] # type: List[Pattern]
|
self.federation_certificate_verification_whitelist: List[Pattern] = []
|
||||||
for entry in fed_whitelist_entries:
|
for entry in fed_whitelist_entries:
|
||||||
try:
|
try:
|
||||||
entry_regex = glob_to_regex(entry.encode("ascii").decode("ascii"))
|
entry_regex = glob_to_regex(entry.encode("ascii").decode("ascii"))
|
||||||
@ -132,8 +130,8 @@ class TlsConfig(Config):
|
|||||||
"use_insecure_ssl_client_just_for_testing_do_not_use"
|
"use_insecure_ssl_client_just_for_testing_do_not_use"
|
||||||
)
|
)
|
||||||
|
|
||||||
self.tls_certificate = None # type: Optional[crypto.X509]
|
self.tls_certificate: Optional[crypto.X509] = None
|
||||||
self.tls_private_key = None # type: Optional[crypto.PKey]
|
self.tls_private_key: Optional[crypto.PKey] = None
|
||||||
|
|
||||||
def is_disk_cert_valid(self, allow_self_signed=True):
|
def is_disk_cert_valid(self, allow_self_signed=True):
|
||||||
"""
|
"""
|
||||||
|
@ -170,11 +170,13 @@ class Keyring:
|
|||||||
)
|
)
|
||||||
self._key_fetchers = key_fetchers
|
self._key_fetchers = key_fetchers
|
||||||
|
|
||||||
self._server_queue = BatchingQueue(
|
self._server_queue: BatchingQueue[
|
||||||
|
_FetchKeyRequest, Dict[str, Dict[str, FetchKeyResult]]
|
||||||
|
] = BatchingQueue(
|
||||||
"keyring_server",
|
"keyring_server",
|
||||||
clock=hs.get_clock(),
|
clock=hs.get_clock(),
|
||||||
process_batch_callback=self._inner_fetch_key_requests,
|
process_batch_callback=self._inner_fetch_key_requests,
|
||||||
) # type: BatchingQueue[_FetchKeyRequest, Dict[str, Dict[str, FetchKeyResult]]]
|
)
|
||||||
|
|
||||||
async def verify_json_for_server(
|
async def verify_json_for_server(
|
||||||
self,
|
self,
|
||||||
@ -330,7 +332,7 @@ class Keyring:
|
|||||||
# First we need to deduplicate requests for the same key. We do this by
|
# First we need to deduplicate requests for the same key. We do this by
|
||||||
# taking the *maximum* requested `minimum_valid_until_ts` for each pair
|
# taking the *maximum* requested `minimum_valid_until_ts` for each pair
|
||||||
# of server name/key ID.
|
# of server name/key ID.
|
||||||
server_to_key_to_ts = {} # type: Dict[str, Dict[str, int]]
|
server_to_key_to_ts: Dict[str, Dict[str, int]] = {}
|
||||||
for request in requests:
|
for request in requests:
|
||||||
by_server = server_to_key_to_ts.setdefault(request.server_name, {})
|
by_server = server_to_key_to_ts.setdefault(request.server_name, {})
|
||||||
for key_id in request.key_ids:
|
for key_id in request.key_ids:
|
||||||
@ -355,7 +357,7 @@ class Keyring:
|
|||||||
|
|
||||||
# We now convert the returned list of results into a map from server
|
# We now convert the returned list of results into a map from server
|
||||||
# name to key ID to FetchKeyResult, to return.
|
# name to key ID to FetchKeyResult, to return.
|
||||||
to_return = {} # type: Dict[str, Dict[str, FetchKeyResult]]
|
to_return: Dict[str, Dict[str, FetchKeyResult]] = {}
|
||||||
for (request, results) in zip(deduped_requests, results_per_request):
|
for (request, results) in zip(deduped_requests, results_per_request):
|
||||||
to_return_by_server = to_return.setdefault(request.server_name, {})
|
to_return_by_server = to_return.setdefault(request.server_name, {})
|
||||||
for key_id, key_result in results.items():
|
for key_id, key_result in results.items():
|
||||||
@ -455,7 +457,7 @@ class StoreKeyFetcher(KeyFetcher):
|
|||||||
)
|
)
|
||||||
|
|
||||||
res = await self.store.get_server_verify_keys(key_ids_to_fetch)
|
res = await self.store.get_server_verify_keys(key_ids_to_fetch)
|
||||||
keys = {} # type: Dict[str, Dict[str, FetchKeyResult]]
|
keys: Dict[str, Dict[str, FetchKeyResult]] = {}
|
||||||
for (server_name, key_id), key in res.items():
|
for (server_name, key_id), key in res.items():
|
||||||
keys.setdefault(server_name, {})[key_id] = key
|
keys.setdefault(server_name, {})[key_id] = key
|
||||||
return keys
|
return keys
|
||||||
@ -603,7 +605,7 @@ class PerspectivesKeyFetcher(BaseV2KeyFetcher):
|
|||||||
).addErrback(unwrapFirstError)
|
).addErrback(unwrapFirstError)
|
||||||
)
|
)
|
||||||
|
|
||||||
union_of_keys = {} # type: Dict[str, Dict[str, FetchKeyResult]]
|
union_of_keys: Dict[str, Dict[str, FetchKeyResult]] = {}
|
||||||
for result in results:
|
for result in results:
|
||||||
for server_name, keys in result.items():
|
for server_name, keys in result.items():
|
||||||
union_of_keys.setdefault(server_name, {}).update(keys)
|
union_of_keys.setdefault(server_name, {}).update(keys)
|
||||||
@ -656,8 +658,8 @@ class PerspectivesKeyFetcher(BaseV2KeyFetcher):
|
|||||||
except HttpResponseException as e:
|
except HttpResponseException as e:
|
||||||
raise KeyLookupError("Remote server returned an error: %s" % (e,))
|
raise KeyLookupError("Remote server returned an error: %s" % (e,))
|
||||||
|
|
||||||
keys = {} # type: Dict[str, Dict[str, FetchKeyResult]]
|
keys: Dict[str, Dict[str, FetchKeyResult]] = {}
|
||||||
added_keys = [] # type: List[Tuple[str, str, FetchKeyResult]]
|
added_keys: List[Tuple[str, str, FetchKeyResult]] = []
|
||||||
|
|
||||||
time_now_ms = self.clock.time_msec()
|
time_now_ms = self.clock.time_msec()
|
||||||
|
|
||||||
@ -805,7 +807,7 @@ class ServerKeyFetcher(BaseV2KeyFetcher):
|
|||||||
Raises:
|
Raises:
|
||||||
KeyLookupError if there was a problem making the lookup
|
KeyLookupError if there was a problem making the lookup
|
||||||
"""
|
"""
|
||||||
keys = {} # type: Dict[str, FetchKeyResult]
|
keys: Dict[str, FetchKeyResult] = {}
|
||||||
|
|
||||||
for requested_key_id in key_ids:
|
for requested_key_id in key_ids:
|
||||||
# we may have found this key as a side-effect of asking for another.
|
# we may have found this key as a side-effect of asking for another.
|
||||||
|
@ -48,6 +48,9 @@ def check(
|
|||||||
room_version_obj: the version of the room
|
room_version_obj: the version of the room
|
||||||
event: the event being checked.
|
event: the event being checked.
|
||||||
auth_events: the existing room state.
|
auth_events: the existing room state.
|
||||||
|
do_sig_check: True if it should be verified that the sending server
|
||||||
|
signed the event.
|
||||||
|
do_size_check: True if the size of the event fields should be verified.
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
AuthError if the checks fail
|
AuthError if the checks fail
|
||||||
@ -528,7 +531,7 @@ def _check_power_levels(
|
|||||||
user_level = get_user_power_level(event.user_id, auth_events)
|
user_level = get_user_power_level(event.user_id, auth_events)
|
||||||
|
|
||||||
# Check other levels:
|
# Check other levels:
|
||||||
levels_to_check = [
|
levels_to_check: List[Tuple[str, Optional[str]]] = [
|
||||||
("users_default", None),
|
("users_default", None),
|
||||||
("events_default", None),
|
("events_default", None),
|
||||||
("state_default", None),
|
("state_default", None),
|
||||||
@ -536,7 +539,7 @@ def _check_power_levels(
|
|||||||
("redact", None),
|
("redact", None),
|
||||||
("kick", None),
|
("kick", None),
|
||||||
("invite", None),
|
("invite", None),
|
||||||
] # type: List[Tuple[str, Optional[str]]]
|
]
|
||||||
|
|
||||||
old_list = current_state.content.get("users", {})
|
old_list = current_state.content.get("users", {})
|
||||||
for user in set(list(old_list) + list(user_list)):
|
for user in set(list(old_list) + list(user_list)):
|
||||||
@ -566,12 +569,12 @@ def _check_power_levels(
|
|||||||
new_loc = new_loc.get(dir, {})
|
new_loc = new_loc.get(dir, {})
|
||||||
|
|
||||||
if level_to_check in old_loc:
|
if level_to_check in old_loc:
|
||||||
old_level = int(old_loc[level_to_check]) # type: Optional[int]
|
old_level: Optional[int] = int(old_loc[level_to_check])
|
||||||
else:
|
else:
|
||||||
old_level = None
|
old_level = None
|
||||||
|
|
||||||
if level_to_check in new_loc:
|
if level_to_check in new_loc:
|
||||||
new_level = int(new_loc[level_to_check]) # type: Optional[int]
|
new_level: Optional[int] = int(new_loc[level_to_check])
|
||||||
else:
|
else:
|
||||||
new_level = None
|
new_level = None
|
||||||
|
|
||||||
|
@ -105,28 +105,28 @@ class _EventInternalMetadata:
|
|||||||
self._dict = dict(internal_metadata_dict)
|
self._dict = dict(internal_metadata_dict)
|
||||||
|
|
||||||
# the stream ordering of this event. None, until it has been persisted.
|
# the stream ordering of this event. None, until it has been persisted.
|
||||||
self.stream_ordering = None # type: Optional[int]
|
self.stream_ordering: Optional[int] = None
|
||||||
|
|
||||||
# whether this event is an outlier (ie, whether we have the state at that point
|
# whether this event is an outlier (ie, whether we have the state at that point
|
||||||
# in the DAG)
|
# in the DAG)
|
||||||
self.outlier = False
|
self.outlier = False
|
||||||
|
|
||||||
out_of_band_membership = DictProperty("out_of_band_membership") # type: bool
|
out_of_band_membership: bool = DictProperty("out_of_band_membership")
|
||||||
send_on_behalf_of = DictProperty("send_on_behalf_of") # type: str
|
send_on_behalf_of: str = DictProperty("send_on_behalf_of")
|
||||||
recheck_redaction = DictProperty("recheck_redaction") # type: bool
|
recheck_redaction: bool = DictProperty("recheck_redaction")
|
||||||
soft_failed = DictProperty("soft_failed") # type: bool
|
soft_failed: bool = DictProperty("soft_failed")
|
||||||
proactively_send = DictProperty("proactively_send") # type: bool
|
proactively_send: bool = DictProperty("proactively_send")
|
||||||
redacted = DictProperty("redacted") # type: bool
|
redacted: bool = DictProperty("redacted")
|
||||||
txn_id = DictProperty("txn_id") # type: str
|
txn_id: str = DictProperty("txn_id")
|
||||||
token_id = DictProperty("token_id") # type: int
|
token_id: int = DictProperty("token_id")
|
||||||
historical = DictProperty("historical") # type: bool
|
historical: bool = DictProperty("historical")
|
||||||
|
|
||||||
# XXX: These are set by StreamWorkerStore._set_before_and_after.
|
# XXX: These are set by StreamWorkerStore._set_before_and_after.
|
||||||
# I'm pretty sure that these are never persisted to the database, so shouldn't
|
# I'm pretty sure that these are never persisted to the database, so shouldn't
|
||||||
# be here
|
# be here
|
||||||
before = DictProperty("before") # type: RoomStreamToken
|
before: RoomStreamToken = DictProperty("before")
|
||||||
after = DictProperty("after") # type: RoomStreamToken
|
after: RoomStreamToken = DictProperty("after")
|
||||||
order = DictProperty("order") # type: Tuple[int, int]
|
order: Tuple[int, int] = DictProperty("order")
|
||||||
|
|
||||||
def get_dict(self) -> JsonDict:
|
def get_dict(self) -> JsonDict:
|
||||||
return dict(self._dict)
|
return dict(self._dict)
|
||||||
@ -291,6 +291,20 @@ class EventBase(metaclass=abc.ABCMeta):
|
|||||||
|
|
||||||
return pdu_json
|
return pdu_json
|
||||||
|
|
||||||
|
def get_templated_pdu_json(self) -> JsonDict:
|
||||||
|
"""
|
||||||
|
Return a JSON object suitable for a templated event, as used in the
|
||||||
|
make_{join,leave,knock} workflow.
|
||||||
|
"""
|
||||||
|
# By using _dict directly we don't pull in signatures/unsigned.
|
||||||
|
template_json = dict(self._dict)
|
||||||
|
# The hashes (similar to the signature) need to be recalculated by the
|
||||||
|
# joining/leaving/knocking server after (potentially) modifying the
|
||||||
|
# event.
|
||||||
|
template_json.pop("hashes")
|
||||||
|
|
||||||
|
return template_json
|
||||||
|
|
||||||
def __set__(self, instance, value):
|
def __set__(self, instance, value):
|
||||||
raise AttributeError("Unrecognized attribute %s" % (instance,))
|
raise AttributeError("Unrecognized attribute %s" % (instance,))
|
||||||
|
|
||||||
|
@ -132,12 +132,12 @@ class EventBuilder:
|
|||||||
format_version = self.room_version.event_format
|
format_version = self.room_version.event_format
|
||||||
if format_version == EventFormatVersions.V1:
|
if format_version == EventFormatVersions.V1:
|
||||||
# The types of auth/prev events changes between event versions.
|
# The types of auth/prev events changes between event versions.
|
||||||
auth_events = await self._store.add_event_hashes(
|
auth_events: Union[
|
||||||
auth_event_ids
|
List[str], List[Tuple[str, Dict[str, str]]]
|
||||||
) # type: Union[List[str], List[Tuple[str, Dict[str, str]]]]
|
] = await self._store.add_event_hashes(auth_event_ids)
|
||||||
prev_events = await self._store.add_event_hashes(
|
prev_events: Union[
|
||||||
prev_event_ids
|
List[str], List[Tuple[str, Dict[str, str]]]
|
||||||
) # type: Union[List[str], List[Tuple[str, Dict[str, str]]]]
|
] = await self._store.add_event_hashes(prev_event_ids)
|
||||||
else:
|
else:
|
||||||
auth_events = auth_event_ids
|
auth_events = auth_event_ids
|
||||||
prev_events = prev_event_ids
|
prev_events = prev_event_ids
|
||||||
@ -156,7 +156,7 @@ class EventBuilder:
|
|||||||
# the db)
|
# the db)
|
||||||
depth = min(depth, MAX_DEPTH)
|
depth = min(depth, MAX_DEPTH)
|
||||||
|
|
||||||
event_dict = {
|
event_dict: Dict[str, Any] = {
|
||||||
"auth_events": auth_events,
|
"auth_events": auth_events,
|
||||||
"prev_events": prev_events,
|
"prev_events": prev_events,
|
||||||
"type": self.type,
|
"type": self.type,
|
||||||
@ -166,7 +166,7 @@ class EventBuilder:
|
|||||||
"unsigned": self.unsigned,
|
"unsigned": self.unsigned,
|
||||||
"depth": depth,
|
"depth": depth,
|
||||||
"prev_state": [],
|
"prev_state": [],
|
||||||
} # type: Dict[str, Any]
|
}
|
||||||
|
|
||||||
if self.is_state():
|
if self.is_state():
|
||||||
event_dict["state_key"] = self._state_key
|
event_dict["state_key"] = self._state_key
|
||||||
|
@ -76,7 +76,7 @@ def load_legacy_spam_checkers(hs: "synapse.server.HomeServer"):
|
|||||||
"""Wrapper that loads spam checkers configured using the old configuration, and
|
"""Wrapper that loads spam checkers configured using the old configuration, and
|
||||||
registers the spam checker hooks they implement.
|
registers the spam checker hooks they implement.
|
||||||
"""
|
"""
|
||||||
spam_checkers = [] # type: List[Any]
|
spam_checkers: List[Any] = []
|
||||||
api = hs.get_module_api()
|
api = hs.get_module_api()
|
||||||
for module, config in hs.config.spam_checkers:
|
for module, config in hs.config.spam_checkers:
|
||||||
# Older spam checkers don't accept the `api` argument, so we
|
# Older spam checkers don't accept the `api` argument, so we
|
||||||
@ -239,7 +239,7 @@ class SpamChecker:
|
|||||||
will be used as the error message returned to the user.
|
will be used as the error message returned to the user.
|
||||||
"""
|
"""
|
||||||
for callback in self._check_event_for_spam_callbacks:
|
for callback in self._check_event_for_spam_callbacks:
|
||||||
res = await callback(event) # type: Union[bool, str]
|
res: Union[bool, str] = await callback(event)
|
||||||
if res:
|
if res:
|
||||||
return res
|
return res
|
||||||
|
|
||||||
|
@ -11,16 +11,124 @@
|
|||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
import logging
|
||||||
|
from typing import TYPE_CHECKING, Awaitable, Callable, List, Optional, Tuple
|
||||||
|
|
||||||
from typing import TYPE_CHECKING, Union
|
from synapse.api.errors import SynapseError
|
||||||
|
|
||||||
from synapse.events import EventBase
|
from synapse.events import EventBase
|
||||||
from synapse.events.snapshot import EventContext
|
from synapse.events.snapshot import EventContext
|
||||||
from synapse.types import Requester, StateMap
|
from synapse.types import Requester, StateMap
|
||||||
|
from synapse.util.async_helpers import maybe_awaitable
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from synapse.server import HomeServer
|
from synapse.server import HomeServer
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
CHECK_EVENT_ALLOWED_CALLBACK = Callable[
|
||||||
|
[EventBase, StateMap[EventBase]], Awaitable[Tuple[bool, Optional[dict]]]
|
||||||
|
]
|
||||||
|
ON_CREATE_ROOM_CALLBACK = Callable[[Requester, dict, bool], Awaitable]
|
||||||
|
CHECK_THREEPID_CAN_BE_INVITED_CALLBACK = Callable[
|
||||||
|
[str, str, StateMap[EventBase]], Awaitable[bool]
|
||||||
|
]
|
||||||
|
CHECK_VISIBILITY_CAN_BE_MODIFIED_CALLBACK = Callable[
|
||||||
|
[str, StateMap[EventBase], str], Awaitable[bool]
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def load_legacy_third_party_event_rules(hs: "HomeServer"):
|
||||||
|
"""Wrapper that loads a third party event rules module configured using the old
|
||||||
|
configuration, and registers the hooks they implement.
|
||||||
|
"""
|
||||||
|
if hs.config.third_party_event_rules is None:
|
||||||
|
return
|
||||||
|
|
||||||
|
module, config = hs.config.third_party_event_rules
|
||||||
|
|
||||||
|
api = hs.get_module_api()
|
||||||
|
third_party_rules = module(config=config, module_api=api)
|
||||||
|
|
||||||
|
# The known hooks. If a module implements a method which name appears in this set,
|
||||||
|
# we'll want to register it.
|
||||||
|
third_party_event_rules_methods = {
|
||||||
|
"check_event_allowed",
|
||||||
|
"on_create_room",
|
||||||
|
"check_threepid_can_be_invited",
|
||||||
|
"check_visibility_can_be_modified",
|
||||||
|
}
|
||||||
|
|
||||||
|
def async_wrapper(f: Optional[Callable]) -> Optional[Callable[..., Awaitable]]:
|
||||||
|
# f might be None if the callback isn't implemented by the module. In this
|
||||||
|
# case we don't want to register a callback at all so we return None.
|
||||||
|
if f is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# We return a separate wrapper for these methods because, in order to wrap them
|
||||||
|
# correctly, we need to await its result. Therefore it doesn't make a lot of
|
||||||
|
# sense to make it go through the run() wrapper.
|
||||||
|
if f.__name__ == "check_event_allowed":
|
||||||
|
|
||||||
|
# We need to wrap check_event_allowed because its old form would return either
|
||||||
|
# a boolean or a dict, but now we want to return the dict separately from the
|
||||||
|
# boolean.
|
||||||
|
async def wrap_check_event_allowed(
|
||||||
|
event: EventBase,
|
||||||
|
state_events: StateMap[EventBase],
|
||||||
|
) -> Tuple[bool, Optional[dict]]:
|
||||||
|
# We've already made sure f is not None above, but mypy doesn't do well
|
||||||
|
# across function boundaries so we need to tell it f is definitely not
|
||||||
|
# None.
|
||||||
|
assert f is not None
|
||||||
|
|
||||||
|
res = await f(event, state_events)
|
||||||
|
if isinstance(res, dict):
|
||||||
|
return True, res
|
||||||
|
else:
|
||||||
|
return res, None
|
||||||
|
|
||||||
|
return wrap_check_event_allowed
|
||||||
|
|
||||||
|
if f.__name__ == "on_create_room":
|
||||||
|
|
||||||
|
# We need to wrap on_create_room because its old form would return a boolean
|
||||||
|
# if the room creation is denied, but now we just want it to raise an
|
||||||
|
# exception.
|
||||||
|
async def wrap_on_create_room(
|
||||||
|
requester: Requester, config: dict, is_requester_admin: bool
|
||||||
|
) -> None:
|
||||||
|
# We've already made sure f is not None above, but mypy doesn't do well
|
||||||
|
# across function boundaries so we need to tell it f is definitely not
|
||||||
|
# None.
|
||||||
|
assert f is not None
|
||||||
|
|
||||||
|
res = await f(requester, config, is_requester_admin)
|
||||||
|
if res is False:
|
||||||
|
raise SynapseError(
|
||||||
|
403,
|
||||||
|
"Room creation forbidden with these parameters",
|
||||||
|
)
|
||||||
|
|
||||||
|
return wrap_on_create_room
|
||||||
|
|
||||||
|
def run(*args, **kwargs):
|
||||||
|
# mypy doesn't do well across function boundaries so we need to tell it
|
||||||
|
# f is definitely not None.
|
||||||
|
assert f is not None
|
||||||
|
|
||||||
|
return maybe_awaitable(f(*args, **kwargs))
|
||||||
|
|
||||||
|
return run
|
||||||
|
|
||||||
|
# Register the hooks through the module API.
|
||||||
|
hooks = {
|
||||||
|
hook: async_wrapper(getattr(third_party_rules, hook, None))
|
||||||
|
for hook in third_party_event_rules_methods
|
||||||
|
}
|
||||||
|
|
||||||
|
api.register_third_party_rules_callbacks(**hooks)
|
||||||
|
|
||||||
|
|
||||||
class ThirdPartyEventRules:
|
class ThirdPartyEventRules:
|
||||||
"""Allows server admins to provide a Python module implementing an extra
|
"""Allows server admins to provide a Python module implementing an extra
|
||||||
@ -35,36 +143,65 @@ class ThirdPartyEventRules:
|
|||||||
|
|
||||||
self.store = hs.get_datastore()
|
self.store = hs.get_datastore()
|
||||||
|
|
||||||
module = None
|
self._check_event_allowed_callbacks: List[CHECK_EVENT_ALLOWED_CALLBACK] = []
|
||||||
config = None
|
self._on_create_room_callbacks: List[ON_CREATE_ROOM_CALLBACK] = []
|
||||||
if hs.config.third_party_event_rules:
|
self._check_threepid_can_be_invited_callbacks: List[
|
||||||
module, config = hs.config.third_party_event_rules
|
CHECK_THREEPID_CAN_BE_INVITED_CALLBACK
|
||||||
|
] = []
|
||||||
|
self._check_visibility_can_be_modified_callbacks: List[
|
||||||
|
CHECK_VISIBILITY_CAN_BE_MODIFIED_CALLBACK
|
||||||
|
] = []
|
||||||
|
|
||||||
if module is not None:
|
def register_third_party_rules_callbacks(
|
||||||
self.third_party_rules = module(
|
self,
|
||||||
config=config,
|
check_event_allowed: Optional[CHECK_EVENT_ALLOWED_CALLBACK] = None,
|
||||||
module_api=hs.get_module_api(),
|
on_create_room: Optional[ON_CREATE_ROOM_CALLBACK] = None,
|
||||||
|
check_threepid_can_be_invited: Optional[
|
||||||
|
CHECK_THREEPID_CAN_BE_INVITED_CALLBACK
|
||||||
|
] = None,
|
||||||
|
check_visibility_can_be_modified: Optional[
|
||||||
|
CHECK_VISIBILITY_CAN_BE_MODIFIED_CALLBACK
|
||||||
|
] = None,
|
||||||
|
):
|
||||||
|
"""Register callbacks from modules for each hook."""
|
||||||
|
if check_event_allowed is not None:
|
||||||
|
self._check_event_allowed_callbacks.append(check_event_allowed)
|
||||||
|
|
||||||
|
if on_create_room is not None:
|
||||||
|
self._on_create_room_callbacks.append(on_create_room)
|
||||||
|
|
||||||
|
if check_threepid_can_be_invited is not None:
|
||||||
|
self._check_threepid_can_be_invited_callbacks.append(
|
||||||
|
check_threepid_can_be_invited,
|
||||||
|
)
|
||||||
|
|
||||||
|
if check_visibility_can_be_modified is not None:
|
||||||
|
self._check_visibility_can_be_modified_callbacks.append(
|
||||||
|
check_visibility_can_be_modified,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def check_event_allowed(
|
async def check_event_allowed(
|
||||||
self, event: EventBase, context: EventContext
|
self, event: EventBase, context: EventContext
|
||||||
) -> Union[bool, dict]:
|
) -> Tuple[bool, Optional[dict]]:
|
||||||
"""Check if a provided event should be allowed in the given context.
|
"""Check if a provided event should be allowed in the given context.
|
||||||
|
|
||||||
The module can return:
|
The module can return:
|
||||||
* True: the event is allowed.
|
* True: the event is allowed.
|
||||||
* False: the event is not allowed, and should be rejected with M_FORBIDDEN.
|
* False: the event is not allowed, and should be rejected with M_FORBIDDEN.
|
||||||
* a dict: replacement event data.
|
|
||||||
|
If the event is allowed, the module can also return a dictionary to use as a
|
||||||
|
replacement for the event.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
event: The event to be checked.
|
event: The event to be checked.
|
||||||
context: The context of the event.
|
context: The context of the event.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The result from the ThirdPartyRules module, as above
|
The result from the ThirdPartyRules module, as above.
|
||||||
"""
|
"""
|
||||||
if self.third_party_rules is None:
|
# Bail out early without hitting the store if we don't have any callbacks to run.
|
||||||
return True
|
if len(self._check_event_allowed_callbacks) == 0:
|
||||||
|
return True, None
|
||||||
|
|
||||||
prev_state_ids = await context.get_prev_state_ids()
|
prev_state_ids = await context.get_prev_state_ids()
|
||||||
|
|
||||||
@ -77,30 +214,47 @@ class ThirdPartyEventRules:
|
|||||||
# the hashes and signatures.
|
# the hashes and signatures.
|
||||||
event.freeze()
|
event.freeze()
|
||||||
|
|
||||||
return await self.third_party_rules.check_event_allowed(event, state_events)
|
for callback in self._check_event_allowed_callbacks:
|
||||||
|
try:
|
||||||
|
res, replacement_data = await callback(event, state_events)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to run module API callback %s: %s", callback, e)
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Return if the event shouldn't be allowed or if the module came up with a
|
||||||
|
# replacement dict for the event.
|
||||||
|
if res is False:
|
||||||
|
return res, None
|
||||||
|
elif isinstance(replacement_data, dict):
|
||||||
|
return True, replacement_data
|
||||||
|
|
||||||
|
return True, None
|
||||||
|
|
||||||
async def on_create_room(
|
async def on_create_room(
|
||||||
self, requester: Requester, config: dict, is_requester_admin: bool
|
self, requester: Requester, config: dict, is_requester_admin: bool
|
||||||
) -> bool:
|
) -> None:
|
||||||
"""Intercept requests to create room to allow, deny or update the
|
"""Intercept requests to create room to maybe deny it (via an exception) or
|
||||||
request config.
|
update the request config.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
requester
|
requester
|
||||||
config: The creation config from the client.
|
config: The creation config from the client.
|
||||||
is_requester_admin: If the requester is an admin
|
is_requester_admin: If the requester is an admin
|
||||||
|
|
||||||
Returns:
|
|
||||||
Whether room creation is allowed or denied.
|
|
||||||
"""
|
"""
|
||||||
|
for callback in self._on_create_room_callbacks:
|
||||||
if self.third_party_rules is None:
|
try:
|
||||||
return True
|
await callback(requester, config, is_requester_admin)
|
||||||
|
except Exception as e:
|
||||||
return await self.third_party_rules.on_create_room(
|
# Don't silence the errors raised by this callback since we expect it to
|
||||||
requester, config, is_requester_admin
|
# raise an exception to deny the creation of the room; instead make sure
|
||||||
|
# it's a SynapseError we can send to clients.
|
||||||
|
if not isinstance(e, SynapseError):
|
||||||
|
e = SynapseError(
|
||||||
|
403, "Room creation forbidden with these parameters"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
raise e
|
||||||
|
|
||||||
async def check_threepid_can_be_invited(
|
async def check_threepid_can_be_invited(
|
||||||
self, medium: str, address: str, room_id: str
|
self, medium: str, address: str, room_id: str
|
||||||
) -> bool:
|
) -> bool:
|
||||||
@ -114,15 +268,20 @@ class ThirdPartyEventRules:
|
|||||||
Returns:
|
Returns:
|
||||||
True if the 3PID can be invited, False if not.
|
True if the 3PID can be invited, False if not.
|
||||||
"""
|
"""
|
||||||
|
# Bail out early without hitting the store if we don't have any callbacks to run.
|
||||||
if self.third_party_rules is None:
|
if len(self._check_threepid_can_be_invited_callbacks) == 0:
|
||||||
return True
|
return True
|
||||||
|
|
||||||
state_events = await self._get_state_map_for_room(room_id)
|
state_events = await self._get_state_map_for_room(room_id)
|
||||||
|
|
||||||
return await self.third_party_rules.check_threepid_can_be_invited(
|
for callback in self._check_threepid_can_be_invited_callbacks:
|
||||||
medium, address, state_events
|
try:
|
||||||
)
|
if await callback(medium, address, state_events) is False:
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to run module API callback %s: %s", callback, e)
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
async def check_visibility_can_be_modified(
|
async def check_visibility_can_be_modified(
|
||||||
self, room_id: str, new_visibility: str
|
self, room_id: str, new_visibility: str
|
||||||
@ -137,18 +296,20 @@ class ThirdPartyEventRules:
|
|||||||
Returns:
|
Returns:
|
||||||
True if the room's visibility can be modified, False if not.
|
True if the room's visibility can be modified, False if not.
|
||||||
"""
|
"""
|
||||||
if self.third_party_rules is None:
|
# Bail out early without hitting the store if we don't have any callback
|
||||||
return True
|
if len(self._check_visibility_can_be_modified_callbacks) == 0:
|
||||||
|
|
||||||
check_func = getattr(
|
|
||||||
self.third_party_rules, "check_visibility_can_be_modified", None
|
|
||||||
)
|
|
||||||
if not check_func or not callable(check_func):
|
|
||||||
return True
|
return True
|
||||||
|
|
||||||
state_events = await self._get_state_map_for_room(room_id)
|
state_events = await self._get_state_map_for_room(room_id)
|
||||||
|
|
||||||
return await check_func(room_id, state_events, new_visibility)
|
for callback in self._check_visibility_can_be_modified_callbacks:
|
||||||
|
try:
|
||||||
|
if await callback(room_id, state_events, new_visibility) is False:
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to run module API callback %s: %s", callback, e)
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
async def _get_state_map_for_room(self, room_id: str) -> StateMap[EventBase]:
|
async def _get_state_map_for_room(self, room_id: str) -> StateMap[EventBase]:
|
||||||
"""Given a room ID, return the state events of that room.
|
"""Given a room ID, return the state events of that room.
|
||||||
|
@ -86,7 +86,7 @@ class FederationClient(FederationBase):
|
|||||||
def __init__(self, hs: "HomeServer"):
|
def __init__(self, hs: "HomeServer"):
|
||||||
super().__init__(hs)
|
super().__init__(hs)
|
||||||
|
|
||||||
self.pdu_destination_tried = {} # type: Dict[str, Dict[str, int]]
|
self.pdu_destination_tried: Dict[str, Dict[str, int]] = {}
|
||||||
self._clock.looping_call(self._clear_tried_cache, 60 * 1000)
|
self._clock.looping_call(self._clear_tried_cache, 60 * 1000)
|
||||||
self.state = hs.get_state_handler()
|
self.state = hs.get_state_handler()
|
||||||
self.transport_layer = hs.get_federation_transport_client()
|
self.transport_layer = hs.get_federation_transport_client()
|
||||||
@ -94,13 +94,13 @@ class FederationClient(FederationBase):
|
|||||||
self.hostname = hs.hostname
|
self.hostname = hs.hostname
|
||||||
self.signing_key = hs.signing_key
|
self.signing_key = hs.signing_key
|
||||||
|
|
||||||
self._get_pdu_cache = ExpiringCache(
|
self._get_pdu_cache: ExpiringCache[str, EventBase] = ExpiringCache(
|
||||||
cache_name="get_pdu_cache",
|
cache_name="get_pdu_cache",
|
||||||
clock=self._clock,
|
clock=self._clock,
|
||||||
max_len=1000,
|
max_len=1000,
|
||||||
expiry_ms=120 * 1000,
|
expiry_ms=120 * 1000,
|
||||||
reset_expiry_on_get=False,
|
reset_expiry_on_get=False,
|
||||||
) # type: ExpiringCache[str, EventBase]
|
)
|
||||||
|
|
||||||
def _clear_tried_cache(self):
|
def _clear_tried_cache(self):
|
||||||
"""Clear pdu_destination_tried cache"""
|
"""Clear pdu_destination_tried cache"""
|
||||||
@ -293,10 +293,10 @@ class FederationClient(FederationBase):
|
|||||||
transaction_data,
|
transaction_data,
|
||||||
)
|
)
|
||||||
|
|
||||||
pdu_list = [
|
pdu_list: List[EventBase] = [
|
||||||
event_from_pdu_json(p, room_version, outlier=outlier)
|
event_from_pdu_json(p, room_version, outlier=outlier)
|
||||||
for p in transaction_data["pdus"]
|
for p in transaction_data["pdus"]
|
||||||
] # type: List[EventBase]
|
]
|
||||||
|
|
||||||
if pdu_list and pdu_list[0]:
|
if pdu_list and pdu_list[0]:
|
||||||
pdu = pdu_list[0]
|
pdu = pdu_list[0]
|
||||||
|
@ -122,12 +122,12 @@ class FederationServer(FederationBase):
|
|||||||
|
|
||||||
# origins that we are currently processing a transaction from.
|
# origins that we are currently processing a transaction from.
|
||||||
# a dict from origin to txn id.
|
# a dict from origin to txn id.
|
||||||
self._active_transactions = {} # type: Dict[str, str]
|
self._active_transactions: Dict[str, str] = {}
|
||||||
|
|
||||||
# We cache results for transaction with the same ID
|
# We cache results for transaction with the same ID
|
||||||
self._transaction_resp_cache = ResponseCache(
|
self._transaction_resp_cache: ResponseCache[Tuple[str, str]] = ResponseCache(
|
||||||
hs.get_clock(), "fed_txn_handler", timeout_ms=30000
|
hs.get_clock(), "fed_txn_handler", timeout_ms=30000
|
||||||
) # type: ResponseCache[Tuple[str, str]]
|
)
|
||||||
|
|
||||||
self.transaction_actions = TransactionActions(self.store)
|
self.transaction_actions = TransactionActions(self.store)
|
||||||
|
|
||||||
@ -135,12 +135,12 @@ class FederationServer(FederationBase):
|
|||||||
|
|
||||||
# We cache responses to state queries, as they take a while and often
|
# We cache responses to state queries, as they take a while and often
|
||||||
# come in waves.
|
# come in waves.
|
||||||
self._state_resp_cache = ResponseCache(
|
self._state_resp_cache: ResponseCache[
|
||||||
hs.get_clock(), "state_resp", timeout_ms=30000
|
Tuple[str, Optional[str]]
|
||||||
) # type: ResponseCache[Tuple[str, Optional[str]]]
|
] = ResponseCache(hs.get_clock(), "state_resp", timeout_ms=30000)
|
||||||
self._state_ids_resp_cache = ResponseCache(
|
self._state_ids_resp_cache: ResponseCache[Tuple[str, str]] = ResponseCache(
|
||||||
hs.get_clock(), "state_ids_resp", timeout_ms=30000
|
hs.get_clock(), "state_ids_resp", timeout_ms=30000
|
||||||
) # type: ResponseCache[Tuple[str, str]]
|
)
|
||||||
|
|
||||||
self._federation_metrics_domains = (
|
self._federation_metrics_domains = (
|
||||||
hs.config.federation.federation_metrics_domains
|
hs.config.federation.federation_metrics_domains
|
||||||
@ -337,7 +337,7 @@ class FederationServer(FederationBase):
|
|||||||
|
|
||||||
origin_host, _ = parse_server_name(origin)
|
origin_host, _ = parse_server_name(origin)
|
||||||
|
|
||||||
pdus_by_room = {} # type: Dict[str, List[EventBase]]
|
pdus_by_room: Dict[str, List[EventBase]] = {}
|
||||||
|
|
||||||
newest_pdu_ts = 0
|
newest_pdu_ts = 0
|
||||||
|
|
||||||
@ -516,9 +516,9 @@ class FederationServer(FederationBase):
|
|||||||
self, room_id: str, event_id: Optional[str]
|
self, room_id: str, event_id: Optional[str]
|
||||||
) -> Dict[str, list]:
|
) -> Dict[str, list]:
|
||||||
if event_id:
|
if event_id:
|
||||||
pdus = await self.handler.get_state_for_pdu(
|
pdus: Iterable[EventBase] = await self.handler.get_state_for_pdu(
|
||||||
room_id, event_id
|
room_id, event_id
|
||||||
) # type: Iterable[EventBase]
|
)
|
||||||
else:
|
else:
|
||||||
pdus = (await self.state.get_current_state(room_id)).values()
|
pdus = (await self.state.get_current_state(room_id)).values()
|
||||||
|
|
||||||
@ -562,8 +562,7 @@ class FederationServer(FederationBase):
|
|||||||
raise IncompatibleRoomVersionError(room_version=room_version)
|
raise IncompatibleRoomVersionError(room_version=room_version)
|
||||||
|
|
||||||
pdu = await self.handler.on_make_join_request(origin, room_id, user_id)
|
pdu = await self.handler.on_make_join_request(origin, room_id, user_id)
|
||||||
time_now = self._clock.time_msec()
|
return {"event": pdu.get_templated_pdu_json(), "room_version": room_version}
|
||||||
return {"event": pdu.get_pdu_json(time_now), "room_version": room_version}
|
|
||||||
|
|
||||||
async def on_invite_request(
|
async def on_invite_request(
|
||||||
self, origin: str, content: JsonDict, room_version_id: str
|
self, origin: str, content: JsonDict, room_version_id: str
|
||||||
@ -611,8 +610,7 @@ class FederationServer(FederationBase):
|
|||||||
|
|
||||||
room_version = await self.store.get_room_version_id(room_id)
|
room_version = await self.store.get_room_version_id(room_id)
|
||||||
|
|
||||||
time_now = self._clock.time_msec()
|
return {"event": pdu.get_templated_pdu_json(), "room_version": room_version}
|
||||||
return {"event": pdu.get_pdu_json(time_now), "room_version": room_version}
|
|
||||||
|
|
||||||
async def on_send_leave_request(
|
async def on_send_leave_request(
|
||||||
self, origin: str, content: JsonDict, room_id: str
|
self, origin: str, content: JsonDict, room_id: str
|
||||||
@ -659,9 +657,8 @@ class FederationServer(FederationBase):
|
|||||||
)
|
)
|
||||||
|
|
||||||
pdu = await self.handler.on_make_knock_request(origin, room_id, user_id)
|
pdu = await self.handler.on_make_knock_request(origin, room_id, user_id)
|
||||||
time_now = self._clock.time_msec()
|
|
||||||
return {
|
return {
|
||||||
"event": pdu.get_pdu_json(time_now),
|
"event": pdu.get_templated_pdu_json(),
|
||||||
"room_version": room_version.identifier,
|
"room_version": room_version.identifier,
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -791,7 +788,7 @@ class FederationServer(FederationBase):
|
|||||||
log_kv({"message": "Claiming one time keys.", "user, device pairs": query})
|
log_kv({"message": "Claiming one time keys.", "user, device pairs": query})
|
||||||
results = await self.store.claim_e2e_one_time_keys(query)
|
results = await self.store.claim_e2e_one_time_keys(query)
|
||||||
|
|
||||||
json_result = {} # type: Dict[str, Dict[str, dict]]
|
json_result: Dict[str, Dict[str, dict]] = {}
|
||||||
for user_id, device_keys in results.items():
|
for user_id, device_keys in results.items():
|
||||||
for device_id, keys in device_keys.items():
|
for device_id, keys in device_keys.items():
|
||||||
for key_id, json_str in keys.items():
|
for key_id, json_str in keys.items():
|
||||||
@ -1119,17 +1116,13 @@ class FederationHandlerRegistry:
|
|||||||
self._get_query_client = ReplicationGetQueryRestServlet.make_client(hs)
|
self._get_query_client = ReplicationGetQueryRestServlet.make_client(hs)
|
||||||
self._send_edu = ReplicationFederationSendEduRestServlet.make_client(hs)
|
self._send_edu = ReplicationFederationSendEduRestServlet.make_client(hs)
|
||||||
|
|
||||||
self.edu_handlers = (
|
self.edu_handlers: Dict[str, Callable[[str, dict], Awaitable[None]]] = {}
|
||||||
{}
|
self.query_handlers: Dict[str, Callable[[dict], Awaitable[JsonDict]]] = {}
|
||||||
) # type: Dict[str, Callable[[str, dict], Awaitable[None]]]
|
|
||||||
self.query_handlers = (
|
|
||||||
{}
|
|
||||||
) # type: Dict[str, Callable[[dict], Awaitable[JsonDict]]]
|
|
||||||
|
|
||||||
# Map from type to instance names that we should route EDU handling to.
|
# Map from type to instance names that we should route EDU handling to.
|
||||||
# We randomly choose one instance from the list to route to for each new
|
# We randomly choose one instance from the list to route to for each new
|
||||||
# EDU received.
|
# EDU received.
|
||||||
self._edu_type_to_instance = {} # type: Dict[str, List[str]]
|
self._edu_type_to_instance: Dict[str, List[str]] = {}
|
||||||
|
|
||||||
def register_edu_handler(
|
def register_edu_handler(
|
||||||
self, edu_type: str, handler: Callable[[str, JsonDict], Awaitable[None]]
|
self, edu_type: str, handler: Callable[[str, JsonDict], Awaitable[None]]
|
||||||
|
@ -71,34 +71,32 @@ class FederationRemoteSendQueue(AbstractFederationSender):
|
|||||||
# We may have multiple federation sender instances, so we need to track
|
# We may have multiple federation sender instances, so we need to track
|
||||||
# their positions separately.
|
# their positions separately.
|
||||||
self._sender_instances = hs.config.worker.federation_shard_config.instances
|
self._sender_instances = hs.config.worker.federation_shard_config.instances
|
||||||
self._sender_positions = {} # type: Dict[str, int]
|
self._sender_positions: Dict[str, int] = {}
|
||||||
|
|
||||||
# Pending presence map user_id -> UserPresenceState
|
# Pending presence map user_id -> UserPresenceState
|
||||||
self.presence_map = {} # type: Dict[str, UserPresenceState]
|
self.presence_map: Dict[str, UserPresenceState] = {}
|
||||||
|
|
||||||
# Stores the destinations we need to explicitly send presence to about a
|
# Stores the destinations we need to explicitly send presence to about a
|
||||||
# given user.
|
# given user.
|
||||||
# Stream position -> (user_id, destinations)
|
# Stream position -> (user_id, destinations)
|
||||||
self.presence_destinations = (
|
self.presence_destinations: SortedDict[
|
||||||
SortedDict()
|
int, Tuple[str, Iterable[str]]
|
||||||
) # type: SortedDict[int, Tuple[str, Iterable[str]]]
|
] = SortedDict()
|
||||||
|
|
||||||
# (destination, key) -> EDU
|
# (destination, key) -> EDU
|
||||||
self.keyed_edu = {} # type: Dict[Tuple[str, tuple], Edu]
|
self.keyed_edu: Dict[Tuple[str, tuple], Edu] = {}
|
||||||
|
|
||||||
# stream position -> (destination, key)
|
# stream position -> (destination, key)
|
||||||
self.keyed_edu_changed = (
|
self.keyed_edu_changed: SortedDict[int, Tuple[str, tuple]] = SortedDict()
|
||||||
SortedDict()
|
|
||||||
) # type: SortedDict[int, Tuple[str, tuple]]
|
|
||||||
|
|
||||||
self.edus = SortedDict() # type: SortedDict[int, Edu]
|
self.edus: SortedDict[int, Edu] = SortedDict()
|
||||||
|
|
||||||
# stream ID for the next entry into keyed_edu_changed/edus.
|
# stream ID for the next entry into keyed_edu_changed/edus.
|
||||||
self.pos = 1
|
self.pos = 1
|
||||||
|
|
||||||
# map from stream ID to the time that stream entry was generated, so that we
|
# map from stream ID to the time that stream entry was generated, so that we
|
||||||
# can clear out entries after a while
|
# can clear out entries after a while
|
||||||
self.pos_time = SortedDict() # type: SortedDict[int, int]
|
self.pos_time: SortedDict[int, int] = SortedDict()
|
||||||
|
|
||||||
# EVERYTHING IS SAD. In particular, python only makes new scopes when
|
# EVERYTHING IS SAD. In particular, python only makes new scopes when
|
||||||
# we make a new function, so we need to make a new function so the inner
|
# we make a new function, so we need to make a new function so the inner
|
||||||
@ -291,7 +289,7 @@ class FederationRemoteSendQueue(AbstractFederationSender):
|
|||||||
|
|
||||||
# list of tuple(int, BaseFederationRow), where the first is the position
|
# list of tuple(int, BaseFederationRow), where the first is the position
|
||||||
# of the federation stream.
|
# of the federation stream.
|
||||||
rows = [] # type: List[Tuple[int, BaseFederationRow]]
|
rows: List[Tuple[int, BaseFederationRow]] = []
|
||||||
|
|
||||||
# Fetch presence to send to destinations
|
# Fetch presence to send to destinations
|
||||||
i = self.presence_destinations.bisect_right(from_token)
|
i = self.presence_destinations.bisect_right(from_token)
|
||||||
@ -445,11 +443,11 @@ class EduRow(BaseFederationRow, namedtuple("EduRow", ("edu",))): # Edu
|
|||||||
buff.edus.setdefault(self.edu.destination, []).append(self.edu)
|
buff.edus.setdefault(self.edu.destination, []).append(self.edu)
|
||||||
|
|
||||||
|
|
||||||
_rowtypes = (
|
_rowtypes: Tuple[Type[BaseFederationRow], ...] = (
|
||||||
PresenceDestinationsRow,
|
PresenceDestinationsRow,
|
||||||
KeyedEduRow,
|
KeyedEduRow,
|
||||||
EduRow,
|
EduRow,
|
||||||
) # type: Tuple[Type[BaseFederationRow], ...]
|
)
|
||||||
|
|
||||||
TypeToRow = {Row.TypeId: Row for Row in _rowtypes}
|
TypeToRow = {Row.TypeId: Row for Row in _rowtypes}
|
||||||
|
|
||||||
|
@ -14,9 +14,12 @@
|
|||||||
|
|
||||||
import abc
|
import abc
|
||||||
import logging
|
import logging
|
||||||
|
from collections import OrderedDict
|
||||||
from typing import TYPE_CHECKING, Dict, Hashable, Iterable, List, Optional, Set, Tuple
|
from typing import TYPE_CHECKING, Dict, Hashable, Iterable, List, Optional, Set, Tuple
|
||||||
|
|
||||||
|
import attr
|
||||||
from prometheus_client import Counter
|
from prometheus_client import Counter
|
||||||
|
from typing_extensions import Literal
|
||||||
|
|
||||||
from twisted.internet import defer
|
from twisted.internet import defer
|
||||||
|
|
||||||
@ -33,8 +36,12 @@ from synapse.metrics import (
|
|||||||
event_processing_loop_room_count,
|
event_processing_loop_room_count,
|
||||||
events_processed_counter,
|
events_processed_counter,
|
||||||
)
|
)
|
||||||
from synapse.metrics.background_process_metrics import run_as_background_process
|
from synapse.metrics.background_process_metrics import (
|
||||||
|
run_as_background_process,
|
||||||
|
wrap_as_background_process,
|
||||||
|
)
|
||||||
from synapse.types import JsonDict, ReadReceipt, RoomStreamToken
|
from synapse.types import JsonDict, ReadReceipt, RoomStreamToken
|
||||||
|
from synapse.util import Clock
|
||||||
from synapse.util.metrics import Measure
|
from synapse.util.metrics import Measure
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@ -137,6 +144,84 @@ class AbstractFederationSender(metaclass=abc.ABCMeta):
|
|||||||
raise NotImplementedError()
|
raise NotImplementedError()
|
||||||
|
|
||||||
|
|
||||||
|
@attr.s
|
||||||
|
class _PresenceQueue:
|
||||||
|
"""A queue of destinations that need to be woken up due to new presence
|
||||||
|
updates.
|
||||||
|
|
||||||
|
Staggers waking up of per destination queues to ensure that we don't attempt
|
||||||
|
to start TLS connections with many hosts all at once, leading to pinned CPU.
|
||||||
|
"""
|
||||||
|
|
||||||
|
# The maximum duration in seconds between queuing up a destination and it
|
||||||
|
# being woken up.
|
||||||
|
_MAX_TIME_IN_QUEUE = 30.0
|
||||||
|
|
||||||
|
# The maximum duration in seconds between waking up consecutive destination
|
||||||
|
# queues.
|
||||||
|
_MAX_DELAY = 0.1
|
||||||
|
|
||||||
|
sender: "FederationSender" = attr.ib()
|
||||||
|
clock: Clock = attr.ib()
|
||||||
|
queue: "OrderedDict[str, Literal[None]]" = attr.ib(factory=OrderedDict)
|
||||||
|
processing: bool = attr.ib(default=False)
|
||||||
|
|
||||||
|
def add_to_queue(self, destination: str) -> None:
|
||||||
|
"""Add a destination to the queue to be woken up."""
|
||||||
|
|
||||||
|
self.queue[destination] = None
|
||||||
|
|
||||||
|
if not self.processing:
|
||||||
|
self._handle()
|
||||||
|
|
||||||
|
@wrap_as_background_process("_PresenceQueue.handle")
|
||||||
|
async def _handle(self) -> None:
|
||||||
|
"""Background process to drain the queue."""
|
||||||
|
|
||||||
|
if not self.queue:
|
||||||
|
return
|
||||||
|
|
||||||
|
assert not self.processing
|
||||||
|
self.processing = True
|
||||||
|
|
||||||
|
try:
|
||||||
|
# We start with a delay that should drain the queue quickly enough that
|
||||||
|
# we process all destinations in the queue in _MAX_TIME_IN_QUEUE
|
||||||
|
# seconds.
|
||||||
|
#
|
||||||
|
# We also add an upper bound to the delay, to gracefully handle the
|
||||||
|
# case where the queue only has a few entries in it.
|
||||||
|
current_sleep_seconds = min(
|
||||||
|
self._MAX_DELAY, self._MAX_TIME_IN_QUEUE / len(self.queue)
|
||||||
|
)
|
||||||
|
|
||||||
|
while self.queue:
|
||||||
|
destination, _ = self.queue.popitem(last=False)
|
||||||
|
|
||||||
|
queue = self.sender._get_per_destination_queue(destination)
|
||||||
|
|
||||||
|
if not queue._new_data_to_send:
|
||||||
|
# The per destination queue has already been woken up.
|
||||||
|
continue
|
||||||
|
|
||||||
|
queue.attempt_new_transaction()
|
||||||
|
|
||||||
|
await self.clock.sleep(current_sleep_seconds)
|
||||||
|
|
||||||
|
if not self.queue:
|
||||||
|
break
|
||||||
|
|
||||||
|
# More destinations may have been added to the queue, so we may
|
||||||
|
# need to reduce the delay to ensure everything gets processed
|
||||||
|
# within _MAX_TIME_IN_QUEUE seconds.
|
||||||
|
current_sleep_seconds = min(
|
||||||
|
current_sleep_seconds, self._MAX_TIME_IN_QUEUE / len(self.queue)
|
||||||
|
)
|
||||||
|
|
||||||
|
finally:
|
||||||
|
self.processing = False
|
||||||
|
|
||||||
|
|
||||||
class FederationSender(AbstractFederationSender):
|
class FederationSender(AbstractFederationSender):
|
||||||
def __init__(self, hs: "HomeServer"):
|
def __init__(self, hs: "HomeServer"):
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
@ -148,14 +233,14 @@ class FederationSender(AbstractFederationSender):
|
|||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
self.is_mine_id = hs.is_mine_id
|
self.is_mine_id = hs.is_mine_id
|
||||||
|
|
||||||
self._presence_router = None # type: Optional[PresenceRouter]
|
self._presence_router: Optional["PresenceRouter"] = None
|
||||||
self._transaction_manager = TransactionManager(hs)
|
self._transaction_manager = TransactionManager(hs)
|
||||||
|
|
||||||
self._instance_name = hs.get_instance_name()
|
self._instance_name = hs.get_instance_name()
|
||||||
self._federation_shard_config = hs.config.worker.federation_shard_config
|
self._federation_shard_config = hs.config.worker.federation_shard_config
|
||||||
|
|
||||||
# map from destination to PerDestinationQueue
|
# map from destination to PerDestinationQueue
|
||||||
self._per_destination_queues = {} # type: Dict[str, PerDestinationQueue]
|
self._per_destination_queues: Dict[str, PerDestinationQueue] = {}
|
||||||
|
|
||||||
LaterGauge(
|
LaterGauge(
|
||||||
"synapse_federation_transaction_queue_pending_destinations",
|
"synapse_federation_transaction_queue_pending_destinations",
|
||||||
@ -192,9 +277,7 @@ class FederationSender(AbstractFederationSender):
|
|||||||
# awaiting a call to flush_read_receipts_for_room. The presence of an entry
|
# awaiting a call to flush_read_receipts_for_room. The presence of an entry
|
||||||
# here for a given room means that we are rate-limiting RR flushes to that room,
|
# here for a given room means that we are rate-limiting RR flushes to that room,
|
||||||
# and that there is a pending call to _flush_rrs_for_room in the system.
|
# and that there is a pending call to _flush_rrs_for_room in the system.
|
||||||
self._queues_awaiting_rr_flush_by_room = (
|
self._queues_awaiting_rr_flush_by_room: Dict[str, Set[PerDestinationQueue]] = {}
|
||||||
{}
|
|
||||||
) # type: Dict[str, Set[PerDestinationQueue]]
|
|
||||||
|
|
||||||
self._rr_txn_interval_per_room_ms = (
|
self._rr_txn_interval_per_room_ms = (
|
||||||
1000.0 / hs.config.federation_rr_transactions_per_room_per_second
|
1000.0 / hs.config.federation_rr_transactions_per_room_per_second
|
||||||
@ -210,6 +293,8 @@ class FederationSender(AbstractFederationSender):
|
|||||||
|
|
||||||
self._external_cache = hs.get_external_cache()
|
self._external_cache = hs.get_external_cache()
|
||||||
|
|
||||||
|
self._presence_queue = _PresenceQueue(self, self.clock)
|
||||||
|
|
||||||
def _get_per_destination_queue(self, destination: str) -> PerDestinationQueue:
|
def _get_per_destination_queue(self, destination: str) -> PerDestinationQueue:
|
||||||
"""Get or create a PerDestinationQueue for the given destination
|
"""Get or create a PerDestinationQueue for the given destination
|
||||||
|
|
||||||
@ -265,7 +350,7 @@ class FederationSender(AbstractFederationSender):
|
|||||||
if not event.internal_metadata.should_proactively_send():
|
if not event.internal_metadata.should_proactively_send():
|
||||||
return
|
return
|
||||||
|
|
||||||
destinations = None # type: Optional[Set[str]]
|
destinations: Optional[Set[str]] = None
|
||||||
if not event.prev_event_ids():
|
if not event.prev_event_ids():
|
||||||
# If there are no prev event IDs then the state is empty
|
# If there are no prev event IDs then the state is empty
|
||||||
# and so no remote servers in the room
|
# and so no remote servers in the room
|
||||||
@ -331,7 +416,7 @@ class FederationSender(AbstractFederationSender):
|
|||||||
for event in events:
|
for event in events:
|
||||||
await handle_event(event)
|
await handle_event(event)
|
||||||
|
|
||||||
events_by_room = {} # type: Dict[str, List[EventBase]]
|
events_by_room: Dict[str, List[EventBase]] = {}
|
||||||
for event in events:
|
for event in events:
|
||||||
events_by_room.setdefault(event.room_id, []).append(event)
|
events_by_room.setdefault(event.room_id, []).append(event)
|
||||||
|
|
||||||
@ -519,7 +604,12 @@ class FederationSender(AbstractFederationSender):
|
|||||||
self._instance_name, destination
|
self._instance_name, destination
|
||||||
):
|
):
|
||||||
continue
|
continue
|
||||||
self._get_per_destination_queue(destination).send_presence(states)
|
|
||||||
|
self._get_per_destination_queue(destination).send_presence(
|
||||||
|
states, start_loop=False
|
||||||
|
)
|
||||||
|
|
||||||
|
self._presence_queue.add_to_queue(destination)
|
||||||
|
|
||||||
def build_and_send_edu(
|
def build_and_send_edu(
|
||||||
self,
|
self,
|
||||||
@ -628,7 +718,7 @@ class FederationSender(AbstractFederationSender):
|
|||||||
In order to reduce load spikes, adds a delay between each destination.
|
In order to reduce load spikes, adds a delay between each destination.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
last_processed = None # type: Optional[str]
|
last_processed: Optional[str] = None
|
||||||
|
|
||||||
while True:
|
while True:
|
||||||
destinations_to_wake = (
|
destinations_to_wake = (
|
||||||
|
@ -105,34 +105,34 @@ class PerDestinationQueue:
|
|||||||
# catch-up at startup.
|
# catch-up at startup.
|
||||||
# New events will only be sent once this is finished, at which point
|
# New events will only be sent once this is finished, at which point
|
||||||
# _catching_up is flipped to False.
|
# _catching_up is flipped to False.
|
||||||
self._catching_up = True # type: bool
|
self._catching_up: bool = True
|
||||||
|
|
||||||
# The stream_ordering of the most recent PDU that was discarded due to
|
# The stream_ordering of the most recent PDU that was discarded due to
|
||||||
# being in catch-up mode.
|
# being in catch-up mode.
|
||||||
self._catchup_last_skipped = 0 # type: int
|
self._catchup_last_skipped: int = 0
|
||||||
|
|
||||||
# Cache of the last successfully-transmitted stream ordering for this
|
# Cache of the last successfully-transmitted stream ordering for this
|
||||||
# destination (we are the only updater so this is safe)
|
# destination (we are the only updater so this is safe)
|
||||||
self._last_successful_stream_ordering = None # type: Optional[int]
|
self._last_successful_stream_ordering: Optional[int] = None
|
||||||
|
|
||||||
# a queue of pending PDUs
|
# a queue of pending PDUs
|
||||||
self._pending_pdus = [] # type: List[EventBase]
|
self._pending_pdus: List[EventBase] = []
|
||||||
|
|
||||||
# XXX this is never actually used: see
|
# XXX this is never actually used: see
|
||||||
# https://github.com/matrix-org/synapse/issues/7549
|
# https://github.com/matrix-org/synapse/issues/7549
|
||||||
self._pending_edus = [] # type: List[Edu]
|
self._pending_edus: List[Edu] = []
|
||||||
|
|
||||||
# Pending EDUs by their "key". Keyed EDUs are EDUs that get clobbered
|
# Pending EDUs by their "key". Keyed EDUs are EDUs that get clobbered
|
||||||
# based on their key (e.g. typing events by room_id)
|
# based on their key (e.g. typing events by room_id)
|
||||||
# Map of (edu_type, key) -> Edu
|
# Map of (edu_type, key) -> Edu
|
||||||
self._pending_edus_keyed = {} # type: Dict[Tuple[str, Hashable], Edu]
|
self._pending_edus_keyed: Dict[Tuple[str, Hashable], Edu] = {}
|
||||||
|
|
||||||
# Map of user_id -> UserPresenceState of pending presence to be sent to this
|
# Map of user_id -> UserPresenceState of pending presence to be sent to this
|
||||||
# destination
|
# destination
|
||||||
self._pending_presence = {} # type: Dict[str, UserPresenceState]
|
self._pending_presence: Dict[str, UserPresenceState] = {}
|
||||||
|
|
||||||
# room_id -> receipt_type -> user_id -> receipt_dict
|
# room_id -> receipt_type -> user_id -> receipt_dict
|
||||||
self._pending_rrs = {} # type: Dict[str, Dict[str, Dict[str, dict]]]
|
self._pending_rrs: Dict[str, Dict[str, Dict[str, dict]]] = {}
|
||||||
self._rrs_pending_flush = False
|
self._rrs_pending_flush = False
|
||||||
|
|
||||||
# stream_id of last successfully sent to-device message.
|
# stream_id of last successfully sent to-device message.
|
||||||
@ -171,13 +171,23 @@ class PerDestinationQueue:
|
|||||||
|
|
||||||
self.attempt_new_transaction()
|
self.attempt_new_transaction()
|
||||||
|
|
||||||
def send_presence(self, states: Iterable[UserPresenceState]) -> None:
|
def send_presence(
|
||||||
"""Add presence updates to the queue. Start the transmission loop if necessary.
|
self, states: Iterable[UserPresenceState], start_loop: bool = True
|
||||||
|
) -> None:
|
||||||
|
"""Add presence updates to the queue.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
states: Presence updates to send
|
||||||
|
start_loop: Whether to start the transmission loop if not already
|
||||||
|
running.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
states: presence to send
|
states: presence to send
|
||||||
"""
|
"""
|
||||||
self._pending_presence.update({state.user_id: state for state in states})
|
self._pending_presence.update({state.user_id: state for state in states})
|
||||||
|
self._new_data_to_send = True
|
||||||
|
|
||||||
|
if start_loop:
|
||||||
self.attempt_new_transaction()
|
self.attempt_new_transaction()
|
||||||
|
|
||||||
def queue_read_receipt(self, receipt: ReadReceipt) -> None:
|
def queue_read_receipt(self, receipt: ReadReceipt) -> None:
|
||||||
@ -243,7 +253,7 @@ class PerDestinationQueue:
|
|||||||
)
|
)
|
||||||
|
|
||||||
async def _transaction_transmission_loop(self) -> None:
|
async def _transaction_transmission_loop(self) -> None:
|
||||||
pending_pdus = [] # type: List[EventBase]
|
pending_pdus: List[EventBase] = []
|
||||||
try:
|
try:
|
||||||
self.transmission_loop_running = True
|
self.transmission_loop_running = True
|
||||||
|
|
||||||
|
@ -395,9 +395,9 @@ class TransportLayerClient:
|
|||||||
# this uses MSC2197 (Search Filtering over Federation)
|
# this uses MSC2197 (Search Filtering over Federation)
|
||||||
path = _create_v1_path("/publicRooms")
|
path = _create_v1_path("/publicRooms")
|
||||||
|
|
||||||
data = {
|
data: Dict[str, Any] = {
|
||||||
"include_all_networks": "true" if include_all_networks else "false"
|
"include_all_networks": "true" if include_all_networks else "false"
|
||||||
} # type: Dict[str, Any]
|
}
|
||||||
if third_party_instance_id:
|
if third_party_instance_id:
|
||||||
data["third_party_instance_id"] = third_party_instance_id
|
data["third_party_instance_id"] = third_party_instance_id
|
||||||
if limit:
|
if limit:
|
||||||
@ -423,9 +423,9 @@ class TransportLayerClient:
|
|||||||
else:
|
else:
|
||||||
path = _create_v1_path("/publicRooms")
|
path = _create_v1_path("/publicRooms")
|
||||||
|
|
||||||
args = {
|
args: Dict[str, Any] = {
|
||||||
"include_all_networks": "true" if include_all_networks else "false"
|
"include_all_networks": "true" if include_all_networks else "false"
|
||||||
} # type: Dict[str, Any]
|
}
|
||||||
if third_party_instance_id:
|
if third_party_instance_id:
|
||||||
args["third_party_instance_id"] = (third_party_instance_id,)
|
args["third_party_instance_id"] = (third_party_instance_id,)
|
||||||
if limit:
|
if limit:
|
||||||
|
@ -1013,7 +1013,7 @@ class PublicRoomList(BaseFederationServlet):
|
|||||||
if not self.allow_access:
|
if not self.allow_access:
|
||||||
raise FederationDeniedError(origin)
|
raise FederationDeniedError(origin)
|
||||||
|
|
||||||
limit = int(content.get("limit", 100)) # type: Optional[int]
|
limit: Optional[int] = int(content.get("limit", 100))
|
||||||
since_token = content.get("since", None)
|
since_token = content.get("since", None)
|
||||||
search_filter = content.get("filter", None)
|
search_filter = content.get("filter", None)
|
||||||
|
|
||||||
@ -1095,7 +1095,9 @@ class FederationGroupsProfileServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1110,7 +1112,9 @@ class FederationGroupsProfileServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1131,7 +1135,9 @@ class FederationGroupsSummaryServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1152,7 +1158,9 @@ class FederationGroupsRoomsServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1174,7 +1182,9 @@ class FederationGroupsAddRoomsServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
room_id: str,
|
room_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1192,7 +1202,9 @@ class FederationGroupsAddRoomsServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
room_id: str,
|
room_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1220,7 +1232,9 @@ class FederationGroupsAddRoomsConfigServlet(BaseGroupsServerServlet):
|
|||||||
room_id: str,
|
room_id: str,
|
||||||
config_key: str,
|
config_key: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1243,7 +1257,9 @@ class FederationGroupsUsersServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1264,7 +1280,9 @@ class FederationGroupsInvitedUsersServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1288,7 +1306,9 @@ class FederationGroupsInviteServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1354,7 +1374,9 @@ class FederationGroupsRemoveUserServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1487,7 +1509,9 @@ class FederationGroupsSummaryRoomsServlet(BaseGroupsServerServlet):
|
|||||||
category_id: str,
|
category_id: str,
|
||||||
room_id: str,
|
room_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1523,7 +1547,9 @@ class FederationGroupsSummaryRoomsServlet(BaseGroupsServerServlet):
|
|||||||
category_id: str,
|
category_id: str,
|
||||||
room_id: str,
|
room_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1549,7 +1575,9 @@ class FederationGroupsCategoriesServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1571,7 +1599,9 @@ class FederationGroupsCategoryServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
category_id: str,
|
category_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1589,7 +1619,9 @@ class FederationGroupsCategoryServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
category_id: str,
|
category_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1618,7 +1650,9 @@ class FederationGroupsCategoryServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
category_id: str,
|
category_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1644,7 +1678,9 @@ class FederationGroupsRolesServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1666,7 +1702,9 @@ class FederationGroupsRoleServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
role_id: str,
|
role_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1682,7 +1720,9 @@ class FederationGroupsRoleServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
role_id: str,
|
role_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1713,7 +1753,9 @@ class FederationGroupsRoleServlet(BaseGroupsServerServlet):
|
|||||||
group_id: str,
|
group_id: str,
|
||||||
role_id: str,
|
role_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1750,7 +1792,9 @@ class FederationGroupsSummaryUsersServlet(BaseGroupsServerServlet):
|
|||||||
role_id: str,
|
role_id: str,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1784,7 +1828,9 @@ class FederationGroupsSummaryUsersServlet(BaseGroupsServerServlet):
|
|||||||
role_id: str,
|
role_id: str,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1825,7 +1871,9 @@ class FederationGroupsSettingJoinPolicyServlet(BaseGroupsServerServlet):
|
|||||||
query: Dict[bytes, List[bytes]],
|
query: Dict[bytes, List[bytes]],
|
||||||
group_id: str,
|
group_id: str,
|
||||||
) -> Tuple[int, JsonDict]:
|
) -> Tuple[int, JsonDict]:
|
||||||
requester_user_id = parse_string_from_args(query, "requester_user_id")
|
requester_user_id = parse_string_from_args(
|
||||||
|
query, "requester_user_id", required=True
|
||||||
|
)
|
||||||
if get_domain_from_id(requester_user_id) != origin:
|
if get_domain_from_id(requester_user_id) != origin:
|
||||||
raise SynapseError(403, "requester_user_id doesn't match origin")
|
raise SynapseError(403, "requester_user_id doesn't match origin")
|
||||||
|
|
||||||
@ -1943,7 +1991,7 @@ class RoomComplexityServlet(BaseFederationServlet):
|
|||||||
return 200, complexity
|
return 200, complexity
|
||||||
|
|
||||||
|
|
||||||
FEDERATION_SERVLET_CLASSES = (
|
FEDERATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||||
FederationSendServlet,
|
FederationSendServlet,
|
||||||
FederationEventServlet,
|
FederationEventServlet,
|
||||||
FederationStateV1Servlet,
|
FederationStateV1Servlet,
|
||||||
@ -1971,15 +2019,13 @@ FEDERATION_SERVLET_CLASSES = (
|
|||||||
FederationSpaceSummaryServlet,
|
FederationSpaceSummaryServlet,
|
||||||
FederationV1SendKnockServlet,
|
FederationV1SendKnockServlet,
|
||||||
FederationMakeKnockServlet,
|
FederationMakeKnockServlet,
|
||||||
) # type: Tuple[Type[BaseFederationServlet], ...]
|
)
|
||||||
|
|
||||||
OPENID_SERVLET_CLASSES = (
|
OPENID_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (OpenIdUserInfo,)
|
||||||
OpenIdUserInfo,
|
|
||||||
) # type: Tuple[Type[BaseFederationServlet], ...]
|
|
||||||
|
|
||||||
ROOM_LIST_CLASSES = (PublicRoomList,) # type: Tuple[Type[PublicRoomList], ...]
|
ROOM_LIST_CLASSES: Tuple[Type[PublicRoomList], ...] = (PublicRoomList,)
|
||||||
|
|
||||||
GROUP_SERVER_SERVLET_CLASSES = (
|
GROUP_SERVER_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||||
FederationGroupsProfileServlet,
|
FederationGroupsProfileServlet,
|
||||||
FederationGroupsSummaryServlet,
|
FederationGroupsSummaryServlet,
|
||||||
FederationGroupsRoomsServlet,
|
FederationGroupsRoomsServlet,
|
||||||
@ -1998,19 +2044,19 @@ GROUP_SERVER_SERVLET_CLASSES = (
|
|||||||
FederationGroupsAddRoomsServlet,
|
FederationGroupsAddRoomsServlet,
|
||||||
FederationGroupsAddRoomsConfigServlet,
|
FederationGroupsAddRoomsConfigServlet,
|
||||||
FederationGroupsSettingJoinPolicyServlet,
|
FederationGroupsSettingJoinPolicyServlet,
|
||||||
) # type: Tuple[Type[BaseFederationServlet], ...]
|
)
|
||||||
|
|
||||||
|
|
||||||
GROUP_LOCAL_SERVLET_CLASSES = (
|
GROUP_LOCAL_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||||
FederationGroupsLocalInviteServlet,
|
FederationGroupsLocalInviteServlet,
|
||||||
FederationGroupsRemoveLocalUserServlet,
|
FederationGroupsRemoveLocalUserServlet,
|
||||||
FederationGroupsBulkPublicisedServlet,
|
FederationGroupsBulkPublicisedServlet,
|
||||||
) # type: Tuple[Type[BaseFederationServlet], ...]
|
)
|
||||||
|
|
||||||
|
|
||||||
GROUP_ATTESTATION_SERVLET_CLASSES = (
|
GROUP_ATTESTATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||||
FederationGroupsRenewAttestaionServlet,
|
FederationGroupsRenewAttestaionServlet,
|
||||||
) # type: Tuple[Type[BaseFederationServlet], ...]
|
)
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_SERVLET_GROUPS = (
|
DEFAULT_SERVLET_GROUPS = (
|
||||||
|
@ -707,9 +707,9 @@ class GroupsServerHandler(GroupsServerWorkerHandler):
|
|||||||
See accept_invite, join_group.
|
See accept_invite, join_group.
|
||||||
"""
|
"""
|
||||||
if not self.hs.is_mine_id(user_id):
|
if not self.hs.is_mine_id(user_id):
|
||||||
local_attestation = self.attestations.create_attestation(
|
local_attestation: Optional[
|
||||||
group_id, user_id
|
JsonDict
|
||||||
) # type: Optional[JsonDict]
|
] = self.attestations.create_attestation(group_id, user_id)
|
||||||
|
|
||||||
remote_attestation = content["attestation"]
|
remote_attestation = content["attestation"]
|
||||||
|
|
||||||
@ -868,9 +868,9 @@ class GroupsServerHandler(GroupsServerWorkerHandler):
|
|||||||
remote_attestation, user_id=requester_user_id, group_id=group_id
|
remote_attestation, user_id=requester_user_id, group_id=group_id
|
||||||
)
|
)
|
||||||
|
|
||||||
local_attestation = self.attestations.create_attestation(
|
local_attestation: Optional[
|
||||||
group_id, requester_user_id
|
JsonDict
|
||||||
) # type: Optional[JsonDict]
|
] = self.attestations.create_attestation(group_id, requester_user_id)
|
||||||
else:
|
else:
|
||||||
local_attestation = None
|
local_attestation = None
|
||||||
remote_attestation = None
|
remote_attestation = None
|
||||||
|
@ -15,8 +15,6 @@
|
|||||||
import logging
|
import logging
|
||||||
from typing import TYPE_CHECKING, Optional
|
from typing import TYPE_CHECKING, Optional
|
||||||
|
|
||||||
import synapse.state
|
|
||||||
import synapse.storage
|
|
||||||
import synapse.types
|
import synapse.types
|
||||||
from synapse.api.constants import EventTypes, Membership
|
from synapse.api.constants import EventTypes, Membership
|
||||||
from synapse.api.ratelimiting import Ratelimiter
|
from synapse.api.ratelimiting import Ratelimiter
|
||||||
@ -38,10 +36,10 @@ class BaseHandler:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, hs: "HomeServer"):
|
def __init__(self, hs: "HomeServer"):
|
||||||
self.store = hs.get_datastore() # type: synapse.storage.DataStore
|
self.store = hs.get_datastore()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.notifier = hs.get_notifier()
|
self.notifier = hs.get_notifier()
|
||||||
self.state_handler = hs.get_state_handler() # type: synapse.state.StateHandler
|
self.state_handler = hs.get_state_handler()
|
||||||
self.distributor = hs.get_distributor()
|
self.distributor = hs.get_distributor()
|
||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
@ -55,12 +53,12 @@ class BaseHandler:
|
|||||||
# Check whether ratelimiting room admin message redaction is enabled
|
# Check whether ratelimiting room admin message redaction is enabled
|
||||||
# by the presence of rate limits in the config
|
# by the presence of rate limits in the config
|
||||||
if self.hs.config.rc_admin_redaction:
|
if self.hs.config.rc_admin_redaction:
|
||||||
self.admin_redaction_ratelimiter = Ratelimiter(
|
self.admin_redaction_ratelimiter: Optional[Ratelimiter] = Ratelimiter(
|
||||||
store=self.store,
|
store=self.store,
|
||||||
clock=self.clock,
|
clock=self.clock,
|
||||||
rate_hz=self.hs.config.rc_admin_redaction.per_second,
|
rate_hz=self.hs.config.rc_admin_redaction.per_second,
|
||||||
burst_count=self.hs.config.rc_admin_redaction.burst_count,
|
burst_count=self.hs.config.rc_admin_redaction.burst_count,
|
||||||
) # type: Optional[Ratelimiter]
|
)
|
||||||
else:
|
else:
|
||||||
self.admin_redaction_ratelimiter = None
|
self.admin_redaction_ratelimiter = None
|
||||||
|
|
||||||
|
@ -15,9 +15,11 @@
|
|||||||
import email.mime.multipart
|
import email.mime.multipart
|
||||||
import email.utils
|
import email.utils
|
||||||
import logging
|
import logging
|
||||||
from typing import TYPE_CHECKING, List, Optional, Tuple
|
from typing import TYPE_CHECKING, Awaitable, Callable, List, Optional, Tuple
|
||||||
|
|
||||||
from synapse.api.errors import StoreError, SynapseError
|
from twisted.web.http import Request
|
||||||
|
|
||||||
|
from synapse.api.errors import AuthError, StoreError, SynapseError
|
||||||
from synapse.metrics.background_process_metrics import wrap_as_background_process
|
from synapse.metrics.background_process_metrics import wrap_as_background_process
|
||||||
from synapse.types import UserID
|
from synapse.types import UserID
|
||||||
from synapse.util import stringutils
|
from synapse.util import stringutils
|
||||||
@ -27,6 +29,15 @@ if TYPE_CHECKING:
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Types for callbacks to be registered via the module api
|
||||||
|
IS_USER_EXPIRED_CALLBACK = Callable[[str], Awaitable[Optional[bool]]]
|
||||||
|
ON_USER_REGISTRATION_CALLBACK = Callable[[str], Awaitable]
|
||||||
|
# Temporary hooks to allow for a transition from `/_matrix/client` endpoints
|
||||||
|
# to `/_synapse/client/account_validity`. See `register_account_validity_callbacks`.
|
||||||
|
ON_LEGACY_SEND_MAIL_CALLBACK = Callable[[str], Awaitable]
|
||||||
|
ON_LEGACY_RENEW_CALLBACK = Callable[[str], Awaitable[Tuple[bool, bool, int]]]
|
||||||
|
ON_LEGACY_ADMIN_REQUEST = Callable[[Request], Awaitable]
|
||||||
|
|
||||||
|
|
||||||
class AccountValidityHandler:
|
class AccountValidityHandler:
|
||||||
def __init__(self, hs: "HomeServer"):
|
def __init__(self, hs: "HomeServer"):
|
||||||
@ -70,6 +81,99 @@ class AccountValidityHandler:
|
|||||||
if hs.config.run_background_tasks:
|
if hs.config.run_background_tasks:
|
||||||
self.clock.looping_call(self._send_renewal_emails, 30 * 60 * 1000)
|
self.clock.looping_call(self._send_renewal_emails, 30 * 60 * 1000)
|
||||||
|
|
||||||
|
self._is_user_expired_callbacks: List[IS_USER_EXPIRED_CALLBACK] = []
|
||||||
|
self._on_user_registration_callbacks: List[ON_USER_REGISTRATION_CALLBACK] = []
|
||||||
|
self._on_legacy_send_mail_callback: Optional[
|
||||||
|
ON_LEGACY_SEND_MAIL_CALLBACK
|
||||||
|
] = None
|
||||||
|
self._on_legacy_renew_callback: Optional[ON_LEGACY_RENEW_CALLBACK] = None
|
||||||
|
|
||||||
|
# The legacy admin requests callback isn't a protected attribute because we need
|
||||||
|
# to access it from the admin servlet, which is outside of this handler.
|
||||||
|
self.on_legacy_admin_request_callback: Optional[ON_LEGACY_ADMIN_REQUEST] = None
|
||||||
|
|
||||||
|
def register_account_validity_callbacks(
|
||||||
|
self,
|
||||||
|
is_user_expired: Optional[IS_USER_EXPIRED_CALLBACK] = None,
|
||||||
|
on_user_registration: Optional[ON_USER_REGISTRATION_CALLBACK] = None,
|
||||||
|
on_legacy_send_mail: Optional[ON_LEGACY_SEND_MAIL_CALLBACK] = None,
|
||||||
|
on_legacy_renew: Optional[ON_LEGACY_RENEW_CALLBACK] = None,
|
||||||
|
on_legacy_admin_request: Optional[ON_LEGACY_ADMIN_REQUEST] = None,
|
||||||
|
):
|
||||||
|
"""Register callbacks from module for each hook."""
|
||||||
|
if is_user_expired is not None:
|
||||||
|
self._is_user_expired_callbacks.append(is_user_expired)
|
||||||
|
|
||||||
|
if on_user_registration is not None:
|
||||||
|
self._on_user_registration_callbacks.append(on_user_registration)
|
||||||
|
|
||||||
|
# The builtin account validity feature exposes 3 endpoints (send_mail, renew, and
|
||||||
|
# an admin one). As part of moving the feature into a module, we need to change
|
||||||
|
# the path from /_matrix/client/unstable/account_validity/... to
|
||||||
|
# /_synapse/client/account_validity, because:
|
||||||
|
#
|
||||||
|
# * the feature isn't part of the Matrix spec thus shouldn't live under /_matrix
|
||||||
|
# * the way we register servlets means that modules can't register resources
|
||||||
|
# under /_matrix/client
|
||||||
|
#
|
||||||
|
# We need to allow for a transition period between the old and new endpoints
|
||||||
|
# in order to allow for clients to update (and for emails to be processed).
|
||||||
|
#
|
||||||
|
# Once the email-account-validity module is loaded, it will take control of account
|
||||||
|
# validity by moving the rows from our `account_validity` table into its own table.
|
||||||
|
#
|
||||||
|
# Therefore, we need to allow modules (in practice just the one implementing the
|
||||||
|
# email-based account validity) to temporarily hook into the legacy endpoints so we
|
||||||
|
# can route the traffic coming into the old endpoints into the module, which is
|
||||||
|
# why we have the following three temporary hooks.
|
||||||
|
if on_legacy_send_mail is not None:
|
||||||
|
if self._on_legacy_send_mail_callback is not None:
|
||||||
|
raise RuntimeError("Tried to register on_legacy_send_mail twice")
|
||||||
|
|
||||||
|
self._on_legacy_send_mail_callback = on_legacy_send_mail
|
||||||
|
|
||||||
|
if on_legacy_renew is not None:
|
||||||
|
if self._on_legacy_renew_callback is not None:
|
||||||
|
raise RuntimeError("Tried to register on_legacy_renew twice")
|
||||||
|
|
||||||
|
self._on_legacy_renew_callback = on_legacy_renew
|
||||||
|
|
||||||
|
if on_legacy_admin_request is not None:
|
||||||
|
if self.on_legacy_admin_request_callback is not None:
|
||||||
|
raise RuntimeError("Tried to register on_legacy_admin_request twice")
|
||||||
|
|
||||||
|
self.on_legacy_admin_request_callback = on_legacy_admin_request
|
||||||
|
|
||||||
|
async def is_user_expired(self, user_id: str) -> bool:
|
||||||
|
"""Checks if a user has expired against third-party modules.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
user_id: The user to check the expiry of.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Whether the user has expired.
|
||||||
|
"""
|
||||||
|
for callback in self._is_user_expired_callbacks:
|
||||||
|
expired = await callback(user_id)
|
||||||
|
if expired is not None:
|
||||||
|
return expired
|
||||||
|
|
||||||
|
if self._account_validity_enabled:
|
||||||
|
# If no module could determine whether the user has expired and the legacy
|
||||||
|
# configuration is enabled, fall back to it.
|
||||||
|
return await self.store.is_account_expired(user_id, self.clock.time_msec())
|
||||||
|
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def on_user_registration(self, user_id: str):
|
||||||
|
"""Tell third-party modules about a user's registration.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
user_id: The ID of the newly registered user.
|
||||||
|
"""
|
||||||
|
for callback in self._on_user_registration_callbacks:
|
||||||
|
await callback(user_id)
|
||||||
|
|
||||||
@wrap_as_background_process("send_renewals")
|
@wrap_as_background_process("send_renewals")
|
||||||
async def _send_renewal_emails(self) -> None:
|
async def _send_renewal_emails(self) -> None:
|
||||||
"""Gets the list of users whose account is expiring in the amount of time
|
"""Gets the list of users whose account is expiring in the amount of time
|
||||||
@ -95,6 +199,17 @@ class AccountValidityHandler:
|
|||||||
Raises:
|
Raises:
|
||||||
SynapseError if the user is not set to renew.
|
SynapseError if the user is not set to renew.
|
||||||
"""
|
"""
|
||||||
|
# If a module supports sending a renewal email from here, do that, otherwise do
|
||||||
|
# the legacy dance.
|
||||||
|
if self._on_legacy_send_mail_callback is not None:
|
||||||
|
await self._on_legacy_send_mail_callback(user_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
if not self._account_validity_renew_by_email_enabled:
|
||||||
|
raise AuthError(
|
||||||
|
403, "Account renewal via email is disabled on this server."
|
||||||
|
)
|
||||||
|
|
||||||
expiration_ts = await self.store.get_expiration_ts_for_user(user_id)
|
expiration_ts = await self.store.get_expiration_ts_for_user(user_id)
|
||||||
|
|
||||||
# If this user isn't set to be expired, raise an error.
|
# If this user isn't set to be expired, raise an error.
|
||||||
@ -209,6 +324,10 @@ class AccountValidityHandler:
|
|||||||
token is considered stale. A token is stale if the 'token_used_ts_ms' db column
|
token is considered stale. A token is stale if the 'token_used_ts_ms' db column
|
||||||
is non-null.
|
is non-null.
|
||||||
|
|
||||||
|
This method exists to support handling the legacy account validity /renew
|
||||||
|
endpoint. If a module implements the on_legacy_renew callback, then this process
|
||||||
|
is delegated to the module instead.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
renewal_token: Token sent with the renewal request.
|
renewal_token: Token sent with the renewal request.
|
||||||
Returns:
|
Returns:
|
||||||
@ -218,6 +337,11 @@ class AccountValidityHandler:
|
|||||||
* An int representing the user's expiry timestamp as milliseconds since the
|
* An int representing the user's expiry timestamp as milliseconds since the
|
||||||
epoch, or 0 if the token was invalid.
|
epoch, or 0 if the token was invalid.
|
||||||
"""
|
"""
|
||||||
|
# If a module supports triggering a renew from here, do that, otherwise do the
|
||||||
|
# legacy dance.
|
||||||
|
if self._on_legacy_renew_callback is not None:
|
||||||
|
return await self._on_legacy_renew_callback(renewal_token)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
(
|
(
|
||||||
user_id,
|
user_id,
|
||||||
|
@ -139,7 +139,7 @@ class AdminHandler(BaseHandler):
|
|||||||
to_key = RoomStreamToken(None, stream_ordering)
|
to_key = RoomStreamToken(None, stream_ordering)
|
||||||
|
|
||||||
# Events that we've processed in this room
|
# Events that we've processed in this room
|
||||||
written_events = set() # type: Set[str]
|
written_events: Set[str] = set()
|
||||||
|
|
||||||
# We need to track gaps in the events stream so that we can then
|
# We need to track gaps in the events stream so that we can then
|
||||||
# write out the state at those events. We do this by keeping track
|
# write out the state at those events. We do this by keeping track
|
||||||
@ -152,7 +152,7 @@ class AdminHandler(BaseHandler):
|
|||||||
# The reverse mapping to above, i.e. map from unseen event to events
|
# The reverse mapping to above, i.e. map from unseen event to events
|
||||||
# that have the unseen event in their prev_events, i.e. the unseen
|
# that have the unseen event in their prev_events, i.e. the unseen
|
||||||
# events "children".
|
# events "children".
|
||||||
unseen_to_child_events = {} # type: Dict[str, Set[str]]
|
unseen_to_child_events: Dict[str, Set[str]] = {}
|
||||||
|
|
||||||
# We fetch events in the room the user could see by fetching *all*
|
# We fetch events in the room the user could see by fetching *all*
|
||||||
# events that we have and then filtering, this isn't the most
|
# events that we have and then filtering, this isn't the most
|
||||||
|
@ -96,7 +96,7 @@ class ApplicationServicesHandler:
|
|||||||
self.current_max, limit
|
self.current_max, limit
|
||||||
)
|
)
|
||||||
|
|
||||||
events_by_room = {} # type: Dict[str, List[EventBase]]
|
events_by_room: Dict[str, List[EventBase]] = {}
|
||||||
for event in events:
|
for event in events:
|
||||||
events_by_room.setdefault(event.room_id, []).append(event)
|
events_by_room.setdefault(event.room_id, []).append(event)
|
||||||
|
|
||||||
@ -275,7 +275,7 @@ class ApplicationServicesHandler:
|
|||||||
async def _handle_presence(
|
async def _handle_presence(
|
||||||
self, service: ApplicationService, users: Collection[Union[str, UserID]]
|
self, service: ApplicationService, users: Collection[Union[str, UserID]]
|
||||||
) -> List[JsonDict]:
|
) -> List[JsonDict]:
|
||||||
events = [] # type: List[JsonDict]
|
events: List[JsonDict] = []
|
||||||
presence_source = self.event_sources.sources["presence"]
|
presence_source = self.event_sources.sources["presence"]
|
||||||
from_key = await self.store.get_type_stream_id_for_appservice(
|
from_key = await self.store.get_type_stream_id_for_appservice(
|
||||||
service, "presence"
|
service, "presence"
|
||||||
@ -375,7 +375,7 @@ class ApplicationServicesHandler:
|
|||||||
self, only_protocol: Optional[str] = None
|
self, only_protocol: Optional[str] = None
|
||||||
) -> Dict[str, JsonDict]:
|
) -> Dict[str, JsonDict]:
|
||||||
services = self.store.get_app_services()
|
services = self.store.get_app_services()
|
||||||
protocols = {} # type: Dict[str, List[JsonDict]]
|
protocols: Dict[str, List[JsonDict]] = {}
|
||||||
|
|
||||||
# Collect up all the individual protocol responses out of the ASes
|
# Collect up all the individual protocol responses out of the ASes
|
||||||
for s in services:
|
for s in services:
|
||||||
|
@ -191,7 +191,7 @@ class AuthHandler(BaseHandler):
|
|||||||
def __init__(self, hs: "HomeServer"):
|
def __init__(self, hs: "HomeServer"):
|
||||||
super().__init__(hs)
|
super().__init__(hs)
|
||||||
|
|
||||||
self.checkers = {} # type: Dict[str, UserInteractiveAuthChecker]
|
self.checkers: Dict[str, UserInteractiveAuthChecker] = {}
|
||||||
for auth_checker_class in INTERACTIVE_AUTH_CHECKERS:
|
for auth_checker_class in INTERACTIVE_AUTH_CHECKERS:
|
||||||
inst = auth_checker_class(hs)
|
inst = auth_checker_class(hs)
|
||||||
if inst.is_enabled():
|
if inst.is_enabled():
|
||||||
@ -296,7 +296,7 @@ class AuthHandler(BaseHandler):
|
|||||||
|
|
||||||
# A mapping of user ID to extra attributes to include in the login
|
# A mapping of user ID to extra attributes to include in the login
|
||||||
# response.
|
# response.
|
||||||
self._extra_attributes = {} # type: Dict[str, SsoLoginExtraAttributes]
|
self._extra_attributes: Dict[str, SsoLoginExtraAttributes] = {}
|
||||||
|
|
||||||
async def validate_user_via_ui_auth(
|
async def validate_user_via_ui_auth(
|
||||||
self,
|
self,
|
||||||
@ -500,7 +500,7 @@ class AuthHandler(BaseHandler):
|
|||||||
all the stages in any of the permitted flows.
|
all the stages in any of the permitted flows.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
sid = None # type: Optional[str]
|
sid: Optional[str] = None
|
||||||
authdict = clientdict.pop("auth", {})
|
authdict = clientdict.pop("auth", {})
|
||||||
if "session" in authdict:
|
if "session" in authdict:
|
||||||
sid = authdict["session"]
|
sid = authdict["session"]
|
||||||
@ -588,9 +588,9 @@ class AuthHandler(BaseHandler):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# check auth type currently being presented
|
# check auth type currently being presented
|
||||||
errordict = {} # type: Dict[str, Any]
|
errordict: Dict[str, Any] = {}
|
||||||
if "type" in authdict:
|
if "type" in authdict:
|
||||||
login_type = authdict["type"] # type: str
|
login_type: str = authdict["type"]
|
||||||
try:
|
try:
|
||||||
result = await self._check_auth_dict(authdict, clientip)
|
result = await self._check_auth_dict(authdict, clientip)
|
||||||
if result:
|
if result:
|
||||||
@ -766,7 +766,7 @@ class AuthHandler(BaseHandler):
|
|||||||
LoginType.TERMS: self._get_params_terms,
|
LoginType.TERMS: self._get_params_terms,
|
||||||
}
|
}
|
||||||
|
|
||||||
params = {} # type: Dict[str, Any]
|
params: Dict[str, Any] = {}
|
||||||
|
|
||||||
for f in public_flows:
|
for f in public_flows:
|
||||||
for stage in f:
|
for stage in f:
|
||||||
@ -1530,9 +1530,9 @@ class AuthHandler(BaseHandler):
|
|||||||
except StoreError:
|
except StoreError:
|
||||||
raise SynapseError(400, "Unknown session ID: %s" % (session_id,))
|
raise SynapseError(400, "Unknown session ID: %s" % (session_id,))
|
||||||
|
|
||||||
user_id_to_verify = await self.get_session_data(
|
user_id_to_verify: str = await self.get_session_data(
|
||||||
session_id, UIAuthSessionDataConstants.REQUEST_USER_ID
|
session_id, UIAuthSessionDataConstants.REQUEST_USER_ID
|
||||||
) # type: str
|
)
|
||||||
|
|
||||||
idps = await self.hs.get_sso_handler().get_identity_providers_for_user(
|
idps = await self.hs.get_sso_handler().get_identity_providers_for_user(
|
||||||
user_id_to_verify
|
user_id_to_verify
|
||||||
|
@ -40,7 +40,7 @@ class CasError(Exception):
|
|||||||
|
|
||||||
def __str__(self):
|
def __str__(self):
|
||||||
if self.error_description:
|
if self.error_description:
|
||||||
return "{}: {}".format(self.error, self.error_description)
|
return f"{self.error}: {self.error_description}"
|
||||||
return self.error
|
return self.error
|
||||||
|
|
||||||
|
|
||||||
@ -171,7 +171,7 @@ class CasHandler:
|
|||||||
|
|
||||||
# Iterate through the nodes and pull out the user and any extra attributes.
|
# Iterate through the nodes and pull out the user and any extra attributes.
|
||||||
user = None
|
user = None
|
||||||
attributes = {} # type: Dict[str, List[Optional[str]]]
|
attributes: Dict[str, List[Optional[str]]] = {}
|
||||||
for child in root[0]:
|
for child in root[0]:
|
||||||
if child.tag.endswith("user"):
|
if child.tag.endswith("user"):
|
||||||
user = child.text
|
user = child.text
|
||||||
|
@ -452,7 +452,7 @@ class DeviceHandler(DeviceWorkerHandler):
|
|||||||
user_id
|
user_id
|
||||||
)
|
)
|
||||||
|
|
||||||
hosts = set() # type: Set[str]
|
hosts: Set[str] = set()
|
||||||
if self.hs.is_mine_id(user_id):
|
if self.hs.is_mine_id(user_id):
|
||||||
hosts.update(get_domain_from_id(u) for u in users_who_share_room)
|
hosts.update(get_domain_from_id(u) for u in users_who_share_room)
|
||||||
hosts.discard(self.server_name)
|
hosts.discard(self.server_name)
|
||||||
@ -613,20 +613,20 @@ class DeviceListUpdater:
|
|||||||
self._remote_edu_linearizer = Linearizer(name="remote_device_list")
|
self._remote_edu_linearizer = Linearizer(name="remote_device_list")
|
||||||
|
|
||||||
# user_id -> list of updates waiting to be handled.
|
# user_id -> list of updates waiting to be handled.
|
||||||
self._pending_updates = (
|
self._pending_updates: Dict[
|
||||||
{}
|
str, List[Tuple[str, str, Iterable[str], JsonDict]]
|
||||||
) # type: Dict[str, List[Tuple[str, str, Iterable[str], JsonDict]]]
|
] = {}
|
||||||
|
|
||||||
# Recently seen stream ids. We don't bother keeping these in the DB,
|
# Recently seen stream ids. We don't bother keeping these in the DB,
|
||||||
# but they're useful to have them about to reduce the number of spurious
|
# but they're useful to have them about to reduce the number of spurious
|
||||||
# resyncs.
|
# resyncs.
|
||||||
self._seen_updates = ExpiringCache(
|
self._seen_updates: ExpiringCache[str, Set[str]] = ExpiringCache(
|
||||||
cache_name="device_update_edu",
|
cache_name="device_update_edu",
|
||||||
clock=self.clock,
|
clock=self.clock,
|
||||||
max_len=10000,
|
max_len=10000,
|
||||||
expiry_ms=30 * 60 * 1000,
|
expiry_ms=30 * 60 * 1000,
|
||||||
iterable=True,
|
iterable=True,
|
||||||
) # type: ExpiringCache[str, Set[str]]
|
)
|
||||||
|
|
||||||
# Attempt to resync out of sync device lists every 30s.
|
# Attempt to resync out of sync device lists every 30s.
|
||||||
self._resync_retry_in_progress = False
|
self._resync_retry_in_progress = False
|
||||||
@ -755,7 +755,7 @@ class DeviceListUpdater:
|
|||||||
"""Given a list of updates for a user figure out if we need to do a full
|
"""Given a list of updates for a user figure out if we need to do a full
|
||||||
resync, or whether we have enough data that we can just apply the delta.
|
resync, or whether we have enough data that we can just apply the delta.
|
||||||
"""
|
"""
|
||||||
seen_updates = self._seen_updates.get(user_id, set()) # type: Set[str]
|
seen_updates: Set[str] = self._seen_updates.get(user_id, set())
|
||||||
|
|
||||||
extremity = await self.store.get_device_list_last_stream_id_for_remote(user_id)
|
extremity = await self.store.get_device_list_last_stream_id_for_remote(user_id)
|
||||||
|
|
||||||
|
@ -203,7 +203,7 @@ class DeviceMessageHandler:
|
|||||||
log_kv({"number_of_to_device_messages": len(messages)})
|
log_kv({"number_of_to_device_messages": len(messages)})
|
||||||
set_tag("sender", sender_user_id)
|
set_tag("sender", sender_user_id)
|
||||||
local_messages = {}
|
local_messages = {}
|
||||||
remote_messages = {} # type: Dict[str, Dict[str, Dict[str, JsonDict]]]
|
remote_messages: Dict[str, Dict[str, Dict[str, JsonDict]]] = {}
|
||||||
for user_id, by_device in messages.items():
|
for user_id, by_device in messages.items():
|
||||||
# Ratelimit local cross-user key requests by the sending device.
|
# Ratelimit local cross-user key requests by the sending device.
|
||||||
if (
|
if (
|
||||||
|
@ -22,6 +22,7 @@ from synapse.api.errors import (
|
|||||||
CodeMessageException,
|
CodeMessageException,
|
||||||
Codes,
|
Codes,
|
||||||
NotFoundError,
|
NotFoundError,
|
||||||
|
RequestSendFailed,
|
||||||
ShadowBanError,
|
ShadowBanError,
|
||||||
StoreError,
|
StoreError,
|
||||||
SynapseError,
|
SynapseError,
|
||||||
@ -236,9 +237,9 @@ class DirectoryHandler(BaseHandler):
|
|||||||
async def get_association(self, room_alias: RoomAlias) -> JsonDict:
|
async def get_association(self, room_alias: RoomAlias) -> JsonDict:
|
||||||
room_id = None
|
room_id = None
|
||||||
if self.hs.is_mine(room_alias):
|
if self.hs.is_mine(room_alias):
|
||||||
result = await self.get_association_from_room_alias(
|
result: Optional[
|
||||||
room_alias
|
RoomAliasMapping
|
||||||
) # type: Optional[RoomAliasMapping]
|
] = await self.get_association_from_room_alias(room_alias)
|
||||||
|
|
||||||
if result:
|
if result:
|
||||||
room_id = result.room_id
|
room_id = result.room_id
|
||||||
@ -252,12 +253,14 @@ class DirectoryHandler(BaseHandler):
|
|||||||
retry_on_dns_fail=False,
|
retry_on_dns_fail=False,
|
||||||
ignore_backoff=True,
|
ignore_backoff=True,
|
||||||
)
|
)
|
||||||
|
except RequestSendFailed:
|
||||||
|
raise SynapseError(502, "Failed to fetch alias")
|
||||||
except CodeMessageException as e:
|
except CodeMessageException as e:
|
||||||
logging.warning("Error retrieving alias")
|
logging.warning("Error retrieving alias")
|
||||||
if e.code == 404:
|
if e.code == 404:
|
||||||
fed_result = None
|
fed_result = None
|
||||||
else:
|
else:
|
||||||
raise
|
raise SynapseError(502, "Failed to fetch alias")
|
||||||
|
|
||||||
if fed_result and "room_id" in fed_result and "servers" in fed_result:
|
if fed_result and "room_id" in fed_result and "servers" in fed_result:
|
||||||
room_id = fed_result["room_id"]
|
room_id = fed_result["room_id"]
|
||||||
|
@ -115,9 +115,9 @@ class E2eKeysHandler:
|
|||||||
the number of in-flight queries at a time.
|
the number of in-flight queries at a time.
|
||||||
"""
|
"""
|
||||||
with await self._query_devices_linearizer.queue((from_user_id, from_device_id)):
|
with await self._query_devices_linearizer.queue((from_user_id, from_device_id)):
|
||||||
device_keys_query = query_body.get(
|
device_keys_query: Dict[str, Iterable[str]] = query_body.get(
|
||||||
"device_keys", {}
|
"device_keys", {}
|
||||||
) # type: Dict[str, Iterable[str]]
|
)
|
||||||
|
|
||||||
# separate users by domain.
|
# separate users by domain.
|
||||||
# make a map from domain to user_id to device_ids
|
# make a map from domain to user_id to device_ids
|
||||||
@ -136,7 +136,7 @@ class E2eKeysHandler:
|
|||||||
|
|
||||||
# First get local devices.
|
# First get local devices.
|
||||||
# A map of destination -> failure response.
|
# A map of destination -> failure response.
|
||||||
failures = {} # type: Dict[str, JsonDict]
|
failures: Dict[str, JsonDict] = {}
|
||||||
results = {}
|
results = {}
|
||||||
if local_query:
|
if local_query:
|
||||||
local_result = await self.query_local_devices(local_query)
|
local_result = await self.query_local_devices(local_query)
|
||||||
@ -151,11 +151,9 @@ class E2eKeysHandler:
|
|||||||
|
|
||||||
# Now attempt to get any remote devices from our local cache.
|
# Now attempt to get any remote devices from our local cache.
|
||||||
# A map of destination -> user ID -> device IDs.
|
# A map of destination -> user ID -> device IDs.
|
||||||
remote_queries_not_in_cache = (
|
remote_queries_not_in_cache: Dict[str, Dict[str, Iterable[str]]] = {}
|
||||||
{}
|
|
||||||
) # type: Dict[str, Dict[str, Iterable[str]]]
|
|
||||||
if remote_queries:
|
if remote_queries:
|
||||||
query_list = [] # type: List[Tuple[str, Optional[str]]]
|
query_list: List[Tuple[str, Optional[str]]] = []
|
||||||
for user_id, device_ids in remote_queries.items():
|
for user_id, device_ids in remote_queries.items():
|
||||||
if device_ids:
|
if device_ids:
|
||||||
query_list.extend(
|
query_list.extend(
|
||||||
@ -362,9 +360,9 @@ class E2eKeysHandler:
|
|||||||
A map from user_id -> device_id -> device details
|
A map from user_id -> device_id -> device details
|
||||||
"""
|
"""
|
||||||
set_tag("local_query", query)
|
set_tag("local_query", query)
|
||||||
local_query = [] # type: List[Tuple[str, Optional[str]]]
|
local_query: List[Tuple[str, Optional[str]]] = []
|
||||||
|
|
||||||
result_dict = {} # type: Dict[str, Dict[str, dict]]
|
result_dict: Dict[str, Dict[str, dict]] = {}
|
||||||
for user_id, device_ids in query.items():
|
for user_id, device_ids in query.items():
|
||||||
# we use UserID.from_string to catch invalid user ids
|
# we use UserID.from_string to catch invalid user ids
|
||||||
if not self.is_mine(UserID.from_string(user_id)):
|
if not self.is_mine(UserID.from_string(user_id)):
|
||||||
@ -402,9 +400,9 @@ class E2eKeysHandler:
|
|||||||
self, query_body: Dict[str, Dict[str, Optional[List[str]]]]
|
self, query_body: Dict[str, Dict[str, Optional[List[str]]]]
|
||||||
) -> JsonDict:
|
) -> JsonDict:
|
||||||
"""Handle a device key query from a federated server"""
|
"""Handle a device key query from a federated server"""
|
||||||
device_keys_query = query_body.get(
|
device_keys_query: Dict[str, Optional[List[str]]] = query_body.get(
|
||||||
"device_keys", {}
|
"device_keys", {}
|
||||||
) # type: Dict[str, Optional[List[str]]]
|
)
|
||||||
res = await self.query_local_devices(device_keys_query)
|
res = await self.query_local_devices(device_keys_query)
|
||||||
ret = {"device_keys": res}
|
ret = {"device_keys": res}
|
||||||
|
|
||||||
@ -421,8 +419,8 @@ class E2eKeysHandler:
|
|||||||
async def claim_one_time_keys(
|
async def claim_one_time_keys(
|
||||||
self, query: Dict[str, Dict[str, Dict[str, str]]], timeout: int
|
self, query: Dict[str, Dict[str, Dict[str, str]]], timeout: int
|
||||||
) -> JsonDict:
|
) -> JsonDict:
|
||||||
local_query = [] # type: List[Tuple[str, str, str]]
|
local_query: List[Tuple[str, str, str]] = []
|
||||||
remote_queries = {} # type: Dict[str, Dict[str, Dict[str, str]]]
|
remote_queries: Dict[str, Dict[str, Dict[str, str]]] = {}
|
||||||
|
|
||||||
for user_id, one_time_keys in query.get("one_time_keys", {}).items():
|
for user_id, one_time_keys in query.get("one_time_keys", {}).items():
|
||||||
# we use UserID.from_string to catch invalid user ids
|
# we use UserID.from_string to catch invalid user ids
|
||||||
@ -439,8 +437,8 @@ class E2eKeysHandler:
|
|||||||
results = await self.store.claim_e2e_one_time_keys(local_query)
|
results = await self.store.claim_e2e_one_time_keys(local_query)
|
||||||
|
|
||||||
# A map of user ID -> device ID -> key ID -> key.
|
# A map of user ID -> device ID -> key ID -> key.
|
||||||
json_result = {} # type: Dict[str, Dict[str, Dict[str, JsonDict]]]
|
json_result: Dict[str, Dict[str, Dict[str, JsonDict]]] = {}
|
||||||
failures = {} # type: Dict[str, JsonDict]
|
failures: Dict[str, JsonDict] = {}
|
||||||
for user_id, device_keys in results.items():
|
for user_id, device_keys in results.items():
|
||||||
for device_id, keys in device_keys.items():
|
for device_id, keys in device_keys.items():
|
||||||
for key_id, json_str in keys.items():
|
for key_id, json_str in keys.items():
|
||||||
@ -768,8 +766,8 @@ class E2eKeysHandler:
|
|||||||
Raises:
|
Raises:
|
||||||
SynapseError: if the input is malformed
|
SynapseError: if the input is malformed
|
||||||
"""
|
"""
|
||||||
signature_list = [] # type: List[SignatureListItem]
|
signature_list: List["SignatureListItem"] = []
|
||||||
failures = {} # type: Dict[str, Dict[str, JsonDict]]
|
failures: Dict[str, Dict[str, JsonDict]] = {}
|
||||||
if not signatures:
|
if not signatures:
|
||||||
return signature_list, failures
|
return signature_list, failures
|
||||||
|
|
||||||
@ -930,8 +928,8 @@ class E2eKeysHandler:
|
|||||||
Raises:
|
Raises:
|
||||||
SynapseError: if the input is malformed
|
SynapseError: if the input is malformed
|
||||||
"""
|
"""
|
||||||
signature_list = [] # type: List[SignatureListItem]
|
signature_list: List["SignatureListItem"] = []
|
||||||
failures = {} # type: Dict[str, Dict[str, JsonDict]]
|
failures: Dict[str, Dict[str, JsonDict]] = {}
|
||||||
if not signatures:
|
if not signatures:
|
||||||
return signature_list, failures
|
return signature_list, failures
|
||||||
|
|
||||||
@ -1300,7 +1298,7 @@ class SigningKeyEduUpdater:
|
|||||||
self._remote_edu_linearizer = Linearizer(name="remote_signing_key")
|
self._remote_edu_linearizer = Linearizer(name="remote_signing_key")
|
||||||
|
|
||||||
# user_id -> list of updates waiting to be handled.
|
# user_id -> list of updates waiting to be handled.
|
||||||
self._pending_updates = {} # type: Dict[str, List[Tuple[JsonDict, JsonDict]]]
|
self._pending_updates: Dict[str, List[Tuple[JsonDict, JsonDict]]] = {}
|
||||||
|
|
||||||
async def incoming_signing_key_update(
|
async def incoming_signing_key_update(
|
||||||
self, origin: str, edu_content: JsonDict
|
self, origin: str, edu_content: JsonDict
|
||||||
@ -1349,7 +1347,7 @@ class SigningKeyEduUpdater:
|
|||||||
# This can happen since we batch updates
|
# This can happen since we batch updates
|
||||||
return
|
return
|
||||||
|
|
||||||
device_ids = [] # type: List[str]
|
device_ids: List[str] = []
|
||||||
|
|
||||||
logger.info("pending updates: %r", pending_updates)
|
logger.info("pending updates: %r", pending_updates)
|
||||||
|
|
||||||
|
@ -93,7 +93,7 @@ class EventStreamHandler(BaseHandler):
|
|||||||
|
|
||||||
# When the user joins a new room, or another user joins a currently
|
# When the user joins a new room, or another user joins a currently
|
||||||
# joined room, we need to send down presence for those users.
|
# joined room, we need to send down presence for those users.
|
||||||
to_add = [] # type: List[JsonDict]
|
to_add: List[JsonDict] = []
|
||||||
for event in events:
|
for event in events:
|
||||||
if not isinstance(event, EventBase):
|
if not isinstance(event, EventBase):
|
||||||
continue
|
continue
|
||||||
@ -103,9 +103,9 @@ class EventStreamHandler(BaseHandler):
|
|||||||
# Send down presence.
|
# Send down presence.
|
||||||
if event.state_key == auth_user_id:
|
if event.state_key == auth_user_id:
|
||||||
# Send down presence for everyone in the room.
|
# Send down presence for everyone in the room.
|
||||||
users = await self.store.get_users_in_room(
|
users: Iterable[str] = await self.store.get_users_in_room(
|
||||||
event.room_id
|
event.room_id
|
||||||
) # type: Iterable[str]
|
)
|
||||||
else:
|
else:
|
||||||
users = [event.state_key]
|
users = [event.state_key]
|
||||||
|
|
||||||
|
@ -181,7 +181,7 @@ class FederationHandler(BaseHandler):
|
|||||||
|
|
||||||
# When joining a room we need to queue any events for that room up.
|
# When joining a room we need to queue any events for that room up.
|
||||||
# For each room, a list of (pdu, origin) tuples.
|
# For each room, a list of (pdu, origin) tuples.
|
||||||
self.room_queues = {} # type: Dict[str, List[Tuple[EventBase, str]]]
|
self.room_queues: Dict[str, List[Tuple[EventBase, str]]] = {}
|
||||||
self._room_pdu_linearizer = Linearizer("fed_room_pdu")
|
self._room_pdu_linearizer = Linearizer("fed_room_pdu")
|
||||||
|
|
||||||
self._room_backfill = Linearizer("room_backfill")
|
self._room_backfill = Linearizer("room_backfill")
|
||||||
@ -368,7 +368,7 @@ class FederationHandler(BaseHandler):
|
|||||||
ours = await self.state_store.get_state_groups_ids(room_id, seen)
|
ours = await self.state_store.get_state_groups_ids(room_id, seen)
|
||||||
|
|
||||||
# state_maps is a list of mappings from (type, state_key) to event_id
|
# state_maps is a list of mappings from (type, state_key) to event_id
|
||||||
state_maps = list(ours.values()) # type: List[StateMap[str]]
|
state_maps: List[StateMap[str]] = list(ours.values())
|
||||||
|
|
||||||
# we don't need this any more, let's delete it.
|
# we don't need this any more, let's delete it.
|
||||||
del ours
|
del ours
|
||||||
@ -735,7 +735,7 @@ class FederationHandler(BaseHandler):
|
|||||||
# we need to make sure we re-load from the database to get the rejected
|
# we need to make sure we re-load from the database to get the rejected
|
||||||
# state correct.
|
# state correct.
|
||||||
fetched_events.update(
|
fetched_events.update(
|
||||||
(await self.store.get_events(missing_desired_events, allow_rejected=True))
|
await self.store.get_events(missing_desired_events, allow_rejected=True)
|
||||||
)
|
)
|
||||||
|
|
||||||
# check for events which were in the wrong room.
|
# check for events which were in the wrong room.
|
||||||
@ -845,7 +845,7 @@ class FederationHandler(BaseHandler):
|
|||||||
# exact key to expect. Otherwise check it matches any key we
|
# exact key to expect. Otherwise check it matches any key we
|
||||||
# have for that device.
|
# have for that device.
|
||||||
|
|
||||||
current_keys = [] # type: Container[str]
|
current_keys: Container[str] = []
|
||||||
|
|
||||||
if device:
|
if device:
|
||||||
keys = device.get("keys", {}).get("keys", {})
|
keys = device.get("keys", {}).get("keys", {})
|
||||||
@ -1185,7 +1185,7 @@ class FederationHandler(BaseHandler):
|
|||||||
if e_type == EventTypes.Member and event.membership == Membership.JOIN
|
if e_type == EventTypes.Member and event.membership == Membership.JOIN
|
||||||
]
|
]
|
||||||
|
|
||||||
joined_domains = {} # type: Dict[str, int]
|
joined_domains: Dict[str, int] = {}
|
||||||
for u, d in joined_users:
|
for u, d in joined_users:
|
||||||
try:
|
try:
|
||||||
dom = get_domain_from_id(u)
|
dom = get_domain_from_id(u)
|
||||||
@ -1314,7 +1314,7 @@ class FederationHandler(BaseHandler):
|
|||||||
|
|
||||||
room_version = await self.store.get_room_version(room_id)
|
room_version = await self.store.get_room_version(room_id)
|
||||||
|
|
||||||
event_map = {} # type: Dict[str, EventBase]
|
event_map: Dict[str, EventBase] = {}
|
||||||
|
|
||||||
async def get_event(event_id: str):
|
async def get_event(event_id: str):
|
||||||
with nested_logging_context(event_id):
|
with nested_logging_context(event_id):
|
||||||
@ -1414,12 +1414,15 @@ class FederationHandler(BaseHandler):
|
|||||||
|
|
||||||
Invites must be signed by the invitee's server before distribution.
|
Invites must be signed by the invitee's server before distribution.
|
||||||
"""
|
"""
|
||||||
|
try:
|
||||||
pdu = await self.federation_client.send_invite(
|
pdu = await self.federation_client.send_invite(
|
||||||
destination=target_host,
|
destination=target_host,
|
||||||
room_id=event.room_id,
|
room_id=event.room_id,
|
||||||
event_id=event.event_id,
|
event_id=event.event_id,
|
||||||
pdu=event,
|
pdu=event,
|
||||||
)
|
)
|
||||||
|
except RequestSendFailed:
|
||||||
|
raise SynapseError(502, f"Can't connect to server {target_host}")
|
||||||
|
|
||||||
return pdu
|
return pdu
|
||||||
|
|
||||||
@ -1593,7 +1596,7 @@ class FederationHandler(BaseHandler):
|
|||||||
|
|
||||||
# Ask the remote server to create a valid knock event for us. Once received,
|
# Ask the remote server to create a valid knock event for us. Once received,
|
||||||
# we sign the event
|
# we sign the event
|
||||||
params = {"ver": supported_room_versions} # type: Dict[str, Iterable[str]]
|
params: Dict[str, Iterable[str]] = {"ver": supported_room_versions}
|
||||||
origin, event, event_format_version = await self._make_and_verify_event(
|
origin, event, event_format_version = await self._make_and_verify_event(
|
||||||
target_hosts, room_id, knockee, Membership.KNOCK, content, params=params
|
target_hosts, room_id, knockee, Membership.KNOCK, content, params=params
|
||||||
)
|
)
|
||||||
@ -1931,7 +1934,7 @@ class FederationHandler(BaseHandler):
|
|||||||
builder=builder
|
builder=builder
|
||||||
)
|
)
|
||||||
|
|
||||||
event_allowed = await self.third_party_event_rules.check_event_allowed(
|
event_allowed, _ = await self.third_party_event_rules.check_event_allowed(
|
||||||
event, context
|
event, context
|
||||||
)
|
)
|
||||||
if not event_allowed:
|
if not event_allowed:
|
||||||
@ -2023,7 +2026,7 @@ class FederationHandler(BaseHandler):
|
|||||||
# for knock events, we run the third-party event rules. It's not entirely clear
|
# for knock events, we run the third-party event rules. It's not entirely clear
|
||||||
# why we don't do this for other sorts of membership events.
|
# why we don't do this for other sorts of membership events.
|
||||||
if event.membership == Membership.KNOCK:
|
if event.membership == Membership.KNOCK:
|
||||||
event_allowed = await self.third_party_event_rules.check_event_allowed(
|
event_allowed, _ = await self.third_party_event_rules.check_event_allowed(
|
||||||
event, context
|
event, context
|
||||||
)
|
)
|
||||||
if not event_allowed:
|
if not event_allowed:
|
||||||
@ -2450,14 +2453,14 @@ class FederationHandler(BaseHandler):
|
|||||||
state_sets_d = await self.state_store.get_state_groups(
|
state_sets_d = await self.state_store.get_state_groups(
|
||||||
event.room_id, extrem_ids
|
event.room_id, extrem_ids
|
||||||
)
|
)
|
||||||
state_sets = list(state_sets_d.values()) # type: List[Iterable[EventBase]]
|
state_sets: List[Iterable[EventBase]] = list(state_sets_d.values())
|
||||||
state_sets.append(state)
|
state_sets.append(state)
|
||||||
current_states = await self.state_handler.resolve_events(
|
current_states = await self.state_handler.resolve_events(
|
||||||
room_version, state_sets, event
|
room_version, state_sets, event
|
||||||
)
|
)
|
||||||
current_state_ids = {
|
current_state_ids: StateMap[str] = {
|
||||||
k: e.event_id for k, e in current_states.items()
|
k: e.event_id for k, e in current_states.items()
|
||||||
} # type: StateMap[str]
|
}
|
||||||
else:
|
else:
|
||||||
current_state_ids = await self.state_handler.get_current_state_ids(
|
current_state_ids = await self.state_handler.get_current_state_ids(
|
||||||
event.room_id, latest_event_ids=extrem_ids
|
event.room_id, latest_event_ids=extrem_ids
|
||||||
@ -2814,7 +2817,7 @@ class FederationHandler(BaseHandler):
|
|||||||
"""
|
"""
|
||||||
# exclude the state key of the new event from the current_state in the context.
|
# exclude the state key of the new event from the current_state in the context.
|
||||||
if event.is_state():
|
if event.is_state():
|
||||||
event_key = (event.type, event.state_key) # type: Optional[Tuple[str, str]]
|
event_key: Optional[Tuple[str, str]] = (event.type, event.state_key)
|
||||||
else:
|
else:
|
||||||
event_key = None
|
event_key = None
|
||||||
state_updates = {
|
state_updates = {
|
||||||
@ -3031,9 +3034,13 @@ class FederationHandler(BaseHandler):
|
|||||||
await member_handler.send_membership_event(None, event, context)
|
await member_handler.send_membership_event(None, event, context)
|
||||||
else:
|
else:
|
||||||
destinations = {x.split(":", 1)[-1] for x in (sender_user_id, room_id)}
|
destinations = {x.split(":", 1)[-1] for x in (sender_user_id, room_id)}
|
||||||
|
|
||||||
|
try:
|
||||||
await self.federation_client.forward_third_party_invite(
|
await self.federation_client.forward_third_party_invite(
|
||||||
destinations, room_id, event_dict
|
destinations, room_id, event_dict
|
||||||
)
|
)
|
||||||
|
except (RequestSendFailed, HttpResponseException):
|
||||||
|
raise SynapseError(502, "Failed to forward third party invite")
|
||||||
|
|
||||||
async def on_exchange_third_party_invite_request(
|
async def on_exchange_third_party_invite_request(
|
||||||
self, event_dict: JsonDict
|
self, event_dict: JsonDict
|
||||||
@ -3149,7 +3156,7 @@ class FederationHandler(BaseHandler):
|
|||||||
|
|
||||||
logger.debug("Checking auth on event %r", event.content)
|
logger.debug("Checking auth on event %r", event.content)
|
||||||
|
|
||||||
last_exception = None # type: Optional[Exception]
|
last_exception: Optional[Exception] = None
|
||||||
|
|
||||||
# for each public key in the 3pid invite event
|
# for each public key in the 3pid invite event
|
||||||
for public_key_object in event_auth.get_public_keys(invite_event):
|
for public_key_object in event_auth.get_public_keys(invite_event):
|
||||||
|
@ -214,7 +214,7 @@ class GroupsLocalWorkerHandler:
|
|||||||
async def bulk_get_publicised_groups(
|
async def bulk_get_publicised_groups(
|
||||||
self, user_ids: Iterable[str], proxy: bool = True
|
self, user_ids: Iterable[str], proxy: bool = True
|
||||||
) -> JsonDict:
|
) -> JsonDict:
|
||||||
destinations = {} # type: Dict[str, Set[str]]
|
destinations: Dict[str, Set[str]] = {}
|
||||||
local_users = set()
|
local_users = set()
|
||||||
|
|
||||||
for user_id in user_ids:
|
for user_id in user_ids:
|
||||||
@ -227,7 +227,7 @@ class GroupsLocalWorkerHandler:
|
|||||||
raise SynapseError(400, "Some user_ids are not local")
|
raise SynapseError(400, "Some user_ids are not local")
|
||||||
|
|
||||||
results = {}
|
results = {}
|
||||||
failed_results = [] # type: List[str]
|
failed_results: List[str] = []
|
||||||
for destination, dest_user_ids in destinations.items():
|
for destination, dest_user_ids in destinations.items():
|
||||||
try:
|
try:
|
||||||
r = await self.transport_client.bulk_get_publicised_groups(
|
r = await self.transport_client.bulk_get_publicised_groups(
|
||||||
|
@ -302,7 +302,7 @@ class IdentityHandler(BaseHandler):
|
|||||||
)
|
)
|
||||||
|
|
||||||
url = "https://%s/_matrix/identity/api/v1/3pid/unbind" % (id_server,)
|
url = "https://%s/_matrix/identity/api/v1/3pid/unbind" % (id_server,)
|
||||||
url_bytes = "/_matrix/identity/api/v1/3pid/unbind".encode("ascii")
|
url_bytes = b"/_matrix/identity/api/v1/3pid/unbind"
|
||||||
|
|
||||||
content = {
|
content = {
|
||||||
"mxid": mxid,
|
"mxid": mxid,
|
||||||
@ -695,7 +695,7 @@ class IdentityHandler(BaseHandler):
|
|||||||
return data["mxid"]
|
return data["mxid"]
|
||||||
except RequestTimedOutError:
|
except RequestTimedOutError:
|
||||||
raise SynapseError(500, "Timed out contacting identity server")
|
raise SynapseError(500, "Timed out contacting identity server")
|
||||||
except IOError as e:
|
except OSError as e:
|
||||||
logger.warning("Error from v1 identity server lookup: %s" % (e,))
|
logger.warning("Error from v1 identity server lookup: %s" % (e,))
|
||||||
|
|
||||||
return None
|
return None
|
||||||
|
@ -46,9 +46,17 @@ class InitialSyncHandler(BaseHandler):
|
|||||||
self.state = hs.get_state_handler()
|
self.state = hs.get_state_handler()
|
||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
self.validator = EventValidator()
|
self.validator = EventValidator()
|
||||||
self.snapshot_cache = ResponseCache(
|
self.snapshot_cache: ResponseCache[
|
||||||
hs.get_clock(), "initial_sync_cache"
|
Tuple[
|
||||||
) # type: ResponseCache[Tuple[str, Optional[StreamToken], Optional[StreamToken], str, Optional[int], bool, bool]]
|
str,
|
||||||
|
Optional[StreamToken],
|
||||||
|
Optional[StreamToken],
|
||||||
|
str,
|
||||||
|
Optional[int],
|
||||||
|
bool,
|
||||||
|
bool,
|
||||||
|
]
|
||||||
|
] = ResponseCache(hs.get_clock(), "initial_sync_cache")
|
||||||
self._event_serializer = hs.get_event_client_serializer()
|
self._event_serializer = hs.get_event_client_serializer()
|
||||||
self.storage = hs.get_storage()
|
self.storage = hs.get_storage()
|
||||||
self.state_store = self.storage.state
|
self.state_store = self.storage.state
|
||||||
|
@ -81,7 +81,7 @@ class MessageHandler:
|
|||||||
|
|
||||||
# The scheduled call to self._expire_event. None if no call is currently
|
# The scheduled call to self._expire_event. None if no call is currently
|
||||||
# scheduled.
|
# scheduled.
|
||||||
self._scheduled_expiry = None # type: Optional[IDelayedCall]
|
self._scheduled_expiry: Optional[IDelayedCall] = None
|
||||||
|
|
||||||
if not hs.config.worker_app:
|
if not hs.config.worker_app:
|
||||||
run_as_background_process(
|
run_as_background_process(
|
||||||
@ -196,9 +196,7 @@ class MessageHandler:
|
|||||||
room_state_events = await self.state_store.get_state_for_events(
|
room_state_events = await self.state_store.get_state_for_events(
|
||||||
[event.event_id], state_filter=state_filter
|
[event.event_id], state_filter=state_filter
|
||||||
)
|
)
|
||||||
room_state = room_state_events[
|
room_state: Mapping[Any, EventBase] = room_state_events[event.event_id]
|
||||||
event.event_id
|
|
||||||
] # type: Mapping[Any, EventBase]
|
|
||||||
else:
|
else:
|
||||||
raise AuthError(
|
raise AuthError(
|
||||||
403,
|
403,
|
||||||
@ -421,9 +419,9 @@ class EventCreationHandler:
|
|||||||
self.action_generator = hs.get_action_generator()
|
self.action_generator = hs.get_action_generator()
|
||||||
|
|
||||||
self.spam_checker = hs.get_spam_checker()
|
self.spam_checker = hs.get_spam_checker()
|
||||||
self.third_party_event_rules = (
|
self.third_party_event_rules: "ThirdPartyEventRules" = (
|
||||||
self.hs.get_third_party_event_rules()
|
self.hs.get_third_party_event_rules()
|
||||||
) # type: ThirdPartyEventRules
|
)
|
||||||
|
|
||||||
self._block_events_without_consent_error = (
|
self._block_events_without_consent_error = (
|
||||||
self.config.block_events_without_consent_error
|
self.config.block_events_without_consent_error
|
||||||
@ -440,7 +438,7 @@ class EventCreationHandler:
|
|||||||
#
|
#
|
||||||
# map from room id to time-of-last-attempt.
|
# map from room id to time-of-last-attempt.
|
||||||
#
|
#
|
||||||
self._rooms_to_exclude_from_dummy_event_insertion = {} # type: Dict[str, int]
|
self._rooms_to_exclude_from_dummy_event_insertion: Dict[str, int] = {}
|
||||||
# The number of forward extremeities before a dummy event is sent.
|
# The number of forward extremeities before a dummy event is sent.
|
||||||
self._dummy_events_threshold = hs.config.dummy_events_threshold
|
self._dummy_events_threshold = hs.config.dummy_events_threshold
|
||||||
|
|
||||||
@ -465,9 +463,7 @@ class EventCreationHandler:
|
|||||||
# Stores the state groups we've recently added to the joined hosts
|
# Stores the state groups we've recently added to the joined hosts
|
||||||
# external cache. Note that the timeout must be significantly less than
|
# external cache. Note that the timeout must be significantly less than
|
||||||
# the TTL on the external cache.
|
# the TTL on the external cache.
|
||||||
self._external_cache_joined_hosts_updates = (
|
self._external_cache_joined_hosts_updates: Optional[ExpiringCache] = None
|
||||||
None
|
|
||||||
) # type: Optional[ExpiringCache]
|
|
||||||
if self._external_cache.is_enabled():
|
if self._external_cache.is_enabled():
|
||||||
self._external_cache_joined_hosts_updates = ExpiringCache(
|
self._external_cache_joined_hosts_updates = ExpiringCache(
|
||||||
"_external_cache_joined_hosts_updates",
|
"_external_cache_joined_hosts_updates",
|
||||||
@ -518,6 +514,9 @@ class EventCreationHandler:
|
|||||||
outlier: Indicates whether the event is an `outlier`, i.e. if
|
outlier: Indicates whether the event is an `outlier`, i.e. if
|
||||||
it's from an arbitrary point and floating in the DAG as
|
it's from an arbitrary point and floating in the DAG as
|
||||||
opposed to being inline with the current DAG.
|
opposed to being inline with the current DAG.
|
||||||
|
historical: Indicates whether the message is being inserted
|
||||||
|
back in time around some existing events. This is used to skip
|
||||||
|
a few checks and mark the event as backfilled.
|
||||||
depth: Override the depth used to order the event in the DAG.
|
depth: Override the depth used to order the event in the DAG.
|
||||||
Should normally be set to None, which will cause the depth to be calculated
|
Should normally be set to None, which will cause the depth to be calculated
|
||||||
based on the prev_events.
|
based on the prev_events.
|
||||||
@ -772,6 +771,7 @@ class EventCreationHandler:
|
|||||||
txn_id: Optional[str] = None,
|
txn_id: Optional[str] = None,
|
||||||
ignore_shadow_ban: bool = False,
|
ignore_shadow_ban: bool = False,
|
||||||
outlier: bool = False,
|
outlier: bool = False,
|
||||||
|
historical: bool = False,
|
||||||
depth: Optional[int] = None,
|
depth: Optional[int] = None,
|
||||||
) -> Tuple[EventBase, int]:
|
) -> Tuple[EventBase, int]:
|
||||||
"""
|
"""
|
||||||
@ -799,6 +799,9 @@ class EventCreationHandler:
|
|||||||
outlier: Indicates whether the event is an `outlier`, i.e. if
|
outlier: Indicates whether the event is an `outlier`, i.e. if
|
||||||
it's from an arbitrary point and floating in the DAG as
|
it's from an arbitrary point and floating in the DAG as
|
||||||
opposed to being inline with the current DAG.
|
opposed to being inline with the current DAG.
|
||||||
|
historical: Indicates whether the message is being inserted
|
||||||
|
back in time around some existing events. This is used to skip
|
||||||
|
a few checks and mark the event as backfilled.
|
||||||
depth: Override the depth used to order the event in the DAG.
|
depth: Override the depth used to order the event in the DAG.
|
||||||
Should normally be set to None, which will cause the depth to be calculated
|
Should normally be set to None, which will cause the depth to be calculated
|
||||||
based on the prev_events.
|
based on the prev_events.
|
||||||
@ -847,6 +850,7 @@ class EventCreationHandler:
|
|||||||
prev_event_ids=prev_event_ids,
|
prev_event_ids=prev_event_ids,
|
||||||
auth_event_ids=auth_event_ids,
|
auth_event_ids=auth_event_ids,
|
||||||
outlier=outlier,
|
outlier=outlier,
|
||||||
|
historical=historical,
|
||||||
depth=depth,
|
depth=depth,
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -945,10 +949,10 @@ class EventCreationHandler:
|
|||||||
if requester:
|
if requester:
|
||||||
context.app_service = requester.app_service
|
context.app_service = requester.app_service
|
||||||
|
|
||||||
third_party_result = await self.third_party_event_rules.check_event_allowed(
|
res, new_content = await self.third_party_event_rules.check_event_allowed(
|
||||||
event, context
|
event, context
|
||||||
)
|
)
|
||||||
if not third_party_result:
|
if res is False:
|
||||||
logger.info(
|
logger.info(
|
||||||
"Event %s forbidden by third-party rules",
|
"Event %s forbidden by third-party rules",
|
||||||
event,
|
event,
|
||||||
@ -956,11 +960,11 @@ class EventCreationHandler:
|
|||||||
raise SynapseError(
|
raise SynapseError(
|
||||||
403, "This event is not allowed in this context", Codes.FORBIDDEN
|
403, "This event is not allowed in this context", Codes.FORBIDDEN
|
||||||
)
|
)
|
||||||
elif isinstance(third_party_result, dict):
|
elif new_content is not None:
|
||||||
# the third-party rules want to replace the event. We'll need to build a new
|
# the third-party rules want to replace the event. We'll need to build a new
|
||||||
# event.
|
# event.
|
||||||
event, context = await self._rebuild_event_after_third_party_rules(
|
event, context = await self._rebuild_event_after_third_party_rules(
|
||||||
third_party_result, event
|
new_content, event
|
||||||
)
|
)
|
||||||
|
|
||||||
self.validator.validate_new(event, self.config)
|
self.validator.validate_new(event, self.config)
|
||||||
@ -1291,7 +1295,7 @@ class EventCreationHandler:
|
|||||||
# Validate a newly added alias or newly added alt_aliases.
|
# Validate a newly added alias or newly added alt_aliases.
|
||||||
|
|
||||||
original_alias = None
|
original_alias = None
|
||||||
original_alt_aliases = [] # type: List[str]
|
original_alt_aliases: List[str] = []
|
||||||
|
|
||||||
original_event_id = event.unsigned.get("replaces_state")
|
original_event_id = event.unsigned.get("replaces_state")
|
||||||
if original_event_id:
|
if original_event_id:
|
||||||
@ -1594,11 +1598,13 @@ class EventCreationHandler:
|
|||||||
for k, v in original_event.internal_metadata.get_dict().items():
|
for k, v in original_event.internal_metadata.get_dict().items():
|
||||||
setattr(builder.internal_metadata, k, v)
|
setattr(builder.internal_metadata, k, v)
|
||||||
|
|
||||||
# the event type hasn't changed, so there's no point in re-calculating the
|
# modules can send new state events, so we re-calculate the auth events just in
|
||||||
# auth events.
|
# case.
|
||||||
|
prev_event_ids = await self.store.get_prev_events_for_room(builder.room_id)
|
||||||
|
|
||||||
event = await builder.build(
|
event = await builder.build(
|
||||||
prev_event_ids=original_event.prev_event_ids(),
|
prev_event_ids=prev_event_ids,
|
||||||
auth_event_ids=original_event.auth_event_ids(),
|
auth_event_ids=None,
|
||||||
)
|
)
|
||||||
|
|
||||||
# we rebuild the event context, to be on the safe side. If nothing else,
|
# we rebuild the event context, to be on the safe side. If nothing else,
|
||||||
|
@ -72,26 +72,26 @@ _SESSION_COOKIES = [
|
|||||||
(b"oidc_session_no_samesite", b"HttpOnly"),
|
(b"oidc_session_no_samesite", b"HttpOnly"),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
#: A token exchanged from the token endpoint, as per RFC6749 sec 5.1. and
|
#: A token exchanged from the token endpoint, as per RFC6749 sec 5.1. and
|
||||||
#: OpenID.Core sec 3.1.3.3.
|
#: OpenID.Core sec 3.1.3.3.
|
||||||
Token = TypedDict(
|
class Token(TypedDict):
|
||||||
"Token",
|
access_token: str
|
||||||
{
|
token_type: str
|
||||||
"access_token": str,
|
id_token: Optional[str]
|
||||||
"token_type": str,
|
refresh_token: Optional[str]
|
||||||
"id_token": Optional[str],
|
expires_in: int
|
||||||
"refresh_token": Optional[str],
|
scope: Optional[str]
|
||||||
"expires_in": int,
|
|
||||||
"scope": Optional[str],
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
#: A JWK, as per RFC7517 sec 4. The type could be more precise than that, but
|
#: A JWK, as per RFC7517 sec 4. The type could be more precise than that, but
|
||||||
#: there is no real point of doing this in our case.
|
#: there is no real point of doing this in our case.
|
||||||
JWK = Dict[str, str]
|
JWK = Dict[str, str]
|
||||||
|
|
||||||
|
|
||||||
#: A JWK Set, as per RFC7517 sec 5.
|
#: A JWK Set, as per RFC7517 sec 5.
|
||||||
JWKS = TypedDict("JWKS", {"keys": List[JWK]})
|
class JWKS(TypedDict):
|
||||||
|
keys: List[JWK]
|
||||||
|
|
||||||
|
|
||||||
class OidcHandler:
|
class OidcHandler:
|
||||||
@ -105,9 +105,9 @@ class OidcHandler:
|
|||||||
assert provider_confs
|
assert provider_confs
|
||||||
|
|
||||||
self._token_generator = OidcSessionTokenGenerator(hs)
|
self._token_generator = OidcSessionTokenGenerator(hs)
|
||||||
self._providers = {
|
self._providers: Dict[str, "OidcProvider"] = {
|
||||||
p.idp_id: OidcProvider(hs, self._token_generator, p) for p in provider_confs
|
p.idp_id: OidcProvider(hs, self._token_generator, p) for p in provider_confs
|
||||||
} # type: Dict[str, OidcProvider]
|
}
|
||||||
|
|
||||||
async def load_metadata(self) -> None:
|
async def load_metadata(self) -> None:
|
||||||
"""Validate the config and load the metadata from the remote endpoint.
|
"""Validate the config and load the metadata from the remote endpoint.
|
||||||
@ -178,7 +178,7 @@ class OidcHandler:
|
|||||||
# are two.
|
# are two.
|
||||||
|
|
||||||
for cookie_name, _ in _SESSION_COOKIES:
|
for cookie_name, _ in _SESSION_COOKIES:
|
||||||
session = request.getCookie(cookie_name) # type: Optional[bytes]
|
session: Optional[bytes] = request.getCookie(cookie_name)
|
||||||
if session is not None:
|
if session is not None:
|
||||||
break
|
break
|
||||||
else:
|
else:
|
||||||
@ -255,7 +255,7 @@ class OidcError(Exception):
|
|||||||
|
|
||||||
def __str__(self):
|
def __str__(self):
|
||||||
if self.error_description:
|
if self.error_description:
|
||||||
return "{}: {}".format(self.error, self.error_description)
|
return f"{self.error}: {self.error_description}"
|
||||||
return self.error
|
return self.error
|
||||||
|
|
||||||
|
|
||||||
@ -277,7 +277,7 @@ class OidcProvider:
|
|||||||
self._token_generator = token_generator
|
self._token_generator = token_generator
|
||||||
|
|
||||||
self._config = provider
|
self._config = provider
|
||||||
self._callback_url = hs.config.oidc_callback_url # type: str
|
self._callback_url: str = hs.config.oidc_callback_url
|
||||||
|
|
||||||
# Calculate the prefix for OIDC callback paths based on the public_baseurl.
|
# Calculate the prefix for OIDC callback paths based on the public_baseurl.
|
||||||
# We'll insert this into the Path= parameter of any session cookies we set.
|
# We'll insert this into the Path= parameter of any session cookies we set.
|
||||||
@ -290,7 +290,7 @@ class OidcProvider:
|
|||||||
self._scopes = provider.scopes
|
self._scopes = provider.scopes
|
||||||
self._user_profile_method = provider.user_profile_method
|
self._user_profile_method = provider.user_profile_method
|
||||||
|
|
||||||
client_secret = None # type: Union[None, str, JwtClientSecret]
|
client_secret: Optional[Union[str, JwtClientSecret]] = None
|
||||||
if provider.client_secret:
|
if provider.client_secret:
|
||||||
client_secret = provider.client_secret
|
client_secret = provider.client_secret
|
||||||
elif provider.client_secret_jwt_key:
|
elif provider.client_secret_jwt_key:
|
||||||
@ -305,7 +305,7 @@ class OidcProvider:
|
|||||||
provider.client_id,
|
provider.client_id,
|
||||||
client_secret,
|
client_secret,
|
||||||
provider.client_auth_method,
|
provider.client_auth_method,
|
||||||
) # type: ClientAuth
|
)
|
||||||
self._client_auth_method = provider.client_auth_method
|
self._client_auth_method = provider.client_auth_method
|
||||||
|
|
||||||
# cache of metadata for the identity provider (endpoint uris, mostly). This is
|
# cache of metadata for the identity provider (endpoint uris, mostly). This is
|
||||||
@ -324,7 +324,7 @@ class OidcProvider:
|
|||||||
self._allow_existing_users = provider.allow_existing_users
|
self._allow_existing_users = provider.allow_existing_users
|
||||||
|
|
||||||
self._http_client = hs.get_proxied_http_client()
|
self._http_client = hs.get_proxied_http_client()
|
||||||
self._server_name = hs.config.server_name # type: str
|
self._server_name: str = hs.config.server_name
|
||||||
|
|
||||||
# identifier for the external_ids table
|
# identifier for the external_ids table
|
||||||
self.idp_id = provider.idp_id
|
self.idp_id = provider.idp_id
|
||||||
@ -639,7 +639,7 @@ class OidcProvider:
|
|||||||
)
|
)
|
||||||
logger.warning(description)
|
logger.warning(description)
|
||||||
# Body was still valid JSON. Might be useful to log it for debugging.
|
# Body was still valid JSON. Might be useful to log it for debugging.
|
||||||
logger.warning("Code exchange response: {resp!r}".format(resp=resp))
|
logger.warning("Code exchange response: %r", resp)
|
||||||
raise OidcError("server_error", description)
|
raise OidcError("server_error", description)
|
||||||
|
|
||||||
return resp
|
return resp
|
||||||
@ -1217,10 +1217,12 @@ class OidcSessionData:
|
|||||||
ui_auth_session_id = attr.ib(type=str)
|
ui_auth_session_id = attr.ib(type=str)
|
||||||
|
|
||||||
|
|
||||||
UserAttributeDict = TypedDict(
|
class UserAttributeDict(TypedDict):
|
||||||
"UserAttributeDict",
|
localpart: Optional[str]
|
||||||
{"localpart": Optional[str], "display_name": Optional[str], "emails": List[str]},
|
display_name: Optional[str]
|
||||||
)
|
emails: List[str]
|
||||||
|
|
||||||
|
|
||||||
C = TypeVar("C")
|
C = TypeVar("C")
|
||||||
|
|
||||||
|
|
||||||
@ -1381,7 +1383,7 @@ class JinjaOidcMappingProvider(OidcMappingProvider[JinjaOidcMappingConfig]):
|
|||||||
if display_name == "":
|
if display_name == "":
|
||||||
display_name = None
|
display_name = None
|
||||||
|
|
||||||
emails = [] # type: List[str]
|
emails: List[str] = []
|
||||||
email = render_template_field(self._config.email_template)
|
email = render_template_field(self._config.email_template)
|
||||||
if email:
|
if email:
|
||||||
emails.append(email)
|
emails.append(email)
|
||||||
@ -1391,7 +1393,7 @@ class JinjaOidcMappingProvider(OidcMappingProvider[JinjaOidcMappingConfig]):
|
|||||||
)
|
)
|
||||||
|
|
||||||
async def get_extra_attributes(self, userinfo: UserInfo, token: Token) -> JsonDict:
|
async def get_extra_attributes(self, userinfo: UserInfo, token: Token) -> JsonDict:
|
||||||
extras = {} # type: Dict[str, str]
|
extras: Dict[str, str] = {}
|
||||||
for key, template in self._config.extra_attributes.items():
|
for key, template in self._config.extra_attributes.items():
|
||||||
try:
|
try:
|
||||||
extras[key] = template.render(user=userinfo).strip()
|
extras[key] = template.render(user=userinfo).strip()
|
||||||
|
@ -81,9 +81,9 @@ class PaginationHandler:
|
|||||||
self._server_name = hs.hostname
|
self._server_name = hs.hostname
|
||||||
|
|
||||||
self.pagination_lock = ReadWriteLock()
|
self.pagination_lock = ReadWriteLock()
|
||||||
self._purges_in_progress_by_room = set() # type: Set[str]
|
self._purges_in_progress_by_room: Set[str] = set()
|
||||||
# map from purge id to PurgeStatus
|
# map from purge id to PurgeStatus
|
||||||
self._purges_by_id = {} # type: Dict[str, PurgeStatus]
|
self._purges_by_id: Dict[str, PurgeStatus] = {}
|
||||||
self._event_serializer = hs.get_event_client_serializer()
|
self._event_serializer = hs.get_event_client_serializer()
|
||||||
|
|
||||||
self._retention_default_max_lifetime = hs.config.retention_default_max_lifetime
|
self._retention_default_max_lifetime = hs.config.retention_default_max_lifetime
|
||||||
|
@ -378,14 +378,14 @@ class WorkerPresenceHandler(BasePresenceHandler):
|
|||||||
|
|
||||||
# The number of ongoing syncs on this process, by user id.
|
# The number of ongoing syncs on this process, by user id.
|
||||||
# Empty if _presence_enabled is false.
|
# Empty if _presence_enabled is false.
|
||||||
self._user_to_num_current_syncs = {} # type: Dict[str, int]
|
self._user_to_num_current_syncs: Dict[str, int] = {}
|
||||||
|
|
||||||
self.notifier = hs.get_notifier()
|
self.notifier = hs.get_notifier()
|
||||||
self.instance_id = hs.get_instance_id()
|
self.instance_id = hs.get_instance_id()
|
||||||
|
|
||||||
# user_id -> last_sync_ms. Lists the users that have stopped syncing but
|
# user_id -> last_sync_ms. Lists the users that have stopped syncing but
|
||||||
# we haven't notified the presence writer of that yet
|
# we haven't notified the presence writer of that yet
|
||||||
self.users_going_offline = {} # type: Dict[str, int]
|
self.users_going_offline: Dict[str, int] = {}
|
||||||
|
|
||||||
self._bump_active_client = ReplicationBumpPresenceActiveTime.make_client(hs)
|
self._bump_active_client = ReplicationBumpPresenceActiveTime.make_client(hs)
|
||||||
self._set_state_client = ReplicationPresenceSetState.make_client(hs)
|
self._set_state_client = ReplicationPresenceSetState.make_client(hs)
|
||||||
@ -650,7 +650,7 @@ class PresenceHandler(BasePresenceHandler):
|
|||||||
|
|
||||||
# Set of users who have presence in the `user_to_current_state` that
|
# Set of users who have presence in the `user_to_current_state` that
|
||||||
# have not yet been persisted
|
# have not yet been persisted
|
||||||
self.unpersisted_users_changes = set() # type: Set[str]
|
self.unpersisted_users_changes: Set[str] = set()
|
||||||
|
|
||||||
hs.get_reactor().addSystemEventTrigger(
|
hs.get_reactor().addSystemEventTrigger(
|
||||||
"before",
|
"before",
|
||||||
@ -664,7 +664,7 @@ class PresenceHandler(BasePresenceHandler):
|
|||||||
|
|
||||||
# Keeps track of the number of *ongoing* syncs on this process. While
|
# Keeps track of the number of *ongoing* syncs on this process. While
|
||||||
# this is non zero a user will never go offline.
|
# this is non zero a user will never go offline.
|
||||||
self.user_to_num_current_syncs = {} # type: Dict[str, int]
|
self.user_to_num_current_syncs: Dict[str, int] = {}
|
||||||
|
|
||||||
# Keeps track of the number of *ongoing* syncs on other processes.
|
# Keeps track of the number of *ongoing* syncs on other processes.
|
||||||
# While any sync is ongoing on another process the user will never
|
# While any sync is ongoing on another process the user will never
|
||||||
@ -674,8 +674,8 @@ class PresenceHandler(BasePresenceHandler):
|
|||||||
# we assume that all the sync requests on that process have stopped.
|
# we assume that all the sync requests on that process have stopped.
|
||||||
# Stored as a dict from process_id to set of user_id, and a dict of
|
# Stored as a dict from process_id to set of user_id, and a dict of
|
||||||
# process_id to millisecond timestamp last updated.
|
# process_id to millisecond timestamp last updated.
|
||||||
self.external_process_to_current_syncs = {} # type: Dict[str, Set[str]]
|
self.external_process_to_current_syncs: Dict[str, Set[str]] = {}
|
||||||
self.external_process_last_updated_ms = {} # type: Dict[str, int]
|
self.external_process_last_updated_ms: Dict[str, int] = {}
|
||||||
|
|
||||||
self.external_sync_linearizer = Linearizer(name="external_sync_linearizer")
|
self.external_sync_linearizer = Linearizer(name="external_sync_linearizer")
|
||||||
|
|
||||||
@ -1581,9 +1581,7 @@ class PresenceEventSource:
|
|||||||
|
|
||||||
# The set of users that we're interested in and that have had a presence update.
|
# The set of users that we're interested in and that have had a presence update.
|
||||||
# We'll actually pull the presence updates for these users at the end.
|
# We'll actually pull the presence updates for these users at the end.
|
||||||
interested_and_updated_users = (
|
interested_and_updated_users: Union[Set[str], FrozenSet[str]] = set()
|
||||||
set()
|
|
||||||
) # type: Union[Set[str], FrozenSet[str]]
|
|
||||||
|
|
||||||
if from_key:
|
if from_key:
|
||||||
# First get all users that have had a presence update
|
# First get all users that have had a presence update
|
||||||
@ -1950,8 +1948,8 @@ async def get_interested_parties(
|
|||||||
A 2-tuple of `(room_ids_to_states, users_to_states)`,
|
A 2-tuple of `(room_ids_to_states, users_to_states)`,
|
||||||
with each item being a dict of `entity_name` -> `[UserPresenceState]`
|
with each item being a dict of `entity_name` -> `[UserPresenceState]`
|
||||||
"""
|
"""
|
||||||
room_ids_to_states = {} # type: Dict[str, List[UserPresenceState]]
|
room_ids_to_states: Dict[str, List[UserPresenceState]] = {}
|
||||||
users_to_states = {} # type: Dict[str, List[UserPresenceState]]
|
users_to_states: Dict[str, List[UserPresenceState]] = {}
|
||||||
for state in states:
|
for state in states:
|
||||||
room_ids = await store.get_rooms_for_user(state.user_id)
|
room_ids = await store.get_rooms_for_user(state.user_id)
|
||||||
for room_id in room_ids:
|
for room_id in room_ids:
|
||||||
@ -2063,12 +2061,12 @@ class PresenceFederationQueue:
|
|||||||
# stream_id, destinations, user_ids)`. We don't store the full states
|
# stream_id, destinations, user_ids)`. We don't store the full states
|
||||||
# for efficiency, and remote workers will already have the full states
|
# for efficiency, and remote workers will already have the full states
|
||||||
# cached.
|
# cached.
|
||||||
self._queue = [] # type: List[Tuple[int, int, Collection[str], Set[str]]]
|
self._queue: List[Tuple[int, int, Collection[str], Set[str]]] = []
|
||||||
|
|
||||||
self._next_id = 1
|
self._next_id = 1
|
||||||
|
|
||||||
# Map from instance name to current token
|
# Map from instance name to current token
|
||||||
self._current_tokens = {} # type: Dict[str, int]
|
self._current_tokens: Dict[str, int] = {}
|
||||||
|
|
||||||
if self._queue_presence_updates:
|
if self._queue_presence_updates:
|
||||||
self._clock.looping_call(self._clear_queue, self._CLEAR_ITEMS_EVERY_MS)
|
self._clock.looping_call(self._clear_queue, self._CLEAR_ITEMS_EVERY_MS)
|
||||||
@ -2168,7 +2166,7 @@ class PresenceFederationQueue:
|
|||||||
# handle the case where `from_token` stream ID has already been dropped.
|
# handle the case where `from_token` stream ID has already been dropped.
|
||||||
start_idx = max(from_token + 1 - self._next_id, -len(self._queue))
|
start_idx = max(from_token + 1 - self._next_id, -len(self._queue))
|
||||||
|
|
||||||
to_send = [] # type: List[Tuple[int, Tuple[str, str]]]
|
to_send: List[Tuple[int, Tuple[str, str]]] = []
|
||||||
limited = False
|
limited = False
|
||||||
new_id = upto_token
|
new_id = upto_token
|
||||||
for _, stream_id, destinations, user_ids in self._queue[start_idx:]:
|
for _, stream_id, destinations, user_ids in self._queue[start_idx:]:
|
||||||
@ -2216,7 +2214,7 @@ class PresenceFederationQueue:
|
|||||||
if not self._federation:
|
if not self._federation:
|
||||||
return
|
return
|
||||||
|
|
||||||
hosts_to_users = {} # type: Dict[str, Set[str]]
|
hosts_to_users: Dict[str, Set[str]] = {}
|
||||||
for row in rows:
|
for row in rows:
|
||||||
hosts_to_users.setdefault(row.destination, set()).add(row.user_id)
|
hosts_to_users.setdefault(row.destination, set()).add(row.user_id)
|
||||||
|
|
||||||
|
@ -197,7 +197,7 @@ class ProfileHandler(BaseHandler):
|
|||||||
400, "Displayname is too long (max %i)" % (MAX_DISPLAYNAME_LEN,)
|
400, "Displayname is too long (max %i)" % (MAX_DISPLAYNAME_LEN,)
|
||||||
)
|
)
|
||||||
|
|
||||||
displayname_to_set = new_displayname # type: Optional[str]
|
displayname_to_set: Optional[str] = new_displayname
|
||||||
if new_displayname == "":
|
if new_displayname == "":
|
||||||
displayname_to_set = None
|
displayname_to_set = None
|
||||||
|
|
||||||
@ -286,7 +286,7 @@ class ProfileHandler(BaseHandler):
|
|||||||
400, "Avatar URL is too long (max %i)" % (MAX_AVATAR_URL_LEN,)
|
400, "Avatar URL is too long (max %i)" % (MAX_AVATAR_URL_LEN,)
|
||||||
)
|
)
|
||||||
|
|
||||||
avatar_url_to_set = new_avatar_url # type: Optional[str]
|
avatar_url_to_set: Optional[str] = new_avatar_url
|
||||||
if new_avatar_url == "":
|
if new_avatar_url == "":
|
||||||
avatar_url_to_set = None
|
avatar_url_to_set = None
|
||||||
|
|
||||||
|
@ -30,6 +30,8 @@ class ReceiptsHandler(BaseHandler):
|
|||||||
|
|
||||||
self.server_name = hs.config.server_name
|
self.server_name = hs.config.server_name
|
||||||
self.store = hs.get_datastore()
|
self.store = hs.get_datastore()
|
||||||
|
self.event_auth_handler = hs.get_event_auth_handler()
|
||||||
|
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
|
|
||||||
# We only need to poke the federation sender explicitly if its on the
|
# We only need to poke the federation sender explicitly if its on the
|
||||||
@ -59,6 +61,19 @@ class ReceiptsHandler(BaseHandler):
|
|||||||
"""Called when we receive an EDU of type m.receipt from a remote HS."""
|
"""Called when we receive an EDU of type m.receipt from a remote HS."""
|
||||||
receipts = []
|
receipts = []
|
||||||
for room_id, room_values in content.items():
|
for room_id, room_values in content.items():
|
||||||
|
# If we're not in the room just ditch the event entirely. This is
|
||||||
|
# probably an old server that has come back and thinks we're still in
|
||||||
|
# the room (or we've been rejoined to the room by a state reset).
|
||||||
|
is_in_room = await self.event_auth_handler.check_host_in_room(
|
||||||
|
room_id, self.server_name
|
||||||
|
)
|
||||||
|
if not is_in_room:
|
||||||
|
logger.info(
|
||||||
|
"Ignoring receipt from %s as we're not in the room",
|
||||||
|
origin,
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
for receipt_type, users in room_values.items():
|
for receipt_type, users in room_values.items():
|
||||||
for user_id, user_values in users.items():
|
for user_id, user_values in users.items():
|
||||||
if get_domain_from_id(user_id) != origin:
|
if get_domain_from_id(user_id) != origin:
|
||||||
@ -83,8 +98,8 @@ class ReceiptsHandler(BaseHandler):
|
|||||||
|
|
||||||
async def _handle_new_receipts(self, receipts: List[ReadReceipt]) -> bool:
|
async def _handle_new_receipts(self, receipts: List[ReadReceipt]) -> bool:
|
||||||
"""Takes a list of receipts, stores them and informs the notifier."""
|
"""Takes a list of receipts, stores them and informs the notifier."""
|
||||||
min_batch_id = None # type: Optional[int]
|
min_batch_id: Optional[int] = None
|
||||||
max_batch_id = None # type: Optional[int]
|
max_batch_id: Optional[int] = None
|
||||||
|
|
||||||
for receipt in receipts:
|
for receipt in receipts:
|
||||||
res = await self.store.insert_receipt(
|
res = await self.store.insert_receipt(
|
||||||
|
@ -55,15 +55,12 @@ login_counter = Counter(
|
|||||||
["guest", "auth_provider"],
|
["guest", "auth_provider"],
|
||||||
)
|
)
|
||||||
|
|
||||||
LoginDict = TypedDict(
|
|
||||||
"LoginDict",
|
class LoginDict(TypedDict):
|
||||||
{
|
device_id: str
|
||||||
"device_id": str,
|
access_token: str
|
||||||
"access_token": str,
|
valid_until_ms: Optional[int]
|
||||||
"valid_until_ms": Optional[int],
|
refresh_token: Optional[str]
|
||||||
"refresh_token": Optional[str],
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class RegistrationHandler(BaseHandler):
|
class RegistrationHandler(BaseHandler):
|
||||||
@ -77,6 +74,7 @@ class RegistrationHandler(BaseHandler):
|
|||||||
self.identity_handler = self.hs.get_identity_handler()
|
self.identity_handler = self.hs.get_identity_handler()
|
||||||
self.ratelimiter = hs.get_registration_ratelimiter()
|
self.ratelimiter = hs.get_registration_ratelimiter()
|
||||||
self.macaroon_gen = hs.get_macaroon_generator()
|
self.macaroon_gen = hs.get_macaroon_generator()
|
||||||
|
self._account_validity_handler = hs.get_account_validity_handler()
|
||||||
self._server_notices_mxid = hs.config.server_notices_mxid
|
self._server_notices_mxid = hs.config.server_notices_mxid
|
||||||
self._server_name = hs.hostname
|
self._server_name = hs.hostname
|
||||||
|
|
||||||
@ -700,6 +698,10 @@ class RegistrationHandler(BaseHandler):
|
|||||||
shadow_banned=shadow_banned,
|
shadow_banned=shadow_banned,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Only call the account validity module(s) on the main process, to avoid
|
||||||
|
# repeating e.g. database writes on all of the workers.
|
||||||
|
await self._account_validity_handler.on_user_registration(user_id)
|
||||||
|
|
||||||
async def register_device(
|
async def register_device(
|
||||||
self,
|
self,
|
||||||
user_id: str,
|
user_id: str,
|
||||||
|
@ -87,7 +87,7 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
self.config = hs.config
|
self.config = hs.config
|
||||||
|
|
||||||
# Room state based off defined presets
|
# Room state based off defined presets
|
||||||
self._presets_dict = {
|
self._presets_dict: Dict[str, Dict[str, Any]] = {
|
||||||
RoomCreationPreset.PRIVATE_CHAT: {
|
RoomCreationPreset.PRIVATE_CHAT: {
|
||||||
"join_rules": JoinRules.INVITE,
|
"join_rules": JoinRules.INVITE,
|
||||||
"history_visibility": HistoryVisibility.SHARED,
|
"history_visibility": HistoryVisibility.SHARED,
|
||||||
@ -109,7 +109,7 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
"guest_can_join": False,
|
"guest_can_join": False,
|
||||||
"power_level_content_override": {},
|
"power_level_content_override": {},
|
||||||
},
|
},
|
||||||
} # type: Dict[str, Dict[str, Any]]
|
}
|
||||||
|
|
||||||
# Modify presets to selectively enable encryption by default per homeserver config
|
# Modify presets to selectively enable encryption by default per homeserver config
|
||||||
for preset_name, preset_config in self._presets_dict.items():
|
for preset_name, preset_config in self._presets_dict.items():
|
||||||
@ -127,9 +127,9 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
# If a user tries to update the same room multiple times in quick
|
# If a user tries to update the same room multiple times in quick
|
||||||
# succession, only process the first attempt and return its result to
|
# succession, only process the first attempt and return its result to
|
||||||
# subsequent requests
|
# subsequent requests
|
||||||
self._upgrade_response_cache = ResponseCache(
|
self._upgrade_response_cache: ResponseCache[Tuple[str, str]] = ResponseCache(
|
||||||
hs.get_clock(), "room_upgrade", timeout_ms=FIVE_MINUTES_IN_MS
|
hs.get_clock(), "room_upgrade", timeout_ms=FIVE_MINUTES_IN_MS
|
||||||
) # type: ResponseCache[Tuple[str, str]]
|
)
|
||||||
self._server_notices_mxid = hs.config.server_notices_mxid
|
self._server_notices_mxid = hs.config.server_notices_mxid
|
||||||
|
|
||||||
self.third_party_event_rules = hs.get_third_party_event_rules()
|
self.third_party_event_rules = hs.get_third_party_event_rules()
|
||||||
@ -377,10 +377,10 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
if not await self.spam_checker.user_may_create_room(user_id):
|
if not await self.spam_checker.user_may_create_room(user_id):
|
||||||
raise SynapseError(403, "You are not permitted to create rooms")
|
raise SynapseError(403, "You are not permitted to create rooms")
|
||||||
|
|
||||||
creation_content = {
|
creation_content: JsonDict = {
|
||||||
"room_version": new_room_version.identifier,
|
"room_version": new_room_version.identifier,
|
||||||
"predecessor": {"room_id": old_room_id, "event_id": tombstone_event_id},
|
"predecessor": {"room_id": old_room_id, "event_id": tombstone_event_id},
|
||||||
} # type: JsonDict
|
}
|
||||||
|
|
||||||
# Check if old room was non-federatable
|
# Check if old room was non-federatable
|
||||||
|
|
||||||
@ -618,15 +618,11 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
else:
|
else:
|
||||||
is_requester_admin = await self.auth.is_server_admin(requester.user)
|
is_requester_admin = await self.auth.is_server_admin(requester.user)
|
||||||
|
|
||||||
# Check whether the third party rules allows/changes the room create
|
# Let the third party rules modify the room creation config if needed, or abort
|
||||||
# request.
|
# the room creation entirely with an exception.
|
||||||
event_allowed = await self.third_party_event_rules.on_create_room(
|
await self.third_party_event_rules.on_create_room(
|
||||||
requester, config, is_requester_admin=is_requester_admin
|
requester, config, is_requester_admin=is_requester_admin
|
||||||
)
|
)
|
||||||
if not event_allowed:
|
|
||||||
raise SynapseError(
|
|
||||||
403, "You are not permitted to create rooms", Codes.FORBIDDEN
|
|
||||||
)
|
|
||||||
|
|
||||||
if not is_requester_admin and not await self.spam_checker.user_may_create_room(
|
if not is_requester_admin and not await self.spam_checker.user_may_create_room(
|
||||||
user_id
|
user_id
|
||||||
@ -936,7 +932,7 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
etype=EventTypes.PowerLevels, content=pl_content
|
etype=EventTypes.PowerLevels, content=pl_content
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
power_level_content = {
|
power_level_content: JsonDict = {
|
||||||
"users": {creator_id: 100},
|
"users": {creator_id: 100},
|
||||||
"users_default": 0,
|
"users_default": 0,
|
||||||
"events": {
|
"events": {
|
||||||
@ -955,7 +951,7 @@ class RoomCreationHandler(BaseHandler):
|
|||||||
"kick": 50,
|
"kick": 50,
|
||||||
"redact": 50,
|
"redact": 50,
|
||||||
"invite": 50,
|
"invite": 50,
|
||||||
} # type: JsonDict
|
}
|
||||||
|
|
||||||
if config["original_invitees_have_ops"]:
|
if config["original_invitees_have_ops"]:
|
||||||
for invitee in invite_list:
|
for invitee in invite_list:
|
||||||
|
@ -20,7 +20,12 @@ import msgpack
|
|||||||
from unpaddedbase64 import decode_base64, encode_base64
|
from unpaddedbase64 import decode_base64, encode_base64
|
||||||
|
|
||||||
from synapse.api.constants import EventTypes, HistoryVisibility, JoinRules
|
from synapse.api.constants import EventTypes, HistoryVisibility, JoinRules
|
||||||
from synapse.api.errors import Codes, HttpResponseException
|
from synapse.api.errors import (
|
||||||
|
Codes,
|
||||||
|
HttpResponseException,
|
||||||
|
RequestSendFailed,
|
||||||
|
SynapseError,
|
||||||
|
)
|
||||||
from synapse.types import JsonDict, ThirdPartyInstanceID
|
from synapse.types import JsonDict, ThirdPartyInstanceID
|
||||||
from synapse.util.caches.descriptors import cached
|
from synapse.util.caches.descriptors import cached
|
||||||
from synapse.util.caches.response_cache import ResponseCache
|
from synapse.util.caches.response_cache import ResponseCache
|
||||||
@ -42,12 +47,12 @@ class RoomListHandler(BaseHandler):
|
|||||||
def __init__(self, hs: "HomeServer"):
|
def __init__(self, hs: "HomeServer"):
|
||||||
super().__init__(hs)
|
super().__init__(hs)
|
||||||
self.enable_room_list_search = hs.config.enable_room_list_search
|
self.enable_room_list_search = hs.config.enable_room_list_search
|
||||||
self.response_cache = ResponseCache(
|
self.response_cache: ResponseCache[
|
||||||
hs.get_clock(), "room_list"
|
Tuple[Optional[int], Optional[str], Optional[ThirdPartyInstanceID]]
|
||||||
) # type: ResponseCache[Tuple[Optional[int], Optional[str], Optional[ThirdPartyInstanceID]]]
|
] = ResponseCache(hs.get_clock(), "room_list")
|
||||||
self.remote_response_cache = ResponseCache(
|
self.remote_response_cache: ResponseCache[
|
||||||
hs.get_clock(), "remote_room_list", timeout_ms=30 * 1000
|
Tuple[str, Optional[int], Optional[str], bool, Optional[str]]
|
||||||
) # type: ResponseCache[Tuple[str, Optional[int], Optional[str], bool, Optional[str]]]
|
] = ResponseCache(hs.get_clock(), "remote_room_list", timeout_ms=30 * 1000)
|
||||||
|
|
||||||
async def get_local_public_room_list(
|
async def get_local_public_room_list(
|
||||||
self,
|
self,
|
||||||
@ -134,10 +139,10 @@ class RoomListHandler(BaseHandler):
|
|||||||
if since_token:
|
if since_token:
|
||||||
batch_token = RoomListNextBatch.from_token(since_token)
|
batch_token = RoomListNextBatch.from_token(since_token)
|
||||||
|
|
||||||
bounds = (
|
bounds: Optional[Tuple[int, str]] = (
|
||||||
batch_token.last_joined_members,
|
batch_token.last_joined_members,
|
||||||
batch_token.last_room_id,
|
batch_token.last_room_id,
|
||||||
) # type: Optional[Tuple[int, str]]
|
)
|
||||||
forwards = batch_token.direction_is_forward
|
forwards = batch_token.direction_is_forward
|
||||||
has_batch_token = True
|
has_batch_token = True
|
||||||
else:
|
else:
|
||||||
@ -177,7 +182,7 @@ class RoomListHandler(BaseHandler):
|
|||||||
|
|
||||||
results = [build_room_entry(r) for r in results]
|
results = [build_room_entry(r) for r in results]
|
||||||
|
|
||||||
response = {} # type: JsonDict
|
response: JsonDict = {}
|
||||||
num_results = len(results)
|
num_results = len(results)
|
||||||
if limit is not None:
|
if limit is not None:
|
||||||
more_to_come = num_results == probing_limit
|
more_to_come = num_results == probing_limit
|
||||||
@ -378,7 +383,11 @@ class RoomListHandler(BaseHandler):
|
|||||||
):
|
):
|
||||||
logger.debug("Falling back to locally-filtered /publicRooms")
|
logger.debug("Falling back to locally-filtered /publicRooms")
|
||||||
else:
|
else:
|
||||||
raise # Not an error that should trigger a fallback.
|
# Not an error that should trigger a fallback.
|
||||||
|
raise SynapseError(502, "Failed to fetch room list")
|
||||||
|
except RequestSendFailed:
|
||||||
|
# Not an error that should trigger a fallback.
|
||||||
|
raise SynapseError(502, "Failed to fetch room list")
|
||||||
|
|
||||||
# if we reach this point, then we fall back to the situation where
|
# if we reach this point, then we fall back to the situation where
|
||||||
# we currently don't support searching across federation, so we have
|
# we currently don't support searching across federation, so we have
|
||||||
@ -417,6 +426,7 @@ class RoomListHandler(BaseHandler):
|
|||||||
repl_layer = self.hs.get_federation_client()
|
repl_layer = self.hs.get_federation_client()
|
||||||
if search_filter:
|
if search_filter:
|
||||||
# We can't cache when asking for search
|
# We can't cache when asking for search
|
||||||
|
try:
|
||||||
return await repl_layer.get_public_rooms(
|
return await repl_layer.get_public_rooms(
|
||||||
server_name,
|
server_name,
|
||||||
limit=limit,
|
limit=limit,
|
||||||
@ -425,6 +435,8 @@ class RoomListHandler(BaseHandler):
|
|||||||
include_all_networks=include_all_networks,
|
include_all_networks=include_all_networks,
|
||||||
third_party_instance_id=third_party_instance_id,
|
third_party_instance_id=third_party_instance_id,
|
||||||
)
|
)
|
||||||
|
except (RequestSendFailed, HttpResponseException):
|
||||||
|
raise SynapseError(502, "Failed to fetch room list")
|
||||||
|
|
||||||
key = (
|
key = (
|
||||||
server_name,
|
server_name,
|
||||||
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user