Merge branch 'master' into develop

2024-10-01 01:36:05 -04:00 · 2019-02-05 18:44:49 +00:00 · 2019-02-05 18:44:49 +00:00 · 2475434080
commit 2475434080
parent 627ecd358e 3bd9daf4b8
18 changed files with 1051 additions and 579 deletions
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@ -4,8 +4,8 @@ jobs:
    machine: true
    steps:
      - checkout
-      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} .
+      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG}-py2 .
-      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG}-py3 --build-arg PYTHON_VERSION=3.6 .
+      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} -t matrixdotorg/synapse:${CIRCLE_TAG}-py3 --build-arg PYTHON_VERSION=3.6 .
      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
      - run: docker push matrixdotorg/synapse:${CIRCLE_TAG}
      - run: docker push matrixdotorg/synapse:${CIRCLE_TAG}-py3
@ -13,8 +13,8 @@ jobs:
    machine: true
    steps:
      - checkout
-      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest .
+      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest-py2 .
-      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest-py3 --build-arg PYTHON_VERSION=3.6 .
+      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest -t matrixdotorg/synapse:latest-py3 --build-arg PYTHON_VERSION=3.6 .
      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
      - run: docker push matrixdotorg/synapse:latest
      - run: docker push matrixdotorg/synapse:latest-py3
--- a/CHANGES.md
+++ b/CHANGES.md
@ -1,41 +1,5 @@
-Synapse 0.99.0rc3 (2019-01-31)
+Synapse 0.99.0 (2019-02-05)
-==============================
+===========================
 Bugfixes
 --------
 - Fix infinite loop when an event is redacted in a v3 room ([\#4535](https://github.com/matrix-org/synapse/issues/4535))
 Improved Documentation
 ----------------------
 - Update debian installation instructions ([\#4526](https://github.com/matrix-org/synapse/issues/4526))
 Internal Changes
 ----------------
 - Add some debug for membership syncing issues ([\#4538](https://github.com/matrix-org/synapse/issues/4538))
 Synapse 0.99.0rc2 (2019-01-30)
 ==============================
 Bugfixes
 --------
 - Fix bug when rejecting remote invites. ([\#4527](https://github.com/matrix-org/synapse/issues/4527))
 - Fix incorrect rendering of server capabilities. ([81b7e7eed](https://github.com/matrix-org/synapse/commit/81b7e7eed323f55d6550e7a270a9dc2c4c7b0fe0))
 Improved Documentation
 ----------------------
 - Add documentation on enabling ACME support when upgrading to v0.99. ([\#4528](https://github.com/matrix-org/synapse/issues/4528))
 Synapse 0.99.0rc1 (2019-01-30)
 ==============================
 Synapse v0.99.x is a precursor to the upcoming Synapse v1.0 release. It contains foundational changes to room architecture and the federation security model necessary to support the upcoming r0 release of the Server to Server API.
@ -43,15 +7,15 @@ Features
 --------
 - Synapse's cipher string has been updated to require ECDH key exchange. Configuring and generating dh_params is no longer required, and they will be ignored. ([\#4229](https://github.com/matrix-org/synapse/issues/4229))
- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt). ([\#4384](https://github.com/matrix-org/synapse/issues/4384), [\#4492](https://github.com/matrix-org/synapse/issues/4492), [\#4525](https://github.com/matrix-org/synapse/issues/4525))
+- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt). ([\#4384](https://github.com/matrix-org/synapse/issues/4384), [\#4492](https://github.com/matrix-org/synapse/issues/4492), [\#4525](https://github.com/matrix-org/synapse/issues/4525), [\#4572](https://github.com/matrix-org/synapse/issues/4572), [\#4564](https://github.com/matrix-org/synapse/issues/4564), [\#4566](https://github.com/matrix-org/synapse/issues/4566), [\#4547](https://github.com/matrix-org/synapse/issues/4547), [\#4557](https://github.com/matrix-org/synapse/issues/4557))
- Implement MSC1708 (.well-known routing for server-server federation) ([\#4408](https://github.com/matrix-org/synapse/issues/4408), [\#4409](https://github.com/matrix-org/synapse/issues/4409), [\#4426](https://github.com/matrix-org/synapse/issues/4426), [\#4427](https://github.com/matrix-org/synapse/issues/4427), [\#4428](https://github.com/matrix-org/synapse/issues/4428), [\#4464](https://github.com/matrix-org/synapse/issues/4464), [\#4468](https://github.com/matrix-org/synapse/issues/4468), [\#4487](https://github.com/matrix-org/synapse/issues/4487), [\#4488](https://github.com/matrix-org/synapse/issues/4488), [\#4489](https://github.com/matrix-org/synapse/issues/4489), [\#4497](https://github.com/matrix-org/synapse/issues/4497), [\#4511](https://github.com/matrix-org/synapse/issues/4511), [\#4516](https://github.com/matrix-org/synapse/issues/4516), [\#4520](https://github.com/matrix-org/synapse/issues/4520), [\#4521](https://github.com/matrix-org/synapse/issues/4521))
+- Implement MSC1708 (.well-known routing for server-server federation) ([\#4408](https://github.com/matrix-org/synapse/issues/4408), [\#4409](https://github.com/matrix-org/synapse/issues/4409), [\#4426](https://github.com/matrix-org/synapse/issues/4426), [\#4427](https://github.com/matrix-org/synapse/issues/4427), [\#4428](https://github.com/matrix-org/synapse/issues/4428), [\#4464](https://github.com/matrix-org/synapse/issues/4464), [\#4468](https://github.com/matrix-org/synapse/issues/4468), [\#4487](https://github.com/matrix-org/synapse/issues/4487), [\#4488](https://github.com/matrix-org/synapse/issues/4488), [\#4489](https://github.com/matrix-org/synapse/issues/4489), [\#4497](https://github.com/matrix-org/synapse/issues/4497), [\#4511](https://github.com/matrix-org/synapse/issues/4511), [\#4516](https://github.com/matrix-org/synapse/issues/4516), [\#4520](https://github.com/matrix-org/synapse/issues/4520), [\#4521](https://github.com/matrix-org/synapse/issues/4521), [\#4539](https://github.com/matrix-org/synapse/issues/4539), [\#4542](https://github.com/matrix-org/synapse/issues/4542), [\#4544](https://github.com/matrix-org/synapse/issues/4544))
 - Search now includes results from predecessor rooms after a room upgrade. ([\#4415](https://github.com/matrix-org/synapse/issues/4415))
 - Config option to disable requesting MSISDN on registration. ([\#4423](https://github.com/matrix-org/synapse/issues/4423))
 - Add a metric for tracking event stream position of the user directory. ([\#4445](https://github.com/matrix-org/synapse/issues/4445))
- Support exposing server capabilities in CS API (MSC1753, MSC1804) ([\#4472](https://github.com/matrix-org/synapse/issues/4472))
+- Support exposing server capabilities in CS API (MSC1753, MSC1804) ([\#4472](https://github.com/matrix-org/synapse/issues/4472), [81b7e7eed](https://github.com/matrix-org/synapse/commit/81b7e7eed323f55d6550e7a270a9dc2c4c7b0fe0)))
- Add support for room version 3 ([\#4483](https://github.com/matrix-org/synapse/issues/4483), [\#4499](https://github.com/matrix-org/synapse/issues/4499), [\#4515](https://github.com/matrix-org/synapse/issues/4515), [\#4523](https://github.com/matrix-org/synapse/issues/4523))
+- Add support for room version 3 ([\#4483](https://github.com/matrix-org/synapse/issues/4483), [\#4499](https://github.com/matrix-org/synapse/issues/4499), [\#4515](https://github.com/matrix-org/synapse/issues/4515), [\#4523](https://github.com/matrix-org/synapse/issues/4523), [\#4535](https://github.com/matrix-org/synapse/issues/4535))
 - Synapse will now reload TLS certificates from disk upon SIGHUP. ([\#4495](https://github.com/matrix-org/synapse/issues/4495), [\#4524](https://github.com/matrix-org/synapse/issues/4524))
-
+- The matrixdotorg/synapse Docker images now use Python 3 by default. ([\#4558](https://github.com/matrix-org/synapse/issues/4558))
 Bugfixes
 --------
@ -60,7 +24,7 @@ Bugfixes
 - Fix typo in ALL_USER_TYPES definition to ensure type is a tuple ([\#4392](https://github.com/matrix-org/synapse/issues/4392))
 - Fix high CPU usage due to remote devicelist updates ([\#4397](https://github.com/matrix-org/synapse/issues/4397))
 - Fix potential bug where creating or joining a room could fail ([\#4404](https://github.com/matrix-org/synapse/issues/4404))
- Fix bug when rejecting remote invites ([\#4405](https://github.com/matrix-org/synapse/issues/4405))
+- Fix bug when rejecting remote invites ([\#4405](https://github.com/matrix-org/synapse/issues/4405), [\#4527](https://github.com/matrix-org/synapse/issues/4527))
 - Fix incorrect logcontexts after a Deferred was cancelled ([\#4407](https://github.com/matrix-org/synapse/issues/4407))
 - Ensure encrypted room state is persisted across room upgrades. ([\#4411](https://github.com/matrix-org/synapse/issues/4411))
 - Copy over whether a room is a direct message and any associated room tags on room upgrade. ([\#4412](https://github.com/matrix-org/synapse/issues/4412))
@ -78,6 +42,12 @@ Deprecations and Removals
 - Synapse no longer generates self-signed TLS certificates when generating a configuration file. ([\#4509](https://github.com/matrix-org/synapse/issues/4509))
 Improved Documentation
 ----------------------
 - Update debian installation instructions ([\#4526](https://github.com/matrix-org/synapse/issues/4526))
 Internal Changes
 ----------------
@ -100,6 +70,8 @@ Internal Changes
 - Make it possible to set the log level for tests via an environment variable ([\#4506](https://github.com/matrix-org/synapse/issues/4506))
 - Reduce the log level of linearizer lock acquirement to DEBUG. ([\#4507](https://github.com/matrix-org/synapse/issues/4507))
 - Fix code to comply with linting in PyFlakes 3.7.1. ([\#4519](https://github.com/matrix-org/synapse/issues/4519))
 - Add some debug for membership syncing issues ([\#4538](https://github.com/matrix-org/synapse/issues/4538))
 - Docker: only copy what we need to the build image ([\#4562](https://github.com/matrix-org/synapse/issues/4562))
 Synapse 0.34.1.1 (2019-01-11)
--- a/INSTALL.md
+++ b/INSTALL.md
@ -0,0 +1,410 @@
 * [Installing Synapse](#installing-synapse)
  * [Installing from source](#installing-from-source)
    * [Platform-Specific Instructions](#platform-specific-instructions)
    * [Troubleshooting Installation](#troubleshooting-installation)
  * [Prebuilt packages](#prebuilt-packages)
 * [Setting up Synapse](#setting-up-synapse)
  * [TLS certificates](#tls-certificates)
  * [Registering a user](#registering-a-user)
  * [Setting up a TURN server](#setting-up-a-turn-server)
  * [URL previews](#url-previews)
 # Installing Synapse
 ## Installing from source
 (Prebuilt packages are available for some platforms - see [Prebuilt packages](#prebuilt-packages).)
 System requirements:
 - POSIX-compliant system (tested on Linux & OS X)
 - Python 3.5, 3.6, 3.7, or 2.7
 - At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
 Synapse is written in Python but some of the libraries it uses are written in
 C. So before we can install Synapse itself we need a working C compiler and the
 header files for Python C extensions. See [Platform-Specific
 Instructions](#platform-specific-instructions) for information on installing
 these on various platforms.
 To install the Synapse homeserver run:
 ```
 mkdir -p ~/synapse
 virtualenv -p python3 ~/synapse/env
 source ~/synapse/env/bin/activate
 pip install --upgrade pip
 pip install --upgrade setuptools
 pip install matrix-synapse[all]
 ```
 This will download Synapse from [PyPI](https://pypi.org/project/matrix-synapse)
 and install it, along with the python libraries it uses, into a virtual environment
 under `~/synapse/env`.  Feel free to pick a different directory if you
 prefer.
 This Synapse installation can then be later upgraded by using pip again with the
 update flag:
 ```
 source ~/synapse/env/bin/activate
 pip install -U matrix-synapse[all]
 ```
 Before you can start Synapse, you will need to generate a configuration
 file. To do this, run (in your virtualenv, as before)::
 ```
 cd ~/synapse
 python -m synapse.app.homeserver \
    --server-name my.domain.name \
    --config-path homeserver.yaml \
    --generate-config \
    --report-stats=[yes|no]
 ```
 ... substituting an appropriate value for `--server-name`. The server name
 determines the "domain" part of user-ids for users on your server: these will
 all be of the format `@user:my.domain.name`. It also determines how other
 matrix servers will reach yours for Federation. For a test configuration,
 set this to the hostname of your server. For a more production-ready setup, you
 will probably want to specify your domain (`example.com`) rather than a
 matrix-specific hostname here (in the same way that your email address is
 probably `user@example.com` rather than `user@email.example.com`) - but
 doing so may require more advanced setup. - see [Setting up Federation](README.rst#setting-up-federation). Beware that the server name cannot be changed later.
 This command will generate you a config file that you can then customise, but it will
 also generate a set of keys for you. These keys will allow your Home Server to
 identify itself to other Home Servers, so don't lose or delete them. It would be
 wise to back them up somewhere safe. (If, for whatever reason, you do need to
 change your Home Server's keys, you may find that other Home Servers have the
 old key cached. If you update the signing key, you should change the name of the
 key in the `<server name>.signing.key` file (the second word) to something
 different. See the
 [spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys)
 for more information on key management.)
 You will need to give Synapse a TLS certficate before it will start - see [TLS
 certificates](#tls-certificates).
 To actually run your new homeserver, pick a working directory for Synapse to
 run (e.g. `~/synapse`), and::
    cd ~/synapse
    source env/bin/activate
    synctl start
 ### Platform-Specific Instructions
 #### Debian/Ubuntu/Raspbian
 Installing prerequisites on Ubuntu or Debian:
 ```
 sudo apt-get install build-essential python3-dev libffi-dev \
                     python-pip python-setuptools sqlite3 \
                     libssl-dev python-virtualenv libjpeg-dev libxslt1-dev
 ```
 #### ArchLinux
 Installing prerequisites on ArchLinux:
 ```
 sudo pacman -S base-devel python python-pip \
               python-setuptools python-virtualenv sqlite3
 ```
 #### CentOS/Fedora
 Installing prerequisites on CentOS 7 or Fedora 25:
 ```
 sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
                 lcms2-devel libwebp-devel tcl-devel tk-devel redhat-rpm-config \
                 python-virtualenv libffi-devel openssl-devel
 sudo yum groupinstall "Development Tools"
 ```
 #### Mac OS X
 Installing prerequisites on Mac OS X:
 ```
 xcode-select --install
 sudo easy_install pip
 sudo pip install virtualenv
 brew install pkg-config libffi
 ```
 #### OpenSUSE
 Installing prerequisites on openSUSE:
 ```
 sudo zypper in -t pattern devel_basis
 sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
               python-devel libffi-devel libopenssl-devel libjpeg62-devel
 ```
 #### OpenBSD
 Installing prerequisites on OpenBSD:
 ```
 doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
              libxslt jpeg
 ```
 There is currently no port for OpenBSD. Additionally, OpenBSD's security
 settings require a slightly more difficult installation process.
 XXX: I suspect this is out of date.
 1. Create a new directory in `/usr/local` called `_synapse`. Also, create a
   new user called `_synapse` and set that directory as the new user's home.
   This is required because, by default, OpenBSD only allows binaries which need
   write and execute permissions on the same memory space to be run from
   `/usr/local`.
 2. `su` to the new `_synapse` user and change to their home directory.
 3. Create a new virtualenv: `virtualenv -p python2.7 ~/.synapse`
 4. Source the virtualenv configuration located at
   `/usr/local/_synapse/.synapse/bin/activate`. This is done in `ksh` by
   using the `.` command, rather than `bash`'s `source`.
 5. Optionally, use `pip` to install `lxml`, which Synapse needs to parse
   webpages for their titles.
 6. Use `pip` to install this repository: `pip install matrix-synapse`
 7. Optionally, change `_synapse`'s shell to `/bin/false` to reduce the
   chance of a compromised Synapse server being used to take over your box.
 After this, you may proceed with the rest of the install directions.
 #### Windows
 If you wish to run or develop Synapse on Windows, the Windows Subsystem For
 Linux provides a Linux environment on Windows 10 which is capable of using the
 Debian, Fedora, or source installation methods. More information about WSL can
 be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
 Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
 for Windows Server.
 ### Troubleshooting Installation
 XXX a bunch of this is no longer relevant.
 Synapse requires pip 8 or later, so if your OS provides too old a version you
 may need to manually upgrade it::
    sudo pip install --upgrade pip
 Installing may fail with `Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)`.
 You can fix this by manually upgrading pip and virtualenv::
    sudo pip install --upgrade virtualenv
 You can next rerun `virtualenv -p python3 synapse` to update the virtual env.
 Installing may fail during installing virtualenv with `InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.`
 You can fix this  by manually installing ndg-httpsclient::
    pip install --upgrade ndg-httpsclient
 Installing may fail with `mock requires setuptools>=17.1. Aborting installation`.
 You can fix this by upgrading setuptools::
    pip install --upgrade setuptools
 If pip crashes mid-installation for reason (e.g. lost terminal), pip may
 refuse to run until you remove the temporary installation directory it
 created. To reset the installation::
    rm -rf /tmp/pip_install_matrix
 pip seems to leak *lots* of memory during installation.  For instance, a Linux
 host with 512MB of RAM may run out of memory whilst installing Twisted.  If this
 happens, you will have to individually install the dependencies which are
 failing, e.g.::
    pip install twisted
 ## Prebuilt packages
 As an alternative to installing from source, prebuilt packages are available
 for a number of platforms.
 ### Docker images and Ansible playbooks
 There is an offical synapse image available at
 https://hub.docker.com/r/matrixdotorg/synapse which can be used with
 the docker-compose file available at [contrib/docker](contrib/docker). Further information on
 this including configuration options is available in the README on
 hub.docker.com.
 Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
 Dockerfile to automate a synapse server in a single Docker image, at
 https://hub.docker.com/r/avhost/docker-matrix/tags/
 Slavi Pantaleev has created an Ansible playbook,
 which installs the offical Docker image of Matrix Synapse
 along with many other Matrix-related services (Postgres database, riot-web, coturn, mxisd, SSL support, etc.).
 For more details, see
 https://github.com/spantaleev/matrix-docker-ansible-deploy
 ### Debian/Ubuntu
 #### Matrix.org packages
 Matrix.org provides Debian/Ubuntu packages of the latest stable version of
 Synapse via https://matrix.org/packages/debian/. To use them:
 ```
 sudo apt install -y lsb-release curl apt-transport-https
 echo "deb https://matrix.org/packages/debian `lsb_release -cs` main" |
    sudo tee /etc/apt/sources.list.d/matrix-org.list
 curl "https://matrix.org/packages/debian/repo-key.asc" |
    sudo apt-key add -
 sudo apt update
 sudo apt install matrix-synapse-py3
 ```
 #### Downstream Debian/Ubuntu packages
 For `buster` and `sid`, Synapse is available in the Debian repositories and
 it should be possible to install it with simply:
 ```
    sudo apt install matrix-synapse
 ```
 There is also a version of `matrix-synapse` in `stretch-backports`. Please see
 the [Debian documentation on
 backports](https://backports.debian.org/Instructions/) for information on how
 to use them.
 We do not recommend using the packages in downstream Ubuntu at this time, as
 they are old and suffer from known security vulnerabilities.
 ### Fedora
 Synapse is in the Fedora repositories as `matrix-synapse`:
 ```
 sudo dnf install matrix-synapse
 ```
 Oleg Girko provides Fedora RPMs at
 https://obs.infoserver.lv/project/monitor/matrix-synapse
 ### OpenSUSE
 Synapse is in the OpenSUSE repositories as `matrix-synapse`:
 ```
 sudo zypper install matrix-synapse
 ```
 ### SUSE Linux Enterprise Server
 Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
 https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/
 ### ArchLinux
 The quickest way to get up and running with ArchLinux is probably with the community package
 https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of
 the necessary dependencies.
 pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):
 ```
 sudo pip install --upgrade pip
 ```
 If you encounter an error with lib bcrypt causing an Wrong ELF Class:
 ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
 compile it under the right architecture. (This should not be needed if
 installing under virtualenv):
 ```
 sudo pip uninstall py-bcrypt
 sudo pip install py-bcrypt
 ```
 ### FreeBSD
 Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
 - Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
 - Packages: `pkg install py27-matrix-synapse`
 ### NixOS
 Robin Lambertz has packaged Synapse for NixOS at:
 https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix
 # Setting up Synapse
 Once you have installed synapse as above, you will need to configure it.
 ## TLS certificates
 The default configuration exposes two HTTP ports: 8008 and 8448. Port 8008 is
 configured without TLS; it should be behind a reverse proxy for TLS/SSL
 termination on port 443 which in turn should be used for clients. Port 8448
 is configured to use TLS for Federation with a self-signed or verified
 certificate, but please be aware that a valid certificate will be required in
 Synapse v1.0. Instructions for having Synapse automatically provision and renew federation certificates through ACME can be found at [ACME.md](docs/ACME.md).
 If you would like to use your own certificates, you can do so by changing
 `tls_certificate_path` and `tls_private_key_path` in `homeserver.yaml`;
 alternatively, you can use a reverse-proxy. Apart from port 8448 using TLS,
 both ports are the same in the default configuration.
 ## Registering a user
 You will need at least one user on your server in order to use a Matrix
 client. Users can be registered either via a Matrix client, or via a
 commandline script.
 To get started, it is easiest to use the command line to register new
 users. This can be done as follows:
 ```
 $ source ~/synapse/env/bin/activate
 $ synctl start # if not already running
 $ register_new_matrix_user -c homeserver.yaml https://localhost:8448
 New user localpart: erikj
 Password:
 Confirm password:
 Make admin [no]:
 Success!
 ```
 This process uses a setting `registration_shared_secret` in
 `homeserver.yaml`, which is shared between Synapse itself and the
 `register_new_matrix_user` script. It doesn't matter what it is (a random
 value is generated by `--generate-config`), but it should be kept secret, as
 anyone with knowledge of it can register users on your server even if
 `enable_registration` is `false`.
 ## Setting up a TURN server
 For reliable VoIP calls to be routed via this homeserver, you MUST configure
 a TURN server.  See [docs/turn-howto.rst](docs/turn-howto.rst) for details.
 ## URL previews
 Synapse includes support for previewing URLs, which is disabled by default.  To
 turn it on you must enable the `url_preview_enabled: True` config parameter
 and explicitly specify the IP ranges that Synapse is not allowed to spider for
 previewing in the `url_preview_ip_range_blacklist` configuration parameter.
 This is critical from a security perspective to stop arbitrary Matrix users
 spidering 'internal' URLs on your network.  At the very least we recommend that
 your loopback and RFC1918 IP addresses are blacklisted.
 This also requires the optional lxml and netaddr python dependencies to be
 installed.  This in turn requires the libxml2 library to be available - on
 Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for
 your OS.
--- a/README.rst
+++ b/README.rst
@ -81,194 +81,8 @@ Thanks for using Matrix!
 Synapse Installation
 ====================
-Synapse is the reference Python/Twisted Matrix homeserver implementation.
+For details on how to install synapse, see `<INSTALL.md>`_.
 System requirements:
 - POSIX-compliant system (tested on Linux & OS X)
 - Python 3.5, 3.6, 3.7, or 2.7
 - At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
 Installing from source
 ----------------------
 (Prebuilt packages are available for some platforms - see `Platform-Specific
 Instructions`_.)
 Synapse is written in Python but some of the libraries it uses are written in
 C. So before we can install Synapse itself we need a working C compiler and the
 header files for Python C extensions.
 Installing prerequisites on Ubuntu or Debian::
    sudo apt-get install build-essential python3-dev libffi-dev \
                         python-pip python-setuptools sqlite3 \
                         libssl-dev python-virtualenv libjpeg-dev libxslt1-dev
 Installing prerequisites on ArchLinux::
    sudo pacman -S base-devel python python-pip \
                   python-setuptools python-virtualenv sqlite3
 Installing prerequisites on CentOS 7 or Fedora 25::
    sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
                     lcms2-devel libwebp-devel tcl-devel tk-devel redhat-rpm-config \
                     python-virtualenv libffi-devel openssl-devel
    sudo yum groupinstall "Development Tools"
 Installing prerequisites on Mac OS X::
    xcode-select --install
    sudo easy_install pip
    sudo pip install virtualenv
    brew install pkg-config libffi
 Installing prerequisites on Raspbian::
    sudo apt-get install build-essential python3-dev libffi-dev \
                         python-pip python-setuptools sqlite3 \
                         libssl-dev python-virtualenv libjpeg-dev
 Installing prerequisites on openSUSE::
    sudo zypper in -t pattern devel_basis
    sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
                   python-devel libffi-devel libopenssl-devel libjpeg62-devel
 Installing prerequisites on OpenBSD::
    doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
                 libxslt jpeg
 To install the Synapse homeserver run::
    mkdir -p ~/synapse
    virtualenv -p python3 ~/synapse/env
    source ~/synapse/env/bin/activate
    pip install --upgrade pip
    pip install --upgrade setuptools
    pip install matrix-synapse[all]
 This installs Synapse, along with the libraries it uses, into a virtual
 environment under ``~/synapse/env``.  Feel free to pick a different directory
 if you prefer.
 This Synapse installation can then be later upgraded by using pip again with the
 update flag::
    source ~/synapse/env/bin/activate
    pip install -U matrix-synapse[all]
 In case of problems, please see the _`Troubleshooting` section below.
 There is an offical synapse image available at
 https://hub.docker.com/r/matrixdotorg/synapse/tags/ which can be used with
 the docker-compose file available at `contrib/docker <contrib/docker>`_. Further information on
 this including configuration options is available in the README on
 hub.docker.com.
 Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
 Dockerfile to automate a synapse server in a single Docker image, at
 https://hub.docker.com/r/avhost/docker-matrix/tags/
 Slavi Pantaleev has created an Ansible playbook,
 which installs the offical Docker image of Matrix Synapse
 along with many other Matrix-related services (Postgres database, riot-web, coturn, mxisd, SSL support, etc.).
 For more details, see
 https://github.com/spantaleev/matrix-docker-ansible-deploy
 Configuring Synapse
 -------------------
 Before you can start Synapse, you will need to generate a configuration
 file. To do this, run (in your virtualenv, as before)::
    cd ~/synapse
    python -m synapse.app.homeserver \
        --server-name my.domain.name \
        --config-path homeserver.yaml \
        --generate-config \
        --report-stats=[yes|no]
 ... substituting an appropriate value for ``--server-name``. The server name
 determines the "domain" part of user-ids for users on your server: these will
 all be of the format ``@user:my.domain.name``. It also determines how other
 matrix servers will reach yours for `Federation`_. For a test configuration,
 set this to the hostname of your server. For a more production-ready setup, you
 will probably want to specify your domain (``example.com``) rather than a
 matrix-specific hostname here (in the same way that your email address is
 probably ``user@example.com`` rather than ``user@email.example.com``) - but
 doing so may require more advanced setup - see `Setting up
 Federation`_. Beware that the server name cannot be changed later.
 This command will generate you a config file that you can then customise, but it will
 also generate a set of keys for you. These keys will allow your Home Server to
 identify itself to other Home Servers, so don't lose or delete them. It would be
 wise to back them up somewhere safe. (If, for whatever reason, you do need to
 change your Home Server's keys, you may find that other Home Servers have the
 old key cached. If you update the signing key, you should change the name of the
 key in the ``<server name>.signing.key`` file (the second word) to something
 different. See `the spec`__ for more information on key management.)
 .. __: `key_management`_
 The default configuration exposes two HTTP ports: 8008 and 8448. Port 8008 is
 configured without TLS; it should be behind a reverse proxy for TLS/SSL
 termination on port 443 which in turn should be used for clients. Port 8448
 is configured to use TLS with a self-signed certificate. If you would like
 to do initial test with a client without having to setup a reverse proxy,
 you can temporarly use another certificate. (Note that a self-signed
 certificate is fine for `Federation`_). You can do so by changing
 ``tls_certificate_path`` and ``tls_private_key_path``
 in ``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure
 to read `Using a reverse proxy with Synapse`_ when doing so.
 Apart from port 8448 using TLS, both ports are the same in the default
 configuration.
 Registering a user
 ------------------
 You will need at least one user on your server in order to use a Matrix
 client. Users can be registered either `via a Matrix client`__, or via a
 commandline script.
 .. __: `client-user-reg`_
 To get started, it is easiest to use the command line to register new users::
    $ source ~/synapse/env/bin/activate
    $ synctl start # if not already running
    $ register_new_matrix_user -c homeserver.yaml https://localhost:8448
    New user localpart: erikj
    Password:
    Confirm password:
    Make admin [no]:
    Success!
 This process uses a setting ``registration_shared_secret`` in
 ``homeserver.yaml``, which is shared between Synapse itself and the
 ``register_new_matrix_user`` script. It doesn't matter what it is (a random
 value is generated by ``--generate-config``), but it should be kept secret, as
 anyone with knowledge of it can register users on your server even if
 ``enable_registration`` is ``false``.
 Setting up a TURN server
 ------------------------
 For reliable VoIP calls to be routed via this homeserver, you MUST configure
 a TURN server.  See `<docs/turn-howto.rst>`_ for details.
 Running Synapse
 ===============
 To actually run your new homeserver, pick a working directory for Synapse to
 run (e.g. ``~/synapse``), and::
    cd ~/synapse
    source env/bin/activate
    synctl start
 Connecting to Synapse from a client
 ===================================
@ -281,10 +95,11 @@ following the recommended setup, or ``https://localhost:8448`` - remember to spe
 port (``:8448``) if not ``:443`` unless you changed the configuration. (Leave the identity
 server as the default - see `Identity servers`_.)
-If using port 8448 you will run into errors until you accept the self-signed
+If using port 8448 you will run into errors if you are using a self-signed
-certificate. You can easily do this by going to ``https://localhost:8448``
+certificate. To overcome this, simply go to ``https://localhost:8448``
 directly with your browser and accept the presented certificate. You can then
-go back in your web client and proceed further.
+go back in your web client and proceed further. Valid federation certificates
 should not have this problem.
 If all goes well you should at least be able to log in, create a room, and
 start sending messages.
@ -312,6 +127,12 @@ create the account. Your name will take the form of::
 As when logging in, you will need to specify a "Custom server".  Specify your
 desired ``localpart`` in the 'User name' box.
 ACME setup
 ==========
 For details on having Synapse manage your federation TLS certificates
 automatically, please see `<docs/ACME.md>`_.
 Security Note
 =============
@ -329,177 +150,11 @@ server on the same domain.
 See https://github.com/vector-im/riot-web/issues/1977 and
 https://developer.github.com/changes/2014-04-25-user-content-security for more details.
 Platform-Specific Instructions
 ==============================
 Debian/Ubuntu
 -------------
 Matrix.org packages
 ~~~~~~~~~~~~~~~~~~~
 Matrix.org provides Debian/Ubuntu packages of the latest stable version of
 Synapse via https://matrix.org/packages/debian/. To use them::
    sudo apt install -y lsb-release curl apt-transport-https
    echo "deb https://matrix.org/packages/debian `lsb_release -cs` main" |
        sudo tee /etc/apt/sources.list.d/matrix-org.list
    curl "https://matrix.org/packages/debian/repo-key.asc" |
        sudo apt-key add -
    sudo apt update
    sudo apt install matrix-synapse-py3
 Downstream Debian/Ubuntu packages
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 For ``buster`` and ``sid``, Synapse is available in the Debian repositories and
 it should be possible to install it with simply::
    sudo apt install matrix-synapse
 There is also a version of ``matrix-synapse`` in ``stretch-backports``. Please
 see the `Debian documentation on backports
 <https://backports.debian.org/Instructions/>`_ for information on how to use
 them.
 We do not recommend using the packages in downstream Ubuntu at this time, as
 they are old and suffer from known security vulnerabilities.
 Fedora
 ------
 Synapse is in the Fedora repositories as ``matrix-synapse``::
    sudo dnf install matrix-synapse
 Oleg Girko provides Fedora RPMs at
 https://obs.infoserver.lv/project/monitor/matrix-synapse
 OpenSUSE
 --------
 Synapse is in the OpenSUSE repositories as ``matrix-synapse``::
    sudo zypper install matrix-synapse
 SUSE Linux Enterprise Server
 ----------------------------
 Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
 https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/
 ArchLinux
 ---------
 The quickest way to get up and running with ArchLinux is probably with the community package
 https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of
 the necessary dependencies.
 pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 )::
    sudo pip install --upgrade pip
 If you encounter an error with lib bcrypt causing an Wrong ELF Class:
 ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
 compile it under the right architecture. (This should not be needed if
 installing under virtualenv)::
    sudo pip uninstall py-bcrypt
    sudo pip install py-bcrypt
 FreeBSD
 -------
 Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
 - Ports: ``cd /usr/ports/net-im/py-matrix-synapse && make install clean``
 - Packages: ``pkg install py27-matrix-synapse``
 OpenBSD
 -------
 There is currently no port for OpenBSD. Additionally, OpenBSD's security
 settings require a slightly more difficult installation process.
 1) Create a new directory in ``/usr/local`` called ``_synapse``. Also, create a
   new user called ``_synapse`` and set that directory as the new user's home.
   This is required because, by default, OpenBSD only allows binaries which need
   write and execute permissions on the same memory space to be run from
   ``/usr/local``.
 2) ``su`` to the new ``_synapse`` user and change to their home directory.
 3) Create a new virtualenv: ``virtualenv -p python2.7 ~/.synapse``
 4) Source the virtualenv configuration located at
   ``/usr/local/_synapse/.synapse/bin/activate``. This is done in ``ksh`` by
   using the ``.`` command, rather than ``bash``'s ``source``.
 5) Optionally, use ``pip`` to install ``lxml``, which Synapse needs to parse
   webpages for their titles.
 6) Use ``pip`` to install this repository: ``pip install matrix-synapse``
 7) Optionally, change ``_synapse``'s shell to ``/bin/false`` to reduce the
   chance of a compromised Synapse server being used to take over your box.
 After this, you may proceed with the rest of the install directions.
 NixOS
 -----
 Robin Lambertz has packaged Synapse for NixOS at:
 https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix
 Windows Install
 ---------------
 If you wish to run or develop Synapse on Windows, the Windows Subsystem For
 Linux provides a Linux environment on Windows 10 which is capable of using the
 Debian, Fedora, or source installation methods. More information about WSL can
 be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
 Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
 for Windows Server.
 Troubleshooting
 ===============
 Troubleshooting Installation
 ----------------------------
 Synapse requires pip 8 or later, so if your OS provides too old a version you
 may need to manually upgrade it::
    sudo pip install --upgrade pip
 Installing may fail with ``Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)``.
 You can fix this by manually upgrading pip and virtualenv::
    sudo pip install --upgrade virtualenv
 You can next rerun ``virtualenv -p python3 synapse`` to update the virtual env.
 Installing may fail during installing virtualenv with ``InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.``
 You can fix this  by manually installing ndg-httpsclient::
    pip install --upgrade ndg-httpsclient
 Installing may fail with ``mock requires setuptools>=17.1. Aborting installation``.
 You can fix this by upgrading setuptools::
    pip install --upgrade setuptools
 If pip crashes mid-installation for reason (e.g. lost terminal), pip may
 refuse to run until you remove the temporary installation directory it
 created. To reset the installation::
    rm -rf /tmp/pip_install_matrix
 pip seems to leak *lots* of memory during installation.  For instance, a Linux
 host with 512MB of RAM may run out of memory whilst installing Twisted.  If this
 happens, you will have to individually install the dependencies which are
 failing, e.g.::
    pip install twisted
 Running out of File Handles
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+---------------------------
 If synapse runs out of filehandles, it typically fails badly - live-locking
 at 100% CPU, and/or failing to accept new TCP connections (blocking the
@ -541,7 +196,7 @@ Federation is the process by which users on different servers can participate
 in the same room. For this to work, those other servers must be able to contact
 yours to send messages.
-As explained in `Configuring synapse`_, the ``server_name`` in your
+The ``server_name`` in your
 ``homeserver.yaml`` file determines the way that other servers will reach
 yours. By default, they will treat it as a hostname and try to connect to
 port 8448. This is easy to set up and will work with the default configuration,
@ -553,9 +208,7 @@ you to run your server on a machine that might not have the same name as your
 domain name. For example, you might want to run your server at
 ``synapse.example.com``, but have your Matrix user-ids look like
 ``@user:example.com``. (A SRV record also allows you to change the port from
-the default 8448. However, if you are thinking of using a reverse-proxy on the
+the default 8448).
 federation port, which is not recommended, be sure to read
 `Reverse-proxying the federation port`_ first.)
 To use a SRV record, first create your SRV record and publish it in DNS. This
 should have the format ``_matrix._tcp.<yourdomain.com> <ttl> IN SRV 10 0 <port>
@ -593,6 +246,8 @@ Troubleshooting
 You can use the federation tester to check if your homeserver is all set:
 ``https://matrix.org/federationtester/api/report?server_name=<your_server_name>``
 If any of the attributes under "checks" is false, federation won't work.
 There is also a nicer interface available from a community member at
 `<https://neo.lain.haus/fed-tester>`_.
 The typical failure mode with federation is that when you try to join a room,
 it is rejected with "401: Unauthorized". Generally this means that other
@ -601,8 +256,6 @@ complicated dance which requires connections in both directions).
 So, things to check are:
 * If you are trying to use a reverse-proxy, read `Reverse-proxying the
  federation port`_.
 * If you are not using a SRV record, check that your ``server_name`` (the part
  of your user-id after the ``:``) matches your hostname, and that port 8448 on
  that hostname is reachable from outside your network.
@ -657,14 +310,8 @@ port. Indeed, clients will use port 443 by default, whereas servers default to
 port 8448. Where these are different, we refer to the 'client port' and the
 'federation port'.
-The next most important thing to know is that using a reverse-proxy on the
+All Matrix endpoints begin with ``/_matrix``, so an example nginx
-federation port has a number of pitfalls. It is possible, but be sure to read
+configuration for forwarding client connections to Synapse might look like::
 `Reverse-proxying the federation port`_.
 The recommended setup is therefore to configure your reverse-proxy on port 443
 to port 8008 of synapse for client connections, but to also directly expose port
 8448 for server-server connections. All the Matrix endpoints begin ``/_matrix``,
 so an example nginx configuration might look like::
  server {
      listen 443 ssl;
@ -705,64 +352,6 @@ Having done so, you can then use ``https://matrix.example.com`` (instead of
 ``https://matrix.example.com:8448``) as the "Custom server" when `Connecting to
 Synapse from a client`_.
 Reverse-proxying the federation port
 ------------------------------------
 There are two issues to consider before using a reverse-proxy on the federation
 port:
 * Due to the way SSL certificates are managed in the Matrix federation protocol
  (see `spec`__), Synapse needs to be configured with the path to the SSL
  certificate, *even if you do not terminate SSL at Synapse*.
  .. __: `key_management`_
 * Until v0.33.3, Synapse did not support SNI on the federation port
  (`bug #1491 <https://github.com/matrix-org/synapse/issues/1491>`_). This bug
  is now fixed, but means that federating with older servers can be unreliable
  when using name-based virtual hosting.
 Furthermore, a number of the normal reasons for using a reverse-proxy do not
 apply:
 * Other servers will connect on port 8448 by default, so there is no need to
  listen on port 443 (for federation, at least), which avoids the need for root
  privileges and virtual hosting.
 * A self-signed SSL certificate is fine for federation, so there is no need to
  automate renewals. (The certificate generated by ``--generate-config`` is
  valid for 10 years.)
 If you want to set up a reverse-proxy on the federation port despite these
 caveats, you will need to do the following:
 * In ``homeserver.yaml``, set ``tls_certificate_path`` to the path to the SSL
  certificate file used by your reverse-proxy, and set ``no_tls`` to ``True``.
  (``tls_private_key_path`` will be ignored if ``no_tls`` is ``True``.)
 * In your reverse-proxy configuration:
  * If there are other virtual hosts on the same port, make sure that the
    *default* one uses the certificate configured above.
  * Forward ``/_matrix`` to Synapse.
 * If your reverse-proxy is not listening on port 8448, publish a SRV record to
  tell other servers how to find you. See `Setting up Federation`_.
 When updating the SSL certificate, just update the file pointed to by
 ``tls_certificate_path`` and then restart Synapse. (You may like to use a symbolic link
 to help make this process atomic.)
 The most common mistake when setting up federation is not to tell Synapse about
 your SSL certificate. To check it, you can visit
 ``https://matrix.org/federationtester/api/report?server_name=<your_server_name>``.
 Unfortunately, there is no UI for this yet, but, you should see
 ``"MatchingTLSFingerprint": true``. If not, check that
 ``Certificates[0].SHA256Fingerprint`` (the fingerprint of the certificate
 presented by your reverse-proxy) matches ``Keys.tls_fingerprints[0].sha256``
 (the fingerprint of the certificate Synapse is using).
 Identity Servers
 ================
@ -794,24 +383,6 @@ an email address with your account, or send an invite to another user via their
 email address.
 URL Previews
 ============
 Synapse 0.15.0 introduces a new API for previewing URLs at
 ``/_matrix/media/r0/preview_url``.  This is disabled by default.  To turn it on
 you must enable the ``url_preview_enabled: True`` config parameter and
 explicitly specify the IP ranges that Synapse is not allowed to spider for
 previewing in the ``url_preview_ip_range_blacklist`` configuration parameter.
 This is critical from a security perspective to stop arbitrary Matrix users
 spidering 'internal' URLs on your network.  At the very least we recommend that
 your loopback and RFC1918 IP addresses are blacklisted.
 This also requires the optional lxml and netaddr python dependencies to be
 installed.  This in turn requires the libxml2 library to be available - on
 Debian/Ubuntu this means ``apt-get install libxml2-dev``, or equivalent for
 your OS.
 Password reset
 ==============
@ -913,5 +484,3 @@ by installing the ``libjemalloc1`` package and adding this line to
 ``/etc/default/matrix-synapse``::
    LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
 .. _`key_management`: https://matrix.org/docs/spec/server_server/unstable.html#retrieving-server-keys
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@ -5,20 +5,20 @@ Before upgrading check if any special steps are required to upgrade from the
 what you currently have installed to current version of synapse. The extra
 instructions that may be required are listed later in this document.
-1. If synapse was installed in a virtualenv then active that virtualenv before
+1. If synapse was installed in a virtualenv then activate that virtualenv before
-   upgrading. If synapse is installed in a virtualenv in ``~/.synapse/`` then
+   upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then
   run:
   .. code:: bash
-       source ~/.synapse/bin/activate
+       source ~/synapse/env/bin/activate
 2. If synapse was installed using pip then upgrade to the latest version by
   running:
   .. code:: bash
-       pip install --upgrade matrix-synapse
+       pip install --upgrade matrix-synapse[all]
       # restart synapse
       synctl restart
@ -31,8 +31,9 @@ instructions that may be required are listed later in this document.
       # Pull the latest version of the master branch.
       git pull
-       # Update the versions of synapse's python dependencies.
+
-       python synapse/python_dependencies.py | xargs pip install --upgrade
+       # Update synapse and its python dependencies.
       pip install --upgrade .[all]
       # restart synapse
       ./synctl restart
@ -51,34 +52,12 @@ returned by the Client-Server API:
 Upgrading to v0.99.0
 ====================
-In preparation for Synapse v1.0, you must update your TLS certificates from
+Please be aware that, before Synapse v1.0 is released around March 2019, you
-self-signed ones to verifiable ones signed by a trusted root CA.
+will need to replace any self-signed certificates with those verified by a
 root CA. Information on how to do so can be found at `the ACME docs
 <docs/ACME.md>`_.
-If you do not already have a certificate for your domain, the easiest way to get
+For more information on configuring TLS certificates see the `FAQ <https://github.com/matrix-org/synapse/blob/master/docs/MSC1711_certificates_FAQ.md>`_
 one is with Synapse's new ACME support, which will use the ACME protocol to
 provision a certificate automatically. By default, certificates will be obtained
 from the publicly trusted CA Let's Encrypt.
 For a sample configuration, please inspect the new ACME section in the example
 generated config by running the ``generate-config`` executable. For example::
  ~/synapse/env3/bin/generate-config
 You will need to provide Let's Encrypt (or other ACME provider) access to your
 Synapse ACME challenge responder on port 80, at the domain of your homeserver.
 This requires you either change the port of the ACME listener provided by
 Synapse to a high port and reverse proxy to it, or use a tool like authbind to
 allow Synapse to listen on port 80 without root access. (Do not run Synapse with
 root permissions!)
 You will need to back up or delete your self signed TLS certificate
 (``example.com.tls.crt`` and ``example.com.tls.key``), Synapse's ACME
 implementation will not overwrite them.
 You may wish to use alternate methods such as Certbot to obtain a certificate
 from Let's Encrypt, depending on your server configuration. Of course, if you
 already have a valid certificate for your homeserver's domain, that can be
 placed in Synapse's config directory without the need for ACME.
 Upgrading to v0.34.0
 ====================
--- a/changelog.d/4539.misc
+++ b/changelog.d/4539.misc
@ -1 +0,0 @@
 Update federation routing logic to check .well-known before SRV
--- a/changelog.d/4542.misc
+++ b/changelog.d/4542.misc
@ -1 +0,0 @@
 Improve performance of handling servers with invalid .well-known
--- a/changelog.d/4544.misc
+++ b/changelog.d/4544.misc
@ -1 +0,0 @@
 Treat an invalid .well-known file the same as an absent one
--- a/debian/changelog
+++ b/debian/changelog
@ -1,3 +1,9 @@
 matrix-synapse-py3 (0.99.0) stable; urgency=medium
  * New synapse release 0.99.0
 -- Synapse Packaging team <packages@matrix.org>  Tue, 5 Feb 2019 18:25:00 +0000
 matrix-synapse-py3 (0.34.1.1++1) stable; urgency=medium
  * Update conflicts specifications to allow smoother transition from matrix-synapse.
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@ -1,3 +1,16 @@
 # Dockerfile to build the matrixdotorg/synapse docker images.
 #
 # To build the image, run `docker build` command from the root of the
 # synapse repository:
 #
 #    docker build -f docker/Dockerfile .
 #
 # There is an optional PYTHON_VERSION build argument which sets the
 # version of python to build against: for example:
 #
 #    docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.6 .
 #
 ARG PYTHON_VERSION=2
 ###
@ -31,7 +44,10 @@ RUN pip install --prefix="/install" --no-warn-script-location \
 # now install synapse and all of the python deps to /install.
-COPY . /synapse
+COPY synapse /synapse/synapse/
 COPY scripts /synapse/scripts/
 COPY MANIFEST.in README.rst setup.py synctl /synapse/
 RUN pip install --prefix="/install" --no-warn-script-location \
        /synapse[all]
@ -56,6 +72,6 @@ COPY ./docker/conf /conf
 VOLUME ["/data"]
-EXPOSE 8008/tcp 8448/tcp
+EXPOSE 8008/tcp 8009/tcp 8448/tcp
 ENTRYPOINT ["/start.py"]
--- a/docker/README.md
+++ b/docker/README.md
@ -1,22 +1,21 @@
 # Synapse Docker
-This Docker image will run Synapse as a single process. It does not provide a database
+This Docker image will run Synapse as a single process. By default it uses a
-server or a TURN server, you should run these separately.
+sqlite database; for production use you should connect it to a separate
 postgres database.
 The image also does *not* provide a TURN server.
 ## Run
 We do not currently offer a `latest` image, as this has somewhat undefined semantics.
 We instead release only tagged versions so upgrading between releases is entirely
 within your control.
 ### Using docker-compose (easier)
-This image is designed to run either with an automatically generated configuration
+This image is designed to run either with an automatically generated
-file or with a custom configuration that requires manual editing.
+configuration file or with a custom configuration that requires manual editing.
 An easy way to make use of this image is via docker-compose. See the
-[contrib/docker](../contrib/docker)
+[contrib/docker](../contrib/docker) section of the synapse project for
-section of the synapse project for examples.
+examples.
 ### Without Compose (harder)
@ -32,7 +31,7 @@ docker run \
    -v ${DATA_PATH}:/data \
    -e SYNAPSE_SERVER_NAME=my.matrix.host \
    -e SYNAPSE_REPORT_STATS=yes \
-    docker.io/matrixdotorg/synapse:latest
+    matrixdotorg/synapse:latest
 ```
 ## Volumes
@ -53,6 +52,28 @@ In order to setup an application service, simply create an ``appservices``
 directory in the data volume and write the application service Yaml
 configuration file there. Multiple application services are supported.
 ## TLS certificates
 Synapse requires a valid TLS certificate. You can do one of the following:
 * Provide your own certificate and key (as
   `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt` and
   `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.key`, or elsewhere by providing an
   entire config as `${SYNAPSE_CONFIG_PATH}`).
 * Use a reverse proxy to terminate incoming TLS, and forward the plain http
   traffic to port 8008 in the container. In this case you should set `-e
   SYNAPSE_NO_TLS=1`.
 * Use the ACME (Let's Encrypt) support built into Synapse. This requires
   `${SYNAPSE_SERVER_NAME}` port 80 to be forwarded to port 8009 in the
   container, for example with `-p 80:8009`. To enable it in the docker
   container, set `-e SYNAPSE_ACME=1`.
 If you don't do any of these, Synapse will fail to start with an error similar to:
    synapse.config._base.ConfigError: Error accessing file '/data/<server_name>.tls.crt' (config for tls_certificate): No such file or directory
 ## Environment
 Unless you specify a custom path for the configuration file, a very generic
@ -71,7 +92,7 @@ then customize it manually. No other environment variable is required.
 Otherwise, a dynamic configuration file will be used. The following environment
 variables are available for configuration:
-* ``SYNAPSE_SERVER_NAME`` (mandatory), the current server public hostname.
+* ``SYNAPSE_SERVER_NAME`` (mandatory), the server public hostname.
 * ``SYNAPSE_REPORT_STATS``, (mandatory, ``yes`` or ``no``), enable anonymous
  statistics reporting back to the Matrix project which helps us to get funding.
 * ``SYNAPSE_NO_TLS``, set this variable to disable TLS in Synapse (use this if
@ -80,7 +101,6 @@ variables are available for configuration:
  the Synapse instance.
 * ``SYNAPSE_ALLOW_GUEST``, set this variable to allow guest joining this server.
 * ``SYNAPSE_EVENT_CACHE_SIZE``, the event cache size [default `10K`].
 * ``SYNAPSE_CACHE_FACTOR``, the cache factor [default `0.5`].
 * ``SYNAPSE_RECAPTCHA_PUBLIC_KEY``, set this variable to the recaptcha public
  key in order to enable recaptcha upon registration.
 * ``SYNAPSE_RECAPTCHA_PRIVATE_KEY``, set this variable to the recaptcha private
@ -88,7 +108,9 @@ variables are available for configuration:
 * ``SYNAPSE_TURN_URIS``, set this variable to the coma-separated list of TURN
  uris to enable TURN for this homeserver.
 * ``SYNAPSE_TURN_SECRET``, set this to the TURN shared secret if required.
-* ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size [default `10M`].
+* ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size
  [default `10M`].
 * ``SYNAPSE_ACME``: set this to enable the ACME certificate renewal support.
 Shared secrets, that will be initialized to random values if not set:
@ -99,27 +121,25 @@ Shared secrets, that will be initialized to random values if not set:
 Database specific values (will use SQLite if not set):
-* `POSTGRES_DB` - The database name for the synapse postgres database. [default: `synapse`]
+* `POSTGRES_DB` - The database name for the synapse postgres
-* `POSTGRES_HOST` - The host of the postgres database if you wish to use postgresql instead of sqlite3. [default: `db` which is useful when using a container on the same docker network in a compose file where the postgres service is called `db`]
+  database. [default: `synapse`]
-* `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If this is set then postgres will be used instead of sqlite3.** [default: none] **NOTE**: You are highly encouraged to use postgresql! Please use the compose file to make it easier to deploy.
+* `POSTGRES_HOST` - The host of the postgres database if you wish to use
-* `POSTGRES_USER` - The user for the synapse postgres database. [default: `matrix`]
+  postgresql instead of sqlite3. [default: `db` which is useful when using a
  container on the same docker network in a compose file where the postgres
  service is called `db`]
 * `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If
  this is set then postgres will be used instead of sqlite3.** [default: none]
  **NOTE**: You are highly encouraged to use postgresql! Please use the compose
  file to make it easier to deploy.
 * `POSTGRES_USER` - The user for the synapse postgres database. [default:
  `matrix`]
 Mail server specific values (will not send emails if not set):
 * ``SYNAPSE_SMTP_HOST``, hostname to the mail server.
-* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default ``25``].
+* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default
-* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if any.
+  ``25``].
-* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail server if any.
+* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if
-
+  any.
-## Build
+* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail
-
+  server if any.
 Build the docker image with the `docker build` command from the root of the synapse repository.
 ```
 docker build -t docker.io/matrixdotorg/synapse . -f docker/Dockerfile
 ```
 The `-t` option sets the image tag. Official images are tagged `matrixdotorg/synapse:<version>` where `<version>` is the same as the release tag in the synapse git repository.
 You may have a local Python wheel cache available, in which case copy the relevant
 packages in the ``cache/`` directory at the root of the project.
--- a/docker/conf/dummy.tls.crt
+++ b/docker/conf/dummy.tls.crt
@ -0,0 +1,17 @@
 -----BEGIN CERTIFICATE-----
 MIICnTCCAYUCAgPoMA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxvY2FsaG9z
 dDAeFw0xOTAxMTUwMDQxNTBaFw0yOTAxMTIwMDQxNTBaMBQxEjAQBgNVBAMMCWxv
 Y2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKqm81/8j5d
 R1s7VZ8ueg12gJrPVCCAOkp0UnuC/ZlXhN0HTvnhQ+B0IlSgB4CcQZyf4jnA6o4M
 rwSc7VX0MPE9x/idoA0g/0WoC6tsxugOrvbzCw8Tv+fnXglm6uVc7aFPfx69wU3q
 lUHGD/8jtEoHxmCG177Pt2lHAfiVLBAyMQGtETzxt/yAfkloaybe316qoljgK5WK
 cokdAt9G84EEqxNeEnx5FG3Vc100bAqJS4GvQlFgtF9KFEqZKEyB1yKBpPMDfPIS
 V9hIV0gswSmYI8dpyBlGf5lPElY68ZGABmOQgr0RI5qHK/h28OpFPE0q3v4AMHgZ
 I36wii4NrAUCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAfD8kcpZ+dn08xh1qtKtp
 X+/YNZaOBIeVdlCzfoZKNblSFAFD/jCfObNJYvZMUQ8NX2UtEJp1lTA6m7ltSsdY
 gpC2k1VD8iN+ooXklJmL0kxc7UUqho8I0l9vn35h+lhLF0ihT6XfZVi/lDHWl+4G
 rG+v9oxvCSCWrNWLearSlFPtQQ8xPtOE0nLwfXtOI/H/2kOuC38ihaIWM4jjbWXK
 E/ksgUfuDv0mFiwf1YdBF5/M3/qOowqzU8HgMJ3WoT/9Po5Ya1pWc+3BcxxytUDf
 XdMu0tWHKX84tZxLcR1nZHzluyvFFM8xNtLi9xV0Z7WbfT76V0C/ulEOybGInYsv
 nQ==
 -----END CERTIFICATE-----
--- a/docker/conf/homeserver.yaml
+++ b/docker/conf/homeserver.yaml
@ -2,10 +2,24 @@
 ## TLS ##
 {% if SYNAPSE_NO_TLS %}
 no_tls: True
 # workaround for https://github.com/matrix-org/synapse/issues/4554
 tls_certificate_path: "/conf/dummy.tls.crt"
 {% else %}
 tls_certificate_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.crt"
 tls_private_key_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.key"
-no_tls: {{ "True" if SYNAPSE_NO_TLS else "False" }}
+
-tls_fingerprints: []
+{% if SYNAPSE_ACME %}
 acme:
    enabled: true
    port: 8009
 {% endif %}
 {% endif %}
 ## Server ##
--- a/docker/start.py
+++ b/docker/start.py
@ -47,9 +47,8 @@ if mode == "generate":
 # In normal mode, generate missing keys if any, then run synapse
 else:
    # Parse the configuration file
    if "SYNAPSE_CONFIG_PATH" in environ:
-        args += ["--config-path", environ["SYNAPSE_CONFIG_PATH"]]
+        config_path = environ["SYNAPSE_CONFIG_PATH"]
    else:
        check_arguments(environ, ("SYNAPSE_SERVER_NAME", "SYNAPSE_REPORT_STATS"))
        generate_secrets(environ, {
@ -58,10 +57,21 @@ else:
        })
        environ["SYNAPSE_APPSERVICES"] = glob.glob("/data/appservices/*.yaml")
        if not os.path.exists("/compiled"): os.mkdir("/compiled")
-        convert("/conf/homeserver.yaml", "/compiled/homeserver.yaml", environ)
+
        config_path = "/compiled/homeserver.yaml"
        convert("/conf/homeserver.yaml", config_path, environ)
        convert("/conf/log.config", "/compiled/log.config", environ)
        subprocess.check_output(["chown", "-R", ownership, "/data"])
-        args += ["--config-path", "/compiled/homeserver.yaml"]
+
    args += [
        "--config-path", config_path,
        # tell synapse to put any generated keys in /data rather than /compiled
        "--keys-directory", "/data",
    ]
    # Generate missing keys and start synapse
    subprocess.check_output(args + ["--generate-keys"])
    os.execv("/sbin/su-exec", ["su-exec", ownership] + args)
--- a/docs/ACME.md
+++ b/docs/ACME.md
@ -0,0 +1,126 @@
 # ACME
 Synapse v1.0 will require valid TLS certificates for communication between
 servers (port `8448` by default) in addition to those that are client-facing
 (port `443`). If you do not already have a valid certificate for your domain,
 the easiest way to get one is with Synapse's new ACME support, which will use
 the ACME protocol to provision a certificate automatically. Synapse v0.99.0+
 will provision server-to-server certificates automatically for you for free
 through [Let's Encrypt](https://letsencrypt.org/) if you tell it to.
 In the case that your `server_name` config variable is the same as
 the hostname that the client connects to, then the same certificate can be
 used between client and federation ports without issue. 
 For a sample configuration, please inspect the new ACME section in the example
 generated config by running the `generate-config` executable. For example:
 ```
 ~/synapse/env3/bin/generate-config
 ```
 You will need to provide Let's Encrypt (or another ACME provider) access to
 your Synapse ACME challenge responder on port 80, at the domain of your
 homeserver. This requires you to either change the port of the ACME listener
 provided by Synapse to a high port and reverse proxy to it, or use a tool
 like `authbind` to allow Synapse to listen on port 80 without root access.
 (Do not run Synapse with root permissions!) Detailed instructions are
 available under "ACME setup" below.
 If you are already using self-signed certificates, you will need to back up
 or delete them (files `example.com.tls.crt` and `example.com.tls.key` in
 Synapse's root directory), Synapse's ACME implementation will not overwrite
 them.
 You may wish to use alternate methods such as Certbot to obtain a certificate
 from Let's Encrypt, depending on your server configuration. Of course, if you
 already have a valid certificate for your homeserver's domain, that can be
 placed in Synapse's config directory without the need for any ACME setup.
 ## ACME setup
 The main steps for enabling ACME support in short summary are:
 1. Allow Synapse to listen on port 80 with authbind, or forward it from a reverse-proxy.
 1. Set `acme:enabled` to `true` in homeserver.yaml.
 1. Move your old certificates (files `example.com.tls.crt` and `example.com.tls.key` out of the way if they currently exist at the paths specified in `homeserver.yaml`.
 1. Restart Synapse
 Detailed instructions for each step are provided below.
 ### Listening on port 80
 In order for Synapse to complete the ACME challenge to provision a
 certificate, it needs access to port 80. Typically listening on port 80 is
 only granted to applications running as root. There are thus two solutions to
 this problem.
 #### Using a reverse proxy
 A reverse proxy such as Apache or nginx allows a single process (the web
 server) to listen on port 80 and proxy traffic to the appropriate program
 running on your server. It is the recommended method for setting up ACME as
 it allows you to use your existing webserver while also allowing Synapse to
 provision certificates as needed.
 For nginx users, add the following line to your existing `server` block:
 ```
 location /.well-known/acme-challenge {
    proxy_pass http://localhost:8009/;
 }
 ```
 For Apache, add the following to your existing webserver config::
 ```
 ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge
 ```
 Make sure to restart/reload your webserver after making changes.
 #### Authbind
 `authbind` allows a program which does not run as root to bind to
 low-numbered ports in a controlled way. The setup is simpler, but requires a
 webserver not to already be running on port 80. **This includes every time
 Synapse renews a certificate**, which may be cumbersome if you usually run a
 web server on port 80. Nevertheless, if you're sure port 80 is not being used
 for any other purpose then all that is necessary is the following:
 Install `authbind`. For example, on Debian/Ubuntu:
 ```
 sudo apt-get install authbind
 ```
 Allow `authbind` to bind port 80:
 ```
 sudo touch /etc/authbind/byport/80
 sudo chmod 777 /etc/authbind/byport/80
 ```
 When Synapse is started, use the following syntax::
 ```
 authbind --deep <synapse start command>
 ```
 ### Config file editing
 Once Synapse is able to listen on port 80 for ACME challenge
 requests, it must be told to perform ACME provisioning by setting `enabled`
 to true under the `acme` section in `homeserver.yaml`:
 ```
 acme:
    enabled: true
 ```
 ### Starting synapse
 Ensure that the certificate paths specified in `homeserver.yaml` (`tls_certificate_path` and `tls_private_key_path`) do not currently point to any files. Synapse will not provision certificates if files exist, as it does not want to overwrite existing certificates.
 Finally, start/restart Synapse.
--- a/docs/MSC1711_certificates_FAQ.md
+++ b/docs/MSC1711_certificates_FAQ.md
@ -0,0 +1,336 @@
 # MSC 1711 Certificates FAQ
 The goal of Synapse 0.99.0 is to act as a stepping stone to Synapse 1.0.0. It
 supports the r0.1 release of the server to server specification, but is
 compatible with both the legacy Matrix federation behaviour (pre-r0.1) as well
 as post-r0.1 behaviour, in order to allow for a smooth upgrade across the
 federation.
 The most important thing to know is that Synapse 1.0.0 will require a valid TLS
 certificate on federation endpoints. Self signed certificates will not be
 sufficient.
 Synapse 0.99.0 makes it easy to configure TLS certificates and will
 interoperate with both >= 1.0.0 servers as well as existing servers yet to
 upgrade.
 **It is critical that all admins upgrade to 0.99.0 and configure a valid TLS
 certificate.** Admins will have 1 month to do so, after which 1.0.0 will be
 released and those servers without a valid certificate will not longer be able
 to federate with >= 1.0.0 servers.
 Full details on how to carry out this configuration change is given
 [below](#configuring-certificates-for-compatibility-with-synapse-100). A
 timeline and some frequently asked questions are also given below.
 For more details and context on the release of the r0.1 Server/Server API and
 imminent Matrix 1.0 release, you can also see our
 [main talk from FOSDEM 2019](https://matrix.org/blog/2019/02/04/matrix-at-fosdem-2019/).
 ## Contents
 * Timeline
 * Configuring certificates for compatibility with Synapse 1.0
 * FAQ
  * Synapse 0.99.0 has just been released, what do I need to do right now?
  * How do I upgrade?
  * What will happen if I do not set up a valid federation certificate
    immediately?
  * What will happen if I do nothing at all?
  * When do I need a SRV record or .well-known URI?
  * Can I still use an SRV record?
  * I have created a .well-known URI. Do I still need an SRV record?
  * It used to work just fine, why are you breaking everything?
  * Can I manage my own certificates rather than having Synapse renew
    certificates itself?
  * Do you still recommend against using a reverse-proxy on the federation port?
  * Do I still need to give my TLS certificates to Synapse if I am using a
    reverse-proxy?
  * Do I need the same certificate for the client and federation port?
  * How do I tell Synapse to reload my keys/certificates after I replace them?
 ## Timeline
 **5th Feb 2019  - Synapse 0.99.0 is released.**
 All server admins are encouraged to upgrade.
 0.99.0:
 -   provides support for ACME to make setting up Let's Encrypt certs easy, as
    well as .well-known support.
 -   does not enforce that a valid CA cert is present on the federation API, but
    rather makes it easy to set one up.
 -   provides support for .well-known
 Admins should upgrade and configure a valid CA cert. Homeservers that require a
 .well-known entry (see below), should retain their SRV record and use it
 alongside their .well-known record.
 **>= 5th March 2019  - Synapse 1.0.0 is released**
 1.0.0 will land no sooner than 1 month after 0.99.0, leaving server admins one
 month after 5th February to upgrade to 0.99.0 and deploy their certificates. In
 accordance with the the [S2S spec](https://matrix.org/docs/spec/server_server/r0.1.0.html)
 1.0.0 will enforce certificate validity. This means that any homeserver without a
 valid certificate after this point will no longer be able to federate with
 1.0.0 servers.
 ## Configuring certificates for compatibility with Synapse 1.0.0
 ### If you do not currently have an SRV record
 In this case, your `server_name` points to the host where your Synapse is
 running. There is no need to create a `.well-known` URI or an SRV record, but
 you will need to give Synapse a valid, signed, certificate.
 The easiest way to do that is with Synapse's built-in ACME (Let's Encrypt)
 support. Full details are in [ACME.md](./ACME.md) but, in a nutshell:
 1. Allow Synapse to listen on port 80 with `authbind`, or forward it from a
    reverse proxy.
 2. Enable acme support in `homeserver.yaml`.
 3. Move your old certificates out of the way.
 4. Restart Synapse.
 ### If you do have an SRV record currently
 If you are using an SRV record, your matrix domain (`server_name`) may not
 point to the same host that your Synapse is running on (the 'target
 domain'). (If it does, you can follow the recommendation above; otherwise, read
 on.)
 Let's assume that your `server_name` is `example.com`, and your Synapse is
 hosted at a target domain of `customer.example.net`. Currently you should have
 an SRV record which looks like:
 ```
 _matrix._tcp.example.com. IN SRV 10 5 443 customer.example.net.
 ```
 In this situation, you have two choices for how to proceed:
 #### Option 1: give Synapse a certificate for your matrix domain
 Synapse 1.0 will expect your server to present a TLS certificate for your
 `server_name` (`example.com` in the above example). You can achieve this by
 doing one of the following:
 * Acquire a certificate for the `server_name` yourself (for example, using
   `certbot`), and give it and the key to Synapse via `tls_certificate_path`
   and `tls_private_key_path`, or:
 * Use Synapse's [ACME support](./ACME.md), and forward port 80 on the
   `server_name` domain to your Synapse instance, or:
 * Set up a reverse-proxy on port 8448 on the `server_name` domain, which
   forwards to Synapse. Once it is set up, you can remove the SRV record.
 #### Option 2: add a .well-known file to delegate your matrix traffic
 This will allow you to keep Synapse on a separate domain, without having to
 give it a certificate for the matrix domain.
 You can do this with a `.well-known` file as follows:
 1. Keep the SRV record in place - it is needed for backwards compatibility
    with Synapse 0.34 and earlier.
 2. Give synapse a certificate corresponding to the target domain
    (`customer.example.net` in the above example). Currently Synapse's ACME
    support [does not support
    this](https://github.com/matrix-org/synapse/issues/4552), so you will have
    to acquire a certificate yourself and give it to Synapse via
    `tls_certificate_path` and `tls_private_key_path`.
 3. Restart Synapse to ensure the new certificate is loaded.
 4. Arrange for a `.well-known` file at
    `https://<server_name>/.well-known/matrix/server` with contents:
    ```json
    {"m.server": "<target domain>:<port>"}
    ```
    In the above example, `https://example.com/.well-known/matrix/server`
    should have the contents:
    ```json
 	{"m.server": "customer.example.net:443"}
    ```
 ## FAQ
 ### Synapse 0.99.0 has just been released, what do I need to do right now?
 Upgrade as soon as you can in preparation for Synapse 1.0.0.
 ### How do I upgrade?
 Follow the upgrade notes here [UPGRADE.rst](https://github.com/matrix-org/synapse/blob/master/UPGRADE.rst)
 ### What will happen if I do not set up a valid federation certificate immediately?
 Nothing initially, but once 1.0.0 is in the wild it will not be possible to
 federate with 1.0.0 servers.
 ### What will happen if I do nothing at all?
 If the admin takes no action at all, and remains on a Synapse < 0.99.0 then the
 homeserver will be unable to federate with those who have implemented
 .well-known. Then, as above, once the month upgrade window has expired the
 homeserver will not be able to federate with any Synapse >= 1.0.0
 ### When do I need a SRV record or .well-known URI?
 If your homeserver listens on the default federation port (8448), and your
 server_name points to the host that your homeserver runs on, you do not need an
 SRV record or .well-known/matrix/server URI.\
 For instance, if you registered example.com and pointed its DNS A record at a
 fresh Upcloud VPS or similar, you could install Synapse 0.99 on that host,
 giving it a server_name of example.com, and it would automatically generate a
 valid TLS certificate for you via Let's Encrypt and no SRV record or
 .well-known URI would be needed.
 This is the common case, although you can add an SRV record or
 .well-known/matrix/server URI for completeness if you wish.
 **However**, if your server does not listen on port 8448, or if your server_name
 does not point to the host that your homeserver runs on, you will need to let
 other servers know how to find it.
 The easiest way to do this is with a .well-known/matrix/server URI on the
 webroot of the domain to advertise your server. For instance, if you ran
 "matrixhosting.com" and you were hosting a Matrix server for `example.com`, you
 would ask `example.com` to create a file at
 `https://example.com/.well-known/matrix/server` with contents:
 ```json
 {"m.server": "example.matrixhosting.com:8448"}
 ```
 ...which would tell servers trying to connect to example.com to instead connect
 to example.matrixhosting.com on port 8448. You would then configure Synapse
 with a server_name of "example.com", but generate a TLS certificate for
 example.matrixhosting.com.
 As an alternative, you can still use an SRV DNS record for the delegation, but
 this will require you to have a certificate for the matrix domain (example.com
 in this example). See "Can I still use an SRV record?".
 ### Can I still use an SRV record?
 Firstly, if you didn't need an SRV record before (because your server is
 listening on port 8448 of your server_name), you certainly don't need one now:
 the defaults are still the same.
 If you previously had an SRV record, you can keep using it provided you are
 able to give Synapse a TLS certificate corresponding to your server name. For
 example, suppose you had the following SRV record, which directs matrix traffic
 for example.com to matrix.example.com:443:
 ```
 _matrix._tcp.example.com. IN SRV 10 5 443 matrix.example.com
 ```
 In this case, Synapse must be given a certificate for example.com - or be
 configured to acquire one from Let's Encrypt.
 If you are unable to give Synapse a certificate for your server_name, you will
 also need to use a .well-known URI instead. However, see also "I have created a
 .well-known URI. Do I still need an SRV record?".
 ### I have created a .well-known URI. Do I still need an SRV record?
 As of Synapse 0.99, Synapse will first check for the existence of a .well-known
 URL and follow any delegation it suggests. It will only then check for the
 existence of an SRV record.
 That means that the SRV record will often be redundant. However, you should
 remember that there may still be older versions of Synapse in the federation
 which do not understand .well-known URIs, so if you removed your SRV record you
 would no longer be able to federate with them.
 It is therefore best to leave the SRV record in place for now. Synapse 0.34 and
 earlier will follow the SRV record (and not care about the invalid
 certificate). Synapse 0.99 and later will follow the .well-known URI, with the
 correct certificate chain.
 ### It used to work just fine, why are you breaking everything?
 We have always wanted Matrix servers to be as easy to set up as possible, and
 so back when we started federation in 2014 we didn't want admins to have to go
 through the cumbersome process of buying a valid TLS certificate to run a
 server. This was before Let's Encrypt came along and made getting a free and
 valid TLS certificate straightforward. So instead, we adopted a system based on
 [Perspectives](https://en.wikipedia.org/wiki/Convergence_(SSL)): an approach
 where you check a set of "notary servers" (in practice, homeservers) to vouch
 for the validity of a certificate rather than having it signed by a CA. As long
 as enough different notaries agree on the certificate's validity, then it is
 trusted.
 However, in practice this has never worked properly. Most people only use the
 default notary server (matrix.org), leading to inadvertent centralisation which
 we want to eliminate. Meanwhile, we never implemented the full consensus
 algorithm to query the servers participating in a room to determine consensus
 on whether a given certificate is valid. This is fiddly to get right
 (especially in face of sybil attacks), and we found ourselves questioning
 whether it was worth the effort to finish the work and commit to maintaining a
 secure certificate validation system as opposed to focusing on core Matrix
 development.
 Meanwhile, Let's Encrypt came along in 2016, and put the final nail in the
 coffin of the Perspectives project (which was already pretty dead). So, the
 Spec Core Team decided that a better approach would be to mandate valid TLS
 certificates for federation alongside the rest of the Web. More details can be
 found in
 [MSC1711](https://github.com/matrix-org/matrix-doc/blob/master/proposals/1711-x509-for-federation.md#background-the-failure-of-the-perspectives-approach).
 This results in a breaking change, which is disruptive, but absolutely critical
 for the security model. However, the existence of Let's Encrypt as a trivial
 way to replace the old self-signed certificates with valid CA-signed ones helps
 smooth things over massively, especially as Synapse can now automate Let's
 Encrypt certificate generation if needed.
 ### Can I manage my own certificates rather than having Synapse renew certificates itself?
 Yes, you are welcome to manage your certificates yourself. Synapse will only
 attempt to obtain certificates from Let's Encrypt if you configure it to do
 so.The only requirement is that there is a valid TLS cert present for
 federation end points.
 ### Do you still recommend against using a reverse-proxy on the federation port?
 We no longer actively recommend against using a reverse proxy. Many admins will
 find it easier to direct federation traffic to a reverse-proxy and manage their
 own TLS certificates, and this is a supported configuration.
 ### Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?
 Practically speaking, this is no longer necessary.
 If you are using a reverse-proxy for all of your TLS traffic, then you can set
 `no_tls: True`. In that case, the only reason Synapse needs the certificate is
 to populate a legacy 'tls_fingerprints' field in the federation API. This is
 ignored by Synapse 0.99.0 and later, and the only time pre-0.99 Synapses will
 check it is when attempting to fetch the server keys - and generally this is
 delegated via `matrix.org`, which is on 0.99.0.
 However, there is a bug in Synapse 0.99.0
 [4554](<https://github.com/matrix-org/synapse/issues/4554>) which prevents
 Synapse from starting if you do not give it a TLS certificate. To work around
 this, you can give it any TLS certificate at all. This will be fixed soon.
 ### Do I need the same certificate for the client and federation port?
 No. There is nothing stopping you doing so, particularly if you are using a
 reverse-proxy. However, Synapse will use the same certificate on any ports
 where TLS is configured.
 ### How do I tell Synapse to reload my keys/certificates after I replace them?
 Synapse will reload the keys and certificates when it receives a SIGHUP - for
 example kill -HUP $(cat homeserver.pid). Alternatively, simply restart Synapse,
 though this will result in downtime while it restarts.
--- a/synapse/init.py
+++ b/synapse/init.py
@ -27,4 +27,4 @@ try:
 except ImportError:
    pass
-__version__ = "0.99.0rc3"
+__version__ = "0.99.0"
--- a/synapse/config/tls.py
+++ b/synapse/config/tls.py
@ -37,7 +37,7 @@ class TlsConfig(Config):
        self.acme_enabled = acme_config.get("enabled", False)
        self.acme_url = acme_config.get(
-            "url", "https://acme-v01.api.letsencrypt.org/directory"
+            "url", u"https://acme-v01.api.letsencrypt.org/directory"
        )
        self.acme_port = acme_config.get("port", 80)
        self.acme_bind_addresses = acme_config.get("bind_addresses", ['::', '0.0.0.0'])
@ -199,10 +199,10 @@ class TlsConfig(Config):
        # If your server runs behind a reverse-proxy which terminates TLS connections
        # (for both client and federation connections), it may be useful to disable
-        # All TLS support for incoming connections. Setting no_tls to False will
+        # All TLS support for incoming connections. Setting no_tls to True will
        # do so (and avoid the need to give synapse a TLS private key).
        #
-        # no_tls: False
+        # no_tls: True
        # List of allowed TLS fingerprints for this server to publish along
        # with the signing keys for this server. Other matrix servers that
		`@ -1 +0,0 @@`
			`Update federation routing logic to check .well-known before SRV`
		`@ -1 +0,0 @@`
			`Improve performance of handling servers with invalid .well-known`
		`@ -1 +0,0 @@`
			`Treat an invalid .well-known file the same as an absent one`