From a451d960cc389649a0f9cb7759801e82a6bf20e1 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 14:07:10 +0000 Subject: [PATCH 01/10] Add docs for ACME setup --- README.rst | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/README.rst b/README.rst index e6354ccba..a3e384ae9 100644 --- a/README.rst +++ b/README.rst @@ -227,6 +227,85 @@ to read `Using a reverse proxy with Synapse`_ when doing so. Apart from port 8448 using TLS, both ports are the same in the default configuration. +ACME setup +---------- + +Synapse requires valid TLS certificates for communication between servers +(port ``8448`` by default) in addition to those that are client-facing (port +``443``). Synapse **will provision server-to-server certificates +automatically for you for free** through `Let's Encrypt +`_ if you tell it to. + + Note: Synapse does not currently hot-renew Let's Encrypt certificates for + you, it only checks for certificates that need renewing on restart. This + functionality will be implemented promptly, but if in the meantime your + federation certificates expire, simply restarting Synapse should renew + them automatically. + +In order for Synapse to complete the ACME challenge to provision a +certificate, it needs access to port 80. Typically listening on port 80 is +only granted to applications running as root. There are thus two solutions to +this problem. + +**Using a reverse proxy** + +A reverse proxy such as Apache or Nginx allows a single process (the web +server) to listen on port 80 and redirect traffic to the appropriate program +running on your server. It is the recommended method for setting up ACME as +it allows you to use your existing webserver while also allowing Synapse to +provision certificates as needed. + +For Nginx users, add the following line to your existing ``server`` block:: + + location /.well-known/acme-challenge { + proxy_pass http://localhost:8009/; + } + +For Apache, add the following to your existing webserver config:: + + ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge + +Make sure to restart/reload your webserver after making changes. + + +**Authbind** + +``authbind`` allows a program which does not or should not run as root to +bind to low-numbered ports in a controlled way. The setup is simpler, but +requires a webserver not to already be running on port 80. **This includes +every time Synapse renews a certificate**, which may be cumbersome if you +usually run a web server on port 80. Nevertheless, if you're sure port 80 is +not being used for any other purpose then all that is necessary is the +following: + +Install ``authbind``:: + + sudo apt-get install authbind + +Allow ``authbind`` to bind port 80:: + + sudo touch /etc/authbind/byport/80 + sudo chmod 777 /etc/authbind/byport/80 + +When Synapse is started (do not start it yet), use the following syntax:: + + # authbind syntax. don't start Synapse yet + authbind --deep + +If using the `Systemd`_ service file above, you can change the following line +from:: + + ExecStart=/home/matrix/matrix-synapse/bin/python -m synapse.app.homeserver + +to:: + + ExecStart=authbind --deep /home/matrix/matrix-synapse/bin/python -m synapse.app.homeserver + + +If you would like to use your own certificates, specifying them in Synapse's +config file is sufficient. + + Registering a user ------------------ From c5fc09322cacfc3fdd49a40ca99b105111dd2914 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 14:10:44 +0000 Subject: [PATCH 02/10] Add changelog --- changelog.d/4547.misc | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/4547.misc diff --git a/changelog.d/4547.misc b/changelog.d/4547.misc new file mode 100644 index 000000000..b6e421d09 --- /dev/null +++ b/changelog.d/4547.misc @@ -0,0 +1 @@ +Add docs for ACME setup to README. \ No newline at end of file From 57fe91f87b1db242d03e1dea278d4156ba65a60d Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 14:39:50 +0000 Subject: [PATCH 03/10] Clean up portions of docs that talk about reversing fed port --- README.rst | 121 +++++++++++------------------------------------------ 1 file changed, 24 insertions(+), 97 deletions(-) diff --git a/README.rst b/README.rst index a3e384ae9..248b5fbfe 100644 --- a/README.rst +++ b/README.rst @@ -212,27 +212,26 @@ key in the ``.signing.key`` file (the second word) to something different. See `the spec`__ for more information on key management.) .. __: `key_management`_ - The default configuration exposes two HTTP ports: 8008 and 8448. Port 8008 is configured without TLS; it should be behind a reverse proxy for TLS/SSL termination on port 443 which in turn should be used for clients. Port 8448 -is configured to use TLS with a self-signed certificate. If you would like -to do initial test with a client without having to setup a reverse proxy, -you can temporarly use another certificate. (Note that a self-signed -certificate is fine for `Federation`_). You can do so by changing -``tls_certificate_path`` and ``tls_private_key_path`` -in ``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure -to read `Using a reverse proxy with Synapse`_ when doing so. +is configured to use TLS for `Federation`_ with a self-signed or verified +certificate, but please be aware that a valid certificate will be required in +Synapse v1.0. -Apart from port 8448 using TLS, both ports are the same in the default -configuration. +If you would like to do initial testing with a client without having to setup +a reverse proxy, you can temporarly use another certificate. You can do so by +changing ``tls_certificate_path`` and ``tls_private_key_path`` in +``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure +to read `Using a reverse proxy with Synapse`_ when doing so. Apart from port +8448 using TLS, both ports are the same in the default configuration. ACME setup ---------- -Synapse requires valid TLS certificates for communication between servers +Synapse v1.0 requires valid TLS certificates for communication between servers (port ``8448`` by default) in addition to those that are client-facing (port -``443``). Synapse **will provision server-to-server certificates +``443``). Synapse v0.99.0+ **will provision server-to-server certificates automatically for you for free** through `Let's Encrypt `_ if you tell it to. @@ -287,23 +286,12 @@ Allow ``authbind`` to bind port 80:: sudo touch /etc/authbind/byport/80 sudo chmod 777 /etc/authbind/byport/80 -When Synapse is started (do not start it yet), use the following syntax:: +When Synapse is started, use the following syntax:: - # authbind syntax. don't start Synapse yet authbind --deep -If using the `Systemd`_ service file above, you can change the following line -from:: - - ExecStart=/home/matrix/matrix-synapse/bin/python -m synapse.app.homeserver - -to:: - - ExecStart=authbind --deep /home/matrix/matrix-synapse/bin/python -m synapse.app.homeserver - - -If you would like to use your own certificates, specifying them in Synapse's -config file is sufficient. +If you would like to use your own certificates, simply specify them in +``homeserver.yaml``. Registering a user @@ -360,10 +348,11 @@ following the recommended setup, or ``https://localhost:8448`` - remember to spe port (``:8448``) if not ``:443`` unless you changed the configuration. (Leave the identity server as the default - see `Identity servers`_.) -If using port 8448 you will run into errors until you accept the self-signed -certificate. You can easily do this by going to ``https://localhost:8448`` +If using port 8448 you will run into errors if you are using a self-signed +certificate. To overcome this, simply go to ``https://localhost:8448`` directly with your browser and accept the presented certificate. You can then -go back in your web client and proceed further. +go back in your web client and proceed further. Valid federation certificates +should not have this problem. If all goes well you should at least be able to log in, create a room, and start sending messages. @@ -632,9 +621,7 @@ you to run your server on a machine that might not have the same name as your domain name. For example, you might want to run your server at ``synapse.example.com``, but have your Matrix user-ids look like ``@user:example.com``. (A SRV record also allows you to change the port from -the default 8448. However, if you are thinking of using a reverse-proxy on the -federation port, which is not recommended, be sure to read -`Reverse-proxying the federation port`_ first.) +the default 8448. To use a SRV record, first create your SRV record and publish it in DNS. This should have the format ``_matrix._tcp. IN SRV 10 0 @@ -736,14 +723,10 @@ port. Indeed, clients will use port 443 by default, whereas servers default to port 8448. Where these are different, we refer to the 'client port' and the 'federation port'. -The next most important thing to know is that using a reverse-proxy on the -federation port has a number of pitfalls. It is possible, but be sure to read -`Reverse-proxying the federation port`_. - -The recommended setup is therefore to configure your reverse-proxy on port 443 -to port 8008 of synapse for client connections, but to also directly expose port -8448 for server-server connections. All the Matrix endpoints begin ``/_matrix``, -so an example nginx configuration might look like:: +The recommended setup is therefore to configure your reverse-proxy on port +443 to port 8008 of synapse for client connections, and port 8448 for +server-server connections. All Matrix endpoints begin with ``/_matrix``, so an +example nginx configuration might look like:: server { listen 443 ssl; @@ -784,63 +767,7 @@ Having done so, you can then use ``https://matrix.example.com`` (instead of ``https://matrix.example.com:8448``) as the "Custom server" when `Connecting to Synapse from a client`_. -Reverse-proxying the federation port ------------------------------------- - -There are two issues to consider before using a reverse-proxy on the federation -port: - -* Due to the way SSL certificates are managed in the Matrix federation protocol - (see `spec`__), Synapse needs to be configured with the path to the SSL - certificate, *even if you do not terminate SSL at Synapse*. - - .. __: `key_management`_ - -* Until v0.33.3, Synapse did not support SNI on the federation port - (`bug #1491 `_). This bug - is now fixed, but means that federating with older servers can be unreliable - when using name-based virtual hosting. - -Furthermore, a number of the normal reasons for using a reverse-proxy do not -apply: - -* Other servers will connect on port 8448 by default, so there is no need to - listen on port 443 (for federation, at least), which avoids the need for root - privileges and virtual hosting. - -* A self-signed SSL certificate is fine for federation, so there is no need to - automate renewals. (The certificate generated by ``--generate-config`` is - valid for 10 years.) - -If you want to set up a reverse-proxy on the federation port despite these -caveats, you will need to do the following: - -* In ``homeserver.yaml``, set ``tls_certificate_path`` to the path to the SSL - certificate file used by your reverse-proxy, and set ``no_tls`` to ``True``. - (``tls_private_key_path`` will be ignored if ``no_tls`` is ``True``.) - -* In your reverse-proxy configuration: - - * If there are other virtual hosts on the same port, make sure that the - *default* one uses the certificate configured above. - - * Forward ``/_matrix`` to Synapse. - -* If your reverse-proxy is not listening on port 8448, publish a SRV record to - tell other servers how to find you. See `Setting up Federation`_. - -When updating the SSL certificate, just update the file pointed to by -``tls_certificate_path`` and then restart Synapse. (You may like to use a symbolic link -to help make this process atomic.) - -The most common mistake when setting up federation is not to tell Synapse about -your SSL certificate. To check it, you can visit -``https://matrix.org/federationtester/api/report?server_name=``. -Unfortunately, there is no UI for this yet, but, you should see -``"MatchingTLSFingerprint": true``. If not, check that -``Certificates[0].SHA256Fingerprint`` (the fingerprint of the certificate -presented by your reverse-proxy) matches ``Keys.tls_fingerprints[0].sha256`` -(the fingerprint of the certificate Synapse is using). +Please see `ACME setup`_ for details on reverse-proxying the federation port. Identity Servers From 7f914a2dbff902b25fbf01a33925329e6a00742e Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 14:52:39 +0000 Subject: [PATCH 04/10] Remove error and add link to foks fed tester project --- README.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.rst b/README.rst index 248b5fbfe..8181ff20c 100644 --- a/README.rst +++ b/README.rst @@ -212,6 +212,7 @@ key in the ``.signing.key`` file (the second word) to something different. See `the spec`__ for more information on key management.) .. __: `key_management`_ + The default configuration exposes two HTTP ports: 8008 and 8448. Port 8008 is configured without TLS; it should be behind a reverse proxy for TLS/SSL termination on port 443 which in turn should be used for clients. Port 8448 @@ -659,6 +660,8 @@ Troubleshooting You can use the federation tester to check if your homeserver is all set: ``https://matrix.org/federationtester/api/report?server_name=`` If any of the attributes under "checks" is false, federation won't work. +There is also a nicer interface available from a community member at +``_. The typical failure mode with federation is that when you try to join a room, it is rejected with "401: Unauthorized". Generally this means that other @@ -667,8 +670,6 @@ complicated dance which requires connections in both directions). So, things to check are: -* If you are trying to use a reverse-proxy, read `Reverse-proxying the - federation port`_. * If you are not using a SRV record, check that your ``server_name`` (the part of your user-id after the ``:``) matches your hostname, and that port 8448 on that hostname is reachable from outside your network. From 0d70288c595bf53a59963be89fcc35f60613c930 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 15:38:09 +0000 Subject: [PATCH 05/10] Address changes --- README.rst | 35 +++++++++++++---------------------- 1 file changed, 13 insertions(+), 22 deletions(-) diff --git a/README.rst b/README.rst index 8181ff20c..02828966c 100644 --- a/README.rst +++ b/README.rst @@ -220,28 +220,19 @@ is configured to use TLS for `Federation`_ with a self-signed or verified certificate, but please be aware that a valid certificate will be required in Synapse v1.0. -If you would like to do initial testing with a client without having to setup -a reverse proxy, you can temporarly use another certificate. You can do so by -changing ``tls_certificate_path`` and ``tls_private_key_path`` in -``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure -to read `Using a reverse proxy with Synapse`_ when doing so. Apart from port -8448 using TLS, both ports are the same in the default configuration. ACME setup ---------- Synapse v1.0 requires valid TLS certificates for communication between servers (port ``8448`` by default) in addition to those that are client-facing (port -``443``). Synapse v0.99.0+ **will provision server-to-server certificates -automatically for you for free** through `Let's Encrypt +``443``). In the case that your `server_name` config variable is the same as +the hostname that the client connects to, then the same certificate can be +used between client and federation ports without issue. Synapse v0.99.0+ +**will provision server-to-server certificates automatically for you for +free** through `Let's Encrypt `_ if you tell it to. - Note: Synapse does not currently hot-renew Let's Encrypt certificates for - you, it only checks for certificates that need renewing on restart. This - functionality will be implemented promptly, but if in the meantime your - federation certificates expire, simply restarting Synapse should renew - them automatically. - In order for Synapse to complete the ACME challenge to provision a certificate, it needs access to port 80. Typically listening on port 80 is only granted to applications running as root. There are thus two solutions to @@ -250,7 +241,7 @@ this problem. **Using a reverse proxy** A reverse proxy such as Apache or Nginx allows a single process (the web -server) to listen on port 80 and redirect traffic to the appropriate program +server) to listen on port 80 and proxy traffic to the appropriate program running on your server. It is the recommended method for setting up ACME as it allows you to use your existing webserver while also allowing Synapse to provision certificates as needed. @@ -278,7 +269,7 @@ usually run a web server on port 80. Nevertheless, if you're sure port 80 is not being used for any other purpose then all that is necessary is the following: -Install ``authbind``:: +Install ``authbind``. For example, on Debian/Ubuntu:: sudo apt-get install authbind @@ -291,9 +282,11 @@ When Synapse is started, use the following syntax:: authbind --deep -If you would like to use your own certificates, simply specify them in -``homeserver.yaml``. - +If you would like to use your own certificates, you can do so by +changing ``tls_certificate_path`` and ``tls_private_key_path`` in +``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure +to read `Using a reverse proxy with Synapse`_ when doing so. Apart from port +8448 using TLS, both ports are the same in the default configuration. Registering a user ------------------ @@ -622,7 +615,7 @@ you to run your server on a machine that might not have the same name as your domain name. For example, you might want to run your server at ``synapse.example.com``, but have your Matrix user-ids look like ``@user:example.com``. (A SRV record also allows you to change the port from -the default 8448. +the default 8448). To use a SRV record, first create your SRV record and publish it in DNS. This should have the format ``_matrix._tcp. IN SRV 10 0 @@ -768,8 +761,6 @@ Having done so, you can then use ``https://matrix.example.com`` (instead of ``https://matrix.example.com:8448``) as the "Custom server" when `Connecting to Synapse from a client`_. -Please see `ACME setup`_ for details on reverse-proxying the federation port. - Identity Servers ================ From 14de15eaa49a91178169fd8b49817360cecb4e97 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 16:48:07 +0000 Subject: [PATCH 06/10] Actually need to enable it --- README.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.rst b/README.rst index 02828966c..34b65b775 100644 --- a/README.rst +++ b/README.rst @@ -288,6 +288,14 @@ changing ``tls_certificate_path`` and ``tls_private_key_path`` in to read `Using a reverse proxy with Synapse`_ when doing so. Apart from port 8448 using TLS, both ports are the same in the default configuration. +Finally, once Synapse's is able to listen on port 80 for ACME challenge +requests, it must be told to perform ACME provisioning by setting ``enabled`` +to true under the ``acme`` section in ``homeserver.yaml``:: + + acme: + enabled: true + + Registering a user ------------------ From 897230f634ec14f3b3becdf3577231554ae5553e Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Feb 2019 16:54:32 +0000 Subject: [PATCH 07/10] Update README.rst Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com> --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index 34b65b775..4e34eaeee 100644 --- a/README.rst +++ b/README.rst @@ -224,7 +224,7 @@ Synapse v1.0. ACME setup ---------- -Synapse v1.0 requires valid TLS certificates for communication between servers +Synapse v1.0 will require valid TLS certificates for communication between servers (port ``8448`` by default) in addition to those that are client-facing (port ``443``). In the case that your `server_name` config variable is the same as the hostname that the client connects to, then the same certificate can be From 9e89a420e841be1559dd1fc98c7843403cb82314 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Feb 2019 16:54:47 +0000 Subject: [PATCH 08/10] Update README.rst Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com> --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index 4e34eaeee..fcf828528 100644 --- a/README.rst +++ b/README.rst @@ -261,7 +261,7 @@ Make sure to restart/reload your webserver after making changes. **Authbind** -``authbind`` allows a program which does not or should not run as root to +``authbind`` allows a program which does not not run as root to bind to low-numbered ports in a controlled way. The setup is simpler, but requires a webserver not to already be running on port 80. **This includes every time Synapse renews a certificate**, which may be cumbersome if you From 57164e17dace3c12706a2393385f6eefb6bc8c74 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 16:59:06 +0000 Subject: [PATCH 09/10] Address comments --- README.rst | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/README.rst b/README.rst index fcf828528..6e3ee01f2 100644 --- a/README.rst +++ b/README.rst @@ -220,6 +220,11 @@ is configured to use TLS for `Federation`_ with a self-signed or verified certificate, but please be aware that a valid certificate will be required in Synapse v1.0. +If you would like to use your own certificates, you can do so by changing +``tls_certificate_path`` and ``tls_private_key_path`` in ``homeserver.yaml``; +alternatively, you can use a reverse-proxy. Apart from port 8448 using TLS, +both ports are the same in the default configuration. + ACME setup ---------- @@ -261,13 +266,12 @@ Make sure to restart/reload your webserver after making changes. **Authbind** -``authbind`` allows a program which does not not run as root to -bind to low-numbered ports in a controlled way. The setup is simpler, but -requires a webserver not to already be running on port 80. **This includes -every time Synapse renews a certificate**, which may be cumbersome if you -usually run a web server on port 80. Nevertheless, if you're sure port 80 is -not being used for any other purpose then all that is necessary is the -following: +``authbind`` allows a program which does not run as root to bind to +low-numbered ports in a controlled way. The setup is simpler, but requires a +webserver not to already be running on port 80. **This includes every time +Synapse renews a certificate**, which may be cumbersome if you usually run a +web server on port 80. Nevertheless, if you're sure port 80 is not being used +for any other purpose then all that is necessary is the following: Install ``authbind``. For example, on Debian/Ubuntu:: @@ -282,12 +286,6 @@ When Synapse is started, use the following syntax:: authbind --deep -If you would like to use your own certificates, you can do so by -changing ``tls_certificate_path`` and ``tls_private_key_path`` in -``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure -to read `Using a reverse proxy with Synapse`_ when doing so. Apart from port -8448 using TLS, both ports are the same in the default configuration. - Finally, once Synapse's is able to listen on port 80 for ACME challenge requests, it must be told to perform ACME provisioning by setting ``enabled`` to true under the ``acme`` section in ``homeserver.yaml``:: @@ -725,10 +723,8 @@ port. Indeed, clients will use port 443 by default, whereas servers default to port 8448. Where these are different, we refer to the 'client port' and the 'federation port'. -The recommended setup is therefore to configure your reverse-proxy on port -443 to port 8008 of synapse for client connections, and port 8448 for -server-server connections. All Matrix endpoints begin with ``/_matrix``, so an -example nginx configuration might look like:: +All Matrix endpoints begin with ``/_matrix``, so an example nginx +configuration for forwarding client connections to Synapse might look like:: server { listen 443 ssl; From da6df65e195bd1a29b2d186d4ed4c988ca4f1f34 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 1 Feb 2019 16:59:23 +0000 Subject: [PATCH 10/10] Fix nginx capatilization --- README.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.rst b/README.rst index 6e3ee01f2..9e3d85de4 100644 --- a/README.rst +++ b/README.rst @@ -245,13 +245,13 @@ this problem. **Using a reverse proxy** -A reverse proxy such as Apache or Nginx allows a single process (the web +A reverse proxy such as Apache or nginx allows a single process (the web server) to listen on port 80 and proxy traffic to the appropriate program running on your server. It is the recommended method for setting up ACME as it allows you to use your existing webserver while also allowing Synapse to provision certificates as needed. -For Nginx users, add the following line to your existing ``server`` block:: +For nginx users, add the following line to your existing ``server`` block:: location /.well-known/acme-challenge { proxy_pass http://localhost:8009/;