Default to blacklisting reserved IP ranges and add a whitelist. (#8870)

This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
This commit is contained in:
Patrick Cloke 2020-12-09 13:56:06 -05:00 committed by GitHub
parent 6ff34e00d9
commit 344ab0b53a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 172 additions and 89 deletions

View File

@ -557,10 +557,9 @@ This is critical from a security perspective to stop arbitrary Matrix users
spidering 'internal' URLs on your network. At the very least we recommend that spidering 'internal' URLs on your network. At the very least we recommend that
your loopback and RFC1918 IP addresses are blacklisted. your loopback and RFC1918 IP addresses are blacklisted.
This also requires the optional `lxml` and `netaddr` python dependencies to be This also requires the optional `lxml` python dependency to be installed. This
installed. This in turn requires the `libxml2` library to be available - on in turn requires the `libxml2` library to be available - on Debian/Ubuntu this
Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for means `apt-get install libxml2-dev`, or equivalent for your OS.
your OS.
# Troubleshooting Installation # Troubleshooting Installation

View File

@ -75,6 +75,27 @@ for example:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.25.0
====================
Blacklisting IP ranges
----------------------
Synapse v1.25.0 includes new settings, ``ip_range_blacklist`` and
``ip_range_whitelist``, for controlling outgoing requests from Synapse for federation,
identity servers, push, and for checking key validity for third-party invite events.
The previous setting, ``federation_ip_range_blacklist``, is deprecated. The new
``ip_range_blacklist`` defaults to private IP ranges if it is not defined.
If you have never customised ``federation_ip_range_blacklist`` it is recommended
that you remove that setting.
If you have customised ``federation_ip_range_blacklist`` you should update the
setting name to ``ip_range_blacklist``.
If you have a custom push server that is reached via private IP space you may
need to customise ``ip_range_blacklist`` or ``ip_range_whitelist``.
Upgrading to v1.24.0 Upgrading to v1.24.0
==================== ====================

View File

@ -1 +1 @@
Apply the `federation_ip_range_blacklist` to push and key revocation requests. Apply an IP range blacklist to push and key revocation requests.

1
changelog.d/8870.bugfix Normal file
View File

@ -0,0 +1 @@
Apply an IP range blacklist to push and key revocation requests.

View File

@ -144,6 +144,35 @@ pid_file: DATADIR/homeserver.pid
# #
#enable_search: false #enable_search: false
# Prevent outgoing requests from being sent to the following blacklisted IP address
# CIDR ranges. If this option is not specified then it defaults to private IP
# address ranges (see the example below).
#
# The blacklist applies to the outbound requests for federation, identity servers,
# push servers, and for checking key validity for third-party invite events.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
# This option replaces federation_ip_range_blacklist in Synapse v1.25.0.
#
#ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
# - '172.16.0.0/12'
# - '192.168.0.0/16'
# - '100.64.0.0/10'
# - '192.0.0.0/24'
# - '169.254.0.0/16'
# - '198.18.0.0/15'
# - '192.0.2.0/24'
# - '198.51.100.0/24'
# - '203.0.113.0/24'
# - '224.0.0.0/4'
# - '::1/128'
# - 'fe80::/10'
# - 'fc00::/7'
# List of ports that Synapse should listen on, their purpose and their # List of ports that Synapse should listen on, their purpose and their
# configuration. # configuration.
# #
@ -642,28 +671,17 @@ acme:
# - nyc.example.com # - nyc.example.com
# - syd.example.com # - syd.example.com
# Prevent outgoing requests from being sent to the following blacklisted IP address # List of IP address CIDR ranges that should be allowed for federation,
# CIDR ranges. If this option is not specified, or specified with an empty list, # identity servers, push servers, and for checking key validity for
# no IP range blacklist will be enforced. # third-party invite events. This is useful for specifying exceptions to
# wide-ranging blacklisted target IP ranges - e.g. for communication with
# a push server only visible in your network.
# #
# The blacklist applies to the outbound requests for federation, identity servers, # This whitelist overrides ip_range_blacklist and defaults to an empty
# push servers, and for checking key validitity for third-party invite events. # list.
# #
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly #ip_range_whitelist:
# listed here, since they correspond to unroutable addresses.) # - '192.168.1.1'
#
# This option replaces federation_ip_range_blacklist in Synapse v1.24.0.
#
ip_range_blacklist:
- '127.0.0.0/8'
- '10.0.0.0/8'
- '172.16.0.0/12'
- '192.168.0.0/16'
- '100.64.0.0/10'
- '169.254.0.0/16'
- '::1/128'
- 'fe80::/64'
- 'fc00::/7'
# Report prometheus metrics on the age of PDUs being sent to and received from # Report prometheus metrics on the age of PDUs being sent to and received from
# the following domains. This can be used to give an idea of "delay" on inbound # the following domains. This can be used to give an idea of "delay" on inbound
@ -955,9 +973,15 @@ media_store_path: "DATADIR/media_store"
# - '172.16.0.0/12' # - '172.16.0.0/12'
# - '192.168.0.0/16' # - '192.168.0.0/16'
# - '100.64.0.0/10' # - '100.64.0.0/10'
# - '192.0.0.0/24'
# - '169.254.0.0/16' # - '169.254.0.0/16'
# - '198.18.0.0/15'
# - '192.0.2.0/24'
# - '198.51.100.0/24'
# - '203.0.113.0/24'
# - '224.0.0.0/4'
# - '::1/128' # - '::1/128'
# - 'fe80::/64' # - 'fe80::/10'
# - 'fc00::/7' # - 'fc00::/7'
# List of IP address CIDR ranges that the URL preview spider is allowed # List of IP address CIDR ranges that the URL preview spider is allowed

View File

@ -12,12 +12,9 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from typing import Optional from typing import Optional
from netaddr import IPSet from synapse.config._base import Config
from synapse.config._base import Config, ConfigError
from synapse.config._util import validate_config from synapse.config._util import validate_config
@ -36,31 +33,6 @@ class FederationConfig(Config):
for domain in federation_domain_whitelist: for domain in federation_domain_whitelist:
self.federation_domain_whitelist[domain] = True self.federation_domain_whitelist[domain] = True
ip_range_blacklist = config.get("ip_range_blacklist", [])
# Attempt to create an IPSet from the given ranges
try:
self.ip_range_blacklist = IPSet(ip_range_blacklist)
except Exception as e:
raise ConfigError("Invalid range(s) provided in ip_range_blacklist: %s" % e)
# Always blacklist 0.0.0.0, ::
self.ip_range_blacklist.update(["0.0.0.0", "::"])
# The federation_ip_range_blacklist is used for backwards-compatibility
# and only applies to federation and identity servers. If it is not given,
# default to ip_range_blacklist.
federation_ip_range_blacklist = config.get(
"federation_ip_range_blacklist", ip_range_blacklist
)
try:
self.federation_ip_range_blacklist = IPSet(federation_ip_range_blacklist)
except Exception as e:
raise ConfigError(
"Invalid range(s) provided in federation_ip_range_blacklist: %s" % e
)
# Always blacklist 0.0.0.0, ::
self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])
federation_metrics_domains = config.get("federation_metrics_domains") or [] federation_metrics_domains = config.get("federation_metrics_domains") or []
validate_config( validate_config(
_METRICS_FOR_DOMAINS_SCHEMA, _METRICS_FOR_DOMAINS_SCHEMA,
@ -84,28 +56,17 @@ class FederationConfig(Config):
# - nyc.example.com # - nyc.example.com
# - syd.example.com # - syd.example.com
# Prevent outgoing requests from being sent to the following blacklisted IP address # List of IP address CIDR ranges that should be allowed for federation,
# CIDR ranges. If this option is not specified, or specified with an empty list, # identity servers, push servers, and for checking key validity for
# no IP range blacklist will be enforced. # third-party invite events. This is useful for specifying exceptions to
# wide-ranging blacklisted target IP ranges - e.g. for communication with
# a push server only visible in your network.
# #
# The blacklist applies to the outbound requests for federation, identity servers, # This whitelist overrides ip_range_blacklist and defaults to an empty
# push servers, and for checking key validitity for third-party invite events. # list.
# #
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly #ip_range_whitelist:
# listed here, since they correspond to unroutable addresses.) # - '192.168.1.1'
#
# This option replaces federation_ip_range_blacklist in Synapse v1.24.0.
#
ip_range_blacklist:
- '127.0.0.0/8'
- '10.0.0.0/8'
- '172.16.0.0/12'
- '192.168.0.0/16'
- '100.64.0.0/10'
- '169.254.0.0/16'
- '::1/128'
- 'fe80::/64'
- 'fc00::/7'
# Report prometheus metrics on the age of PDUs being sent to and received from # Report prometheus metrics on the age of PDUs being sent to and received from
# the following domains. This can be used to give an idea of "delay" on inbound # the following domains. This can be used to give an idea of "delay" on inbound

View File

@ -17,6 +17,9 @@ import os
from collections import namedtuple from collections import namedtuple
from typing import Dict, List from typing import Dict, List
from netaddr import IPSet
from synapse.config.server import DEFAULT_IP_RANGE_BLACKLIST
from synapse.python_dependencies import DependencyException, check_requirements from synapse.python_dependencies import DependencyException, check_requirements
from synapse.util.module_loader import load_module from synapse.util.module_loader import load_module
@ -184,9 +187,6 @@ class ContentRepositoryConfig(Config):
"to work" "to work"
) )
# netaddr is a dependency for url_preview
from netaddr import IPSet
self.url_preview_ip_range_blacklist = IPSet( self.url_preview_ip_range_blacklist = IPSet(
config["url_preview_ip_range_blacklist"] config["url_preview_ip_range_blacklist"]
) )
@ -215,6 +215,10 @@ class ContentRepositoryConfig(Config):
# strip final NL # strip final NL
formatted_thumbnail_sizes = formatted_thumbnail_sizes[:-1] formatted_thumbnail_sizes = formatted_thumbnail_sizes[:-1]
ip_range_blacklist = "\n".join(
" # - '%s'" % ip for ip in DEFAULT_IP_RANGE_BLACKLIST
)
return ( return (
r""" r"""
## Media Store ## ## Media Store ##
@ -285,15 +289,7 @@ class ContentRepositoryConfig(Config):
# you uncomment the following list as a starting point. # you uncomment the following list as a starting point.
# #
#url_preview_ip_range_blacklist: #url_preview_ip_range_blacklist:
# - '127.0.0.0/8' %(ip_range_blacklist)s
# - '10.0.0.0/8'
# - '172.16.0.0/12'
# - '192.168.0.0/16'
# - '100.64.0.0/10'
# - '169.254.0.0/16'
# - '::1/128'
# - 'fe80::/64'
# - 'fc00::/7'
# List of IP address CIDR ranges that the URL preview spider is allowed # List of IP address CIDR ranges that the URL preview spider is allowed
# to access even if they are specified in url_preview_ip_range_blacklist. # to access even if they are specified in url_preview_ip_range_blacklist.

View File

@ -23,6 +23,7 @@ from typing import Any, Dict, Iterable, List, Optional, Set
import attr import attr
import yaml import yaml
from netaddr import IPSet
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS from synapse.api.room_versions import KNOWN_ROOM_VERSIONS
from synapse.http.endpoint import parse_and_validate_server_name from synapse.http.endpoint import parse_and_validate_server_name
@ -39,6 +40,34 @@ logger = logging.Logger(__name__)
# in the list. # in the list.
DEFAULT_BIND_ADDRESSES = ["::", "0.0.0.0"] DEFAULT_BIND_ADDRESSES = ["::", "0.0.0.0"]
DEFAULT_IP_RANGE_BLACKLIST = [
# Localhost
"127.0.0.0/8",
# Private networks.
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
# Carrier grade NAT.
"100.64.0.0/10",
# Address registry.
"192.0.0.0/24",
# Link-local networks.
"169.254.0.0/16",
# Testing networks.
"198.18.0.0/15",
"192.0.2.0/24",
"198.51.100.0/24",
"203.0.113.0/24",
# Multicast.
"224.0.0.0/4",
# Localhost
"::1/128",
# Link-local addresses.
"fe80::/10",
# Unique local addresses.
"fc00::/7",
]
DEFAULT_ROOM_VERSION = "6" DEFAULT_ROOM_VERSION = "6"
ROOM_COMPLEXITY_TOO_GREAT = ( ROOM_COMPLEXITY_TOO_GREAT = (
@ -256,6 +285,38 @@ class ServerConfig(Config):
# due to resource constraints # due to resource constraints
self.admin_contact = config.get("admin_contact", None) self.admin_contact = config.get("admin_contact", None)
ip_range_blacklist = config.get(
"ip_range_blacklist", DEFAULT_IP_RANGE_BLACKLIST
)
# Attempt to create an IPSet from the given ranges
try:
self.ip_range_blacklist = IPSet(ip_range_blacklist)
except Exception as e:
raise ConfigError("Invalid range(s) provided in ip_range_blacklist.") from e
# Always blacklist 0.0.0.0, ::
self.ip_range_blacklist.update(["0.0.0.0", "::"])
try:
self.ip_range_whitelist = IPSet(config.get("ip_range_whitelist", ()))
except Exception as e:
raise ConfigError("Invalid range(s) provided in ip_range_whitelist.") from e
# The federation_ip_range_blacklist is used for backwards-compatibility
# and only applies to federation and identity servers. If it is not given,
# default to ip_range_blacklist.
federation_ip_range_blacklist = config.get(
"federation_ip_range_blacklist", ip_range_blacklist
)
try:
self.federation_ip_range_blacklist = IPSet(federation_ip_range_blacklist)
except Exception as e:
raise ConfigError(
"Invalid range(s) provided in federation_ip_range_blacklist."
) from e
# Always blacklist 0.0.0.0, ::
self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])
if self.public_baseurl is not None: if self.public_baseurl is not None:
if self.public_baseurl[-1] != "/": if self.public_baseurl[-1] != "/":
self.public_baseurl += "/" self.public_baseurl += "/"
@ -561,6 +622,10 @@ class ServerConfig(Config):
def generate_config_section( def generate_config_section(
self, server_name, data_dir_path, open_private_ports, listeners, **kwargs self, server_name, data_dir_path, open_private_ports, listeners, **kwargs
): ):
ip_range_blacklist = "\n".join(
" # - '%s'" % ip for ip in DEFAULT_IP_RANGE_BLACKLIST
)
_, bind_port = parse_and_validate_server_name(server_name) _, bind_port = parse_and_validate_server_name(server_name)
if bind_port is not None: if bind_port is not None:
unsecure_port = bind_port - 400 unsecure_port = bind_port - 400
@ -752,6 +817,21 @@ class ServerConfig(Config):
# #
#enable_search: false #enable_search: false
# Prevent outgoing requests from being sent to the following blacklisted IP address
# CIDR ranges. If this option is not specified then it defaults to private IP
# address ranges (see the example below).
#
# The blacklist applies to the outbound requests for federation, identity servers,
# push servers, and for checking key validity for third-party invite events.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
# This option replaces federation_ip_range_blacklist in Synapse v1.25.0.
#
#ip_range_blacklist:
%(ip_range_blacklist)s
# List of ports that Synapse should listen on, their purpose and their # List of ports that Synapse should listen on, their purpose and their
# configuration. # configuration.
# #

View File

@ -370,10 +370,11 @@ class HomeServer(metaclass=abc.ABCMeta):
def get_proxied_blacklisted_http_client(self) -> SimpleHttpClient: def get_proxied_blacklisted_http_client(self) -> SimpleHttpClient:
""" """
An HTTP client that uses configured HTTP(S) proxies and blacklists IPs An HTTP client that uses configured HTTP(S) proxies and blacklists IPs
based on the IP range blacklist. based on the IP range blacklist/whitelist.
""" """
return SimpleHttpClient( return SimpleHttpClient(
self, self,
ip_whitelist=self.config.ip_range_whitelist,
ip_blacklist=self.config.ip_range_blacklist, ip_blacklist=self.config.ip_range_blacklist,
http_proxy=os.getenvb(b"http_proxy"), http_proxy=os.getenvb(b"http_proxy"),
https_proxy=os.getenvb(b"HTTPS_PROXY"), https_proxy=os.getenvb(b"HTTPS_PROXY"),

View File

@ -48,7 +48,7 @@ class MediaRepoShardTestCase(BaseMultiWorkerStreamTestCase):
self.user_id = self.register_user("user", "pass") self.user_id = self.register_user("user", "pass")
self.access_token = self.login("user", "pass") self.access_token = self.login("user", "pass")
self.reactor.lookups["example.com"] = "127.0.0.2" self.reactor.lookups["example.com"] = "1.2.3.4"
def default_config(self): def default_config(self):
conf = super().default_config() conf = super().default_config()