documentation/docs/hide-instance-behind-proxy-server.md

Hiding Invidious instance behind proxy servers (for escaping DMCA requests)

This tutorial has been writen by unixfox, owner of yewtu.be

Synopsis

This tutorial will explain how to hide your Invidious (public) instance behind another server, useful for escaping the DMCA requests.

This proxy server will only redirect the TCP connections, allowing you to keep everything from your existing infrastructure.

??? note "It's possible to do it by forwarding the HTTP connections directly but..."

By forwarding the actual HTTP protocol this is more compute intensive and won't be covered in this tutorial.

You need to proxy the HTTP protocol normally like you would already do with your current web server for invidious. But in this case from the proxy server to your existing infrastructure. Then also setup the certificates for HTTPS.

Then you optionally preserve the IP address of your clients using for example on NGINX set_real_ip_from and real_ip_header.

Warning

Adding an extra layer like a proxy server will add more latency, thus increase the response time from your final server to your browser. That's why for the part where you are going to rent a proxy server, you should choose a server close to your existing server(s).

It should be noted, in the 4th step of the instructions, you will be able to remove this latency for a major part of the users of your instance (around 90% - people using modern browsers). The rest will experience this added latency which won't result in a decrease of the user experience if you choose your server wisely.

Requirements

• Your main web server should support the proxy protocol, it's possible to do it without it (later on in the tutorial). Non-exhaustive list of web servers that support it:

  • NGINX (recommended web server for this tutorial)
  • Traefik
  • Apache (doesn't come in the official version)
  • Caddy (need to build with a custom module)

• You will have to spend a bit more money per month for renting this new proxy server.

Instructions

1) Renting your proxy server

There are a lot of providers that don't care about DMCA requests, you can find them by doing a search on your favorite search engine.
Here are some lists compiled from lowendtalk users:

• https://lowendtalk.com/discussion/182615/are-there-any-active-dmca-free-vps-offers-from-top-providers (webarchive link)
• https://lowendtalk.com/discussion/181443/options-for-dmca-free-vps (webarchive link)
• https://lowendtalk.com/discussion/179472/what-are-my-options-for-a-dmca-free-vps (webarchive link)

I personally chose BuyVM as it's close to my existing infrastructure and it's cheap, but please try to at least find another one, it's much better to avoid the centralisation.

Here are the recommended requirements for this proxy server:

• At least 256MB of RAM in total if you are running Debian or 512MB with Ubuntu.
• The outgoing bandwidth limit/quota should be the same as the outgoing bandwidth consumption of your existing server that run Invidious.
  Later in the tutorial I'll explain you how to reduce the bandwidth consumption on the proxy server but this won't drastically reduce it by a lot.
  TL;DR: If you have "unlimited" bandwidth limit it's much better.
• 1 CPU core.
• The proxy server should be close to your existing server(s), for minimal latency overhead.

??? note "You can monitor the bandwidth usage of your servers using vnStat."

Documentation: https://wiki.archlinux.org/title/vnStat  
Column `rx`: receive or incoming bandwidth  
Column `tx`: Transfer or outgoing bandwidth  

1. Install vnStat from your package manager, example on debian/ubuntu: `apt install vnstat`
2. Start vnStat: `systemctl enable --now vnstat`
3. In a few hours execute the command `vnstat`

If you run into performance issue on the proxy server, feel free to switch to a new provider.

And if you need help in choosing the ideal server, please seek for help on our Matrix room or IRC channel.

2) Listen to new ports on your web server of your existing infrastructure

On the web server of your existing infrastructure/server that currently host your Invidious, listen on another port with the proxy protocol enabled for the HTTPS/TLS port.

In this tutorial I'll use the port 8443, but you can choose any port you would want.
You don't need to setup a new port for the HTTP (cleartext) port as the proxy protocol is only for preserving the IP address of the clients that will connect your web server and nowadays all the requests are permanently redirected to HTTPS.

??? note "If you don't want to preserve the IP address of the users of your instance."

Meaning keeping your instance truly 100% anonymous, then you don't need to enable the proxy protocol on another port, you can keep your current configuration without touching anything and jump to the 3rd step.  

But preserving the IP address is in my opinion essential for blocking bots, bad actors.

NGINX

In the NGINX configuration file for Invidious, just after the line listen 443 ssl http2 add this line:

listen 8443 ssl http2 proxy_protocol;
set_real_ip_from PUBLIC_IPV4_ADDRESS_OF_PROXY_SERVER/32;
real_ip_header proxy_protocol;

Very simple example of a final result:

http {
    server {
        listen 443 ssl http2;
        listen 8443 ssl http2 proxy_protocol;
        set_real_ip_from PUBLIC_IPV4_ADDRESS_OF_PROXY_SERVER/32;
        real_ip_header proxy_protocol;
    }
}

Note: You may not have http2 parameter for the listen 443 ssl line, if you don't have it enabled, it's hugely recommended to enable it for better performance and user experience.

If you are going to follow the 4th step, please remove http2 parameter from the line listen 8443 ssl http2 proxy_protocol, it may clash with the technique.

Traefik

CLI parameters, adapt to your platform (docker compose, kubernetes or local traefik):

--entryPoints.websecure.address=:8443
--entryPoints.websecure.proxyProtocol.trustedIPs=PUBLIC_IPV4_ADDRESS_OF_PROXY_SERVER/32

Please see the documentation: https://doc.traefik.io/traefik/routing/entrypoints/#proxyprotocol

For people on Kubernetes, this may help you: https://github.com/traefik/traefik-helm-chart/issues/404

Caddy

(Not tested) On caddy it is not possible to listen on a port that support proxy protocol and another port without the proxy protocol, thus you won't be able to follow the step 4.
Or you can follow the step 4 but you need to not use the proxy protocol (not touching your configuration), thus losing the ability to preserve the IP address of the clients.

1. Compile caddy with this module: https://github.com/mastercactapus/caddy2-proxyprotocol
2. Enable the proxy protocol like so: https://github.com/mastercactapus/caddy2-proxyprotocol#caddyfile

Apache

Apache is not the ideal web server for using proxy protocol because it's still in alpha stage and you need to compile apache using an external module like explained here: https://www.scaleway.com/en/docs/tutorials/proxy-protocol-v2-load-balancer/#configuring-proxy-protocol-in-apache-web-server

Either you do that or you don't preserve the IP address of the clients by not touching your configuration but I can't help you more.

Other web servers

I won't cover all the possible web servers, look at the documentation of your web server and make it listen on port 8443 (or another port) with the proxy protocol enabled on it. Also try to find a way to preserve the IP address if needed.

You can also not touch anything about your web server configuration and follow the note in the 3rd step.

3) Install HAProxy on the proxy server

On the proxy server that you just rented.

1. Install haproxy from your package manager, on debian/ubuntu: apt install haproxy.

2. Edit the file /etc/haproxy/haproxy.cfg and replace it with this configuration:

   global
       log /dev/null    local0 info alert
       log /dev/null    local1 notice alert
       chroot /var/lib/haproxy
       stats socket /run/haproxy/admin.sock mode 660 level admin
       stats timeout 30s
       user haproxy
       group haproxy
       daemon
   
   defaults
       log	global
       mode	http
   
   frontend https
       bind :::443 v4v6
       mode tcp
       use_backend https
   
   backend https
       balance roundrobin
       mode tcp
       server server1 PUBLIC_IPV4_ADDRESS_OF_ORIGINAL_SERVER:8443 check send-proxy
   
   frontend http
       bind :::80 v4v6
       mode tcp
       use_backend http
   
   backend http
       balance roundrobin
       mode tcp
       server server1 PUBLIC_IPV4_ADDRESS_OF_ORIGINAL_SERVER:80 check
   

   If you didn't enable the proxy protocol on your server then remove send-proxy for the backend https and replace the port 8443 with 443.

   In case you have more servers, add the other servers after the two lines server server1 with their corresponding IP addresses.

   For caddy if you have enabled the proxy protocol you need to also add send-proxy for the backend http.

3. Restart HAProxy using systemctl restart haproxy.

4. Test if your setup is working by either temporarily update your local Hosts file to the IP address of your proxy server. Or you can use curl for that:

   curl https://yourdomain.com/ --resolve yourdomain.com:443:PUBLIC_IPV4_ADDRESS_OF_PROXY_SERVER
   

5. If everything works well, you can now update the DNS entries of the domain for your Invidious instance to the IPv4 (and IPv6) address of your proxy server.

4) Optionally reduce the traffic going through the proxy server

There is a technique in the HTTP protocol that allow to redirect a client (e.g. a browser) to another server, it's the Alt-Svc header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Alt-Svc.

Unfortunately it only works for Firefox browsers as Chrome doesn't support the ability to use a different domain on HTTP2 with Alt-Svc but it does support it for HTTP3 which you can later on enable too with Alt-Svc.

I'll use the terminology original server(s) for the server(s) of your existing infrastructure, not the proxy server.

Requirement

You need to have HTTP2 enabled on your web server, on NGINX it's as simple as adding http2 in the listen ssl line.
You can check that in your config or here: https://tools.keycdn.com/http2-test

Instructions

1. Go to your instance and press F12 for the dev tools, then go to the "Network" tab and click on the latest request.
   Then check in Remote Address (Chrome) or Address (Firefox you need to click on >) if you see the IP address of your proxy server.
2. Add a new DNS record for the IPv4 (and IPv6) of your original server(s) to a subdomain, like original.yourdomain.com.
3. In your existing web server add this new HTTP header:

   alt-svc: h2="original.yourdomain.com:443"; ma=86400
   

   Here is how to do it for:
   • NGINX: add_header alt-svc 'h2="original.yourdomain.com:443"; ma=86400';
   • Traefik (or read the doc): traefik.http.middlewares.altsvc.headers.customresponseheaders.alt-svc=h2="original.yourdomain.com:443"; ma=86400
   • Caddy: header alt-svc h2="original.yourdomain.com:443"; ma=86400
   • Apache: Header set alt-svc 'h2="original.yourdomain.com:443"; ma=86400'
4. Restart your web server
5. Go to your instance and press F12 for the dev tools, then go to the "Network" tab and click on the latest request.
   Then check in Remote Address (Chrome) or Address (Firefox you need to click on >) if you see the IP address of your original server(s).
   You may need to refresh the page.

Bonus point if you have HTTP3 on your web server

Different web servers that support HTTP3:

• Traefik, you can enable HTTP3: https://doc.traefik.io/traefik/routing/entrypoints/#http3
• NGINX, there is a tutorial here: https://www.nginx.com/blog/binary-packages-for-preview-nginx-quic-http3-implementation/
• Caddy, it's already enabled by default

1. Whitelist the port UDP 443 in your firewall.
2. Edit the previously added HTTP header by adapting it like this:

   alt-svc: h3="original.yourdomain.com:443"; ma=86400, h2="original.yourdomain.com:443"; ma=86400
   

3. Restart your web server. Check in the dev tools if you still see the IP address of your original server(s) and the HTTP3 protocol should be also displayed.