Add full deployment with docker-compose & comprehensive instructions

2023-07-12 16:43:29 -07:00 · 2023-07-12 16:43:29 -07:00 · e2fa816ab6
commit e2fa816ab6
parent 2f7da4ed26
6 changed files with 429 additions and 158 deletions
--- a/README.md
+++ b/README.md
@ -1,36 +1,115 @@
 # Relayer for Tornado Cash [![Build Status](https://github.com/tornadocash/relayer/workflows/build/badge.svg)](https://github.com/tornadocash/relayer/actions) [![Docker Image Version (latest semver)](https://img.shields.io/docker/v/tornadocash/relayer/light?logo=docker&logoColor=%23FFFFFF&sort=semver)](https://hub.docker.com/repository/docker/tornadocash/relayer)

+***Tornado Cash was sanctioned by the US Treasury on  08/08/2022, this makes it illegal for US citizens to interact with  Tornado Cash and all of it's associated deployed smart contracts. Please understand the laws where you live and take all necessary steps to  protect and anonymize yourself.**
+
+***It is recommended to run your Relayer on a VPS instance ([Virtual Private Server](https://njal.la/)). Ensure SSH configuration is enabled for security, you can find information about SSH keygen and management [here](https://www.ssh.com/academy/ssh/keygen).**
+
 ## Deploy with docker-compose

-docker-compose.yml contains a stack that will automatically provision SSL certificates for your domain name and will add a https redirect to port 80.
+__PREREQUISITES__

-1. Download [docker-compose.yml](/docker-compose.yml) and [.env.example](/.env.example)
+1. Update core dependencies 

-```
-wget https://raw.githubusercontent.com/tornadocash/tornado-relayer/light/docker-compose.yml
-wget https://raw.githubusercontent.com/tornadocash/tornado-relayer/light/.env.example -O .env
-```
+  - `sudo apt-get update` 

-2. Setup environment variables
+2. Install docker-compose

-   - set `NET_ID` (1 for mainnet, 5 for Goerli)
-   - set `HTTP_RPC_URL` rpc url for your ethereum node
-   - set `PRIVATE_KEY` for your relayer address (without 0x prefix)
-   - set `VIRTUAL_HOST` and `LETSENCRYPT_HOST` to your domain and add DNS record pointing to your relayer ip address
-   - set `REGULAR_TORNADO_WITHDRAW_FEE` - fee in % that is used for tornado pool withdrawals
-   - set `REWARD_ACCOUNT` - eth address that is used to collect fees
-   - update `CONFIRMATIONS` if needed - how many block confirmations to wait before processing an event. Not recommended to set less than 3
-   - update `MAX_GAS_PRICE` if needed - maximum value of gwei value for relayer's transaction
+  - `curl -SL https://github.com/docker/compose/releases/download/v2.16.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose`

-     If you want to use more than 1 eth address for relaying transactions, please add as many `workers` as you want. For example, you can comment out `worker2` in docker-compose.yml file, but please use a different `PRIVATE_KEY` for each worker.
+3. Install Docker

-3. Run `docker-compose up -d`
+  - `curl -fsSL https://get.docker.com -o get-docker.sh && chmod +x get-docker.sh && ./get-docker.sh` 

-## Run locally
+4. Install git
+
+  - `sudo apt-get install git-all` 
+
+5. Install nginx
+
+  - `sudo apt install nginx` 
+
+6. Stop apache2 instance (enabled by default)
+
+  - `sudo systemctl stop apache2`
+
+__FIREWALL CONFIGURATION__ 
+
+_* Warning: Failure to configure SSH as the first UFW rule, will lock you out of the instance_ 
+
+1. Make sure UFW is installed by running `apt update` and `apt install ufw` 
+2. Allow SSH in the first position in UFW by running `ufw insert 1 allow ssh`*
+3. Allow HTTP, and HTTPS by running `ufw allow https/tcp/http`
+4. Finalize changes and enable firewall `ufw enable`
+
+__NGINX REVERSE PROXY__
+
+1. Copy the pre-modified nginx policy as your default policy 
+
+  - `cp tornado.conf /etc/nginx/sites-available/default` 
+
+2. Append the default nginx configuration to include streams
+
+  - `echo "stream {  map_hash_bucket_size 128;  map_hash_max_size 128;  include /etc/nginx/conf.d/streams/*.conf; }" >> /etc/nginx/nginx.conf`
+
+3. Create the stream configuration
+
+  - `mkdir /etc/nginx/conf.d/streams && cp tornado-stream.conf /etc/nginx/conf.d/streams/tornado-stream.conf`
+
+4. Start nginx to make sure the configuration is correct 
+
+  - `sudo systemctl restart nginx`
+
+5. Stop nginx
+
+  - `sudo systemctl stop nginx`
+
+__DEPLOYMENT__
+
+1. Clone the repository and enter the directory 
+
+  - `git clone https://git.tornado.ws/tornadocash/classic-relayer -b sidechain-v4 && cd classic-relayer`
+
+2. Check environment files:
+
+   By default each network is preconfigured the naming of `.env.<NETWORK>`
+
+   - `.env.bsc` for Binance Smart Chain
+
+   - `.env.arb` for Arbitrum
+
+   - `.env.op` for Optimism
+
+   - `.env.gnosis` for Gnosis (xdai)
+
+   - `.env.polygon` for Polygon (matic)
+
+   - `.env.avax` for Avalanche C-Chain
+
+ 3. Configure (fill) environment files for those networks on which the relayer will be deployed:
+
+    - Set `PRIVATE_KEY` to your relayer address (remove the 0x from your private key) to each environment file (*It is recommended not to reuse the same private keys for each network as a security measure*)
+
+    - Set `VIRTUAL_HOST` and `LETSENCRYPT_HOST` a unique subndomain for every network to each environment file, eg: `mainnet.example.com` for Ethereum, `binance.example.com` for Binance etc
+    - Add a A wildcard record DNS record with the value assigned to your instance IP address to configure subdomains
+    - Set `RELAYER_FEE` to what you would like to charge as your fee
+    - Set `RPC_URL` to a non-censoring RPC (You can [run your own](https://github.com/feshchenkod/rpc-nodes), or use a [free option](https://chainnodes.org/))
+    - Set `ORACLE_RPC_URL` to an Ethereum native RPC endpoint
+
+ 4. Build docker image for sidechain with simple `npm run build` command
+
+ 5. Uncomment the `env_file` lines (remove `# `) for the associated network services in `docker-compose.yml` for chosen chains (networks)
+
+ 6. Run docker-compose for the configured networks specified via `--profile <NETWORK_SYMBOL>`, for example (if you run relayer only Binance Smart Chain and Arbitrum):
+
+    - `docker-compose --profile bsc --profile arb up -d`
+
+ 7. Visit your domain addresses and check each `/status` endpoint to ensure there is no errors in the `status` fields
+
+## Run locally for one chain

 1. `yarn`
 2. `cp .env.example .env`
-3. Modify `.env` as needed
+3. Modify `.env` as needed (described above)
 4. `yarn start`
 5. Go to `http://127.0.0.1:8000`
 6. In order to execute withdraw request, you can run following command
--- a/docker-compose.test.yml
+++ b/docker-compose.test.yml
@ -1,53 +0,0 @@
-version: '2'
-
-# ssh-agent && ssh-add -K ~/.ssh/id_rsa
-# DOCKER_BUILDKIT=1 docker build --ssh default -t tornadocash/relayer .
-services:
-  server:
-    image: tornadocash/relayer
-    restart: always
-    command: server
-    env_file: .env
-    ports:
-      - 8000:8000
-    environment:
-      REDIS_URL: redis://redis/0
-      nginx_proxy_read_timeout: 600
-    depends_on: [redis]
-
-  healthWatcher:
-    image: tornadocash/relayer
-    restart: always
-    command: healthWatcher
-    env_file: .env
-    environment:
-      REDIS_URL: redis://redis/0
-    depends_on: [redis]
-
-  worker1:
-    image: tornadocash/relayer
-    restart: always
-    command: worker
-    env_file: .env
-    environment:
-      REDIS_URL: redis://redis/0
-    depends_on: [redis]
-
-  #  worker2:
-  #    image: tornadocash/relayer
-  #    restart: always
-  #    command: worker
-  #    env_file: .env
-  #    environment:
-  #      PRIVATE_KEY: qwe
-  #      REDIS_URL: redis://redis/0
-
-  redis:
-    image: redis
-    restart: always
-    command: [redis-server, --appendonly, 'yes']
-    volumes:
-      - redis:/data
-
-volumes:
-  redis:
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -1,91 +1,6 @@
 version: '2'

 services:
-  server:
-    image: tornadocash/relayer:sidechain-beta
-    restart: always
-    command: server
-    env_file: .env
-    environment:
-      REDIS_URL: redis://redis/0
-      nginx_proxy_read_timeout: 600
-    depends_on: [redis]
-
-  healthWatcher:
-    image: tornadocash/relayer:sidechain-beta
-    restart: always
-    command: healthWatcher
-    env_file: .env
-    environment:
-      REDIS_URL: redis://redis/0
-    depends_on: [redis]
-
-  worker1:
-    image: tornadocash/relayer:sidechain-beta
-    restart: always
-    command: worker
-    env_file: .env
-    environment:
-      REDIS_URL: redis://redis/0
-    depends_on: [redis]
-
-  #  worker2:
-  #    image: tornadocash/relayer:light
-  #    restart: always
-  #    command: worker
-  #    env_file: .env
-  #    environment:
-  #      PRIVATE_KEY: qwe
-  #      REDIS_URL: redis://redis/0
-
-  #  # this container will proxy *.onion domain to the server container
-  #  # if you want to run *only* as .onion service, you don't need `nginx`, `letsencrypt`, `dockergen` containers
-  #  tor:
-  #    image: strm/tor
-  #    restart: always
-  #    depends_on: [server]
-  #    environment:
-  #      LISTEN_PORT: 80
-  #      REDIRECT: server:8000
-  #      # Generate a new key with
-  #      # docker run --rm --entrypoint shallot strm/tor-hiddenservice-nginx ^foo
-  #      PRIVATE_KEY: |
-  #        -----BEGIN RSA PRIVATE KEY-----
-  #        ...
-  #        -----END RSA PRIVATE KEY-----
-
-  #  # auto update docker containers when new image is pushed to docker hub (be careful with that)
-  #  watchtower:
-  #    image: v2tec/watchtower
-  #    restart: always
-  #    volumes:
-  #      - /var/run/docker.sock:/var/run/docker.sock
-
-  #  # this container will send Telegram notifications when other containers are stopped/restarted
-  #  # it's best to run this container on some other instance, otherwise it can't notify if the whole instance goes down
-  #  notifier:
-  #    image: poma/docker-telegram-notifier
-  #    restart: always
-  #    volumes:
-  #      - /var/run/docker.sock:/var/run/docker.sock:ro
-  #    environment:
-  #      # How to create bot: https://core.telegram.org/bots#3-how-do-i-create-a-bot
-  #      # How to get chat id: https://stackoverflow.com/questions/32423837/telegram-bot-how-to-get-a-group-chat-id/32572159#32572159
-  #      TELEGRAM_NOTIFIER_BOT_TOKEN: ...
-  #      TELEGRAM_NOTIFIER_CHAT_ID: ...
-
-  #  # this container will send Telegram notifications if specified address doesn't have enough funds
-  #  monitor_mainnet:
-  #    image: peppersec/monitor_eth
-  #    restart: always
-  #    environment:
-  #      TELEGRAM_NOTIFIER_BOT_TOKEN: ...
-  #      TELEGRAM_NOTIFIER_CHAT_ID: ...
-  #      ADDRESS: '0x0000000000000000000000000000000000000000'
-  #      THRESHOLD: 0.5 # ETH
-  #      RPC_URL: https://mainnet.infura.io
-  #      BLOCK_EXPLORER: etherscan.io
-
  redis:
    image: redis
    restart: always
@ -128,6 +43,234 @@ services:
      - nginx
      - dockergen

+  # ---------------------- BSC (Binance Smart Chain) ----------------------- #
+
+  bsc-server:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['bsc']
+    restart: always
+    command: server
+    env_file: .env.bsc
+    environment:
+      NET_ID: 56
+      REDIS_URL: redis://redis/1
+      nginx_proxy_read_timeout: 600
+    depends_on: [redis]
+
+  bsc-healthWatcher:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['bsc']
+    restart: always
+    command: healthWatcher
+    env_file: .env.bsc
+    environment:
+      NET_ID: 56
+      REDIS_URL: redis://redis/1
+    depends_on: [redis, bsc-server]
+
+  bsc-worker1:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['bsc']
+    restart: always
+    command: worker
+    env_file: .env.bsc
+    environment:
+      NET_ID: 56
+      REDIS_URL: redis://redis/1
+    depends_on: [redis, bsc-server]
+
+  # -------------------------------------------------- #
+
+  # ---------------------- Polygon (MATIC) --------------------- #
+
+  polygon-server:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['polygon']
+    restart: always
+    command: server
+    env_file: .env.polygon
+    environment:
+      NET_ID: 137
+      REDIS_URL: redis://redis/2
+      nginx_proxy_read_timeout: 600
+    depends_on: [redis]
+
+  polygon-healthWatcher:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['polygon']
+    restart: always
+    command: healthWatcher
+    env_file: .env.polygon
+    environment:
+      NET_ID: 137
+      REDIS_URL: redis://redis/2
+    depends_on: [redis, polygon-server]
+
+  polygon-worker1:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['polygon']
+    restart: always
+    command: worker
+    env_file: .env.polygon
+    environment:
+      NET_ID: 137
+      REDIS_URL: redis://redis/2
+    depends_on: [redis, polygon-server]
+
+  # -------------------------------------------------- #
+
+  # ---------------------- Gnosis (XDAI) ---------------------- #
+
+  gnosis-server:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['gnosis']
+    restart: always
+    command: server
+    env_file: .env.gnosis
+    environment:
+      NET_ID: 100
+      REDIS_URL: redis://redis/3
+      nginx_proxy_read_timeout: 600
+    depends_on: [redis]
+
+  gnosis-healthWatcher:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['gnosis']
+    restart: always
+    command: healthWatcher
+    env_file: .env.gnosis
+    environment:
+      NET_ID: 100
+      REDIS_URL: redis://redis/3
+    depends_on: [redis, gnosis-server]
+
+  gnosis-worker1:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['gnosis']
+    restart: always
+    command: worker
+    env_file: .env.gnosis
+    environment:
+      NET_ID: 100
+      REDIS_URL: redis://redis/3
+    depends_on: [redis, gnosis-server]
+
+  # -------------------------------------------------- #
+
+  # ---------------------- AVAX ---------------------- #
+
+  avax-server:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['avax']
+    restart: always
+    command: server
+    env_file: .env.avax
+    environment:
+      NET_ID: 43114
+      REDIS_URL: redis://redis/4
+      nginx_proxy_read_timeout: 600
+    depends_on: [redis]
+
+  avax-healthWatcher:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['avax']
+    restart: always
+    command: healthWatcher
+    env_file: .env.avax
+    environment:
+      NET_ID: 43114
+      REDIS_URL: redis://redis/4
+    depends_on: [redis, avax-server]
+
+  avax-worker1:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['avax']
+    restart: always
+    command: worker
+    env_file: .env.avax
+    environment:
+      NET_ID: 43114
+      REDIS_URL: redis://redis/4
+    depends_on: [redis, avax-server]
+
+  # -------------------------------------------------- #
+
+  # ---------------------- OP ------------------------ #
+
+  op-server:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['op']
+    restart: always
+    command: server
+    env_file: .env.op
+    environment:
+      NET_ID: 10
+      REDIS_URL: redis://redis/5
+      nginx_proxy_read_timeout: 600
+    depends_on: [redis]
+
+  op-healthWatcher:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['op']
+    restart: always
+    command: healthWatcher
+    env_file: .env.op
+    environment:
+      NET_ID: 10
+      REDIS_URL: redis://redis/5
+    depends_on: [redis, op-server]
+
+  op-worker1:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['op']
+    restart: always
+    command: worker
+    env_file: .env.op
+    environment:
+      NET_ID: 10
+      REDIS_URL: redis://redis/5
+    depends_on: [redis, op-server]
+
+  # -------------------------------------------------- #
+
+  # ---------------------- Arbitrum ----------------------- #
+
+  arb-server:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['arb']
+    restart: always
+    command: server
+    env_file: .env.arb
+    environment:
+      NET_ID: 42161
+      REDIS_URL: redis://redis/6
+      nginx_proxy_read_timeout: 600
+    depends_on: [redis]
+
+  arb-healthWatcher:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['arb']
+    restart: always
+    command: healthWatcher
+    env_file: .env.arb
+    environment:
+      NET_ID: 42161
+      REDIS_URL: redis://redis/6
+    depends_on: [redis, arb-server]
+
+  arb-worker1:
+    image: tornadocash/relayer:sidechain-beta
+    profiles: ['arb']
+    restart: always
+    command: worker
+    env_file: .env.arb
+    environment:
+      NET_ID: 42161
+      REDIS_URL: redis://redis/6
+    depends_on: [redis, arb-server]
+
+  # -------------------------------------------------- #
+
 volumes:
  conf:
  vhost:
--- a/package.json
+++ b/package.json
@ -11,7 +11,7 @@
    "prettier:fix": "npx prettier --write . --config .prettierrc",
    "lint": "yarn eslint && yarn prettier:check",
    "test": "mocha",
-    "build": "docker build -t tornadocash/relayer:sidechain-beta",
+    "build": "docker build -t tornadocash/relayer:sidechain-beta .",
    "start": "concurrently \"yarn server\" \"yarn worker\" \"yarn healthWatcher\""
  },
  "author": "tornado.cash",
--- a/tornado-stream.conf
+++ b/tornado-stream.conf
@ -0,0 +1,15 @@
+map $ssl_preread_server_name $name {
+    yourdomain.com tornado_mainnet;
+
+    default tornado_mainnet;
+}
+
+upstream tornado_mainnet {
+    server 127.0.0.1:4380;
+}
+
+server {
+    listen 0.0.0.0:443;
+    proxy_pass $name;
+    ssl_preread on;
+}
--- a/tornado.conf
+++ b/tornado.conf
@ -0,0 +1,87 @@
+# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
+# scheme used to connect to this server
+map $http_x_forwarded_proto $proxy_x_forwarded_proto {
+  default $http_x_forwarded_proto;
+  ''      $scheme;
+}
+# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
+# server port the client connected to
+map $http_x_forwarded_port $proxy_x_forwarded_port {
+  default $http_x_forwarded_port;
+  ''      $server_port;
+}
+# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
+# Connection header that may have been passed to this server
+map $http_upgrade $proxy_connection {
+  default upgrade;
+  '' close;
+}
+# Apply fix for very long server names
+server_names_hash_bucket_size 128;
+# Default dhparam
+# Set appropriate X-Forwarded-Ssl header based on $proxy_x_forwarded_proto
+map $proxy_x_forwarded_proto $proxy_x_forwarded_ssl {
+  default off;
+  https on;
+}
+gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
+log_format vhost '$host $remote_addr - $remote_user [$time_local] '
+                 '"$request" $status $body_bytes_sent '
+                 '"$http_referer" "$http_user_agent" '
+                 '"$upstream_addr"';
+# HTTP 1.1 support
+proxy_http_version 1.1;
+proxy_buffering off;
+proxy_set_header Host $http_host;
+proxy_set_header Upgrade $http_upgrade;
+proxy_set_header Connection $proxy_connection;
+proxy_set_header X-Real-IP $remote_addr;
+proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
+proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
+proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
+proxy_set_header X-Original-URI $request_uri;
+# Mitigate httpoxy attack (see README for details)
+proxy_set_header Proxy "";
+
+# Request rate limiting per second, 2Mb zone @ 5 requests per second
+limit_req_zone $binary_remote_addr zone=one:2m rate=5r/s;
+# Connections per IP limited to 2 
+limit_conn_zone $binary_remote_addr zone=two:2m;
+
+server {
+	server_name _; # This is just an invalid value which will never trigger on a real hostname.
+	server_tokens off;
+	listen 80;
+	access_log /var/log/nginx/access.log vhost;
+	return 503;
+}
+
+server {
+	server_name yourdomain.com;
+
+	# Connection timeouts
+	client_body_timeout 10s;
+  client_header_timeout 10s;
+
+	listen 80;
+	access_log /var/log/nginx/access.log vhost;
+
+	# Do not HTTPS redirect LetsEncrypt ACME challenge
+	location ^~ /.well-known/acme-challenge/ {
+    limit_req zone=one;
+    limit_conn two 1;
+
+    proxy_pass http://127.0.0.1:8000;
+
+		break;
+	}
+
+	location / {
+	  limit_req zone=one;
+    limit_conn two 1;
+
+		return 301 https://$host$request_uri;
+	}
+
+}