From 14643b9fce3f114dbffd945a4b780d68881e736c Mon Sep 17 00:00:00 2001 From: TheFrenchGhosty <47571719+TheFrenchGhosty@users.noreply.github.com> Date: Thu, 17 Feb 2022 21:11:02 +0000 Subject: [PATCH] Massively enhance the Installation documentation (#160) * Massively enhance the Installation documentation Co-authored-by: Samantaz Fox --- Installation.md | 245 ++++++++++++++++++++---------------------------- Updating.md | 44 --------- 2 files changed, 102 insertions(+), 187 deletions(-) delete mode 100644 Updating.md diff --git a/Installation.md b/Installation.md index b7ae37a..f0255d7 100644 --- a/Installation.md +++ b/Installation.md @@ -10,42 +10,38 @@ dateCreated: 2021-02-25T11:24:06.655Z # Installation -Compiling invidious requires at least 2GB of free RAM (We recommend to have at least 3GB installed). +Compiling Invidious requires at least 2GB of free RAM (We recommend to have at least 3GB installed). If you have less (e.g on a cheap VPS) you can setup a SWAP file or partition, so the combined amount is >= 3GB. After installation take a look at the [Post-install steps](#post-install-configuration). -### Note on blocking bots +Note: Any [PaaS](https://en.wikipedia.org/wiki/Platform_as_a_service) or [SaaS](https://en.wikipedia.org/wiki/Software_as_a_service) provider/software (Heroku, YunoHost, Repli...) are unsupported. Use them at your own risk. They **WILL** cause problems with Invidious and might even suspend your account for "abuse" since Invidious is heavy, bandwidth intensive and technically a proxy (and most providers don't like them). If you use one and want to report an issue, please mention which one you use. -Allowing bots that excessively crawl (Semrush, webmeup, etc.) will lead your instance to get blocked very fast. While not required, it is a good idea to consider using bot blocking software such as [Nginx Bad Bot Blocker](https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker). -## Automated installation +## Automated Installation [Invidious-Updater](https://github.com/tmiland/Invidious-Updater) is a self-contained script that can automatically install and update Invidious. + ## Docker -> The Invidious docker image is only [available on Quay](https://quay.io/repository/invidious/invidious) because, unlike Docker Hub, [Quay is open source](https://github.com/quay/quay/blob/master/LICENSE). This is reflected in the `docker-compose.yml` file used in this walkthrough.{.is-warning} +> The Invidious docker image is only [available on Quay](https://quay.io/repository/invidious/invidious) because, unlike Docker Hub, [Quay is Free and Open Source Software](https://github.com/quay/quay/blob/master/LICENSE). This is reflected in the `docker-compose.yml` file used in this walk-through.{.is-warning} -Ensure [Docker Engine](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed before beginning. +Ensure [Docker Engine](https://docs.docker.com/engine/install) and [Docker Compose](https://docs.docker.com/compose/install) are installed before beginning. -### Make directory +### Docker-compose method (production) + +**This method uses the pre-built Docker image from quay** + +Note: Currently the repository has to be cloned, this is because the `init-invidious-db.sh` file and the `config/sql` directory have to be mounted to the postgres container (See the volumes section in the docker-compose file below). This "problem" will be solved in the future. ```bash -$ mkdir invidious +git clone https://github.com/iv-org/invidious.git +cd invidious ``` -### Create Docker Compose file +Edit the docker-compose.yml with this content: -```bash -$ cd invidious -``` - -```bash -$ nano docker-compose.yml -``` - -Here is a working Compose setup: ```docker version: "2.4" services: @@ -74,7 +70,10 @@ services: ports: - "127.0.0.1:3000:3000" environment: - INVIDIOUS_CONFIG: | + INVIDIOUS_CONFIG: | + # Please read the following file for a comprehensive list of all available + # configuration options and their associated syntax: + # https://github.com/iv-org/invidious/blob/master/config/config.example.yml channel_threads: 1 check_tables: true feed_threads: 1 @@ -110,58 +109,31 @@ networks: invidious: ``` +Note: This compose is made for a true "production" setup, where Invidious is behind a reverse proxy. If you prefer to directly access Invidious, replace `127.0.0.1:3000:3000` with `3000:3000` under the `ports:` section. + > The environment variable `POSTGRES_USER` cannot be changed. The SQL config files that run the initial database migrations are hard-coded with the username `kemal`. {.is-warning} -### Start Invidious + +### Docker-compose method (development) + +**This method builds a Docker image from source** ```bash -$ docker-compose up -``` -or -```bash -$ docker-compose up -d -``` -to run it in the background. - -Then, visit `localhost:3000` in your browser. - -### Stop Invidious - -```bash -$ docker-compose down +git clone https://github.com/iv-org/invidious.git +cd invidious +docker-compose up ``` -### Delete data -```bash -$ docker volume rm invidious_postgresdata -``` - -## Manual installation +## Manual Installation ### Linux -#### Install crystal +#### Install Crystal Follow the instructions for your distribution here: https://crystal-lang.org/install/ -If you're in a hurry, here are one-liner commands for some common distributions: -* Arch linux `sudo pacman -S crystal shards` -* Debian/Ubuntu: `curl -fsSL https://crystal-lang.org/install.sh | sudo bash` -* Fedora: `sudo brew update && sudo brew install crystal-lang` - -Or you can do a tarball install: -```bash -cd ~/Downloads -wget https://github.com/crystal-lang/crystal/releases/download/1.1.1/crystal-1.1.1-1-linux-x86_64.tar.gz -cd /opt -sudo tar -xzf ~/Downloads/crystal-1.1.1-1-linux-x86_64.tar.gz -sudo cp /opt/crystal-1.1.1-1/bin/{crystal,shards} /usr/local/bin/ -sudo cp -r /opt/crystal-1.1.1-1/lib/crystal /usr/local/lib/crystal -sudo cp -r /opt/crystal-1.1.1-1/share/crystal /usr/local/share/crystal -``` - #### Install the dependencies Arch Linux @@ -169,13 +141,12 @@ Arch Linux sudo pacman -S base-devel librsvg postgresql ``` -Ubuntu or Debian +Debian/Ubuntu ```bash -sudo apt-get update sudo apt install libssl-dev libxml2-dev libyaml-dev libgmp-dev libreadline-dev postgresql librsvg2-bin libsqlite3-dev zlib1g-dev libpcre3-dev libevent-dev ``` -Fedora +RHEL based and RHEL-like systems (RHEL, Fedora, AlmaLinux, RockyLinux...) ```bash sudo dnf install -y openssl-devel libevent-devel libxml2-devel libyaml-devel gmp-devel readline-devel postgresql librsvg2-devel sqlite-devel zlib-devel gcc ``` @@ -183,134 +154,122 @@ sudo dnf install -y openssl-devel libevent-devel libxml2-devel libyaml-devel gmp #### Add an Invidious user and clone the repository ```bash -$ useradd -m invidious -$ sudo -i -u invidious -$ git clone https://github.com/iv-org/invidious -$ exit +useradd -m invidious +su - invidious +git clone https://github.com/iv-org/invidious +exit ``` #### Set up PostgresSQL ```bash -$ sudo systemctl enable --now postgresql -$ sudo -i -u postgres -$ psql -c "CREATE USER kemal WITH PASSWORD 'kemal';" # Change 'kemal' here to a stronger password, and update `password` in config/config.yml -$ createdb -O kemal invidious -$ psql invidious kemal < /home/invidious/invidious/config/sql/channels.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/videos.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/channel_videos.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/users.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/session_ids.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/nonces.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/annotations.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/playlists.sql -$ psql invidious kemal < /home/invidious/invidious/config/sql/playlist_videos.sql -$ exit +systemctl enable --now postgresql +sudo -i -u postgres +psql -c "CREATE USER kemal WITH PASSWORD 'kemal';" # Change 'kemal' here to a stronger password, and update `password` in config/config.yml +createdb -O kemal invidious +psql invidious kemal < /home/invidious/invidious/config/sql/channels.sql +psql invidious kemal < /home/invidious/invidious/config/sql/videos.sql +psql invidious kemal < /home/invidious/invidious/config/sql/channel_videos.sql +psql invidious kemal < /home/invidious/invidious/config/sql/users.sql +psql invidious kemal < /home/invidious/invidious/config/sql/session_ids.sql +psql invidious kemal < /home/invidious/invidious/config/sql/nonces.sql +psql invidious kemal < /home/invidious/invidious/config/sql/annotations.sql +psql invidious kemal < /home/invidious/invidious/config/sql/playlists.sql +psql invidious kemal < /home/invidious/invidious/config/sql/playlist_videos.sql +exit ``` #### Set up Invidious ```bash -$ sudo -i -u invidious -$ cd invidious -$ shards update && shards install -$ crystal build src/invidious.cr --release -# test compiled binary -$ ./invidious # stop with ctrl c -$ exit +su - invidious +cd invidious +shards update && shards install && crystal build src/invidious.cr --release +exit ``` #### Systemd service ```bash -$ sudo cp /home/invidious/invidious/invidious.service /etc/systemd/system/invidious.service -$ sudo systemctl enable --now invidious.service -``` - -#### Logrotate - -```bash -$ echo "/home/invidious/invidious/invidious.log { -rotate 4 -weekly -notifempty -missingok -compress -minsize 1048576 -}" | sudo tee /etc/logrotate.d/invidious.logrotate -$ sudo chmod 0644 /etc/logrotate.d/invidious.logrotate +cp /home/invidious/invidious/invidious.service /etc/systemd/system/invidious.service +systemctl enable --now invidious.service ``` ### MacOS ```bash # Install dependencies -$ brew update -$ brew install shards crystal postgres imagemagick librsvg +brew update +brew install shards crystal postgres imagemagick librsvg # Clone the repository and set up a PostgreSQL database -$ git clone https://github.com/iv-org/invidious -$ cd invidious -$ brew services start postgresql -$ psql -c "CREATE ROLE kemal WITH PASSWORD 'kemal';" # Change 'kemal' here to a stronger password, and update `password` in config/config.yml -$ createdb -O kemal invidious -$ psql invidious kemal < config/sql/channels.sql -$ psql invidious kemal < config/sql/videos.sql -$ psql invidious kemal < config/sql/channel_videos.sql -$ psql invidious kemal < config/sql/users.sql -$ psql invidious kemal < config/sql/session_ids.sql -$ psql invidious kemal < config/sql/nonces.sql -$ psql invidious kemal < config/sql/annotations.sql -$ psql invidious kemal < config/sql/privacy.sql -$ psql invidious kemal < config/sql/playlists.sql -$ psql invidious kemal < config/sql/playlist_videos.sql +git clone https://github.com/iv-org/invidious +cd invidious +brew services start postgresql +psql -c "CREATE ROLE kemal WITH PASSWORD 'kemal';" # Change 'kemal' here to a stronger password, and update `password` in config/config.yml +createdb -O kemal invidious +psql invidious kemal < config/sql/channels.sql +psql invidious kemal < config/sql/videos.sql +psql invidious kemal < config/sql/channel_videos.sql +psql invidious kemal < config/sql/users.sql +psql invidious kemal < config/sql/session_ids.sql +psql invidious kemal < config/sql/nonces.sql +psql invidious kemal < config/sql/annotations.sql +psql invidious kemal < config/sql/privacy.sql +psql invidious kemal < config/sql/playlists.sql +psql invidious kemal < config/sql/playlist_videos.sql # Set up Invidious -$ shards update && shards install -$ crystal build src/invidious.cr --release +shards update && shards install && crystal build src/invidious.cr --release ``` ## Post-install configuration: Detailed configuration available in the [configuration guide](./Configuration.md). -Because of various issues Invidious **must** be restarted often, at least once a day, ideally every hours. +Because of various issues Invidious **must** be restarted often, at least once a day, ideally every hour. If you use a reverse proxy, you **must** configure invidious to properly serve request through it: -`https_only: true` : if your are serving your instance via https, set it to true +`https_only: true` : if you are serving your instance via https, set it to true `domain: domain.ext`: if you are serving your instance via a domain name, set it here -`external_port: 443`: if your are serving your instance via https, set it to 443 +`external_port: 443`: if you are serving your instance via https, set it to 443 ## Update Invidious -Instructions are available in the [updating guide](./Updating.md). +#### Updating a Docker install +```bash +docker-compose pull && docker-compose up && docker image prune -f +``` + +#### Update a manual install +```bash +sudo - invidious +cd invidious +shards update && shards install && crystal build src/invidious.cr --release +exit +systemctl restart invidious.service +``` ## Usage: ```bash -$ ./invidious -h -Usage: invidious [arguments] - -b HOST, --bind HOST Host to bind (defaults to 0.0.0.0) - -p PORT, --port PORT Port to listen for connections (defaults to 3000) - -s, --ssl Enables SSL - --ssl-key-file FILE SSL key file - --ssl-cert-file FILE SSL certificate file - -h, --help Shows this help - -c THREADS, --channel-threads=THREADS - Number of threads for refreshing channels (default: 1) - -f THREADS, --feed-threads=THREADS - Number of threads for refreshing feeds (default: 1) - -o OUTPUT, --output=OUTPUT Redirect output (default: STDOUT) - -v, --version Print version +./invidious ``` -Or for development: + +#### Logrotate configuration ```bash -$ curl -fsSLo- https://raw.githubusercontent.com/samueleaton/sentry/master/install.cr | crystal eval -$ ./sentry -🤖 Your SentryBot is vigilant. beep-boop... -``` +echo "/home/invidious/invidious/invidious.log { +rotate 4 +weekly +notifempty +missingok +compress +minsize 1048576 +}" | tee /etc/logrotate.d/invidious.logrotate +chmod 0644 /etc/logrotate.d/invidious.logrotate +``` diff --git a/Updating.md b/Updating.md deleted file mode 100644 index 0f2a6cc..0000000 --- a/Updating.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Updating -description: -published: true -date: 2021-05-23T16:59:22.409Z -tags: -editor: markdown -dateCreated: 2021-01-28T20:40:27.192Z ---- - -## Invidious releases are based on tags. You can use them if you want to be sure your instance is stable. - -#### With release tags -```bash -$ sudo -i -u invidious -$ cd invidious -$ currentVersion=$(git rev-list --max-count=1 --abbrev-commit HEAD) -$ git pull -$ latestVersion=$(git describe --tags --abbrev=0) -$ git checkout $latestVersion -$ for i in `git rev-list --reverse --abbrev-commit $currentVersion..HEAD` ; do file=./config/migrate-scripts/migrate-db-$i.sh ; [ -f $file ] && $file ; done -$ shards update && shards install -$ crystal build src/invidious.cr --release -$ exit -$ sudo systemctl restart invidious.service -``` - -#### With master branch -```bash -$ sudo -i -u invidious -$ cd invidious -$ currentVersion=$(git rev-list --max-count=1 --abbrev-commit HEAD) -$ git pull -$ for i in `git rev-list --reverse --abbrev-commit $currentVersion..HEAD` ; do file=./config/migrate-scripts/migrate-db-$i.sh ; [ -f $file ] && $file ; done -$ shards update && shards install -$ crystal build src/invidious.cr --release -$ exit -$ sudo systemctl restart invidious.service -``` - -#### With Docker: -```bash -$ docker-compose pull && docker-compose up --force-recreate --build && docker image prune -f -``` \ No newline at end of file