mat2-web/README.md

                 _   ___                     _     
                | | |__ \                   | |    
 _ __ ___   __ _| |_   ) | ___ __      _____| |__   Trashing your meta,
| '_ ` _ \ / _` | __| / / |___|\ \ /\ / / _ \ '_ \    keeping your data,
| | | | | | (_| | |_ / /_       \ V  V /  __/ |_) |     within your browser.
|_| |_| |_|\__,_|\__|____|       \_/\_/ \___|_.__/ 

This is an online version of mat2. Keep in mind that this is a beta version, don't rely on it for anything serious, yet.

Demo instance

There is a demo instance deployed a mat2-web.dustri.org. Please don't upload any sensitive files to it.

Vue Frontend

There is a SPA Frontend available at https://0xacab.org/jfriedli/mat2-quasar-frontend. It consumes the RESTful API of this project. As a fallback for non JS users it redirects to this web app. To set it up checkout the Readme.

How to deploy it?

mat2 is available in Debian stable.

# apt install uwsgi uwsgi-plugin-python3 git mat2
# apt install nginx-light  # if you prefer nginx
# apt install apache2 libapache2-mod-proxy-uwsgi  # if you prefer Apache2
# cd /var/www/
# git clone https://0xacab.org/jvoisin/mat2-web.git
# mkdir ./mat2-web/uploads/
# chown -R www-data:www-data ./mat2-web

Since uWSGI isn't fun to configure, feel free to copy this file to /etc/uwsgi/apps-enabled/mat2-web.ini and this one to /etc/nginx/sites-enabled/mat2-web.

Nginx is the recommended web engine, but you can also use Apache if you prefer, by copying this file to your /etc/apache2/sites-enabled/mat2-web file.

Then configure the environment variable: MAT2_ALLOW_ORIGIN_WHITELIST=https://myhost1.org https://myhost2.org Note that you can add multiple hosts from which you want to accept API requests. These need to be separated by a space. IMPORTANT: The default value if the variable is not set is: Access-Control-Allow-Origin: *

Configure the following environment variables:

• MAT2_MAX_FILES_BULK_DOWNLOAD=10 Max number of files that can be grouped for a bulk download.
• MAT2_MAX_FILE_AGE_FOR_REMOVAL=900 Seconds a file in the upload folder is kept. After that it will be deleted. Default 15 * 60

This specifies the max number of files that can be bulk downloaded using the api. Note: Each file has a max file size of 16mb

Finally, restart uWSGI and your web server:

systemctl restart uwsgi
systemctl restart nginx/apache/…

It should now be working.

Deploy via Ansible

If you happen to be using Ansible, there's an Ansible role to deploy mat2-web on Debian, thanks to the amazing systemli people: ansible-role-mat2-web

The role installs mat2-web as a uWSGI service, and runs it as a dedicated system user, installs bubblewrap to sandbox mat2 and creates a garbage collector cronjob to remove leftover files. Besides, it can create a dm-crypt volume with random key for the uploads folder, to ensure that the uploaded files won't be recoverable between reboots.

Deploy using Docker

You can find the ready to run docker image here: https://0xacab.org/jvoisin/mat2-web/container_registry

Example: docker run -p 80:80 -d -e MAT2_ALLOW_ORIGIN_WHITELIST='https://myhost1.org' registry.0xacab.org/jvoisin/mat2-web:latest

Development

Install docker and docker-compose and then run docker-compose up to setup the docker dev environment. Mat2-web is now accessible on your host machine at localhost:5000. Every code change triggers a restart of the app. If you want to add/remove dependencies you have to rebuild the container.

RESTful API

Upload Endpoint

Endpoint: /api/upload

HTTP Verbs: POST

Body:

{
	"file_name": "my-filename.jpg",
	"file": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
}

The file_name parameter takes the file name. The file parameter is the base64 encoded file which will be cleaned.

Example Response:

{
    "output_filename": "fancy.cleaned.jpg",
    "mime": "image/jpg",
    "key": "81a541f9ebc0233d419d25ed39908b16f82be26a783f32d56c381559e84e6161",
    "secret": "44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c",
    "meta": {
        "BitDepth": 8,
        "ColorType": "RGB with Alpha",
        "Compression": "Deflate/Inflate",
        "Filter": "Adaptive",
        "Interlace": "Noninterlaced"
    },
    "meta_after": {},
    "download_link": "http://localhost:5000/download/81a541f9ebc0233d419d25ed39908b16f82be26a783f32d56c381559e84e6161/44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c/fancy.cleaned.jpg"
}

Supported Extensions Endpoint

Endpoint: /api/extension

HTTP Verbs: GET

Example Response (shortened):

[
    ".asc",
    ".avi",
    ".bat",
    ".bmp",
    ".brf",
    ".c",
    ".css",
    ".docx",
    ".epub"
]

Endpoint: /api/download/bulk

This endpoint allows you to bulk download several files which you uploaded beforehand. Note that the download_list MUST contain more than two files. The max length is configurable (default is 10).

HTTP Verbs: POST

Body:

{
  "download_list": [
    {
        "file_name": "uploaded_file_name.jpg",
        "key": "uploaded_file_key",
        "secret": "uploaded_file_secret"
    }
  ]
}

The file_name parameter takes the file name from a previously uploaded file. The key parameter is the key from a previously uploaded file.

Example Response:

{
    "output_filename": "files.2cd225d5-2d75-44a2-9f26-e120a87e4279.cleaned.zip",
    "mime": "application/zip",
    "key": "5ee4cf8821226340d3d5ed16bd2e1b435234a9ad218f282b489a85d116e7a4c4",
    "secret": "44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c",
    "meta_after": {},
    "download_link": "http://localhost/api/download/5ee4cf8821226340d3d5ed16bd2e1b435234a9ad218f282b489a85d116e7a4c4/files.2cd225d5-2d75-44a2-9f26-e120a87e4279.cleaned.zip"
}

Docker

There are two Dockerfiles present in this repository. The file called Dockerfile.development is used for development and Dockerfile.production is used for production deployments.

You can find the automated docker builds in the registry of this repository: https://0xacab.org/jvoisin/mat2-web/container_registry

Building the production image

Build command: docker build -f Dockerfile.production -t mat-web .

Run it: docker run -ti -p8181:8080 --security-opt=no-new-privileges --read-only --tmpfs /tmp --tmpfs=/var/www/mat2-web/uploads mat-web:latest

This does mount the upload folder as tmpfs and servers the app on localhost:8181.

Configuration

The default settings from main.py may be overridden by adding a config.py file and add custom values for the relevant flask config variables. E.g.:

MAX_CONTENT_LENGTH = 32 * 1024 * 1024  # 32MB

See Flask configuration docs for further information.

Custom templates

You can override the default templates from templates/ by putting replacements into the directory path that's configured in app.config['CUSTOM_TEMPLATES_DIR'] (default custom_templates/).

Threat model

• An attacker in possession of the very same file that a user wants to clean, along with its names, can perform a denial of service by continually requesting this file, and getting it before the user.
• An attacker in possession of only the name of a file that a user wants to clean can't perform a denial of service attack, since the path to download the cleaned file is not only dependent of the name, but also the content.
• The server should do its very best to delete files as soon as possible.

Licenses

• mat2-web is under MIT
• The raleway font is under OFL1.1
• normalize.css is under MIT
• skeleton is under MIT