mirror of
https://0xacab.org/jvoisin/mat2-web.git
synced 2025-02-24 00:59:59 -05:00
246 lines
8.4 KiB
Markdown
246 lines
8.4 KiB
Markdown
```
|
|
_ ___ _
|
|
| | |__ \ | |
|
|
_ __ ___ __ _| |_ ) | ___ __ _____| |__ Trashing your meta,
|
|
| '_ ` _ \ / _` | __| / / |___|\ \ /\ / / _ \ '_ \ keeping your data,
|
|
| | | | | | (_| | |_ / /_ \ V V / __/ |_) | within your browser.
|
|
|_| |_| |_|\__,_|\__|____| \_/\_/ \___|_.__/
|
|
```
|
|
|
|
This is an online version of [mat2](https://0xacab.org/jvoisin/mat2).
|
|
Keep in mind that this is a beta version, don't rely on it for anything
|
|
serious, yet.
|
|
|
|
# Demo instance
|
|
|
|
There is a demo instance deployed a [mat2-web.dustri.org](https://mat2-web.dustri.org).
|
|
Please don't upload any sensitive files to it.
|
|
|
|
# Vue Frontend
|
|
|
|
There is a SPA Frontend available at https://0xacab.org/jfriedli/mat2-quasar-frontend. It consumes
|
|
the RESTful API of this project. As a fallback for non JS users it redirects to this web app.
|
|
To set it up checkout the [Readme](https://0xacab.org/jfriedli/mat2-quasar-frontend/blob/master/README.md).
|
|
|
|
# How to deploy it?
|
|
|
|
mat2 is available in [Debian stable](https://packages.debian.org/stable/mat2).
|
|
|
|
```
|
|
# apt install uwsgi uwsgi-plugin-python3 git mat2
|
|
# apt install nginx-light # if you prefer nginx
|
|
# apt install apache2 libapache2-mod-proxy-uwsgi # if you prefer Apache2
|
|
# cd /var/www/
|
|
# git clone https://0xacab.org/jvoisin/mat2-web.git
|
|
# mkdir ./mat2-web/uploads/
|
|
# chown -R www-data:www-data ./mat2-web
|
|
```
|
|
|
|
Since [uWSGI](https://uwsgi-docs.readthedocs.io/en/latest/) isn't fun to
|
|
configure, feel free to copy
|
|
[this file](https://0xacab.org/jvoisin/mat2-web/tree/master/config/uwsgi.config)
|
|
to `/etc/uwsgi/apps-enabled/mat2-web.ini` and
|
|
[this one](https://0xacab.org/jvoisin/mat2-web/tree/master/config/nginx.config)
|
|
to `/etc/nginx/sites-enabled/mat2-web`.
|
|
|
|
Nginx is the recommended web engine, but you can also use Apache if you prefer,
|
|
by copying [this file](https://0xacab.org/jvoisin/mat2-web/tree/master/config/apache2.config)
|
|
to your `/etc/apache2/sites-enabled/mat2-web` file.
|
|
|
|
Then configure the environment variable: `MAT2_ALLOW_ORIGIN_WHITELIST=https://myhost1.org https://myhost2.org`
|
|
Note that you can add multiple hosts from which you want to accept API requests. These need to be separated by
|
|
a space.
|
|
**IMPORTANT:** The default value if the variable is not set is: `Access-Control-Allow-Origin: *`
|
|
|
|
Configure the following environment variables:
|
|
|
|
- `MAT2_MAX_FILES_BULK_DOWNLOAD=10` Max number of files that can be grouped for a bulk download.
|
|
- `MAT2_MAX_FILE_AGE_FOR_REMOVAL=900` Seconds a file in the upload folder is kept.
|
|
After that it will be deleted. Default `15 * 60`
|
|
|
|
This specifies the max number of files that can be bulk downloaded using the api.
|
|
Note: Each file has a max file size of 16mb
|
|
|
|
Finally, restart uWSGI and your web server:
|
|
|
|
```
|
|
systemctl restart uwsgi
|
|
systemctl restart nginx/apache/…
|
|
```
|
|
|
|
It should now be working.
|
|
|
|
# Deploy via Ansible
|
|
|
|
If you happen to be using [Ansible](https://www.ansible.com/), there's an
|
|
Ansible role to deploy mat2-web on Debian, thanks to the amazing
|
|
[systemli](https://www.systemli.org/en/index.html) people:
|
|
[ansible-role-mat2-web](https://github.com/systemli/ansible-role-mat2-web)
|
|
|
|
The role installs mat2-web as a uWSGI service, and runs it as a dedicated
|
|
system user, installs bubblewrap to sandbox mat2 and creates a garbage
|
|
collector cronjob to remove leftover files. Besides, it can create a
|
|
[dm-crypt](https://en.wikipedia.org/wiki/Dm-crypt) volume with random key for
|
|
the uploads folder, to ensure that the uploaded files won't be recoverable
|
|
between reboots.
|
|
|
|
|
|
# Deploy using Docker
|
|
You can find the ready to run docker image here:
|
|
https://0xacab.org/jvoisin/mat2-web/container_registry
|
|
|
|
Example:
|
|
`docker run -p 80:80 -d -e MAT2_ALLOW_ORIGIN_WHITELIST='https://myhost1.org' registry.0xacab.org/jvoisin/mat2-web:latest`
|
|
|
|
# Development
|
|
Install docker and docker-compose and then run `docker-compose up` to setup
|
|
the docker dev environment. Mat2-web is now accessible on your host machine at `localhost:5000`.
|
|
Every code change triggers a restart of the app.
|
|
If you want to add/remove dependencies you have to rebuild the container.
|
|
|
|
# RESTful API
|
|
|
|
## Upload Endpoint
|
|
|
|
**Endpoint:** `/api/upload`
|
|
|
|
**HTTP Verbs:** POST
|
|
|
|
**Body:**
|
|
```json
|
|
{
|
|
"file_name": "my-filename.jpg",
|
|
"file": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
|
|
}
|
|
```
|
|
|
|
The `file_name` parameter takes the file name.
|
|
The `file` parameter is the base64 encoded file which will be cleaned.
|
|
|
|
**Example Response:**
|
|
```json
|
|
{
|
|
"output_filename": "fancy.cleaned.jpg",
|
|
"mime": "image/jpg",
|
|
"key": "81a541f9ebc0233d419d25ed39908b16f82be26a783f32d56c381559e84e6161",
|
|
"secret": "44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c",
|
|
"meta": {
|
|
"BitDepth": 8,
|
|
"ColorType": "RGB with Alpha",
|
|
"Compression": "Deflate/Inflate",
|
|
"Filter": "Adaptive",
|
|
"Interlace": "Noninterlaced"
|
|
},
|
|
"meta_after": {},
|
|
"download_link": "http://localhost:5000/download/81a541f9ebc0233d419d25ed39908b16f82be26a783f32d56c381559e84e6161/44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c/fancy.cleaned.jpg"
|
|
}
|
|
```
|
|
|
|
## Supported Extensions Endpoint
|
|
|
|
**Endpoint:** `/api/extension`
|
|
|
|
**HTTP Verbs:** GET
|
|
|
|
**Example Response (shortened):**
|
|
```json
|
|
[
|
|
".asc",
|
|
".avi",
|
|
".bat",
|
|
".bmp",
|
|
".brf",
|
|
".c",
|
|
".css",
|
|
".docx",
|
|
".epub"
|
|
]
|
|
```
|
|
|
|
**Endpoint:** `/api/download/bulk`
|
|
|
|
This endpoint allows you to bulk download several files
|
|
which you uploaded beforehand. Note that the `download_list`
|
|
MUST contain more than two files. The max length is configurable
|
|
(default is 10).
|
|
|
|
**HTTP Verbs:** POST
|
|
|
|
**Body:**
|
|
```json
|
|
{
|
|
"download_list": [
|
|
{
|
|
"file_name": "uploaded_file_name.jpg",
|
|
"key": "uploaded_file_key",
|
|
"secret": "uploaded_file_secret"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
The `file_name` parameter takes the file name from a previously uploaded file.
|
|
The `key` parameter is the key from a previously uploaded file.
|
|
|
|
**Example Response:**
|
|
```json
|
|
{
|
|
"output_filename": "files.2cd225d5-2d75-44a2-9f26-e120a87e4279.cleaned.zip",
|
|
"mime": "application/zip",
|
|
"key": "5ee4cf8821226340d3d5ed16bd2e1b435234a9ad218f282b489a85d116e7a4c4",
|
|
"secret": "44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c",
|
|
"meta_after": {},
|
|
"download_link": "http://localhost/api/download/5ee4cf8821226340d3d5ed16bd2e1b435234a9ad218f282b489a85d116e7a4c4/files.2cd225d5-2d75-44a2-9f26-e120a87e4279.cleaned.zip"
|
|
}
|
|
```
|
|
|
|
# Docker
|
|
|
|
There are two Dockerfiles present in this repository. The file called `Dockerfile.development` is used for development
|
|
and `Dockerfile.production` is used for production deployments.
|
|
|
|
You can find the automated docker builds in the registry of this
|
|
repository: https://0xacab.org/jvoisin/mat2-web/container_registry
|
|
|
|
### Building the production image
|
|
Build command: `docker build -f Dockerfile.production -t mat-web .`
|
|
|
|
Run it: ` docker run -ti -p8181:8080 --read-only --tmpfs /tmp --tmpfs=/var/www/mat2-web/uploads mat-web:latest`
|
|
|
|
This does mount the upload folder as tmpfs and servers the app on `localhost:8181`
|
|
|
|
# Configuration
|
|
|
|
The default settings from `main.py` may be overridden by adding a `config.py`
|
|
file and add custom values for the relevant flask config variables. E.g.:
|
|
|
|
```
|
|
MAX_CONTENT_LENGTH = 32 * 1024 * 1024 # 32MB
|
|
```
|
|
|
|
See [Flask configuration docs](http://exploreflask.com/en/latest/configuration.html)
|
|
for further information.
|
|
|
|
# Custom templates
|
|
|
|
You can override the default templates from `templates/` by putting replacements
|
|
into the directory path that's configured in `app.config['CUSTOM_TEMPLATES_DIR']`
|
|
(default `custom_templates/`).
|
|
|
|
# Threat model
|
|
|
|
- An attacker in possession of the very same file that a user wants to clean,
|
|
along with its names, can perform a denial of service by continually
|
|
requesting this file, and getting it before the user.
|
|
- An attacker in possession of only the name of a file that a user wants to
|
|
clean can't perform a denial of service attack, since the path to download
|
|
the cleaned file is not only dependent of the name, but also the content.
|
|
- The server should do its very best to delete files as soon as possible.
|
|
|
|
# Licenses
|
|
|
|
- mat2-web is under MIT
|
|
- The [raleway](https://github.com/impallari/Raleway/) font is under OFL1.1
|
|
- [normalize.css](https://github.com/necolas/normalize.css/) is under MIT
|
|
- [skeleton](http://getskeleton.com/) is under MIT
|