Add onionshare CLI to cli folder, move GUI to desktop folder, and start refactoring it to work with briefcase

README.md

@@ -22,4 +22,4 @@ To learn how OnionShare works, what its security properties are, how to use it,
 
 ---
 
-Test status: [![CircleCI](https://circleci.com/gh/micahflee/onionshare.svg?style=svg)](https://circleci.com/gh/micahflee/onionshare)
+Test status: [![CircleCI](https://circleci.com/gh/micahflee/onionshare.svg?style=svg)](https://circleci.com/gh/micahflee/onionshare)

cli/.circleci/config.yml

@@ -0,0 +1,38 @@
+version: 2
+workflows:
+  version: 2
+  test:
+    jobs:
+      - test-3.6
+      - test-3.7
+      - test-3.8
+
+jobs:
+  test-3.6: &test-template
+    docker:
+      - image: circleci/python:3.6-buster
+
+    working_directory: ~/repo
+
+    steps:
+      - checkout
+
+      - run:
+          name: Install dependencies
+          command: |
+            poetry install
+
+      - run:
+          name: Run unit tests
+          command: |
+            poetry run pytest -vvv ./tests
+
+  test-3.7:
+    <<: *test-template
+    docker:
+      - image: circleci/python:3.7-buster
+
+  test-3.8:
+    <<: *test-template
+    docker:
+      - image: circleci/python:3.8-buster

cli/CHANGELOG.md

@@ -0,0 +1,3 @@
+# OnionShare CLI Changelog
+
+Coming soon.

cli/README.md

@@ -0,0 +1,87 @@
+```
+                     @@@@@@@@@                      
+                @@@@@@@@@@@@@@@@@@@                 
+             @@@@@@@@@@@@@@@@@@@@@@@@@              
+           @@@@@@@@@@@@@@@@@@@@@@@@@@@@@            
+             @@@@@@@@@@@@@@@@@@@@@@@@@@@@@           ___        _               
+               @@@@@@         @@@@@@@@@@@@@         / _ \      (_)              
+         @@@@    @               @@@@@@@@@@@       | | | |_ __  _  ___  _ __    
+       @@@@@@@@                   @@@@@@@@@@       | | | | '_ \| |/ _ \| '_ \   
+     @@@@@@@@@@@@                  @@@@@@@@@@      \ \_/ / | | | | (_) | | | |  
+   @@@@@@@@@@@@@@@@                 @@@@@@@@@       \___/|_| |_|_|\___/|_| |_|  
+      @@@@@@@@@                 @@@@@@@@@@@@@@@@    _____ _                     
+      @@@@@@@@@@                  @@@@@@@@@@@@     /  ___| |                    
+       @@@@@@@@@@                   @@@@@@@@       \ `--.| |__   __ _ _ __ ___ 
+       @@@@@@@@@@@               @    @@@@          `--. \ '_ \ / _` | '__/ _ \
+        @@@@@@@@@@@@@         @@@@@@               /\__/ / | | | (_| | | |  __/
+         @@@@@@@@@@@@@@@@@@@@@@@@@@@@@             \____/|_| |_|\__,_|_|  \___|
+           @@@@@@@@@@@@@@@@@@@@@@@@@@@@@            
+             @@@@@@@@@@@@@@@@@@@@@@@@@              
+                @@@@@@@@@@@@@@@@@@@                 
+                     @@@@@@@@@                      
+```
+
+# OnionShare CLI
+
+_This project is under development and not ready for use._
+
+OnionShare is an open source tool that lets you securely and anonymously share files, host websites, and chat with friends using the Tor network.
+
+This is the command line version of OnionShare. [Click here](https://github.com/micahflee/onionshare) for the graphical version.
+
+## Installing OnionShare CLI
+
+First, make sure you have `tor` installed. In Linux, install it through your package manager. In macOS, install it with [Homebrew](https://brew.sh): `brew install tor`.
+
+Then install OnionShare CLI:
+
+```
+pip install onionshare-cli
+```
+
+Then run it with:
+
+```
+onionshare-cli --help
+```
+
+## Developing OnionShare CLI
+
+You must have python3 and [poetry](https://python-poetry.org/) installed.
+
+Install dependencies with poetry:
+
+```
+poetry install
+```
+
+To run from the source tree:
+
+```
+poetry run onionshare-cli
+```
+
+To run tests:
+
+```
+poetry run pytest -vvv ./tests
+```
+
+### Making a release
+
+Before making a release, make update the version in these places:
+
+- `pyproject.toml`
+- `onionshare_cli/resources/version.txt`
+
+And edit `CHANGELOG.md` to include a list of all major changes since the last release.
+
+Create a PGP-signed git tag. For example for OnionShare CLI 0.1.0, the tag must be `v0.1.0`.
+
+Build and publish to PyPi:
+
+```
+poetry publish --build
+```
+
+Test status: [![CircleCI](https://circleci.com/gh/micahflee/onionshare-cli.svg?style=svg)](https://circleci.com/gh/micahflee/onionshare-cli)

cli/onionshare_cli/common.py

@@ -0,0 +1,300 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+import base64
+import hashlib
+import inspect
+import os
+import platform
+import random
+import socket
+import sys
+import tempfile
+import threading
+import time
+import shutil
+
+from .settings import Settings
+
+
+class Common:
+    """
+    The Common object is shared amongst all parts of OnionShare.
+    """
+
+    def __init__(self, verbose=False):
+        self.verbose = verbose
+
+        # The platform OnionShare is running on
+        self.platform = platform.system()
+        if self.platform.endswith("BSD") or self.platform == "DragonFly":
+            self.platform = "BSD"
+
+        # The current version of OnionShare
+        with open(self.get_resource_path("version.txt")) as f:
+            self.version = f.read().strip()
+
+    def load_settings(self, config=None):
+        """
+        Loading settings, optionally from a custom config json file.
+        """
+        self.settings = Settings(self, config)
+        self.settings.load()
+
+    def log(self, module, func, msg=None):
+        """
+        If verbose mode is on, log error messages to stdout
+        """
+        if self.verbose:
+            timestamp = time.strftime("%b %d %Y %X")
+
+            final_msg = f"[{timestamp}] {module}.{func}"
+            if msg:
+                final_msg = f"{final_msg}: {msg}"
+            print(final_msg)
+
+    def get_resource_path(self, filename):
+        """
+        Returns the absolute path of a resource
+        """
+        resources_path = os.path.join(
+            os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe()))),
+            "resources",
+        )
+        return os.path.join(resources_path, filename)
+
+    def get_tor_paths(self):
+        if self.platform == "Linux":
+            tor_path = shutil.which("tor")
+            obfs4proxy_file_path = shutil.which("obfs4proxy")
+            prefix = os.path.dirname(os.path.dirname(tor_path))
+            tor_geo_ip_file_path = os.path.join(prefix, "share/tor/geoip")
+            tor_geo_ipv6_file_path = os.path.join(prefix, "share/tor/geoip6")
+        elif self.platform == "Windows":
+            base_path = os.path.join(
+                os.path.dirname(os.path.dirname(self.get_resource_path(""))), "tor"
+            )
+            tor_path = os.path.join(os.path.join(base_path, "Tor"), "tor.exe")
+            obfs4proxy_file_path = os.path.join(
+                os.path.join(base_path, "Tor"), "obfs4proxy.exe"
+            )
+            tor_geo_ip_file_path = os.path.join(
+                os.path.join(os.path.join(base_path, "Data"), "Tor"), "geoip"
+            )
+            tor_geo_ipv6_file_path = os.path.join(
+                os.path.join(os.path.join(base_path, "Data"), "Tor"), "geoip6"
+            )
+        elif self.platform == "Darwin":
+            base_path = os.path.dirname(
+                os.path.dirname(os.path.dirname(self.get_resource_path("")))
+            )
+            tor_path = os.path.join(base_path, "Resources", "Tor", "tor")
+            tor_geo_ip_file_path = os.path.join(base_path, "Resources", "Tor", "geoip")
+            tor_geo_ipv6_file_path = os.path.join(
+                base_path, "Resources", "Tor", "geoip6"
+            )
+            obfs4proxy_file_path = os.path.join(
+                base_path, "Resources", "Tor", "obfs4proxy"
+            )
+        elif self.platform == "BSD":
+            tor_path = "/usr/local/bin/tor"
+            tor_geo_ip_file_path = "/usr/local/share/tor/geoip"
+            tor_geo_ipv6_file_path = "/usr/local/share/tor/geoip6"
+            obfs4proxy_file_path = "/usr/local/bin/obfs4proxy"
+
+        return (
+            tor_path,
+            tor_geo_ip_file_path,
+            tor_geo_ipv6_file_path,
+            obfs4proxy_file_path,
+        )
+
+    def build_data_dir(self):
+        """
+        Returns the path of the OnionShare data directory.
+        """
+        if self.platform == "Windows":
+            try:
+                appdata = os.environ["APPDATA"]
+                onionshare_data_dir = f"{appdata}\\OnionShare"
+            except:
+                # If for some reason we don't have the 'APPDATA' environment variable
+                # (like running tests in Linux while pretending to be in Windows)
+                onionshare_data_dir = os.path.expanduser("~/.config/onionshare")
+        elif self.platform == "Darwin":
+            onionshare_data_dir = os.path.expanduser(
+                "~/Library/Application Support/OnionShare"
+            )
+        else:
+            onionshare_data_dir = os.path.expanduser("~/.config/onionshare")
+
+        # Modify the data dir if running tests
+        if getattr(sys, "onionshare_test_mode", False):
+            onionshare_data_dir += "-testdata"
+
+        os.makedirs(onionshare_data_dir, 0o700, True)
+        return onionshare_data_dir
+
+    def build_tmp_dir(self):
+        """
+        Returns path to a folder that can hold temporary files
+        """
+        tmp_dir = os.path.join(self.build_data_dir(), "tmp")
+        os.makedirs(tmp_dir, 0o700, True)
+        return tmp_dir
+
+    def build_persistent_dir(self):
+        """
+        Returns the path to the folder that holds persistent files
+        """
+        persistent_dir = os.path.join(self.build_data_dir(), "persistent")
+        os.makedirs(persistent_dir, 0o700, True)
+        return persistent_dir
+
+    def build_tor_dir(self):
+        """
+        Returns path to the tor data directory
+        """
+        tor_dir = os.path.join(self.build_data_dir(), "tor_data")
+        os.makedirs(tor_dir, 0o700, True)
+        return tor_dir
+
+    def build_password(self, word_count=2):
+        """
+        Returns a random string made of words from the wordlist, such as "deter-trig".
+        """
+        with open(self.get_resource_path("wordlist.txt")) as f:
+            wordlist = f.read().split()
+
+        r = random.SystemRandom()
+        return "-".join(r.choice(wordlist) for _ in range(word_count))
+
+    def build_username(self, word_count=2):
+        """
+        Returns a random string made of words from the wordlist, such as "deter-trig".
+        """
+        with open(self.get_resource_path("wordlist.txt")) as f:
+            wordlist = f.read().split()
+
+        r = random.SystemRandom()
+        return "-".join(r.choice(wordlist) for _ in range(word_count))
+
+    @staticmethod
+    def random_string(num_bytes, output_len=None):
+        """
+        Returns a random string with a specified number of bytes.
+        """
+        b = os.urandom(num_bytes)
+        h = hashlib.sha256(b).digest()[:16]
+        s = base64.b32encode(h).lower().replace(b"=", b"").decode("utf-8")
+        if not output_len:
+            return s
+        return s[:output_len]
+
+    @staticmethod
+    def human_readable_filesize(b):
+        """
+        Returns filesize in a human readable format.
+        """
+        thresh = 1024.0
+        if b < thresh:
+            return "{:.1f} B".format(b)
+        units = ("KiB", "MiB", "GiB", "TiB", "PiB", "EiB", "ZiB", "YiB")
+        u = 0
+        b /= thresh
+        while b >= thresh:
+            b /= thresh
+            u += 1
+        return "{:.1f} {}".format(b, units[u])
+
+    @staticmethod
+    def format_seconds(seconds):
+        """Return a human-readable string of the format 1d2h3m4s"""
+        days, seconds = divmod(seconds, 86400)
+        hours, seconds = divmod(seconds, 3600)
+        minutes, seconds = divmod(seconds, 60)
+
+        human_readable = []
+        if days:
+            human_readable.append("{:.0f}d".format(days))
+        if hours:
+            human_readable.append("{:.0f}h".format(hours))
+        if minutes:
+            human_readable.append("{:.0f}m".format(minutes))
+        if seconds or not human_readable:
+            human_readable.append("{:.0f}s".format(seconds))
+        return "".join(human_readable)
+
+    @staticmethod
+    def estimated_time_remaining(bytes_downloaded, total_bytes, started):
+        now = time.time()
+        time_elapsed = now - started  # in seconds
+        download_rate = bytes_downloaded / time_elapsed
+        remaining_bytes = total_bytes - bytes_downloaded
+        eta = remaining_bytes / download_rate
+        return Common.format_seconds(eta)
+
+    @staticmethod
+    def get_available_port(min_port, max_port):
+        """
+        Find a random available port within the given range.
+        """
+        with socket.socket() as tmpsock:
+            while True:
+                try:
+                    tmpsock.bind(("127.0.0.1", random.randint(min_port, max_port)))
+                    break
+                except OSError as e:
+                    pass
+            _, port = tmpsock.getsockname()
+        return port
+
+    @staticmethod
+    def dir_size(start_path):
+        """
+        Calculates the total size, in bytes, of all of the files in a directory.
+        """
+        total_size = 0
+        for dirpath, dirnames, filenames in os.walk(start_path):
+            for f in filenames:
+                fp = os.path.join(dirpath, f)
+                if not os.path.islink(fp):
+                    total_size += os.path.getsize(fp)
+        return total_size
+
+
+class AutoStopTimer(threading.Thread):
+    """
+    Background thread sleeps t hours and returns.
+    """
+
+    def __init__(self, common, time):
+        threading.Thread.__init__(self)
+
+        self.common = common
+
+        self.setDaemon(True)
+        self.time = time
+
+    def run(self):
+        self.common.log(
+            "AutoStopTimer", f"Server will shut down after {self.time} seconds"
+        )
+        time.sleep(self.time)
+        return 1

cli/onionshare_cli/onion.py

@@ -0,0 +1,805 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from stem.control import Controller
+from stem import ProtocolError, SocketClosed
+from stem.connection import MissingPassword, UnreadableCookieFile, AuthenticationFailure
+from Crypto.PublicKey import RSA
+import base64, os, sys, tempfile, shutil, urllib, platform, subprocess, time, shlex
+
+from distutils.version import LooseVersion as Version
+from . import common
+from .settings import Settings
+
+# TODO: Figure out how to localize this for the GUI
+
+
+class TorErrorAutomatic(Exception):
+    """
+    OnionShare is failing to connect and authenticate to the Tor controller,
+    using automatic settings that should work with Tor Browser.
+    """
+
+    pass
+
+
+class TorErrorInvalidSetting(Exception):
+    """
+    This exception is raised if the settings just don't make sense.
+    """
+
+    pass
+
+
+class TorErrorSocketPort(Exception):
+    """
+    OnionShare can't connect to the Tor controller using the supplied address and port.
+    """
+
+    pass
+
+
+class TorErrorSocketFile(Exception):
+    """
+    OnionShare can't connect to the Tor controller using the supplied socket file.
+    """
+
+    pass
+
+
+class TorErrorMissingPassword(Exception):
+    """
+    OnionShare connected to the Tor controller, but it requires a password.
+    """
+
+    pass
+
+
+class TorErrorUnreadableCookieFile(Exception):
+    """
+    OnionShare connected to the Tor controller, but your user does not have permission
+    to access the cookie file.
+    """
+
+    pass
+
+
+class TorErrorAuthError(Exception):
+    """
+    OnionShare connected to the address and port, but can't authenticate. It's possible
+    that a Tor controller isn't listening on this port.
+    """
+
+    pass
+
+
+class TorErrorProtocolError(Exception):
+    """
+    This exception is raised if onionshare connects to the Tor controller, but it
+    isn't acting like a Tor controller (such as in Whonix).
+    """
+
+    pass
+
+
+class TorTooOld(Exception):
+    """
+    This exception is raised if onionshare needs to use a feature of Tor or stem
+    (like stealth ephemeral onion services) but the version you have installed
+    is too old.
+    """
+
+    pass
+
+
+class BundledTorNotSupported(Exception):
+    """
+    This exception is raised if onionshare is set to use the bundled Tor binary,
+    but it's not supported on that platform, or in dev mode.
+    """
+
+
+class BundledTorTimeout(Exception):
+    """
+    This exception is raised if onionshare is set to use the bundled Tor binary,
+    but Tor doesn't finish connecting promptly.
+    """
+
+
+class BundledTorCanceled(Exception):
+    """
+    This exception is raised if onionshare is set to use the bundled Tor binary,
+    and the user cancels connecting to Tor
+    """
+
+
+class BundledTorBroken(Exception):
+    """
+    This exception is raised if onionshare is set to use the bundled Tor binary,
+    but the process seems to fail to run.
+    """
+
+
+class Onion(object):
+    """
+    Onion is an abstraction layer for connecting to the Tor control port and
+    creating onion services. OnionShare supports creating onion services by
+    connecting to the Tor controller and using ADD_ONION, DEL_ONION.
+
+    stealth: Should the onion service be stealth?
+
+    settings: A Settings object. If it's not passed in, load from disk.
+
+    bundled_connection_func: If the tor connection type is bundled, optionally
+    call this function and pass in a status string while connecting to tor. This
+    is necessary for status updates to reach the GUI.
+    """
+
+    def __init__(self, common, use_tmp_dir=False):
+        self.common = common
+        self.common.log("Onion", "__init__")
+
+        self.use_tmp_dir = use_tmp_dir
+
+        # Is bundled tor supported?
+        if (
+            self.common.platform == "Windows" or self.common.platform == "Darwin"
+        ) and getattr(sys, "onionshare_dev_mode", False):
+            self.bundle_tor_supported = False
+        else:
+            self.bundle_tor_supported = True
+
+        # Set the path of the tor binary, for bundled tor
+        (
+            self.tor_path,
+            self.tor_geo_ip_file_path,
+            self.tor_geo_ipv6_file_path,
+            self.obfs4proxy_file_path,
+        ) = self.common.get_tor_paths()
+
+        # The tor process
+        self.tor_proc = None
+
+        # The Tor controller
+        self.c = None
+
+        # Start out not connected to Tor
+        self.connected_to_tor = False
+
+        # Assigned later if we are using stealth mode
+        self.auth_string = None
+
+    def connect(
+        self,
+        custom_settings=None,
+        config=None,
+        tor_status_update_func=None,
+        connect_timeout=120,
+        local_only=False,
+    ):
+        if local_only:
+            self.common.log(
+                "Onion", "connect", "--local-only, so skip trying to connect"
+            )
+            return
+
+        self.common.log("Onion", "connect")
+
+        # Either use settings that are passed in, or use them from common
+        if custom_settings:
+            self.settings = custom_settings
+        elif config:
+            self.common.load_settings(config)
+            self.settings = self.common.settings
+        else:
+            self.common.load_settings()
+            self.settings = self.common.settings
+
+        # The Tor controller
+        self.c = None
+
+        if self.settings.get("connection_type") == "bundled":
+            if not self.bundle_tor_supported:
+                raise BundledTorNotSupported(
+                    # strings._("settings_error_bundled_tor_not_supported")
+                    "Using the Tor version that comes with OnionShare does not work in developer mode on Windows or macOS."
+                )
+
+            # Create a torrc for this session
+            if self.use_tmp_dir:
+                self.tor_data_directory = tempfile.TemporaryDirectory(
+                    dir=self.common.build_tmp_dir()
+                )
+                self.tor_data_directory_name = self.tor_data_directory.name
+            else:
+                self.tor_data_directory_name = self.common.build_tor_dir()
+            self.common.log(
+                "Onion",
+                "connect",
+                f"tor_data_directory_name={self.tor_data_directory_name}",
+            )
+
+            # Create the torrc
+            with open(self.common.get_resource_path("torrc_template")) as f:
+                torrc_template = f.read()
+            self.tor_cookie_auth_file = os.path.join(
+                self.tor_data_directory_name, "cookie"
+            )
+            try:
+                self.tor_socks_port = self.common.get_available_port(1000, 65535)
+            except:
+                raise OSError("OnionShare port not available")
+            self.tor_torrc = os.path.join(self.tor_data_directory_name, "torrc")
+
+            if self.common.platform == "Windows" or self.common.platform == "Darwin":
+                # Windows doesn't support unix sockets, so it must use a network port.
+                # macOS can't use unix sockets either because socket filenames are limited to
+                # 100 chars, and the macOS sandbox forces us to put the socket file in a place
+                # with a really long path.
+                torrc_template += "ControlPort {{control_port}}\n"
+                try:
+                    self.tor_control_port = self.common.get_available_port(1000, 65535)
+                except:
+                    raise OSError("OnionShare port not available")
+                self.tor_control_socket = None
+            else:
+                # Linux and BSD can use unix sockets
+                torrc_template += "ControlSocket {{control_socket}}\n"
+                self.tor_control_port = None
+                self.tor_control_socket = os.path.join(
+                    self.tor_data_directory_name, "control_socket"
+                )
+
+            torrc_template = torrc_template.replace(
+                "{{data_directory}}", self.tor_data_directory_name
+            )
+            torrc_template = torrc_template.replace(
+                "{{control_port}}", str(self.tor_control_port)
+            )
+            torrc_template = torrc_template.replace(
+                "{{control_socket}}", str(self.tor_control_socket)
+            )
+            torrc_template = torrc_template.replace(
+                "{{cookie_auth_file}}", self.tor_cookie_auth_file
+            )
+            torrc_template = torrc_template.replace(
+                "{{geo_ip_file}}", self.tor_geo_ip_file_path
+            )
+            torrc_template = torrc_template.replace(
+                "{{geo_ipv6_file}}", self.tor_geo_ipv6_file_path
+            )
+            torrc_template = torrc_template.replace(
+                "{{socks_port}}", str(self.tor_socks_port)
+            )
+
+            with open(self.tor_torrc, "w") as f:
+                f.write(torrc_template)
+
+                # Bridge support
+                if self.settings.get("tor_bridges_use_obfs4"):
+                    f.write(
+                        f"ClientTransportPlugin obfs4 exec {self.obfs4proxy_file_path}\n"
+                    )
+                    with open(
+                        self.common.get_resource_path("torrc_template-obfs4")
+                    ) as o:
+                        for line in o:
+                            f.write(line)
+                elif self.settings.get("tor_bridges_use_meek_lite_azure"):
+                    f.write(
+                        f"ClientTransportPlugin meek_lite exec {self.obfs4proxy_file_path}\n"
+                    )
+                    with open(
+                        self.common.get_resource_path("torrc_template-meek_lite_azure")
+                    ) as o:
+                        for line in o:
+                            f.write(line)
+
+                if self.settings.get("tor_bridges_use_custom_bridges"):
+                    if "obfs4" in self.settings.get("tor_bridges_use_custom_bridges"):
+                        f.write(
+                            f"ClientTransportPlugin obfs4 exec {self.obfs4proxy_file_path}\n"
+                        )
+                    elif "meek_lite" in self.settings.get(
+                        "tor_bridges_use_custom_bridges"
+                    ):
+                        f.write(
+                            f"ClientTransportPlugin meek_lite exec {self.obfs4proxy_file_path}\n"
+                        )
+                    f.write(self.settings.get("tor_bridges_use_custom_bridges"))
+                    f.write("\nUseBridges 1")
+
+            # Execute a tor subprocess
+            start_ts = time.time()
+            if self.common.platform == "Windows":
+                # In Windows, hide console window when opening tor.exe subprocess
+                startupinfo = subprocess.STARTUPINFO()
+                startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
+                self.tor_proc = subprocess.Popen(
+                    [self.tor_path, "-f", self.tor_torrc],
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.PIPE,
+                    startupinfo=startupinfo,
+                )
+            else:
+                self.tor_proc = subprocess.Popen(
+                    [self.tor_path, "-f", self.tor_torrc],
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.PIPE,
+                )
+
+            # Wait for the tor controller to start
+            time.sleep(2)
+
+            # Connect to the controller
+            try:
+                if (
+                    self.common.platform == "Windows"
+                    or self.common.platform == "Darwin"
+                ):
+                    self.c = Controller.from_port(port=self.tor_control_port)
+                    self.c.authenticate()
+                else:
+                    self.c = Controller.from_socket_file(path=self.tor_control_socket)
+                    self.c.authenticate()
+            except Exception as e:
+                raise BundledTorBroken(
+                    # strings._("settings_error_bundled_tor_broken").format(e.args[0])
+                    "OnionShare could not connect to Tor:\n{}".format(e.args[0])
+                )
+
+            while True:
+                try:
+                    res = self.c.get_info("status/bootstrap-phase")
+                except SocketClosed:
+                    raise BundledTorCanceled()
+
+                res_parts = shlex.split(res)
+                progress = res_parts[2].split("=")[1]
+                summary = res_parts[4].split("=")[1]
+
+                # "\033[K" clears the rest of the line
+                print(
+                    f"\rConnecting to the Tor network: {progress}% - {summary}\033[K",
+                    end="",
+                )
+
+                if callable(tor_status_update_func):
+                    if not tor_status_update_func(progress, summary):
+                        # If the dialog was canceled, stop connecting to Tor
+                        self.common.log(
+                            "Onion",
+                            "connect",
+                            "tor_status_update_func returned false, canceling connecting to Tor",
+                        )
+                        print()
+                        return False
+
+                if summary == "Done":
+                    print("")
+                    break
+                time.sleep(0.2)
+
+                # If using bridges, it might take a bit longer to connect to Tor
+                if (
+                    self.settings.get("tor_bridges_use_custom_bridges")
+                    or self.settings.get("tor_bridges_use_obfs4")
+                    or self.settings.get("tor_bridges_use_meek_lite_azure")
+                ):
+                    # Only override timeout if a custom timeout has not been passed in
+                    if connect_timeout == 120:
+                        connect_timeout = 150
+                if time.time() - start_ts > connect_timeout:
+                    print("")
+                    try:
+                        self.tor_proc.terminate()
+                        raise BundledTorTimeout(
+                            # strings._("settings_error_bundled_tor_timeout")
+                            "Taking too long to connect to Tor. Maybe you aren't connected to the Internet, or have an inaccurate system clock?"
+                        )
+                    except FileNotFoundError:
+                        pass
+
+        elif self.settings.get("connection_type") == "automatic":
+            # Automatically try to guess the right way to connect to Tor Browser
+
+            # Try connecting to control port
+            found_tor = False
+
+            # If the TOR_CONTROL_PORT environment variable is set, use that
+            env_port = os.environ.get("TOR_CONTROL_PORT")
+            if env_port:
+                try:
+                    self.c = Controller.from_port(port=int(env_port))
+                    found_tor = True
+                except:
+                    pass
+
+            else:
+                # Otherwise, try default ports for Tor Browser, Tor Messenger, and system tor
+                try:
+                    ports = [9151, 9153, 9051]
+                    for port in ports:
+                        self.c = Controller.from_port(port=port)
+                        found_tor = True
+                except:
+                    pass
+
+                # If this still didn't work, try guessing the default socket file path
+                socket_file_path = ""
+                if not found_tor:
+                    try:
+                        if self.common.platform == "Darwin":
+                            socket_file_path = os.path.expanduser(
+                                "~/Library/Application Support/TorBrowser-Data/Tor/control.socket"
+                            )
+
+                        self.c = Controller.from_socket_file(path=socket_file_path)
+                        found_tor = True
+                    except:
+                        pass
+
+            # If connecting to default control ports failed, so let's try
+            # guessing the socket file name next
+            if not found_tor:
+                try:
+                    if self.common.platform == "Linux" or self.common.platform == "BSD":
+                        socket_file_path = (
+                            f"/run/user/{os.geteuid()}/Tor/control.socket"
+                        )
+                    elif self.common.platform == "Darwin":
+                        socket_file_path = (
+                            f"/run/user/{os.geteuid()}/Tor/control.socket"
+                        )
+                    elif self.common.platform == "Windows":
+                        # Windows doesn't support unix sockets
+                        raise TorErrorAutomatic(
+                            # strings._("settings_error_automatic")
+                            "Could not connect to the Tor controller. Is Tor Browser (available from torproject.org) running in the background?"
+                        )
+
+                    self.c = Controller.from_socket_file(path=socket_file_path)
+
+                except:
+                    raise TorErrorAutomatic(
+                        # strings._("settings_error_automatic")
+                        "Could not connect to the Tor controller. Is Tor Browser (available from torproject.org) running in the background?"
+                    )
+
+            # Try authenticating
+            try:
+                self.c.authenticate()
+            except:
+                raise TorErrorAutomatic(
+                    # strings._("settings_error_automatic")
+                    "Could not connect to the Tor controller. Is Tor Browser (available from torproject.org) running in the background?"
+                )
+
+        else:
+            # Use specific settings to connect to tor
+
+            # Try connecting
+            try:
+                if self.settings.get("connection_type") == "control_port":
+                    self.c = Controller.from_port(
+                        address=self.settings.get("control_port_address"),
+                        port=self.settings.get("control_port_port"),
+                    )
+                elif self.settings.get("connection_type") == "socket_file":
+                    self.c = Controller.from_socket_file(
+                        path=self.settings.get("socket_file_path")
+                    )
+                else:
+                    raise TorErrorInvalidSetting(
+                        # strings._("settings_error_unknown")
+                        "Can't connect to Tor controller because your settings don't make sense."
+                    )
+
+            except:
+                if self.settings.get("connection_type") == "control_port":
+                    raise TorErrorSocketPort(
+                        # strings._("settings_error_socket_port")
+                        "Can't connect to the Tor controller at {}:{}.".format(
+                            self.settings.get("control_port_address"),
+                            self.settings.get("control_port_port"),
+                        )
+                    )
+                else:
+                    raise TorErrorSocketFile(
+                        # strings._("settings_error_socket_file")
+                        "Can't connect to the Tor controller using socket file {}.".format(
+                            self.settings.get("socket_file_path")
+                        )
+                    )
+
+            # Try authenticating
+            try:
+                if self.settings.get("auth_type") == "no_auth":
+                    self.c.authenticate()
+                elif self.settings.get("auth_type") == "password":
+                    self.c.authenticate(self.settings.get("auth_password"))
+                else:
+                    raise TorErrorInvalidSetting(
+                        # strings._("settings_error_unknown")
+                        "Can't connect to Tor controller because your settings don't make sense."
+                    )
+
+            except MissingPassword:
+                raise TorErrorMissingPassword(
+                    # strings._("settings_error_missing_password")
+                    "Connected to Tor controller, but it requires a password to authenticate."
+                )
+            except UnreadableCookieFile:
+                raise TorErrorUnreadableCookieFile(
+                    # strings._("settings_error_unreadable_cookie_file")
+                    "Connected to the Tor controller, but password may be wrong, or your user is not permitted to read the cookie file."
+                )
+            except AuthenticationFailure:
+                raise TorErrorAuthError(
+                    # strings._("settings_error_auth")
+                    "Connected to {}:{}, but can't authenticate. Maybe this isn't a Tor controller?".format(
+                        self.settings.get("control_port_address"),
+                        self.settings.get("control_port_port"),
+                    )
+                )
+
+        # If we made it this far, we should be connected to Tor
+        self.connected_to_tor = True
+
+        # Get the tor version
+        self.tor_version = self.c.get_version().version_str
+        self.common.log("Onion", "connect", f"Connected to tor {self.tor_version}")
+
+        # Do the versions of stem and tor that I'm using support ephemeral onion services?
+        list_ephemeral_hidden_services = getattr(
+            self.c, "list_ephemeral_hidden_services", None
+        )
+        self.supports_ephemeral = (
+            callable(list_ephemeral_hidden_services) and self.tor_version >= "0.2.7.1"
+        )
+
+        # Do the versions of stem and tor that I'm using support stealth onion services?
+        try:
+            res = self.c.create_ephemeral_hidden_service(
+                {1: 1},
+                basic_auth={"onionshare": None},
+                await_publication=False,
+                key_type="NEW",
+                key_content="RSA1024",
+            )
+            tmp_service_id = res.service_id
+            self.c.remove_ephemeral_hidden_service(tmp_service_id)
+            self.supports_stealth = True
+        except:
+            # ephemeral stealth onion services are not supported
+            self.supports_stealth = False
+
+        # Does this version of Tor support next-gen ('v3') onions?
+        # Note, this is the version of Tor where this bug was fixed:
+        # https://trac.torproject.org/projects/tor/ticket/28619
+        self.supports_v3_onions = self.tor_version >= Version("0.3.5.7")
+
+    def is_authenticated(self):
+        """
+        Returns True if the Tor connection is still working, or False otherwise.
+        """
+        if self.c is not None:
+            return self.c.is_authenticated()
+        else:
+            return False
+
+    def start_onion_service(self, mode_settings, port, await_publication):
+        """
+        Start a onion service on port 80, pointing to the given port, and
+        return the onion hostname.
+        """
+        self.common.log("Onion", "start_onion_service", f"port={port}")
+
+        if not self.supports_ephemeral:
+            raise TorTooOld(
+                # strings._("error_ephemeral_not_supported")
+                "Your version of Tor is too old, ephemeral onion services are not supported"
+            )
+        if mode_settings.get("general", "client_auth") and not self.supports_stealth:
+            raise TorTooOld(
+                # strings._("error_stealth_not_supported")
+                "Your version of Tor is too old, stealth onion services are not supported"
+            )
+
+        auth_cookie = None
+        if mode_settings.get("general", "client_auth"):
+            if mode_settings.get("onion", "hidservauth_string"):
+                auth_cookie = mode_settings.get("onion", "hidservauth_string").split()[
+                    2
+                ]
+            if auth_cookie:
+                basic_auth = {"onionshare": auth_cookie}
+            else:
+                # If we had neither a scheduled auth cookie or a persistent hidservauth string,
+                # set the cookie to 'None', which means Tor will create one for us
+                basic_auth = {"onionshare": None}
+        else:
+            # Not using client auth at all
+            basic_auth = None
+
+        if mode_settings.get("onion", "private_key"):
+            key_content = mode_settings.get("onion", "private_key")
+            if self.is_v2_key(key_content):
+                key_type = "RSA1024"
+            else:
+                # Assume it was a v3 key. Stem will throw an error if it's something illegible
+                key_type = "ED25519-V3"
+        else:
+            key_type = "NEW"
+            # Work out if we can support v3 onion services, which are preferred
+            if self.supports_v3_onions and not mode_settings.get("general", "legacy"):
+                key_content = "ED25519-V3"
+            else:
+                # fall back to v2 onion services
+                key_content = "RSA1024"
+
+        # v3 onions don't yet support basic auth. Our ticket:
+        # https://github.com/micahflee/onionshare/issues/697
+        if (
+            key_type == "NEW"
+            and key_content == "ED25519-V3"
+            and not mode_settings.get("general", "legacy")
+        ):
+            basic_auth = None
+
+        debug_message = f"key_type={key_type}"
+        if key_type == "NEW":
+            debug_message += f", key_content={key_content}"
+        self.common.log("Onion", "start_onion_service", debug_message)
+        try:
+            res = self.c.create_ephemeral_hidden_service(
+                {80: port},
+                await_publication=await_publication,
+                basic_auth=basic_auth,
+                key_type=key_type,
+                key_content=key_content,
+            )
+
+        except ProtocolError as e:
+            raise TorErrorProtocolError(
+                # strings._("error_tor_protocol_error")
+                "Tor error: {}".format(e.args[0])
+            )
+
+        onion_host = res.service_id + ".onion"
+
+        # Save the service_id
+        mode_settings.set("general", "service_id", res.service_id)
+
+        # Save the private key and hidservauth string
+        if not mode_settings.get("onion", "private_key"):
+            mode_settings.set("onion", "private_key", res.private_key)
+        if mode_settings.get("general", "client_auth") and not mode_settings.get(
+            "onion", "hidservauth_string"
+        ):
+            auth_cookie = list(res.client_auth.values())[0]
+            self.auth_string = f"HidServAuth {onion_host} {auth_cookie}"
+            mode_settings.set("onion", "hidservauth_string", self.auth_string)
+
+        return onion_host
+
+    def stop_onion_service(self, mode_settings):
+        """
+        Stop a specific onion service
+        """
+        onion_host = mode_settings.get("general", "service_id")
+        if onion_host:
+            self.common.log("Onion", "stop_onion_service", f"onion host: {onion_host}")
+            try:
+                self.c.remove_ephemeral_hidden_service(
+                    mode_settings.get("general", "service_id")
+                )
+            except:
+                self.common.log(
+                    "Onion", "stop_onion_service", f"failed to remove {onion_host}"
+                )
+
+    def cleanup(self, stop_tor=True):
+        """
+        Stop onion services that were created earlier. If there's a tor subprocess running, kill it.
+        """
+        self.common.log("Onion", "cleanup")
+
+        # Cleanup the ephemeral onion services, if we have any
+        try:
+            onions = self.c.list_ephemeral_hidden_services()
+            for service_id in onions:
+                onion_host = f"{service_id}.onion"
+                try:
+                    self.common.log(
+                        "Onion", "cleanup", f"trying to remove onion {onion_host}"
+                    )
+                    self.c.remove_ephemeral_hidden_service(service_id)
+                except:
+                    self.common.log(
+                        "Onion", "cleanup", f"failed to remove onion {onion_host}"
+                    )
+                    pass
+        except:
+            pass
+
+        if stop_tor:
+            # Stop tor process
+            if self.tor_proc:
+                self.tor_proc.terminate()
+                time.sleep(0.2)
+                if self.tor_proc.poll() is None:
+                    self.common.log(
+                        "Onion",
+                        "cleanup",
+                        "Tried to terminate tor process but it's still running",
+                    )
+                    try:
+                        self.tor_proc.kill()
+                        time.sleep(0.2)
+                        if self.tor_proc.poll() is None:
+                            self.common.log(
+                                "Onion",
+                                "cleanup",
+                                "Tried to kill tor process but it's still running",
+                            )
+                    except:
+                        self.common.log(
+                            "Onion", "cleanup", "Exception while killing tor process"
+                        )
+                self.tor_proc = None
+
+            # Reset other Onion settings
+            self.connected_to_tor = False
+
+            try:
+                # Delete the temporary tor data directory
+                if self.use_tmp_dir:
+                    self.tor_data_directory.cleanup()
+            except:
+                pass
+
+    def get_tor_socks_port(self):
+        """
+        Returns a (address, port) tuple for the Tor SOCKS port
+        """
+        self.common.log("Onion", "get_tor_socks_port")
+
+        if self.settings.get("connection_type") == "bundled":
+            return ("127.0.0.1", self.tor_socks_port)
+        elif self.settings.get("connection_type") == "automatic":
+            return ("127.0.0.1", 9150)
+        else:
+            return (self.settings.get("socks_address"), self.settings.get("socks_port"))
+
+    def is_v2_key(self, key):
+        """
+        Helper function for determining if a key is RSA1024 (v2) or not.
+        """
+        try:
+            # Import the key
+            key = RSA.importKey(base64.b64decode(key))
+            # Is this a v2 Onion key? (1024 bits) If so, we should keep using it.
+            if key.n.bit_length() == 1024:
+                return True
+            else:
+                return False
+        except:
+            return False

cli/onionshare_cli/onionshare.py

@@ -0,0 +1,111 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import os, shutil
+
+from . import common
+from .onion import TorTooOld, TorErrorProtocolError
+from .common import AutoStopTimer
+
+
+class OnionShare(object):
+    """
+    OnionShare is the main application class. Pass in options and run
+    start_onion_service and it will do the magic.
+    """
+
+    def __init__(self, common, onion, local_only=False, autostop_timer=0):
+        self.common = common
+
+        self.common.log("OnionShare", "__init__")
+
+        # The Onion object
+        self.onion = onion
+
+        self.hidserv_dir = None
+        self.onion_host = None
+        self.port = None
+
+        # files and dirs to delete on shutdown
+        self.cleanup_filenames = []
+
+        # do not use tor -- for development
+        self.local_only = local_only
+
+        # optionally shut down after N hours
+        self.autostop_timer = autostop_timer
+        # init auto-stop timer thread
+        self.autostop_timer_thread = None
+
+    def choose_port(self):
+        """
+        Choose a random port.
+        """
+        try:
+            self.port = self.common.get_available_port(17600, 17650)
+        except:
+            raise OSError("Cannot find an available OnionShare port")
+
+    def start_onion_service(self, mode_settings, await_publication=True):
+        """
+        Start the onionshare onion service.
+        """
+        self.common.log("OnionShare", "start_onion_service")
+
+        if not self.port:
+            self.choose_port()
+
+        if self.autostop_timer > 0:
+            self.autostop_timer_thread = AutoStopTimer(self.common, self.autostop_timer)
+
+        if self.local_only:
+            self.onion_host = f"127.0.0.1:{self.port}"
+            return
+
+        self.onion_host = self.onion.start_onion_service(
+            mode_settings, self.port, await_publication
+        )
+
+        if mode_settings.get("general", "client_auth"):
+            self.auth_string = self.onion.auth_string
+
+    def stop_onion_service(self, mode_settings):
+        """
+        Stop the onion service
+        """
+        self.onion.stop_onion_service(mode_settings)
+
+    def cleanup(self):
+        """
+        Shut everything down and clean up temporary files, etc.
+        """
+        self.common.log("OnionShare", "cleanup")
+
+        # Cleanup files
+        try:
+            for filename in self.cleanup_filenames:
+                if os.path.isfile(filename):
+                    os.remove(filename)
+                elif os.path.isdir(filename):
+                    shutil.rmtree(filename)
+        except:
+            # Don't crash if file is still in use
+            pass
+        self.cleanup_filenames = []

cli/onionshare_cli/resources/version.txt

@@ -0,0 +1 @@
+0.1.3

cli/onionshare_cli/settings.py

@@ -0,0 +1,197 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import json
+import os
+import platform
+import locale
+
+try:
+    # We only need pwd module in macOS, and it's not available in Windows
+    import pwd
+except:
+    pass
+
+
+class Settings(object):
+    """
+    This class stores all of the settings for OnionShare, specifically for how
+    to connect to Tor. If it can't find the settings file, it uses the default,
+    which is to attempt to connect automatically using default Tor Browser
+    settings.
+    """
+
+    def __init__(self, common, config=False):
+        self.common = common
+
+        self.common.log("Settings", "__init__")
+
+        # If a readable config file was provided, use that instead
+        if config:
+            if os.path.isfile(config):
+                self.filename = config
+            else:
+                self.common.log(
+                    "Settings",
+                    "__init__",
+                    "Supplied config does not exist or is unreadable. Falling back to default location",
+                )
+                self.filename = self.build_filename()
+
+        else:
+            # Default config
+            self.filename = self.build_filename()
+
+        # Dictionary of available languages in this version of OnionShare,
+        # mapped to the language name, in that language
+        self.available_locales = {
+            "ar": "العربية",  # Arabic
+            #'bn': 'বাংলা', # Bengali (commented out because not at 90% translation)
+            "ca": "Català",  # Catalan
+            "zh_Hant": "正體中文 (繁體)",  # Traditional Chinese
+            "zh_Hans": "中文 (简体)",  # Simplified Chinese
+            "da": "Dansk",  # Danish
+            "nl": "Nederlands",  # Dutch
+            "en": "English",  # English
+            # "fi": "Suomi",  # Finnish (commented out because not at 90% translation)
+            "fr": "Français",  # French
+            "de": "Deutsch",  # German
+            "el": "Ελληνικά",  # Greek
+            "is": "Íslenska",  # Icelandic
+            "ga": "Gaeilge",  # Irish
+            "it": "Italiano",  # Italian
+            "ja": "日本語",  # Japanese
+            "nb_NO": "Norsk Bokmål",  # Norwegian Bokmål
+            "fa": "فارسی",  # Persian
+            "pl": "Polski",  # Polish
+            "pt_BR": "Português (Brasil)",  # Portuguese Brazil
+            "pt_PT": "Português (Portugal)",  # Portuguese Portugal
+            "ro": "Română",  # Romanian
+            "ru": "Русский",  # Russian
+            "sr_Latn": "Srpska (latinica)",  #  Serbian (latin)
+            "es": "Español",  # Spanish
+            "sv": "Svenska",  # Swedish
+            "te": "తెలుగు",  # Telugu
+            "tr": "Türkçe",  # Turkish
+            "uk": "Українська",  # Ukrainian
+        }
+
+        # These are the default settings. They will get overwritten when loading from disk
+        self.default_settings = {
+            "version": self.common.version,
+            "connection_type": "bundled",
+            "control_port_address": "127.0.0.1",
+            "control_port_port": 9051,
+            "socks_address": "127.0.0.1",
+            "socks_port": 9050,
+            "socket_file_path": "/var/run/tor/control",
+            "auth_type": "no_auth",
+            "auth_password": "",
+            "use_autoupdate": True,
+            "autoupdate_timestamp": None,
+            "no_bridges": True,
+            "tor_bridges_use_obfs4": False,
+            "tor_bridges_use_meek_lite_azure": False,
+            "tor_bridges_use_custom_bridges": "",
+            "persistent_tabs": [],
+            "locale": None,  # this gets defined in fill_in_defaults()
+        }
+        self._settings = {}
+        self.fill_in_defaults()
+
+    def fill_in_defaults(self):
+        """
+        If there are any missing settings from self._settings, replace them with
+        their default values.
+        """
+        for key in self.default_settings:
+            if key not in self._settings:
+                self._settings[key] = self.default_settings[key]
+
+        # Choose the default locale based on the OS preference, and fall-back to English
+        if self._settings["locale"] is None:
+            language_code, encoding = locale.getdefaultlocale()
+
+            # Default to English
+            if not language_code:
+                language_code = "en_US"
+
+            if language_code == "pt_PT" and language_code == "pt_BR":
+                # Portuguese locales include country code
+                default_locale = language_code
+            else:
+                # All other locales cut off the country code
+                default_locale = language_code[:2]
+
+            if default_locale not in self.available_locales:
+                default_locale = "en"
+            self._settings["locale"] = default_locale
+
+    def build_filename(self):
+        """
+        Returns the path of the settings file.
+        """
+        return os.path.join(self.common.build_data_dir(), "onionshare.json")
+
+    def load(self):
+        """
+        Load the settings from file.
+        """
+        self.common.log("Settings", "load")
+
+        # If the settings file exists, load it
+        if os.path.exists(self.filename):
+            try:
+                self.common.log("Settings", "load", f"Trying to load {self.filename}")
+                with open(self.filename, "r") as f:
+                    self._settings = json.load(f)
+                    self.fill_in_defaults()
+            except:
+                pass
+
+        # Make sure data_dir exists
+        try:
+            os.makedirs(self.get("data_dir"), exist_ok=True)
+        except:
+            pass
+
+    def save(self):
+        """
+        Save settings to file.
+        """
+        self.common.log("Settings", "save")
+        open(self.filename, "w").write(json.dumps(self._settings, indent=2))
+        self.common.log("Settings", "save", f"Settings saved in {self.filename}")
+
+    def get(self, key):
+        return self._settings[key]
+
+    def set(self, key, val):
+        # If typecasting int values fails, fallback to default values
+        if key == "control_port_port" or key == "socks_port":
+            try:
+                val = int(val)
+            except:
+                if key == "control_port_port":
+                    val = self.default_settings["control_port_port"]
+                elif key == "socks_port":
+                    val = self.default_settings["socks_port"]
+
+        self._settings[key] = val

cli/onionshare_cli/web/receive_mode.py

@@ -0,0 +1,488 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import os
+import tempfile
+import json
+from datetime import datetime
+from flask import Request, request, render_template, make_response, flash, redirect
+from werkzeug.utils import secure_filename
+
+
+class ReceiveModeWeb:
+    """
+    All of the web logic for receive mode
+    """
+
+    def __init__(self, common, web):
+        self.common = common
+        self.common.log("ReceiveModeWeb", "__init__")
+
+        self.web = web
+
+        self.can_upload = True
+        self.uploads_in_progress = []
+
+        # This tracks the history id
+        self.cur_history_id = 0
+
+        self.define_routes()
+
+    def define_routes(self):
+        """
+        The web app routes for receiving files
+        """
+
+        @self.web.app.route("/")
+        def index():
+            history_id = self.cur_history_id
+            self.cur_history_id += 1
+            self.web.add_request(
+                self.web.REQUEST_INDIVIDUAL_FILE_STARTED,
+                request.path,
+                {"id": history_id, "status_code": 200},
+            )
+
+            self.web.add_request(self.web.REQUEST_LOAD, request.path)
+            r = make_response(
+                render_template(
+                    "receive.html", static_url_path=self.web.static_url_path
+                )
+            )
+            return self.web.add_security_headers(r)
+
+        @self.web.app.route("/upload", methods=["POST"])
+        def upload(ajax=False):
+            """
+            Handle the upload files POST request, though at this point, the files have
+            already been uploaded and saved to their correct locations.
+            """
+            files = request.files.getlist("file[]")
+            filenames = []
+            for f in files:
+                if f.filename != "":
+                    filename = secure_filename(f.filename)
+                    filenames.append(filename)
+                    local_path = os.path.join(request.receive_mode_dir, filename)
+                    basename = os.path.basename(local_path)
+
+                    # Tell the GUI the receive mode directory for this file
+                    self.web.add_request(
+                        self.web.REQUEST_UPLOAD_SET_DIR,
+                        request.path,
+                        {
+                            "id": request.history_id,
+                            "filename": basename,
+                            "dir": request.receive_mode_dir,
+                        },
+                    )
+
+                    self.common.log(
+                        "ReceiveModeWeb",
+                        "define_routes",
+                        f"/upload, uploaded {f.filename}, saving to {local_path}",
+                    )
+                    print(f"\nReceived: {local_path}")
+
+            if request.upload_error:
+                self.common.log(
+                    "ReceiveModeWeb",
+                    "define_routes",
+                    "/upload, there was an upload error",
+                )
+
+                self.web.add_request(
+                    self.web.REQUEST_ERROR_DATA_DIR_CANNOT_CREATE,
+                    request.path,
+                    {"receive_mode_dir": request.receive_mode_dir},
+                )
+                print(
+                    f"Could not create OnionShare data folder: {request.receive_mode_dir}"
+                )
+
+                msg = "Error uploading, please inform the OnionShare user"
+                if ajax:
+                    return json.dumps({"error_flashes": [msg]})
+                else:
+                    flash(msg, "error")
+                    return redirect("/")
+
+            if ajax:
+                info_flashes = []
+
+            if len(filenames) == 0:
+                msg = "No files uploaded"
+                if ajax:
+                    info_flashes.append(msg)
+                else:
+                    flash(msg, "info")
+            else:
+                msg = "Sent "
+                for filename in filenames:
+                    msg += f"{filename}, "
+                msg = msg.rstrip(", ")
+                if ajax:
+                    info_flashes.append(msg)
+                else:
+                    flash(msg, "info")
+
+            if self.can_upload:
+                if ajax:
+                    return json.dumps({"info_flashes": info_flashes})
+                else:
+                    return redirect("/")
+            else:
+                if ajax:
+                    return json.dumps(
+                        {
+                            "new_body": render_template(
+                                "thankyou.html",
+                                static_url_path=self.web.static_url_path,
+                            )
+                        }
+                    )
+                else:
+                    # It was the last upload and the timer ran out
+                    r = make_response(
+                        render_template("thankyou.html"),
+                        static_url_path=self.web.static_url_path,
+                    )
+                    return self.web.add_security_headers(r)
+
+        @self.web.app.route("/upload-ajax", methods=["POST"])
+        def upload_ajax_public():
+            if not self.can_upload:
+                return self.web.error403()
+            return upload(ajax=True)
+
+
+class ReceiveModeWSGIMiddleware(object):
+    """
+    Custom WSGI middleware in order to attach the Web object to environ, so
+    ReceiveModeRequest can access it.
+    """
+
+    def __init__(self, app, web):
+        self.app = app
+        self.web = web
+
+    def __call__(self, environ, start_response):
+        environ["web"] = self.web
+        environ["stop_q"] = self.web.stop_q
+        return self.app(environ, start_response)
+
+
+class ReceiveModeFile(object):
+    """
+    A custom file object that tells ReceiveModeRequest every time data gets
+    written to it, in order to track the progress of uploads. It starts out with
+    a .part file extension, and when it's complete it removes that extension.
+    """
+
+    def __init__(self, request, filename, write_func, close_func):
+        self.onionshare_request = request
+        self.onionshare_filename = filename
+        self.onionshare_write_func = write_func
+        self.onionshare_close_func = close_func
+
+        self.filename = os.path.join(self.onionshare_request.receive_mode_dir, filename)
+        self.filename_in_progress = f"{self.filename}.part"
+
+        # Open the file
+        self.upload_error = False
+        try:
+            self.f = open(self.filename_in_progress, "wb+")
+        except:
+            # This will only happen if someone is messing with the data dir while
+            # OnionShare is running, but if it does make sure to throw an error
+            self.upload_error = True
+            self.f = tempfile.TemporaryFile("wb+")
+
+        # Make all the file-like methods and attributes actually access the
+        # TemporaryFile, except for write
+        attrs = [
+            "closed",
+            "detach",
+            "fileno",
+            "flush",
+            "isatty",
+            "mode",
+            "name",
+            "peek",
+            "raw",
+            "read",
+            "read1",
+            "readable",
+            "readinto",
+            "readinto1",
+            "readline",
+            "readlines",
+            "seek",
+            "seekable",
+            "tell",
+            "truncate",
+            "writable",
+            "writelines",
+        ]
+        for attr in attrs:
+            setattr(self, attr, getattr(self.f, attr))
+
+    def write(self, b):
+        """
+        Custom write method that calls out to onionshare_write_func
+        """
+        if self.upload_error or (not self.onionshare_request.stop_q.empty()):
+            self.close()
+            self.onionshare_request.close()
+            return
+
+        try:
+            bytes_written = self.f.write(b)
+            self.onionshare_write_func(self.onionshare_filename, bytes_written)
+
+        except:
+            self.upload_error = True
+
+    def close(self):
+        """
+        Custom close method that calls out to onionshare_close_func
+        """
+        try:
+            self.f.close()
+
+            if not self.upload_error:
+                # Rename the in progress file to the final filename
+                os.rename(self.filename_in_progress, self.filename)
+
+        except:
+            self.upload_error = True
+
+        self.onionshare_close_func(self.onionshare_filename, self.upload_error)
+
+
+class ReceiveModeRequest(Request):
+    """
+    A custom flask Request object that keeps track of how much data has been
+    uploaded for each file, for receive mode.
+    """
+
+    def __init__(self, environ, populate_request=True, shallow=False):
+        super(ReceiveModeRequest, self).__init__(environ, populate_request, shallow)
+        self.web = environ["web"]
+        self.stop_q = environ["stop_q"]
+
+        self.web.common.log("ReceiveModeRequest", "__init__")
+
+        # Prevent running the close() method more than once
+        self.closed = False
+
+        # Is this a valid upload request?
+        self.upload_request = False
+        if self.method == "POST":
+            if self.path == "/upload" or self.path == "/upload-ajax":
+                self.upload_request = True
+
+        if self.upload_request:
+            # No errors yet
+            self.upload_error = False
+
+            # Figure out what files should be saved
+            now = datetime.now()
+            date_dir = now.strftime("%Y-%m-%d")
+            time_dir = now.strftime("%H.%M.%S")
+            self.receive_mode_dir = os.path.join(
+                self.web.settings.get("receive", "data_dir"), date_dir, time_dir
+            )
+
+            # Create that directory, which shouldn't exist yet
+            try:
+                os.makedirs(self.receive_mode_dir, 0o700, exist_ok=False)
+            except OSError:
+                # If this directory already exists, maybe someone else is uploading files at
+                # the same second, so use a different name in that case
+                if os.path.exists(self.receive_mode_dir):
+                    # Keep going until we find a directory name that's available
+                    i = 1
+                    while True:
+                        new_receive_mode_dir = f"{self.receive_mode_dir}-{i}"
+                        try:
+                            os.makedirs(new_receive_mode_dir, 0o700, exist_ok=False)
+                            self.receive_mode_dir = new_receive_mode_dir
+                            break
+                        except OSError:
+                            pass
+                        i += 1
+                        # Failsafe
+                        if i == 100:
+                            self.web.common.log(
+                                "ReceiveModeRequest",
+                                "__init__",
+                                "Error finding available receive mode directory",
+                            )
+                            self.upload_error = True
+                            break
+            except PermissionError:
+                self.web.add_request(
+                    self.web.REQUEST_ERROR_DATA_DIR_CANNOT_CREATE,
+                    request.path,
+                    {"receive_mode_dir": self.receive_mode_dir},
+                )
+                print(
+                    f"Could not create OnionShare data folder: {self.receive_mode_dir}"
+                )
+                self.web.common.log(
+                    "ReceiveModeRequest",
+                    "__init__",
+                    "Permission denied creating receive mode directory",
+                )
+                self.upload_error = True
+
+            # If there's an error so far, finish early
+            if self.upload_error:
+                return
+
+            # A dictionary that maps filenames to the bytes uploaded so far
+            self.progress = {}
+
+            # Prevent new uploads if we've said so (timer expired)
+            if self.web.receive_mode.can_upload:
+
+                # Create an history_id, attach it to the request
+                self.history_id = self.web.receive_mode.cur_history_id
+                self.web.receive_mode.cur_history_id += 1
+
+                # Figure out the content length
+                try:
+                    self.content_length = int(self.headers["Content-Length"])
+                except:
+                    self.content_length = 0
+
+                date_str = datetime.now().strftime("%b %d, %I:%M%p")
+                size_str = self.web.common.human_readable_filesize(self.content_length)
+                print(f"{date_str}: Upload of total size {size_str} is starting")
+
+                # Don't tell the GUI that a request has started until we start receiving files
+                self.told_gui_about_request = False
+
+                self.previous_file = None
+
+    def _get_file_stream(
+        self, total_content_length, content_type, filename=None, content_length=None
+    ):
+        """
+        This gets called for each file that gets uploaded, and returns an file-like
+        writable stream.
+        """
+        if self.upload_request:
+            if not self.told_gui_about_request:
+                # Tell the GUI about the request
+                self.web.add_request(
+                    self.web.REQUEST_STARTED,
+                    self.path,
+                    {"id": self.history_id, "content_length": self.content_length},
+                )
+                self.web.receive_mode.uploads_in_progress.append(self.history_id)
+
+                self.told_gui_about_request = True
+
+            self.filename = secure_filename(filename)
+
+            self.progress[self.filename] = {"uploaded_bytes": 0, "complete": False}
+
+        f = ReceiveModeFile(
+            self, self.filename, self.file_write_func, self.file_close_func
+        )
+        if f.upload_error:
+            self.web.common.log(
+                "ReceiveModeRequest", "_get_file_stream", "Error creating file"
+            )
+            self.upload_error = True
+        return f
+
+    def close(self):
+        """
+        Closing the request.
+        """
+        super(ReceiveModeRequest, self).close()
+
+        # Prevent calling this method more than once per request
+        if self.closed:
+            return
+        self.closed = True
+
+        self.web.common.log("ReceiveModeRequest", "close")
+
+        try:
+            if self.told_gui_about_request:
+                history_id = self.history_id
+
+                if (
+                    not self.web.stop_q.empty()
+                    or not self.progress[self.filename]["complete"]
+                ):
+                    # Inform the GUI that the upload has canceled
+                    self.web.add_request(
+                        self.web.REQUEST_UPLOAD_CANCELED, self.path, {"id": history_id}
+                    )
+                else:
+                    # Inform the GUI that the upload has finished
+                    self.web.add_request(
+                        self.web.REQUEST_UPLOAD_FINISHED, self.path, {"id": history_id}
+                    )
+                self.web.receive_mode.uploads_in_progress.remove(history_id)
+
+        except AttributeError:
+            pass
+
+    def file_write_func(self, filename, length):
+        """
+        This function gets called when a specific file is written to.
+        """
+        if self.closed:
+            return
+
+        if self.upload_request:
+            self.progress[filename]["uploaded_bytes"] += length
+
+            if self.previous_file != filename:
+                self.previous_file = filename
+
+            size_str = self.web.common.human_readable_filesize(
+                self.progress[filename]["uploaded_bytes"]
+            )
+            print(f"\r=> {size_str} {filename}          ", end="")
+
+            # Update the GUI on the upload progress
+            if self.told_gui_about_request:
+                self.web.add_request(
+                    self.web.REQUEST_PROGRESS,
+                    self.path,
+                    {"id": self.history_id, "progress": self.progress},
+                )
+
+    def file_close_func(self, filename, upload_error=False):
+        """
+        This function gets called when a specific file is closed.
+        """
+        self.progress[filename]["complete"] = True
+
+        # If the file tells us there was an upload error, let the request know as well
+        if upload_error:
+            self.upload_error = True

cli/onionshare_cli/web/send_base_mode.py

@@ -0,0 +1,321 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import os
+import sys
+import tempfile
+import mimetypes
+import gzip
+from flask import Response, request, render_template, make_response
+
+
+class SendBaseModeWeb:
+    """
+    All of the web logic shared between share and website mode (modes where the user sends files)
+    """
+
+    def __init__(self, common, web):
+        super(SendBaseModeWeb, self).__init__()
+        self.common = common
+        self.web = web
+
+        # Information about the file to be shared
+        self.is_zipped = False
+        self.download_filename = None
+        self.download_filesize = None
+        self.gzip_filename = None
+        self.gzip_filesize = None
+        self.zip_writer = None
+
+        # If autostop_sharing, only allow one download at a time
+        self.download_in_progress = False
+
+        # This tracks the history id
+        self.cur_history_id = 0
+
+        self.define_routes()
+        self.init()
+
+    def set_file_info(self, filenames, processed_size_callback=None):
+        """
+        Build a data structure that describes the list of files
+        """
+        # If there's just one folder, replace filenames with a list of files inside that folder
+        if len(filenames) == 1 and os.path.isdir(filenames[0]):
+            filenames = [
+                os.path.join(filenames[0], x) for x in os.listdir(filenames[0])
+            ]
+
+        # Re-initialize
+        self.files = {}  # Dictionary mapping file paths to filenames on disk
+        self.root_files = (
+            {}
+        )  # This is only the root files and dirs, as opposed to all of them
+        self.cleanup_filenames = []
+        self.cur_history_id = 0
+        self.file_info = {"files": [], "dirs": []}
+        self.gzip_individual_files = {}
+        self.init()
+
+        # Build the file list
+        for filename in filenames:
+            basename = os.path.basename(filename.rstrip("/"))
+
+            # If it's a filename, add it
+            if os.path.isfile(filename):
+                self.files[basename] = filename
+                self.root_files[basename] = filename
+
+            # If it's a directory, add it recursively
+            elif os.path.isdir(filename):
+                self.root_files[basename + "/"] = filename
+
+                for root, _, nested_filenames in os.walk(filename):
+                    # Normalize the root path. So if the directory name is "/home/user/Documents/some_folder",
+                    # and it has a nested folder foobar, the root is "/home/user/Documents/some_folder/foobar".
+                    # The normalized_root should be "some_folder/foobar"
+                    normalized_root = os.path.join(
+                        basename, root[len(filename) :].lstrip("/")
+                    ).rstrip("/")
+
+                    # Add the dir itself
+                    self.files[normalized_root + "/"] = root
+
+                    # Add the files in this dir
+                    for nested_filename in nested_filenames:
+                        self.files[
+                            os.path.join(normalized_root, nested_filename)
+                        ] = os.path.join(root, nested_filename)
+
+        self.set_file_info_custom(filenames, processed_size_callback)
+
+    def directory_listing(self, filenames, path="", filesystem_path=None):
+        # Tell the GUI about the directory listing
+        history_id = self.cur_history_id
+        self.cur_history_id += 1
+        self.web.add_request(
+            self.web.REQUEST_INDIVIDUAL_FILE_STARTED,
+            f"/{path}",
+            {"id": history_id, "method": request.method, "status_code": 200},
+        )
+
+        breadcrumbs = [("☗", "/")]
+        parts = path.split("/")[:-1]
+        for i in range(len(parts)):
+            breadcrumbs.append((parts[i], f"/{'/'.join(parts[0 : i + 1])}/"))
+        breadcrumbs_leaf = breadcrumbs.pop()[0]
+
+        # If filesystem_path is None, this is the root directory listing
+        files, dirs = self.build_directory_listing(filenames, filesystem_path)
+        r = self.directory_listing_template(
+            path, files, dirs, breadcrumbs, breadcrumbs_leaf
+        )
+        return self.web.add_security_headers(r)
+
+    def build_directory_listing(self, filenames, filesystem_path):
+        files = []
+        dirs = []
+
+        for filename in filenames:
+            if filesystem_path:
+                this_filesystem_path = os.path.join(filesystem_path, filename)
+            else:
+                this_filesystem_path = self.files[filename]
+
+            is_dir = os.path.isdir(this_filesystem_path)
+
+            if is_dir:
+                dirs.append({"basename": filename})
+            else:
+                size = os.path.getsize(this_filesystem_path)
+                size_human = self.common.human_readable_filesize(size)
+                files.append({"basename": filename, "size_human": size_human})
+        return files, dirs
+
+    def stream_individual_file(self, filesystem_path):
+        """
+        Return a flask response that's streaming the download of an individual file, and gzip
+        compressing it if the browser supports it.
+        """
+        use_gzip = self.should_use_gzip()
+
+        # gzip compress the individual file, if it hasn't already been compressed
+        if use_gzip:
+            if filesystem_path not in self.gzip_individual_files:
+                gzip_filename = tempfile.mkstemp("wb+")[1]
+                self._gzip_compress(filesystem_path, gzip_filename, 6, None)
+                self.gzip_individual_files[filesystem_path] = gzip_filename
+
+                # Make sure the gzip file gets cleaned up when onionshare stops
+                self.cleanup_filenames.append(gzip_filename)
+
+            file_to_download = self.gzip_individual_files[filesystem_path]
+            filesize = os.path.getsize(self.gzip_individual_files[filesystem_path])
+        else:
+            file_to_download = filesystem_path
+            filesize = os.path.getsize(filesystem_path)
+
+        path = request.path
+
+        # Tell GUI the individual file started
+        history_id = self.cur_history_id
+        self.cur_history_id += 1
+
+        # Only GET requests are allowed, any other method should fail
+        if request.method != "GET":
+            return self.web.error405(history_id)
+
+        self.web.add_request(
+            self.web.REQUEST_INDIVIDUAL_FILE_STARTED,
+            path,
+            {"id": history_id, "filesize": filesize},
+        )
+
+        def generate():
+            chunk_size = 102400  # 100kb
+
+            fp = open(file_to_download, "rb")
+            done = False
+            while not done:
+                chunk = fp.read(chunk_size)
+                if chunk == b"":
+                    done = True
+                else:
+                    try:
+                        yield chunk
+
+                        # Tell GUI the progress
+                        downloaded_bytes = fp.tell()
+                        percent = (1.0 * downloaded_bytes / filesize) * 100
+                        if (
+                            not self.web.is_gui
+                            or self.common.platform == "Linux"
+                            or self.common.platform == "BSD"
+                        ):
+                            sys.stdout.write(
+                                "\r{0:s}, {1:.2f}%          ".format(
+                                    self.common.human_readable_filesize(
+                                        downloaded_bytes
+                                    ),
+                                    percent,
+                                )
+                            )
+                            sys.stdout.flush()
+
+                        self.web.add_request(
+                            self.web.REQUEST_INDIVIDUAL_FILE_PROGRESS,
+                            path,
+                            {
+                                "id": history_id,
+                                "bytes": downloaded_bytes,
+                                "filesize": filesize,
+                            },
+                        )
+                        done = False
+                    except:
+                        # Looks like the download was canceled
+                        done = True
+
+                        # Tell the GUI the individual file was canceled
+                        self.web.add_request(
+                            self.web.REQUEST_INDIVIDUAL_FILE_CANCELED,
+                            path,
+                            {"id": history_id},
+                        )
+
+            fp.close()
+
+            if self.common.platform != "Darwin":
+                sys.stdout.write("\n")
+
+        basename = os.path.basename(filesystem_path)
+
+        r = Response(generate())
+        if use_gzip:
+            r.headers.set("Content-Encoding", "gzip")
+        r.headers.set("Content-Length", filesize)
+        r.headers.set("Content-Disposition", "inline", filename=basename)
+        r = self.web.add_security_headers(r)
+        (content_type, _) = mimetypes.guess_type(basename, strict=False)
+        if content_type is not None:
+            r.headers.set("Content-Type", content_type)
+        return r
+
+    def should_use_gzip(self):
+        """
+        Should we use gzip for this browser?
+        """
+        return (not self.is_zipped) and (
+            "gzip" in request.headers.get("Accept-Encoding", "").lower()
+        )
+
+    def _gzip_compress(
+        self, input_filename, output_filename, level, processed_size_callback=None
+    ):
+        """
+        Compress a file with gzip, without loading the whole thing into memory
+        Thanks: https://stackoverflow.com/questions/27035296/python-how-to-gzip-a-large-text-file-without-memoryerror
+        """
+        bytes_processed = 0
+        blocksize = 1 << 16  # 64kB
+        with open(input_filename, "rb") as input_file:
+            output_file = gzip.open(output_filename, "wb", level)
+            while True:
+                if processed_size_callback is not None:
+                    processed_size_callback(bytes_processed)
+
+                block = input_file.read(blocksize)
+                if len(block) == 0:
+                    break
+                output_file.write(block)
+                bytes_processed += blocksize
+
+            output_file.close()
+
+    def init(self):
+        """
+        Inherited class will implement this
+        """
+        pass
+
+    def define_routes(self):
+        """
+        Inherited class will implement this
+        """
+        pass
+
+    def directory_listing_template(self):
+        """
+        Inherited class will implement this. It should call render_template and return
+        the response.
+        """
+        pass
+
+    def set_file_info_custom(self, filenames, processed_size_callback):
+        """
+        Inherited class will implement this.
+        """
+        pass
+
+    def render_logic(self, path=""):
+        """
+        Inherited class will implement this.
+        """
+        pass

cli/onionshare_cli/web/share_mode.py

@@ -0,0 +1,411 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import os
+import sys
+import tempfile
+import zipfile
+import mimetypes
+from flask import Response, request, render_template, make_response
+
+from .send_base_mode import SendBaseModeWeb
+
+
+class ShareModeWeb(SendBaseModeWeb):
+    """
+    All of the web logic for share mode
+    """
+
+    def init(self):
+        self.common.log("ShareModeWeb", "init")
+
+        # Allow downloading individual files if "Stop sharing after files have been sent" is unchecked
+        self.download_individual_files = not self.web.settings.get(
+            "share", "autostop_sharing"
+        )
+
+    def define_routes(self):
+        """
+        The web app routes for sharing files
+        """
+
+        @self.web.app.route("/", defaults={"path": ""})
+        @self.web.app.route("/<path:path>")
+        def index(path):
+            """
+            Render the template for the onionshare landing page.
+            """
+            self.web.add_request(self.web.REQUEST_LOAD, request.path)
+
+            # Deny new downloads if "Stop sharing after files have been sent" is checked and there is
+            # currently a download
+            deny_download = (
+                self.web.settings.get("share", "autostop_sharing")
+                and self.download_in_progress
+            )
+            if deny_download:
+                r = make_response(
+                    render_template("denied.html"),
+                    static_url_path=self.web.static_url_path,
+                )
+                return self.web.add_security_headers(r)
+
+            # If download is allowed to continue, serve download page
+            if self.should_use_gzip():
+                self.filesize = self.gzip_filesize
+            else:
+                self.filesize = self.download_filesize
+
+            return self.render_logic(path)
+
+        @self.web.app.route("/download")
+        def download():
+            """
+            Download the zip file.
+            """
+            # Deny new downloads if "Stop After First Download" is checked and there is
+            # currently a download
+            deny_download = (
+                self.web.settings.get("share", "autostop_sharing")
+                and self.download_in_progress
+            )
+            if deny_download:
+                r = make_response(
+                    render_template(
+                        "denied.html", static_url_path=self.web.static_url_path
+                    )
+                )
+                return self.web.add_security_headers(r)
+
+            # Prepare some variables to use inside generate() function below
+            # which is outside of the request context
+            shutdown_func = request.environ.get("werkzeug.server.shutdown")
+            path = request.path
+
+            # If this is a zipped file, then serve as-is. If it's not zipped, then,
+            # if the http client supports gzip compression, gzip the file first
+            # and serve that
+            use_gzip = self.should_use_gzip()
+            if use_gzip:
+                file_to_download = self.gzip_filename
+                self.filesize = self.gzip_filesize
+            else:
+                file_to_download = self.download_filename
+                self.filesize = self.download_filesize
+
+            # Tell GUI the download started
+            history_id = self.cur_history_id
+            self.cur_history_id += 1
+            self.web.add_request(
+                self.web.REQUEST_STARTED, path, {"id": history_id, "use_gzip": use_gzip}
+            )
+
+            basename = os.path.basename(self.download_filename)
+
+            def generate():
+                # Starting a new download
+                if self.web.settings.get("share", "autostop_sharing"):
+                    self.download_in_progress = True
+
+                chunk_size = 102400  # 100kb
+
+                fp = open(file_to_download, "rb")
+                self.web.done = False
+                canceled = False
+                while not self.web.done:
+                    # The user has canceled the download, so stop serving the file
+                    if not self.web.stop_q.empty():
+                        self.web.add_request(
+                            self.web.REQUEST_CANCELED, path, {"id": history_id}
+                        )
+                        break
+
+                    chunk = fp.read(chunk_size)
+                    if chunk == b"":
+                        self.web.done = True
+                    else:
+                        try:
+                            yield chunk
+
+                            # tell GUI the progress
+                            downloaded_bytes = fp.tell()
+                            percent = (1.0 * downloaded_bytes / self.filesize) * 100
+
+                            # only output to stdout if running onionshare in CLI mode, or if using Linux (#203, #304)
+                            if (
+                                not self.web.is_gui
+                                or self.common.platform == "Linux"
+                                or self.common.platform == "BSD"
+                            ):
+                                sys.stdout.write(
+                                    "\r{0:s}, {1:.2f}%          ".format(
+                                        self.common.human_readable_filesize(
+                                            downloaded_bytes
+                                        ),
+                                        percent,
+                                    )
+                                )
+                                sys.stdout.flush()
+
+                            self.web.add_request(
+                                self.web.REQUEST_PROGRESS,
+                                path,
+                                {"id": history_id, "bytes": downloaded_bytes},
+                            )
+                            self.web.done = False
+                        except:
+                            # looks like the download was canceled
+                            self.web.done = True
+                            canceled = True
+
+                            # tell the GUI the download has canceled
+                            self.web.add_request(
+                                self.web.REQUEST_CANCELED, path, {"id": history_id}
+                            )
+
+                fp.close()
+
+                if self.common.platform != "Darwin":
+                    sys.stdout.write("\n")
+
+                # Download is finished
+                if self.web.settings.get("share", "autostop_sharing"):
+                    self.download_in_progress = False
+
+                # Close the server, if necessary
+                if self.web.settings.get("share", "autostop_sharing") and not canceled:
+                    print("Stopped because transfer is complete")
+                    self.web.running = False
+                    try:
+                        if shutdown_func is None:
+                            raise RuntimeError("Not running with the Werkzeug Server")
+                        shutdown_func()
+                    except:
+                        pass
+
+            r = Response(generate())
+            if use_gzip:
+                r.headers.set("Content-Encoding", "gzip")
+            r.headers.set("Content-Length", self.filesize)
+            r.headers.set("Content-Disposition", "attachment", filename=basename)
+            r = self.web.add_security_headers(r)
+            # guess content type
+            (content_type, _) = mimetypes.guess_type(basename, strict=False)
+            if content_type is not None:
+                r.headers.set("Content-Type", content_type)
+            return r
+
+    def directory_listing_template(
+        self, path, files, dirs, breadcrumbs, breadcrumbs_leaf
+    ):
+        return make_response(
+            render_template(
+                "send.html",
+                file_info=self.file_info,
+                files=files,
+                dirs=dirs,
+                breadcrumbs=breadcrumbs,
+                breadcrumbs_leaf=breadcrumbs_leaf,
+                filename=os.path.basename(self.download_filename),
+                filesize=self.filesize,
+                filesize_human=self.common.human_readable_filesize(
+                    self.download_filesize
+                ),
+                is_zipped=self.is_zipped,
+                static_url_path=self.web.static_url_path,
+                download_individual_files=self.download_individual_files,
+            )
+        )
+
+    def set_file_info_custom(self, filenames, processed_size_callback):
+        self.common.log("ShareModeWeb", "set_file_info_custom")
+        self.web.cancel_compression = False
+        self.build_zipfile_list(filenames, processed_size_callback)
+
+    def render_logic(self, path=""):
+        if path in self.files:
+            filesystem_path = self.files[path]
+
+            # If it's a directory
+            if os.path.isdir(filesystem_path):
+                # Render directory listing
+                filenames = []
+                for filename in os.listdir(filesystem_path):
+                    if os.path.isdir(os.path.join(filesystem_path, filename)):
+                        filenames.append(filename + "/")
+                    else:
+                        filenames.append(filename)
+                filenames.sort()
+                return self.directory_listing(filenames, path, filesystem_path)
+
+            # If it's a file
+            elif os.path.isfile(filesystem_path):
+                if self.download_individual_files:
+                    return self.stream_individual_file(filesystem_path)
+                else:
+                    history_id = self.cur_history_id
+                    self.cur_history_id += 1
+                    return self.web.error404(history_id)
+
+            # If it's not a directory or file, throw a 404
+            else:
+                history_id = self.cur_history_id
+                self.cur_history_id += 1
+                return self.web.error404(history_id)
+        else:
+            # Special case loading /
+
+            if path == "":
+                # Root directory listing
+                filenames = list(self.root_files)
+                filenames.sort()
+                return self.directory_listing(filenames, path)
+
+            else:
+                # If the path isn't found, throw a 404
+                history_id = self.cur_history_id
+                self.cur_history_id += 1
+                return self.web.error404(history_id)
+
+    def build_zipfile_list(self, filenames, processed_size_callback=None):
+        self.common.log("ShareModeWeb", "build_zipfile_list")
+        for filename in filenames:
+            info = {
+                "filename": filename,
+                "basename": os.path.basename(filename.rstrip("/")),
+            }
+            if os.path.isfile(filename):
+                info["size"] = os.path.getsize(filename)
+                info["size_human"] = self.common.human_readable_filesize(info["size"])
+                self.file_info["files"].append(info)
+            if os.path.isdir(filename):
+                info["size"] = self.common.dir_size(filename)
+                info["size_human"] = self.common.human_readable_filesize(info["size"])
+                self.file_info["dirs"].append(info)
+        self.file_info["files"] = sorted(
+            self.file_info["files"], key=lambda k: k["basename"]
+        )
+        self.file_info["dirs"] = sorted(
+            self.file_info["dirs"], key=lambda k: k["basename"]
+        )
+
+        # Check if there's only 1 file and no folders
+        if len(self.file_info["files"]) == 1 and len(self.file_info["dirs"]) == 0:
+            self.download_filename = self.file_info["files"][0]["filename"]
+            self.download_filesize = self.file_info["files"][0]["size"]
+
+            # Compress the file with gzip now, so we don't have to do it on each request
+            self.gzip_filename = tempfile.mkstemp("wb+")[1]
+            self._gzip_compress(
+                self.download_filename, self.gzip_filename, 6, processed_size_callback
+            )
+            self.gzip_filesize = os.path.getsize(self.gzip_filename)
+
+            # Make sure the gzip file gets cleaned up when onionshare stops
+            self.cleanup_filenames.append(self.gzip_filename)
+
+            self.is_zipped = False
+
+        else:
+            # Zip up the files and folders
+            self.zip_writer = ZipWriter(
+                self.common, processed_size_callback=processed_size_callback
+            )
+            self.download_filename = self.zip_writer.zip_filename
+            for info in self.file_info["files"]:
+                self.zip_writer.add_file(info["filename"])
+                # Canceling early?
+                if self.web.cancel_compression:
+                    self.zip_writer.close()
+                    return False
+
+            for info in self.file_info["dirs"]:
+                if not self.zip_writer.add_dir(info["filename"]):
+                    return False
+
+            self.zip_writer.close()
+            self.download_filesize = os.path.getsize(self.download_filename)
+
+            # Make sure the zip file gets cleaned up when onionshare stops
+            self.cleanup_filenames.append(self.zip_writer.zip_filename)
+
+            self.is_zipped = True
+
+        return True
+
+
+class ZipWriter(object):
+    """
+    ZipWriter accepts files and directories and compresses them into a zip file
+    with. If a zip_filename is not passed in, it will use the default onionshare
+    filename.
+    """
+
+    def __init__(self, common, zip_filename=None, processed_size_callback=None):
+        self.common = common
+        self.cancel_compression = False
+
+        if zip_filename:
+            self.zip_filename = zip_filename
+        else:
+            self.zip_filename = (
+                f"{tempfile.mkdtemp()}/onionshare_{self.common.random_string(4, 6)}.zip"
+            )
+
+        self.z = zipfile.ZipFile(self.zip_filename, "w", allowZip64=True)
+        self.processed_size_callback = processed_size_callback
+        if self.processed_size_callback is None:
+            self.processed_size_callback = lambda _: None
+        self._size = 0
+        self.processed_size_callback(self._size)
+
+    def add_file(self, filename):
+        """
+        Add a file to the zip archive.
+        """
+        self.z.write(filename, os.path.basename(filename), zipfile.ZIP_DEFLATED)
+        self._size += os.path.getsize(filename)
+        self.processed_size_callback(self._size)
+
+    def add_dir(self, filename):
+        """
+        Add a directory, and all of its children, to the zip archive.
+        """
+        dir_to_strip = os.path.dirname(filename.rstrip("/")) + "/"
+        for dirpath, dirnames, filenames in os.walk(filename):
+            for f in filenames:
+                # Canceling early?
+                if self.cancel_compression:
+                    return False
+
+                full_filename = os.path.join(dirpath, f)
+                if not os.path.islink(full_filename):
+                    arc_filename = full_filename[len(dir_to_strip) :]
+                    self.z.write(full_filename, arc_filename, zipfile.ZIP_DEFLATED)
+                    self._size += os.path.getsize(full_filename)
+                    self.processed_size_callback(self._size)
+
+        return True
+
+    def close(self):
+        """
+        Close the zip archive.
+        """
+        self.z.close()

cli/onionshare_cli/web/web.py

@@ -0,0 +1,424 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import hmac
+import logging
+import os
+import queue
+import socket
+import sys
+import tempfile
+import requests
+from distutils.version import LooseVersion as Version
+from urllib.request import urlopen
+
+import flask
+from flask import (
+    Flask,
+    request,
+    render_template,
+    abort,
+    make_response,
+    send_file,
+    __version__ as flask_version,
+)
+from flask_httpauth import HTTPBasicAuth
+from flask_socketio import SocketIO
+
+from .share_mode import ShareModeWeb
+from .receive_mode import ReceiveModeWeb, ReceiveModeWSGIMiddleware, ReceiveModeRequest
+from .website_mode import WebsiteModeWeb
+from .chat_mode import ChatModeWeb
+
+# Stub out flask's show_server_banner function, to avoiding showing warnings that
+# are not applicable to OnionShare
+def stubbed_show_server_banner(env, debug, app_import_path, eager_loading):
+    pass
+
+
+try:
+    flask.cli.show_server_banner = stubbed_show_server_banner
+except:
+    pass
+
+
+class Web:
+    """
+    The Web object is the OnionShare web server, powered by flask
+    """
+
+    REQUEST_LOAD = 0
+    REQUEST_STARTED = 1
+    REQUEST_PROGRESS = 2
+    REQUEST_CANCELED = 3
+    REQUEST_RATE_LIMIT = 4
+    REQUEST_UPLOAD_FILE_RENAMED = 5
+    REQUEST_UPLOAD_SET_DIR = 6
+    REQUEST_UPLOAD_FINISHED = 7
+    REQUEST_UPLOAD_CANCELED = 8
+    REQUEST_INDIVIDUAL_FILE_STARTED = 9
+    REQUEST_INDIVIDUAL_FILE_PROGRESS = 10
+    REQUEST_INDIVIDUAL_FILE_CANCELED = 11
+    REQUEST_ERROR_DATA_DIR_CANNOT_CREATE = 12
+    REQUEST_OTHER = 13
+    REQUEST_INVALID_PASSWORD = 14
+
+    def __init__(self, common, is_gui, mode_settings, mode="share"):
+        self.common = common
+        self.common.log("Web", "__init__", f"is_gui={is_gui}, mode={mode}")
+
+        self.settings = mode_settings
+
+        # The flask app
+        self.app = Flask(
+            __name__,
+            static_folder=self.common.get_resource_path("static"),
+            static_url_path=f"/static_{self.common.random_string(16)}",  # randomize static_url_path to avoid making /static unusable
+            template_folder=self.common.get_resource_path("templates"),
+        )
+        self.app.secret_key = self.common.random_string(8)
+        self.generate_static_url_path()
+        self.auth = HTTPBasicAuth()
+        self.auth.error_handler(self.error401)
+
+        # Verbose mode?
+        if self.common.verbose:
+            self.verbose_mode()
+
+        # Are we running in GUI mode?
+        self.is_gui = is_gui
+
+        # If the user stops the server while a transfer is in progress, it should
+        # immediately stop the transfer. In order to make it thread-safe, stop_q
+        # is a queue. If anything is in it, then the user stopped the server
+        self.stop_q = queue.Queue()
+
+        # Are we using receive mode?
+        self.mode = mode
+        if self.mode == "receive":
+            # Use custom WSGI middleware, to modify environ
+            self.app.wsgi_app = ReceiveModeWSGIMiddleware(self.app.wsgi_app, self)
+            # Use a custom Request class to track upload progess
+            self.app.request_class = ReceiveModeRequest
+
+        # Starting in Flask 0.11, render_template_string autoescapes template variables
+        # by default. To prevent content injection through template variables in
+        # earlier versions of Flask, we force autoescaping in the Jinja2 template
+        # engine if we detect a Flask version with insecure default behavior.
+        if Version(flask_version) < Version("0.11"):
+            # Monkey-patch in the fix from https://github.com/pallets/flask/commit/99c99c4c16b1327288fd76c44bc8635a1de452bc
+            Flask.select_jinja_autoescape = self._safe_select_jinja_autoescape
+
+        self.security_headers = [
+            ("X-Frame-Options", "DENY"),
+            ("X-Xss-Protection", "1; mode=block"),
+            ("X-Content-Type-Options", "nosniff"),
+            ("Referrer-Policy", "no-referrer"),
+            ("Server", "OnionShare"),
+        ]
+
+        self.q = queue.Queue()
+        self.password = None
+
+        self.reset_invalid_passwords()
+
+        self.done = False
+
+        # shutting down the server only works within the context of flask, so the easiest way to do it is over http
+        self.shutdown_password = self.common.random_string(16)
+
+        # Keep track if the server is running
+        self.running = False
+
+        # Define the web app routes
+        self.define_common_routes()
+
+        # Create the mode web object, which defines its own routes
+        self.share_mode = None
+        self.receive_mode = None
+        self.website_mode = None
+        self.chat_mode = None
+        if self.mode == "share":
+            self.share_mode = ShareModeWeb(self.common, self)
+        elif self.mode == "receive":
+            self.receive_mode = ReceiveModeWeb(self.common, self)
+        elif self.mode == "website":
+            self.website_mode = WebsiteModeWeb(self.common, self)
+        elif self.mode == "chat":
+            self.socketio = SocketIO()
+            self.socketio.init_app(self.app)
+            self.chat_mode = ChatModeWeb(self.common, self)
+
+    def get_mode(self):
+        if self.mode == "share":
+            return self.share_mode
+        elif self.mode == "receive":
+            return self.receive_mode
+        elif self.mode == "website":
+            return self.website_mode
+        elif self.mode == "chat":
+            return self.chat_mode
+        else:
+            return None
+
+    def generate_static_url_path(self):
+        # The static URL path has a 128-bit random number in it to avoid having name
+        # collisions with files that might be getting shared
+        self.static_url_path = f"/static_{self.common.random_string(16)}"
+        self.common.log(
+            "Web",
+            "generate_static_url_path",
+            f"new static_url_path is {self.static_url_path}",
+        )
+
+        # Update the flask route to handle the new static URL path
+        self.app.static_url_path = self.static_url_path
+        self.app.add_url_rule(
+            self.static_url_path + "/<path:filename>",
+            endpoint="static",
+            view_func=self.app.send_static_file,
+        )
+
+    def define_common_routes(self):
+        """
+        Common web app routes between all modes.
+        """
+
+        @self.auth.get_password
+        def get_pw(username):
+            if username == "onionshare":
+                return self.password
+            else:
+                return None
+
+        @self.app.before_request
+        def conditional_auth_check():
+            # Allow static files without basic authentication
+            if request.path.startswith(self.static_url_path + "/"):
+                return None
+
+            # If public mode is disabled, require authentication
+            if not self.settings.get("general", "public"):
+
+                @self.auth.login_required
+                def _check_login():
+                    return None
+
+                return _check_login()
+
+        @self.app.errorhandler(404)
+        def not_found(e):
+            mode = self.get_mode()
+            history_id = mode.cur_history_id
+            mode.cur_history_id += 1
+            return self.error404(history_id)
+
+        @self.app.route("/<password_candidate>/shutdown")
+        def shutdown(password_candidate):
+            """
+            Stop the flask web server, from the context of an http request.
+            """
+            if password_candidate == self.shutdown_password:
+                self.force_shutdown()
+                return ""
+            abort(404)
+
+        if self.mode != "website":
+
+            @self.app.route("/favicon.ico")
+            def favicon():
+                return send_file(
+                    f"{self.common.get_resource_path('static')}/img/favicon.ico"
+                )
+
+    def error401(self):
+        auth = request.authorization
+        if auth:
+            if (
+                auth["username"] == "onionshare"
+                and auth["password"] not in self.invalid_passwords
+            ):
+                print(f"Invalid password guess: {auth['password']}")
+                self.add_request(Web.REQUEST_INVALID_PASSWORD, data=auth["password"])
+
+                self.invalid_passwords.append(auth["password"])
+                self.invalid_passwords_count += 1
+
+                if self.invalid_passwords_count == 20:
+                    self.add_request(Web.REQUEST_RATE_LIMIT)
+                    self.force_shutdown()
+                    print(
+                        "Someone has made too many wrong attempts to guess your password, so OnionShare has stopped the server. Start sharing again and send the recipient a new address to share."
+                    )
+
+        r = make_response(
+            render_template("401.html", static_url_path=self.static_url_path), 401
+        )
+        return self.add_security_headers(r)
+
+    def error403(self):
+        self.add_request(Web.REQUEST_OTHER, request.path)
+        r = make_response(
+            render_template("403.html", static_url_path=self.static_url_path), 403
+        )
+        return self.add_security_headers(r)
+
+    def error404(self, history_id):
+        self.add_request(
+            self.REQUEST_INDIVIDUAL_FILE_STARTED,
+            request.path,
+            {"id": history_id, "status_code": 404},
+        )
+
+        self.add_request(Web.REQUEST_OTHER, request.path)
+        r = make_response(
+            render_template("404.html", static_url_path=self.static_url_path), 404
+        )
+        return self.add_security_headers(r)
+
+    def error405(self, history_id):
+        self.add_request(
+            self.REQUEST_INDIVIDUAL_FILE_STARTED,
+            request.path,
+            {"id": history_id, "status_code": 405},
+        )
+
+        self.add_request(Web.REQUEST_OTHER, request.path)
+        r = make_response(
+            render_template("405.html", static_url_path=self.static_url_path), 405
+        )
+        return self.add_security_headers(r)
+
+    def add_security_headers(self, r):
+        """
+        Add security headers to a request
+        """
+        for header, value in self.security_headers:
+            r.headers.set(header, value)
+        # Set a CSP header unless in website mode and the user has disabled it
+        if not self.settings.get("website", "disable_csp") or self.mode != "website":
+            r.headers.set(
+                "Content-Security-Policy",
+                "default-src 'self'; style-src 'self'; script-src 'self'; img-src 'self' data:;",
+            )
+        return r
+
+    def _safe_select_jinja_autoescape(self, filename):
+        if filename is None:
+            return True
+        return filename.endswith((".html", ".htm", ".xml", ".xhtml"))
+
+    def add_request(self, request_type, path=None, data=None):
+        """
+        Add a request to the queue, to communicate with the GUI.
+        """
+        self.q.put({"type": request_type, "path": path, "data": data})
+
+    def generate_password(self, saved_password=None):
+        self.common.log("Web", "generate_password", f"saved_password={saved_password}")
+        if saved_password != None and saved_password != "":
+            self.password = saved_password
+            self.common.log(
+                "Web",
+                "generate_password",
+                f'saved_password sent, so password is: "{self.password}"',
+            )
+        else:
+            self.password = self.common.build_password()
+            self.common.log(
+                "Web", "generate_password", f'built random password: "{self.password}"'
+            )
+
+    def verbose_mode(self):
+        """
+        Turn on verbose mode, which will log flask errors to a file.
+        """
+        flask_log_filename = os.path.join(self.common.build_data_dir(), "flask.log")
+        log_handler = logging.FileHandler(flask_log_filename)
+        log_handler.setLevel(logging.WARNING)
+        self.app.logger.addHandler(log_handler)
+
+    def reset_invalid_passwords(self):
+        self.invalid_passwords_count = 0
+        self.invalid_passwords = []
+
+    def force_shutdown(self):
+        """
+        Stop the flask web server, from the context of the flask app.
+        """
+        # Shutdown the flask service
+        try:
+            func = request.environ.get("werkzeug.server.shutdown")
+            if func is None:
+                raise RuntimeError("Not running with the Werkzeug Server")
+            func()
+        except:
+            pass
+        self.running = False
+
+    def start(self, port):
+        """
+        Start the flask web server.
+        """
+        self.common.log("Web", "start", f"port={port}")
+
+        # Make sure the stop_q is empty when starting a new server
+        while not self.stop_q.empty():
+            try:
+                self.stop_q.get(block=False)
+            except queue.Empty:
+                pass
+
+        # In Whonix, listen on 0.0.0.0 instead of 127.0.0.1 (#220)
+        if os.path.exists("/usr/share/anon-ws-base-files/workstation"):
+            host = "0.0.0.0"
+        else:
+            host = "127.0.0.1"
+
+        self.running = True
+        if self.mode == "chat":
+            self.socketio.run(self.app, host=host, port=port)
+        else:
+            self.app.run(host=host, port=port, threaded=True)
+
+    def stop(self, port):
+        """
+        Stop the flask web server by loading /shutdown.
+        """
+        self.common.log("Web", "stop", "stopping server")
+
+        # Let the mode know that the user stopped the server
+        self.stop_q.put(True)
+
+        # To stop flask, load http://shutdown:[shutdown_password]@127.0.0.1/[shutdown_password]/shutdown
+        # (We're putting the shutdown_password in the path as well to make routing simpler)
+        if self.running:
+            if self.password:
+                requests.get(
+                    f"http://127.0.0.1:{port}/{self.shutdown_password}/shutdown",
+                    auth=requests.auth.HTTPBasicAuth("onionshare", self.password),
+                )
+            else:
+                requests.get(
+                    f"http://127.0.0.1:{port}/{self.shutdown_password}/shutdown"
+                )
+
+        # Reset any password that was in use
+        self.password = None

cli/onionshare_cli/web/website_mode.py

@@ -0,0 +1,123 @@
+# -*- coding: utf-8 -*-
+"""
+OnionShare | https://onionshare.org/
+
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import os
+import sys
+import tempfile
+import mimetypes
+from flask import Response, request, render_template, make_response
+
+from .send_base_mode import SendBaseModeWeb
+
+
+class WebsiteModeWeb(SendBaseModeWeb):
+    """
+    All of the web logic for website mode
+    """
+
+    def init(self):
+        pass
+
+    def define_routes(self):
+        """
+        The web app routes for sharing a website
+        """
+
+        @self.web.app.route("/", defaults={"path": ""})
+        @self.web.app.route("/<path:path>")
+        def path_public(path):
+            return path_logic(path)
+
+        def path_logic(path=""):
+            """
+            Render the onionshare website.
+            """
+            return self.render_logic(path)
+
+    def directory_listing_template(
+        self, path, files, dirs, breadcrumbs, breadcrumbs_leaf
+    ):
+        return make_response(
+            render_template(
+                "listing.html",
+                path=path,
+                files=files,
+                dirs=dirs,
+                breadcrumbs=breadcrumbs,
+                breadcrumbs_leaf=breadcrumbs_leaf,
+                static_url_path=self.web.static_url_path,
+            )
+        )
+
+    def set_file_info_custom(self, filenames, processed_size_callback):
+        self.common.log("WebsiteModeWeb", "set_file_info_custom")
+        self.web.cancel_compression = True
+
+    def render_logic(self, path=""):
+        if path in self.files:
+            filesystem_path = self.files[path]
+
+            # If it's a directory
+            if os.path.isdir(filesystem_path):
+                # Is there an index.html?
+                index_path = os.path.join(path, "index.html")
+                if index_path in self.files:
+                    # Render it
+                    return self.stream_individual_file(self.files[index_path])
+
+                else:
+                    # Otherwise, render directory listing
+                    filenames = []
+                    for filename in os.listdir(filesystem_path):
+                        if os.path.isdir(os.path.join(filesystem_path, filename)):
+                            filenames.append(filename + "/")
+                        else:
+                            filenames.append(filename)
+                    filenames.sort()
+                    return self.directory_listing(filenames, path, filesystem_path)
+
+            # If it's a file
+            elif os.path.isfile(filesystem_path):
+                return self.stream_individual_file(filesystem_path)
+
+            # If it's not a directory or file, throw a 404
+            else:
+                history_id = self.cur_history_id
+                self.cur_history_id += 1
+                return self.web.error404(history_id)
+        else:
+            # Special case loading /
+
+            if path == "":
+                index_path = "index.html"
+                if index_path in self.files:
+                    # Render it
+                    return self.stream_individual_file(self.files[index_path])
+                else:
+                    # Root directory listing
+                    filenames = list(self.root_files)
+                    filenames.sort()
+                    return self.directory_listing(filenames, path)
+
+            else:
+                # If the path isn't found, throw a 404
+                history_id = self.cur_history_id
+                self.cur_history_id += 1
+                return self.web.error404(history_id)

cli/poetry.lock

@@ -1,112 +1,78 @@
 [[package]]
-name = "altgraph"
-version = "0.17"
-description = "Python graph (network) package"
 category = "dev"
-optional = false
-python-versions = "*"
-
-[[package]]
-name = "appdirs"
-version = "1.4.4"
-description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
-category = "dev"
-optional = false
-python-versions = "*"
-
-[[package]]
-name = "atomicwrites"
-version = "1.4.0"
 description = "Atomic file writes."
-category = "dev"
+marker = "sys_platform == \"win32\""
+name = "atomicwrites"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "1.4.0"
 
 [[package]]
-name = "attrs"
+category = "dev"
-version = "20.2.0"
 description = "Classes Without Boilerplate"
-category = "dev"
+name = "attrs"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "20.2.0"
 
 [package.extras]
-dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "zope.interface", "sphinx", "sphinx-rtd-theme", "pre-commit"]
+dev = ["coverage (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "zope.interface", "sphinx", "sphinx-rtd-theme", "pre-commit"]
 docs = ["sphinx", "sphinx-rtd-theme", "zope.interface"]
-tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "zope.interface"]
+tests = ["coverage (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "zope.interface"]
-tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six"]
+tests_no_zope = ["coverage (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six"]
 
 [[package]]
-name = "black"
+category = "main"
-version = "19.10b0"
-description = "The uncompromising code formatter."
-category = "dev"
-optional = false
-python-versions = ">=3.6"
-
-[package.dependencies]
-appdirs = "*"
-attrs = ">=18.1.0"
-click = ">=6.5"
-pathspec = ">=0.6,<1"
-regex = "*"
-toml = ">=0.9.4"
-typed-ast = ">=1.4.0"
-
-[package.extras]
-d = ["aiohttp (>=3.3.2)", "aiohttp-cors"]
-
-[[package]]
-name = "certifi"
-version = "2020.6.20"
 description = "Python package for providing Mozilla's CA Bundle."
-category = "main"
+name = "certifi"
 optional = false
 python-versions = "*"
+version = "2020.6.20"
 
 [[package]]
-name = "chardet"
+category = "main"
-version = "3.0.4"
 description = "Universal encoding detector for Python 2 and 3"
-category = "main"
+name = "chardet"
 optional = false
 python-versions = "*"
+version = "3.0.4"
 
 [[package]]
-name = "click"
+category = "main"
-version = "7.1.2"
 description = "Composable command line interface toolkit"
-category = "main"
+name = "click"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+version = "7.1.2"
 
 [[package]]
-name = "colorama"
+category = "dev"
-version = "0.4.3"
 description = "Cross-platform colored terminal text."
-category = "main"
+marker = "sys_platform == \"win32\""
+name = "colorama"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+version = "0.4.4"
 
 [[package]]
-name = "dnspython"
-version = "1.16.0"
-description = "DNS toolkit"
 category = "main"
+description = "DNS toolkit"
+name = "dnspython"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "1.16.0"
 
 [package.extras]
 DNSSEC = ["pycryptodome", "ecdsa (>=0.13)"]
 IDNA = ["idna (>=2.1)"]
 
 [[package]]
-name = "eventlet"
-version = "0.28.0"
-description = "Highly concurrent networking library"
 category = "main"
+description = "Highly concurrent networking library"
+name = "eventlet"
 optional = false
 python-versions = "*"
+version = "0.28.0"
 
 [package.dependencies]
 dnspython = ">=1.15.0,<2.0.0"
@@ -115,18 +81,18 @@ monotonic = ">=1.4"
 six = ">=1.10.0"
 
 [[package]]
-name = "flask"
-version = "1.1.2"
-description = "A simple framework for building complex web applications."
 category = "main"
+description = "A simple framework for building complex web applications."
+name = "flask"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+version = "1.1.2"
 
 [package.dependencies]
-click = ">=5.1"
-itsdangerous = ">=0.24"
 Jinja2 = ">=2.10.1"
 Werkzeug = ">=0.15"
+click = ">=5.1"
+itsdangerous = ">=0.24"
 
 [package.extras]
 dev = ["pytest", "coverage", "tox", "sphinx", "pallets-sphinx-themes", "sphinxcontrib-log-cabinet", "sphinx-issues"]
@@ -134,51 +100,52 @@ docs = ["sphinx", "pallets-sphinx-themes", "sphinxcontrib-log-cabinet", "sphinx-
 dotenv = ["python-dotenv"]
 
 [[package]]
-name = "flask-httpauth"
-version = "4.1.0"
-description = "Basic and Digest HTTP authentication for Flask routes"
 category = "main"
+description = "Basic and Digest HTTP authentication for Flask routes"
+name = "flask-httpauth"
 optional = false
 python-versions = "*"
+version = "4.1.0"
 
 [package.dependencies]
 Flask = "*"
 
 [[package]]
-name = "flask-socketio"
-version = "4.3.1"
-description = "Socket.IO integration for Flask applications"
 category = "main"
+description = "Socket.IO integration for Flask applications"
+name = "flask-socketio"
 optional = false
 python-versions = "*"
+version = "4.3.1"
 
 [package.dependencies]
 Flask = ">=0.9"
 python-socketio = ">=4.3.0"
 
 [[package]]
-name = "greenlet"
-version = "0.4.17"
-description = "Lightweight in-process concurrent programming"
 category = "main"
+description = "Lightweight in-process concurrent programming"
+name = "greenlet"
 optional = false
 python-versions = "*"
+version = "0.4.17"
 
 [[package]]
-name = "idna"
-version = "2.10"
-description = "Internationalized Domain Names in Applications (IDNA)"
 category = "main"
+description = "Internationalized Domain Names in Applications (IDNA)"
+name = "idna"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "2.10"
 
 [[package]]
-name = "importlib-metadata"
-version = "2.0.0"
-description = "Read metadata from Python packages"
 category = "dev"
+description = "Read metadata from Python packages"
+marker = "python_version < \"3.8\""
+name = "importlib-metadata"
 optional = false
 python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7"
+version = "2.0.0"
 
 [package.dependencies]
 zipp = ">=0.5"
@@ -188,28 +155,28 @@ docs = ["sphinx", "rst.linker"]
 testing = ["packaging", "pep517", "importlib-resources (>=1.3)"]
 
 [[package]]
-name = "iniconfig"
-version = "1.0.1"
-description = "iniconfig: brain-dead simple config-ini parsing"
 category = "dev"
+description = "iniconfig: brain-dead simple config-ini parsing"
+name = "iniconfig"
 optional = false
 python-versions = "*"
+version = "1.0.1"
 
 [[package]]
-name = "itsdangerous"
-version = "1.1.0"
-description = "Various helpers to pass data to untrusted environments and back."
 category = "main"
+description = "Various helpers to pass data to untrusted environments and back."
+name = "itsdangerous"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "1.1.0"
 
 [[package]]
-name = "jinja2"
-version = "2.11.2"
-description = "A very fast and expressive template engine."
 category = "main"
+description = "A very fast and expressive template engine."
+name = "jinja2"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+version = "2.11.2"
 
 [package.dependencies]
 MarkupSafe = ">=0.23"
@@ -218,217 +185,114 @@ MarkupSafe = ">=0.23"
 i18n = ["Babel (>=0.8)"]
 
 [[package]]
-name = "macholib"
-version = "1.14"
-description = "Mach-O header analysis and editing"
-category = "dev"
-optional = false
-python-versions = "*"
-
-[package.dependencies]
-altgraph = ">=0.15"
-
-[[package]]
-name = "markupsafe"
-version = "1.1.1"
-description = "Safely add untrusted strings to HTML/XML markup."
 category = "main"
+description = "Safely add untrusted strings to HTML/XML markup."
+name = "markupsafe"
 optional = false
 python-versions = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*"
+version = "1.1.1"
 
 [[package]]
-name = "monotonic"
-version = "1.5"
-description = "An implementation of time.monotonic() for Python 2 & < 3.3"
 category = "main"
+description = "An implementation of time.monotonic() for Python 2 & < 3.3"
+name = "monotonic"
 optional = false
 python-versions = "*"
+version = "1.5"
 
 [[package]]
-name = "more-itertools"
-version = "8.5.0"
-description = "More routines for operating on iterables, beyond itertools"
 category = "dev"
-optional = false
-python-versions = ">=3.5"
-
-[[package]]
-name = "packaging"
-version = "20.4"
 description = "Core utilities for Python packages"
-category = "dev"
+name = "packaging"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "20.4"
 
 [package.dependencies]
 pyparsing = ">=2.0.2"
 six = "*"
 
 [[package]]
-name = "pathspec"
-version = "0.8.0"
-description = "Utility library for gitignore style pattern matching of file paths."
 category = "dev"
-optional = false
-python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
-
-[[package]]
-name = "pluggy"
-version = "0.13.1"
 description = "plugin and hook calling mechanisms for python"
-category = "dev"
+name = "pluggy"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "0.13.1"
 
 [package.dependencies]
-importlib-metadata = {version = ">=0.12", markers = "python_version < \"3.8\""}
+[package.dependencies.importlib-metadata]
+python = "<3.8"
+version = ">=0.12"
 
 [package.extras]
 dev = ["pre-commit", "tox"]
 
 [[package]]
-name = "psutil"
-version = "5.7.2"
-description = "Cross-platform lib for process and system monitoring in Python."
-category = "main"
-optional = false
-python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
-
-[package.extras]
-test = ["ipaddress", "mock", "unittest2", "enum34", "pywin32", "wmi"]
-
-[[package]]
-name = "py"
-version = "1.9.0"
-description = "library with cross-python path, ini-parsing, io, code, log facilities"
 category = "dev"
+description = "library with cross-python path, ini-parsing, io, code, log facilities"
+name = "py"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "1.9.0"
 
 [[package]]
-name = "pycryptodome"
-version = "3.9.8"
-description = "Cryptographic library for Python"
 category = "main"
+description = "Cryptographic library for Python"
+name = "pycryptodome"
 optional = false
 python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "3.9.8"
 
 [[package]]
-name = "pyinstaller"
-version = "4.0"
-description = "PyInstaller bundles a Python application and all its dependencies into a single package."
 category = "dev"
-optional = false
-python-versions = "*"
-
-[package.dependencies]
-altgraph = "*"
-macholib = {version = ">=1.8", markers = "sys_platform == \"darwin\""}
-pyinstaller-hooks-contrib = ">=2020.6"
-
-[package.extras]
-encryption = ["tinyaes (>=1.0.0)"]
-hook_testing = ["pytest (>=2.7.3)", "execnet (>=1.5.0)", "psutil"]
-
-[[package]]
-name = "pyinstaller-hooks-contrib"
-version = "2020.9"
-description = "Community maintained hooks for PyInstaller"
-category = "dev"
-optional = false
-python-versions = "*"
-
-[[package]]
-name = "pyparsing"
-version = "2.4.7"
 description = "Python parsing module"
-category = "dev"
+name = "pyparsing"
 optional = false
 python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*"
+version = "2.4.7"
 
 [[package]]
-name = "pyqt5"
-version = "5.14.0"
-description = "Python bindings for the Qt cross platform application toolkit"
 category = "main"
-optional = false
-python-versions = ">=3.5"
-
-[package.dependencies]
-PyQt5-sip = ">=12.7,<13"
-
-[[package]]
-name = "pyqt5-sip"
-version = "12.8.1"
-description = "The sip module support for PyQt5"
-category = "main"
-optional = false
-python-versions = ">=3.5"
-
-[[package]]
-name = "pysocks"
-version = "1.7.1"
 description = "A Python SOCKS client module. See https://github.com/Anorov/PySocks for more information."
-category = "main"
+name = "pysocks"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+version = "1.7.1"
 
 [[package]]
-name = "pytest"
-version = "6.1.1"
-description = "pytest: simple powerful testing with Python"
 category = "dev"
+description = "pytest: simple powerful testing with Python"
+name = "pytest"
 optional = false
 python-versions = ">=3.5"
+version = "6.1.1"
 
 [package.dependencies]
-atomicwrites = {version = ">=1.0", markers = "sys_platform == \"win32\""}
+atomicwrites = ">=1.0"
 attrs = ">=17.4.0"
-colorama = {version = "*", markers = "sys_platform == \"win32\""}
+colorama = "*"
-importlib-metadata = {version = ">=0.12", markers = "python_version < \"3.8\""}
 iniconfig = "*"
 packaging = "*"
 pluggy = ">=0.12,<1.0"
 py = ">=1.8.2"
 toml = "*"
 
+[package.dependencies.importlib-metadata]
+python = "<3.8"
+version = ">=0.12"
+
 [package.extras]
 checkqa_mypy = ["mypy (0.780)"]
 testing = ["argcomplete", "hypothesis (>=3.56)", "mock", "nose", "requests", "xmlschema"]
 
 [[package]]
-name = "pytest-faulthandler"
-version = "2.0.1"
-description = "py.test plugin that activates the fault handler module for tests (dummy package)"
-category = "dev"
-optional = false
-python-versions = "*"
-
-[package.dependencies]
-pytest = ">=5.0"
-
-[[package]]
-name = "pytest-qt"
-version = "3.3.0"
-description = "pytest support for PyQt and PySide applications"
-category = "dev"
-optional = false
-python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
-
-[package.dependencies]
-pytest = ">=3.0.0"
-
-[package.extras]
-dev = ["pre-commit", "tox"]
-doc = ["sphinx", "sphinx-rtd-theme"]
-
-[[package]]
-name = "python-engineio"
-version = "3.13.2"
-description = "Engine.IO server"
 category = "main"
+description = "Engine.IO server"
+name = "python-engineio"
 optional = false
 python-versions = "*"
+version = "3.13.2"
 
 [package.dependencies]
 six = ">=1.9.0"
@@ -438,12 +302,12 @@ asyncio_client = ["aiohttp (>=3.4)"]
 client = ["requests (>=2.21.0)", "websocket-client (>=0.54.0)"]
 
 [[package]]
-name = "python-socketio"
-version = "4.6.0"
-description = "Socket.IO server"
 category = "main"
+description = "Socket.IO server"
+name = "python-socketio"
 optional = false
 python-versions = "*"
+version = "4.6.0"
 
 [package.dependencies]
 python-engineio = ">=3.13.0"
@@ -454,46 +318,12 @@ asyncio_client = ["aiohttp (>=3.4)", "websockets (>=7.0)"]
 client = ["requests (>=2.21.0)", "websocket-client (>=0.54.0)"]
 
 [[package]]
-name = "pywin32"
-version = "228"
-description = "Python for Window Extensions"
 category = "main"
-optional = false
-python-versions = "*"
-
-[[package]]
-name = "qrcode"
-version = "6.1"
-description = "QR Code image generator"
-category = "main"
-optional = false
-python-versions = "*"
-
-[package.dependencies]
-colorama = {version = "*", markers = "platform_system == \"Windows\""}
-six = "*"
-
-[package.extras]
-dev = ["tox", "pytest", "mock"]
-maintainer = ["zest.releaser"]
-pil = ["pillow"]
-test = ["pytest", "pytest-cov", "mock"]
-
-[[package]]
-name = "regex"
-version = "2020.10.11"
-description = "Alternative regular expression module, to replace re."
-category = "dev"
-optional = false
-python-versions = "*"
-
-[[package]]
-name = "requests"
-version = "2.24.0"
 description = "Python HTTP for Humans."
-category = "main"
+name = "requests"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+version = "2.24.0"
 
 [package.dependencies]
 certifi = ">=2017.4.17"
@@ -506,44 +336,36 @@ security = ["pyOpenSSL (>=0.14)", "cryptography (>=1.3.4)"]
 socks = ["PySocks (>=1.5.6,<1.5.7 || >1.5.7)", "win-inet-pton"]
 
 [[package]]
-name = "six"
-version = "1.15.0"
-description = "Python 2 and 3 compatibility utilities"
 category = "main"
+description = "Python 2 and 3 compatibility utilities"
+name = "six"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*"
+version = "1.15.0"
 
 [[package]]
-name = "stem"
+category = "main"
-version = "1.8.0"
 description = "Stem is a Python controller library that allows applications to interact with Tor (https://www.torproject.org/)."
-category = "main"
+name = "stem"
 optional = false
 python-versions = "*"
+version = "1.8.0"
 
 [[package]]
-name = "toml"
+category = "dev"
-version = "0.10.1"
 description = "Python Library for Tom's Obvious, Minimal Language"
-category = "dev"
+name = "toml"
 optional = false
 python-versions = "*"
+version = "0.10.1"
 
 [[package]]
-name = "typed-ast"
-version = "1.4.1"
-description = "a fork of Python 2 and 3 ast modules with type comment support"
-category = "dev"
-optional = false
-python-versions = "*"
-
-[[package]]
-name = "urllib3"
-version = "1.25.10"
-description = "HTTP library with thread-safe connection pooling, file post, and more."
 category = "main"
+description = "HTTP library with thread-safe connection pooling, file post, and more."
+name = "urllib3"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4"
+version = "1.25.10"
 
 [package.extras]
 brotli = ["brotlipy (>=0.6.0)"]
@@ -551,43 +373,35 @@ secure = ["certifi", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "pyOpenSSL (>=0
 socks = ["PySocks (>=1.5.6,<1.5.7 || >1.5.7,<2.0)"]
 
 [[package]]
-name = "werkzeug"
-version = "1.0.1"
-description = "The comprehensive WSGI web application library."
 category = "main"
+description = "The comprehensive WSGI web application library."
+name = "werkzeug"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+version = "1.0.1"
 
 [package.extras]
 dev = ["pytest", "pytest-timeout", "coverage", "tox", "sphinx", "pallets-sphinx-themes", "sphinx-issues"]
 watchdog = ["watchdog"]
 
 [[package]]
-name = "zipp"
-version = "3.3.0"
-description = "Backport of pathlib-compatible object wrapper for zip files"
 category = "dev"
+description = "Backport of pathlib-compatible object wrapper for zip files"
+marker = "python_version < \"3.8\""
+name = "zipp"
 optional = false
 python-versions = ">=3.6"
+version = "3.3.0"
 
 [package.extras]
 docs = ["sphinx", "jaraco.packaging (>=3.2)", "rst.linker (>=1.9)"]
 testing = ["pytest (>=3.5,<3.7.3 || >3.7.3)", "pytest-checkdocs (>=1.2.3)", "pytest-flake8", "pytest-cov", "jaraco.test (>=3.2.0)", "jaraco.itertools", "func-timeout", "pytest-black (>=0.3.7)", "pytest-mypy"]
 
 [metadata]
-lock-version = "1.1"
+content-hash = "5c1dfb397d3520827e3fd1a0b53903a87ba486750758ca532fb7c13f9eab0d35"
 python-versions = "^3.7"
-content-hash = "5cd6d77db75f2f433392e4a2f166cf14236fd24e2a7b59a5863937ab8b7faf80"
 
 [metadata.files]
-altgraph = [
-    {file = "altgraph-0.17-py2.py3-none-any.whl", hash = "sha256:c623e5f3408ca61d4016f23a681b9adb100802ca3e3da5e718915a9e4052cebe"},
-    {file = "altgraph-0.17.tar.gz", hash = "sha256:1f05a47122542f97028caf78775a095fbe6a2699b5089de8477eb583167d69aa"},
-]
-appdirs = [
-    {file = "appdirs-1.4.4-py2.py3-none-any.whl", hash = "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"},
-    {file = "appdirs-1.4.4.tar.gz", hash = "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41"},
-]
 atomicwrites = [
     {file = "atomicwrites-1.4.0-py2.py3-none-any.whl", hash = "sha256:6d1784dea7c0c8d4a5172b6c620f40b6e4cbfdf96d783691f2e1302a7b88e197"},
     {file = "atomicwrites-1.4.0.tar.gz", hash = "sha256:ae70396ad1a434f9c7046fd2dd196fc04b12f9e91ffb859164193be8b6168a7a"},
@@ -596,10 +410,6 @@ attrs = [
     {file = "attrs-20.2.0-py2.py3-none-any.whl", hash = "sha256:fce7fc47dfc976152e82d53ff92fa0407700c21acd20886a13777a0d20e655dc"},
     {file = "attrs-20.2.0.tar.gz", hash = "sha256:26b54ddbbb9ee1d34d5d3668dd37d6cf74990ab23c828c2888dccdceee395594"},
 ]
-black = [
-    {file = "black-19.10b0-py36-none-any.whl", hash = "sha256:1b30e59be925fafc1ee4565e5e08abef6b03fe455102883820fe5ee2e4734e0b"},
-    {file = "black-19.10b0.tar.gz", hash = "sha256:c2edb73a08e9e0e6f65a0e6af18b059b8b1cdd5bef997d7a0b181df93dc81539"},
-]
 certifi = [
     {file = "certifi-2020.6.20-py2.py3-none-any.whl", hash = "sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41"},
     {file = "certifi-2020.6.20.tar.gz", hash = "sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3"},
@@ -613,8 +423,7 @@ click = [
     {file = "click-7.1.2.tar.gz", hash = "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a"},
 ]
 colorama = [
-    {file = "colorama-0.4.3-py2.py3-none-any.whl", hash = "sha256:7d73d2a99753107a36ac6b455ee49046802e59d9d076ef8e47b61499fa29afff"},
+    {file = "colorama-0.4.4-py2.py3-none-any.whl", hash = "sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"},
-    {file = "colorama-0.4.3.tar.gz", hash = "sha256:e96da0d330793e2cb9485e9ddfd918d456036c7149416295932478192f4436a1"},
 ]
 dnspython = [
     {file = "dnspython-1.16.0-py2.py3-none-any.whl", hash = "sha256:f69c21288a962f4da86e56c4905b49d11aba7938d3d740e80d9e366ee4f1632d"},
@@ -676,10 +485,6 @@ jinja2 = [
     {file = "Jinja2-2.11.2-py2.py3-none-any.whl", hash = "sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"},
     {file = "Jinja2-2.11.2.tar.gz", hash = "sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0"},
 ]
-macholib = [
-    {file = "macholib-1.14-py2.py3-none-any.whl", hash = "sha256:c500f02867515e6c60a27875b408920d18332ddf96b4035ef03beddd782d4281"},
-    {file = "macholib-1.14.tar.gz", hash = "sha256:0c436bc847e7b1d9bda0560351bf76d7caf930fb585a828d13608839ef42c432"},
-]
 markupsafe = [
     {file = "MarkupSafe-1.1.1-cp27-cp27m-macosx_10_6_intel.whl", hash = "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161"},
     {file = "MarkupSafe-1.1.1-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7"},
@@ -719,35 +524,14 @@ monotonic = [
     {file = "monotonic-1.5-py2.py3-none-any.whl", hash = "sha256:552a91f381532e33cbd07c6a2655a21908088962bb8fa7239ecbcc6ad1140cc7"},
     {file = "monotonic-1.5.tar.gz", hash = "sha256:23953d55076df038541e648a53676fb24980f7a1be290cdda21300b3bc21dfb0"},
 ]
-more-itertools = [
-    {file = "more-itertools-8.5.0.tar.gz", hash = "sha256:6f83822ae94818eae2612063a5101a7311e68ae8002005b5e05f03fd74a86a20"},
-    {file = "more_itertools-8.5.0-py3-none-any.whl", hash = "sha256:9b30f12df9393f0d28af9210ff8efe48d10c94f73e5daf886f10c4b0b0b4f03c"},
-]
 packaging = [
     {file = "packaging-20.4-py2.py3-none-any.whl", hash = "sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181"},
     {file = "packaging-20.4.tar.gz", hash = "sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8"},
 ]
-pathspec = [
-    {file = "pathspec-0.8.0-py2.py3-none-any.whl", hash = "sha256:7d91249d21749788d07a2d0f94147accd8f845507400749ea19c1ec9054a12b0"},
-    {file = "pathspec-0.8.0.tar.gz", hash = "sha256:da45173eb3a6f2a5a487efba21f050af2b41948be6ab52b6a1e3ff22bb8b7061"},
-]
 pluggy = [
     {file = "pluggy-0.13.1-py2.py3-none-any.whl", hash = "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"},
     {file = "pluggy-0.13.1.tar.gz", hash = "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0"},
 ]
-psutil = [
-    {file = "psutil-5.7.2-cp27-none-win32.whl", hash = "sha256:f2018461733b23f308c298653c8903d32aaad7873d25e1d228765e91ae42c3f2"},
-    {file = "psutil-5.7.2-cp27-none-win_amd64.whl", hash = "sha256:66c18ca7680a31bf16ee22b1d21b6397869dda8059dbdb57d9f27efa6615f195"},
-    {file = "psutil-5.7.2-cp35-cp35m-win32.whl", hash = "sha256:5e9d0f26d4194479a13d5f4b3798260c20cecf9ac9a461e718eb59ea520a360c"},
-    {file = "psutil-5.7.2-cp35-cp35m-win_amd64.whl", hash = "sha256:4080869ed93cce662905b029a1770fe89c98787e543fa7347f075ade761b19d6"},
-    {file = "psutil-5.7.2-cp36-cp36m-win32.whl", hash = "sha256:d8a82162f23c53b8525cf5f14a355f5d1eea86fa8edde27287dd3a98399e4fdf"},
-    {file = "psutil-5.7.2-cp36-cp36m-win_amd64.whl", hash = "sha256:0ee3c36428f160d2d8fce3c583a0353e848abb7de9732c50cf3356dd49ad63f8"},
-    {file = "psutil-5.7.2-cp37-cp37m-win32.whl", hash = "sha256:ff1977ba1a5f71f89166d5145c3da1cea89a0fdb044075a12c720ee9123ec818"},
-    {file = "psutil-5.7.2-cp37-cp37m-win_amd64.whl", hash = "sha256:a5b120bb3c0c71dfe27551f9da2f3209a8257a178ed6c628a819037a8df487f1"},
-    {file = "psutil-5.7.2-cp38-cp38-win32.whl", hash = "sha256:10512b46c95b02842c225f58fa00385c08fa00c68bac7da2d9a58ebe2c517498"},
-    {file = "psutil-5.7.2-cp38-cp38-win_amd64.whl", hash = "sha256:68d36986ded5dac7c2dcd42f2682af1db80d4bce3faa126a6145c1637e1b559f"},
-    {file = "psutil-5.7.2.tar.gz", hash = "sha256:90990af1c3c67195c44c9a889184f84f5b2320dce3ee3acbd054e3ba0b4a7beb"},
-]
 py = [
     {file = "py-1.9.0-py2.py3-none-any.whl", hash = "sha256:366389d1db726cd2fcfc79732e75410e5fe4d31db13692115529d34069a043c2"},
     {file = "py-1.9.0.tar.gz", hash = "sha256:9ca6883ce56b4e8da7e79ac18787889fa5206c79dcc67fb065376cd2fe03f342"},
@@ -763,73 +547,31 @@ pycryptodome = [
     {file = "pycryptodome-3.9.8-cp35-cp35m-macosx_10_6_intel.whl", hash = "sha256:6276478ada411aca97c0d5104916354b3d740d368407912722bd4d11aa9ee4c2"},
     {file = "pycryptodome-3.9.8-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:54bdedd28476dea8a3cd86cb67c0df1f0e3d71cae8022354b0f879c41a3d27b2"},
     {file = "pycryptodome-3.9.8-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:f521178e5a991ffd04182ed08f552daca1affcb826aeda0e1945cd989a9d4345"},
-    {file = "pycryptodome-3.9.8-cp35-cp35m-manylinux2014_aarch64.whl", hash = "sha256:f2e045224074d5664dc9cbabbf4f4d4d46f1ee90f24780e3a9a668fd096ff17f"},
     {file = "pycryptodome-3.9.8-cp35-cp35m-win32.whl", hash = "sha256:a207231a52426de3ff20f5608f0687261a3329d97a036c51f7d4c606a6f30c23"},
     {file = "pycryptodome-3.9.8-cp35-cp35m-win_amd64.whl", hash = "sha256:2b998dc45ef5f4e5cf5248a6edfcd8d8e9fb5e35df8e4259b13a1b10eda7b16b"},
     {file = "pycryptodome-3.9.8-cp36-cp36m-macosx_10_6_intel.whl", hash = "sha256:03d5cca8618620f45fd40f827423f82b86b3a202c8d44108601b0f5f56b04299"},
     {file = "pycryptodome-3.9.8-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:f78a68c2c820e4731e510a2df3eef0322f24fde1781ced970bf497b6c7d92982"},
     {file = "pycryptodome-3.9.8-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:132a56abba24e2e06a479d8e5db7a48271a73a215f605017bbd476d31f8e71c1"},
-    {file = "pycryptodome-3.9.8-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:cecbf67e81d6144a50dc615629772859463b2e4f815d0c082fa421db362f040e"},
     {file = "pycryptodome-3.9.8-cp36-cp36m-win32.whl", hash = "sha256:67dcad1b8b201308586a8ca2ffe89df1e4f731d5a4cdd0610cc4ea790351c739"},
     {file = "pycryptodome-3.9.8-cp36-cp36m-win_amd64.whl", hash = "sha256:b56638d58a3a4be13229c6a815cd448f9e3ce40c00880a5398471b42ee86f50e"},
     {file = "pycryptodome-3.9.8-cp37-cp37m-macosx_10_6_intel.whl", hash = "sha256:bec2bcdf7c9ce7f04d718e51887f3b05dc5c1cfaf5d2c2e9065ecddd1b2f6c9a"},
     {file = "pycryptodome-3.9.8-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:abc2e126c9490e58a36a0f83516479e781d83adfb134576a5cbe5c6af2a3e93c"},
     {file = "pycryptodome-3.9.8-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:ef39c98d9b8c0736d91937d193653e47c3b19ddf4fc3bccdc5e09aaa4b0c5d21"},
-    {file = "pycryptodome-3.9.8-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:87006cf0d81505408f1ae4f55cf8a5d95a8e029a4793360720ae17c6500f7ecc"},
     {file = "pycryptodome-3.9.8-cp37-cp37m-win32.whl", hash = "sha256:4350a42028240c344ee855f032c7d4ad6ff4f813bfbe7121547b7dc579ecc876"},
     {file = "pycryptodome-3.9.8-cp37-cp37m-win_amd64.whl", hash = "sha256:c8bf40cf6e281a4378e25846924327e728a887e8bf0ee83b2604a0f4b61692e8"},
     {file = "pycryptodome-3.9.8-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:d8074c8448cfd0705dfa71ca333277fce9786d0b9cac75d120545de6253f996a"},
     {file = "pycryptodome-3.9.8-cp38-cp38-manylinux1_i686.whl", hash = "sha256:8063a712fba642f78d3c506b0896846601b6de7f5c3d534e388ad0cc07f5a149"},
     {file = "pycryptodome-3.9.8-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:dd302b6ae3965afeb5ef1b0d92486f986c0e65183cd7835973f0b593800590e6"},
-    {file = "pycryptodome-3.9.8-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:663f8de2b3df2e744d6e1610506e0ea4e213bde906795953c1e82279c169f0a7"},
     {file = "pycryptodome-3.9.8-cp38-cp38-win32.whl", hash = "sha256:02e51e1d5828d58f154896ddfd003e2e7584869c275e5acbe290443575370fba"},
     {file = "pycryptodome-3.9.8-cp38-cp38-win_amd64.whl", hash = "sha256:55eb61aca2c883db770999f50d091ff7c14016f2769ad7bca3d9b75d1d7c1b68"},
     {file = "pycryptodome-3.9.8-cp39-cp39-manylinux1_i686.whl", hash = "sha256:39ef9fb52d6ec7728fce1f1693cb99d60ce302aeebd59bcedea70ca3203fda60"},
     {file = "pycryptodome-3.9.8-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:de6e1cd75677423ff64712c337521e62e3a7a4fc84caabbd93207752e831a85a"},
-    {file = "pycryptodome-3.9.8-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:9f62d21bc693f3d7d444f17ed2ad7a913b4c37c15cd807895d013c39c0517dfd"},
     {file = "pycryptodome-3.9.8.tar.gz", hash = "sha256:0e24171cf01021bc5dc17d6a9d4f33a048f09d62cc3f62541e95ef104588bda4"},
 ]
-pyinstaller = [
-    {file = "pyinstaller-4.0.tar.gz", hash = "sha256:970beb07115761d5e4ec317c1351b712fd90ae7f23994db914c633281f99bab0"},
-]
-pyinstaller-hooks-contrib = [
-    {file = "pyinstaller-hooks-contrib-2020.9.tar.gz", hash = "sha256:a5fd45a920012802e3f2089e1d3501ef2f49265dfea8fc46c3310f18e3326c91"},
-    {file = "pyinstaller_hooks_contrib-2020.9-py2.py3-none-any.whl", hash = "sha256:c382f3ac1a42b45cfecd581475c36db77da90e479b2f5bcb6d840d21fa545114"},
-]
 pyparsing = [
     {file = "pyparsing-2.4.7-py2.py3-none-any.whl", hash = "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"},
     {file = "pyparsing-2.4.7.tar.gz", hash = "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1"},
 ]
-pyqt5 = [
-    {file = "PyQt5-5.14.0-5.14.0-cp35.cp36.cp37.cp38-abi3-macosx_10_6_intel.whl", hash = "sha256:895d4101f7f8c82bc728d7eb9da1c756955ce27a0c945eafe7f234dd03402853"},
-    {file = "PyQt5-5.14.0-5.14.0-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl", hash = "sha256:a757ba71c51f428b52ba404e781e2f19b4436b2c31298b8313339d5817781b65"},
-    {file = "PyQt5-5.14.0-5.14.0-cp35.cp36.cp37.cp38-none-win32.whl", hash = "sha256:cc3529c0f7cbbe7491073458d5d15e7518ce544ad8c627f485e5db8a27fcaf61"},
-    {file = "PyQt5-5.14.0-5.14.0-cp35.cp36.cp37.cp38-none-win_amd64.whl", hash = "sha256:0dcc128b72f83cce0fc7926c83f05a9b74b652b5eb31a4ab71693ac8829e73c8"},
-    {file = "PyQt5-5.14.0.tar.gz", hash = "sha256:0145a6b7de15756366decb736c349a0cb510d706c83fda5b8cd9e0557bc1da72"},
-]
-pyqt5-sip = [
-    {file = "PyQt5_sip-12.8.1-cp35-cp35m-macosx_10_6_intel.whl", hash = "sha256:bb5a87b66fc1445915104ee97f7a20a69decb42f52803e3b0795fa17ff88226c"},
-    {file = "PyQt5_sip-12.8.1-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:a29e2ac399429d3b7738f73e9081e50783e61ac5d29344e0802d0dcd6056c5a2"},
-    {file = "PyQt5_sip-12.8.1-cp35-cp35m-win32.whl", hash = "sha256:0304ca9114b9817a270f67f421355075b78ff9fc25ac58ffd72c2601109d2194"},
-    {file = "PyQt5_sip-12.8.1-cp35-cp35m-win_amd64.whl", hash = "sha256:84ba7746762bd223bed22428e8561aa267a229c28344c2d28c5d5d3f8970cffb"},
-    {file = "PyQt5_sip-12.8.1-cp36-cp36m-macosx_10_6_intel.whl", hash = "sha256:7b81382ce188d63890a0e35abe0f9bb946cabc873a31873b73583b0fc84ac115"},
-    {file = "PyQt5_sip-12.8.1-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:b6d42250baec52a5f77de64e2951d001c5501c3a2df2179f625b241cbaec3369"},
-    {file = "PyQt5_sip-12.8.1-cp36-cp36m-win32.whl", hash = "sha256:6c1ebee60f1d2b3c70aff866b7933d8d8d7646011f7c32f9321ee88c290aa4f9"},
-    {file = "PyQt5_sip-12.8.1-cp36-cp36m-win_amd64.whl", hash = "sha256:34dcd29be47553d5f016ff86e89e24cbc5eebae92eb2f96fb32d2d7ba028c43c"},
-    {file = "PyQt5_sip-12.8.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:ed897c58acf4a3cdca61469daa31fe6e44c33c6c06a37c3f21fab31780b3b86a"},
-    {file = "PyQt5_sip-12.8.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:a1b8ef013086e224b8e86c93f880f776d01b59195bdfa2a8e0b23f0480678fec"},
-    {file = "PyQt5_sip-12.8.1-cp37-cp37m-win32.whl", hash = "sha256:0cd969be528c27bbd4755bd323dff4a79a8fdda28215364e6ce3e069cb56c2a9"},
-    {file = "PyQt5_sip-12.8.1-cp37-cp37m-win_amd64.whl", hash = "sha256:c9800729badcb247765e4ffe2241549d02da1fa435b9db224845bc37c3e99cb0"},
-    {file = "PyQt5_sip-12.8.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:9312ec47cac4e33c11503bc1cbeeb0bdae619620472f38e2078c5a51020a930f"},
-    {file = "PyQt5_sip-12.8.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:2f35e82fd7ec1e1f6716e9154721c7594956a4f5bd4f826d8c6a6453833cc2f0"},
-    {file = "PyQt5_sip-12.8.1-cp38-cp38-win32.whl", hash = "sha256:da9c9f1e65b9d09e73bd75befc82961b6b61b5a3b9d0a7c832168e1415f163c6"},
-    {file = "PyQt5_sip-12.8.1-cp38-cp38-win_amd64.whl", hash = "sha256:832fd60a264de4134c2824d393320838f3ab648180c9c357ec58a74524d24507"},
-    {file = "PyQt5_sip-12.8.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:c317ab1263e6417c498b81f5c970a9b1af7acefab1f80b4cc0f2f8e661f29fc5"},
-    {file = "PyQt5_sip-12.8.1-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:c9d6d448c29dc6606bb7974696608f81f4316c8234f7c7216396ed110075e777"},
-    {file = "PyQt5_sip-12.8.1-cp39-cp39-win32.whl", hash = "sha256:5a011aeff89660622a6d5c3388d55a9d76932f3b82c95e82fc31abd8b1d2990d"},
-    {file = "PyQt5_sip-12.8.1-cp39-cp39-win_amd64.whl", hash = "sha256:f168f0a7f32b81bfeffdf003c36f25d81c97dee5eb67072a5183e761fe250f13"},
-    {file = "PyQt5_sip-12.8.1.tar.gz", hash = "sha256:30e944db9abee9cc757aea16906d4198129558533eb7fadbe48c5da2bd18e0bd"},
-]
 pysocks = [
     {file = "PySocks-1.7.1-py27-none-any.whl", hash = "sha256:08e69f092cc6dbe92a0fdd16eeb9b9ffbc13cadfe5ca4c7bd92ffb078b293299"},
     {file = "PySocks-1.7.1-py3-none-any.whl", hash = "sha256:2725bd0a9925919b9b51739eea5f9e2bae91e83288108a9ad338b2e3a4435ee5"},
@@ -839,14 +581,6 @@ pytest = [
     {file = "pytest-6.1.1-py3-none-any.whl", hash = "sha256:7a8190790c17d79a11f847fba0b004ee9a8122582ebff4729a082c109e81a4c9"},
     {file = "pytest-6.1.1.tar.gz", hash = "sha256:8f593023c1a0f916110285b6efd7f99db07d59546e3d8c36fc60e2ab05d3be92"},
 ]
-pytest-faulthandler = [
-    {file = "pytest-faulthandler-2.0.1.tar.gz", hash = "sha256:ed72bbce87ac344da81eb7d882196a457d4a1026a3da4a57154dacd85cd71ae5"},
-    {file = "pytest_faulthandler-2.0.1-py2.py3-none-any.whl", hash = "sha256:236430ba962fd1c910d670922be55fe5b25ea9bc3fc6561a0cafbb8759e7504d"},
-]
-pytest-qt = [
-    {file = "pytest-qt-3.3.0.tar.gz", hash = "sha256:714b0bf86c5313413f2d300ac613515db3a1aef595051ab8ba2ffe619dbe8925"},
-    {file = "pytest_qt-3.3.0-py2.py3-none-any.whl", hash = "sha256:5f8928288f50489d83f5d38caf2d7d9fcd6e7cf769947902caa4661dc7c851e3"},
-]
 python-engineio = [
     {file = "python-engineio-3.13.2.tar.gz", hash = "sha256:36b33c6aa702d9b6a7f527eec6387a2da1a9a24484ec2f086d76576413cef04b"},
     {file = "python_engineio-3.13.2-py2.py3-none-any.whl", hash = "sha256:cfded18156862f94544a9f8ef37f56727df731c8552d7023f5afee8369be2db6"},
@@ -855,53 +589,6 @@ python-socketio = [
     {file = "python-socketio-4.6.0.tar.gz", hash = "sha256:358d8fbbc029c4538ea25bcaa283e47f375be0017fcba829de8a3a731c9df25a"},
     {file = "python_socketio-4.6.0-py2.py3-none-any.whl", hash = "sha256:d437f797c44b6efba2f201867cf02b8c96b97dff26d4e4281ac08b45817cd522"},
 ]
-pywin32 = [
-    {file = "pywin32-228-cp27-cp27m-win32.whl", hash = "sha256:37dc9935f6a383cc744315ae0c2882ba1768d9b06700a70f35dc1ce73cd4ba9c"},
-    {file = "pywin32-228-cp27-cp27m-win_amd64.whl", hash = "sha256:11cb6610efc2f078c9e6d8f5d0f957620c333f4b23466931a247fb945ed35e89"},
-    {file = "pywin32-228-cp35-cp35m-win32.whl", hash = "sha256:1f45db18af5d36195447b2cffacd182fe2d296849ba0aecdab24d3852fbf3f80"},
-    {file = "pywin32-228-cp35-cp35m-win_amd64.whl", hash = "sha256:6e38c44097a834a4707c1b63efa9c2435f5a42afabff634a17f563bc478dfcc8"},
-    {file = "pywin32-228-cp36-cp36m-win32.whl", hash = "sha256:ec16d44b49b5f34e99eb97cf270806fdc560dff6f84d281eb2fcb89a014a56a9"},
-    {file = "pywin32-228-cp36-cp36m-win_amd64.whl", hash = "sha256:a60d795c6590a5b6baeacd16c583d91cce8038f959bd80c53bd9a68f40130f2d"},
-    {file = "pywin32-228-cp37-cp37m-win32.whl", hash = "sha256:af40887b6fc200eafe4d7742c48417529a8702dcc1a60bf89eee152d1d11209f"},
-    {file = "pywin32-228-cp37-cp37m-win_amd64.whl", hash = "sha256:00eaf43dbd05ba6a9b0080c77e161e0b7a601f9a3f660727a952e40140537de7"},
-    {file = "pywin32-228-cp38-cp38-win32.whl", hash = "sha256:fa6ba028909cfc64ce9e24bcf22f588b14871980d9787f1e2002c99af8f1850c"},
-    {file = "pywin32-228-cp38-cp38-win_amd64.whl", hash = "sha256:9b3466083f8271e1a5eb0329f4e0d61925d46b40b195a33413e0905dccb285e8"},
-    {file = "pywin32-228-cp39-cp39-win32.whl", hash = "sha256:ed74b72d8059a6606f64842e7917aeee99159ebd6b8d6261c518d002837be298"},
-    {file = "pywin32-228-cp39-cp39-win_amd64.whl", hash = "sha256:8319bafdcd90b7202c50d6014efdfe4fde9311b3ff15fd6f893a45c0868de203"},
-]
-qrcode = [
-    {file = "qrcode-6.1-py2.py3-none-any.whl", hash = "sha256:3996ee560fc39532910603704c82980ff6d4d5d629f9c3f25f34174ce8606cf5"},
-    {file = "qrcode-6.1.tar.gz", hash = "sha256:505253854f607f2abf4d16092c61d4e9d511a3b4392e60bff957a68592b04369"},
-]
-regex = [
-    {file = "regex-2020.10.11-cp27-cp27m-win32.whl", hash = "sha256:4f5c0fe46fb79a7adf766b365cae56cafbf352c27358fda811e4a1dc8216d0db"},
-    {file = "regex-2020.10.11-cp27-cp27m-win_amd64.whl", hash = "sha256:39a5ef30bca911f5a8a3d4476f5713ed4d66e313d9fb6755b32bec8a2e519635"},
-    {file = "regex-2020.10.11-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:7c4fc5a8ec91a2254bb459db27dbd9e16bba1dabff638f425d736888d34aaefa"},
-    {file = "regex-2020.10.11-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:d537e270b3e6bfaea4f49eaf267984bfb3628c86670e9ad2a257358d3b8f0955"},
-    {file = "regex-2020.10.11-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:a8240df4957a5b0e641998a5d78b3c4ea762c845d8cb8997bf820626826fde9a"},
-    {file = "regex-2020.10.11-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:4302153abb96859beb2c778cc4662607a34175065fc2f33a21f49eb3fbd1ccd3"},
-    {file = "regex-2020.10.11-cp36-cp36m-win32.whl", hash = "sha256:c077c9d04a040dba001cf62b3aff08fd85be86bccf2c51a770c77377662a2d55"},
-    {file = "regex-2020.10.11-cp36-cp36m-win_amd64.whl", hash = "sha256:46ab6070b0d2cb85700b8863b3f5504c7f75d8af44289e9562195fe02a8dd72d"},
-    {file = "regex-2020.10.11-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:d629d750ebe75a88184db98f759633b0a7772c2e6f4da529f0027b4a402c0e2f"},
-    {file = "regex-2020.10.11-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:8e7ef296b84d44425760fe813cabd7afbb48c8dd62023018b338bbd9d7d6f2f0"},
-    {file = "regex-2020.10.11-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:e490f08897cb44e54bddf5c6e27deca9b58c4076849f32aaa7a0b9f1730f2c20"},
-    {file = "regex-2020.10.11-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:850339226aa4fec04916386577674bb9d69abe0048f5d1a99f91b0004bfdcc01"},
-    {file = "regex-2020.10.11-cp37-cp37m-win32.whl", hash = "sha256:60c4f64d9a326fe48e8738c3dbc068e1edc41ff7895a9e3723840deec4bc1c28"},
-    {file = "regex-2020.10.11-cp37-cp37m-win_amd64.whl", hash = "sha256:8ba3efdd60bfee1aa784dbcea175eb442d059b576934c9d099e381e5a9f48930"},
-    {file = "regex-2020.10.11-cp38-cp38-manylinux1_i686.whl", hash = "sha256:2308491b3e6c530a3bb38a8a4bb1dc5fd32cbf1e11ca623f2172ba17a81acef1"},
-    {file = "regex-2020.10.11-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:b8806649983a1c78874ec7e04393ef076805740f6319e87a56f91f1767960212"},
-    {file = "regex-2020.10.11-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:a2a31ee8a354fa3036d12804730e1e20d58bc4e250365ead34b9c30bbe9908c3"},
-    {file = "regex-2020.10.11-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:d9d53518eeed12190744d366ec4a3f39b99d7daa705abca95f87dd8b442df4ad"},
-    {file = "regex-2020.10.11-cp38-cp38-win32.whl", hash = "sha256:3d5a8d007116021cf65355ada47bf405656c4b3b9a988493d26688275fde1f1c"},
-    {file = "regex-2020.10.11-cp38-cp38-win_amd64.whl", hash = "sha256:f579caecbbca291b0fcc7d473664c8c08635da2f9b1567c22ea32311c86ef68c"},
-    {file = "regex-2020.10.11-cp39-cp39-manylinux1_i686.whl", hash = "sha256:8c8c42aa5d3ac9a49829c4b28a81bebfa0378996f9e0ca5b5ab8a36870c3e5ee"},
-    {file = "regex-2020.10.11-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:c529ba90c1775697a65b46c83d47a2d3de70f24d96da5d41d05a761c73b063af"},
-    {file = "regex-2020.10.11-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:6cf527ec2f3565248408b61dd36e380d799c2a1047eab04e13a2b0c15dd9c767"},
-    {file = "regex-2020.10.11-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:671c51d352cfb146e48baee82b1ee8d6ffe357c292f5e13300cdc5c00867ebfc"},
-    {file = "regex-2020.10.11-cp39-cp39-win32.whl", hash = "sha256:a63907332531a499b8cdfd18953febb5a4c525e9e7ca4ac147423b917244b260"},
-    {file = "regex-2020.10.11-cp39-cp39-win_amd64.whl", hash = "sha256:1a16afbfadaadc1397353f9b32e19a65dc1d1804c80ad73a14f435348ca017ad"},
-    {file = "regex-2020.10.11.tar.gz", hash = "sha256:463e770c48da76a8da82b8d4a48a541f314e0df91cbb6d873a341dbe578efafd"},
-]
 requests = [
     {file = "requests-2.24.0-py2.py3-none-any.whl", hash = "sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898"},
     {file = "requests-2.24.0.tar.gz", hash = "sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b"},
@@ -917,29 +604,6 @@ toml = [
     {file = "toml-0.10.1-py2.py3-none-any.whl", hash = "sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88"},
     {file = "toml-0.10.1.tar.gz", hash = "sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f"},
 ]
-typed-ast = [
-    {file = "typed_ast-1.4.1-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3"},
-    {file = "typed_ast-1.4.1-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb"},
-    {file = "typed_ast-1.4.1-cp35-cp35m-win32.whl", hash = "sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919"},
-    {file = "typed_ast-1.4.1-cp35-cp35m-win_amd64.whl", hash = "sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01"},
-    {file = "typed_ast-1.4.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75"},
-    {file = "typed_ast-1.4.1-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652"},
-    {file = "typed_ast-1.4.1-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7"},
-    {file = "typed_ast-1.4.1-cp36-cp36m-win32.whl", hash = "sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1"},
-    {file = "typed_ast-1.4.1-cp36-cp36m-win_amd64.whl", hash = "sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa"},
-    {file = "typed_ast-1.4.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614"},
-    {file = "typed_ast-1.4.1-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41"},
-    {file = "typed_ast-1.4.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b"},
-    {file = "typed_ast-1.4.1-cp37-cp37m-win32.whl", hash = "sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe"},
-    {file = "typed_ast-1.4.1-cp37-cp37m-win_amd64.whl", hash = "sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355"},
-    {file = "typed_ast-1.4.1-cp38-cp38-macosx_10_15_x86_64.whl", hash = "sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6"},
-    {file = "typed_ast-1.4.1-cp38-cp38-manylinux1_i686.whl", hash = "sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907"},
-    {file = "typed_ast-1.4.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d"},
-    {file = "typed_ast-1.4.1-cp38-cp38-win32.whl", hash = "sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c"},
-    {file = "typed_ast-1.4.1-cp38-cp38-win_amd64.whl", hash = "sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4"},
-    {file = "typed_ast-1.4.1-cp39-cp39-macosx_10_15_x86_64.whl", hash = "sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34"},
-    {file = "typed_ast-1.4.1.tar.gz", hash = "sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b"},
-]
 urllib3 = [
     {file = "urllib3-1.25.10-py2.py3-none-any.whl", hash = "sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461"},
     {file = "urllib3-1.25.10.tar.gz", hash = "sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a"},

cli/pyproject.toml

@@ -1,9 +1,19 @@
 [tool.poetry]
-name = "onionshare"
+name = "onionshare_cli"
-version = "2.3"
+version = "0.1.3"
 description = "OnionShare lets you securely and anonymously send and receive files. It works by starting a web server, making it accessible as a Tor onion service, and generating an unguessable web address so others can download files from you, or upload files to you. It does _not_ require setting up a separate server or using a third party file-sharing service."
 authors = ["Micah Lee <micah@micahflee.com>"]
 license = "GPLv3+"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Framework :: Flask",
+    "Topic :: Communications :: File Sharing",
+    "Topic :: Security :: Cryptography",
+    "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
+    "Intended Audience :: End Users/Desktop",
+    "Operating System :: OS Independent",
+    "Environment :: Web Environment",
+]
 
 [tool.poetry.dependencies]
 python = "^3.7"
@@ -12,31 +22,18 @@ Flask = "*"
 Flask-HTTPAuth = "*"
 flask-socketio = "*"
 pycryptodome = "*"
-PyQt5 = "5.14"
-PyQt5-sip = "*"
 PySocks = "*"
 requests = "*"
 stem = "*"
 urllib3 = "*"
 eventlet = "*"
-qrcode = "*"
-psutil = "*"
-pywin32 = {version = "*", platform = "win32"}
 
 [tool.poetry.dev-dependencies]
-atomicwrites = "*"
-attrs = "*"
-more-itertools = "*"
-pluggy = "*"
-py = "*"
 pytest = "*"
-pytest-faulthandler = "*"
-pytest-qt = "*"
-six = "*"
-urllib3 = "*"
 setuptools = "*"
-pyinstaller = {version = "*", platform = "darwin"}
+
-black = {version = "^19.10b0", allow-prereleases = true}
+[tool.poetry.scripts]
+onionshare-cli = 'onionshare_cli:main'
 
 [build-system]
 requires = ["poetry>=0.12"]

cli/tests/conftest.py

@@ -0,0 +1,198 @@
+import sys
+
+# Force tests to look for resources in the source code tree
+sys.onionshare_dev_mode = True
+
+# Let OnionShare know the tests are running, to avoid colliding with settings files
+sys.onionshare_test_mode = True
+
+import os
+import shutil
+import tempfile
+
+import pytest
+
+from onionshare_cli import common, web
+
+
+# The temporary directory for CLI tests
+test_temp_dir = None
+
+
+def pytest_addoption(parser):
+    parser.addoption(
+        "--runtor", action="store_true", default=False, help="run tor tests"
+    )
+
+
+def pytest_collection_modifyitems(config, items):
+    if not config.getoption("--runtor"):
+        # --runtor given in cli: do not skip tor tests
+        skip_tor = pytest.mark.skip(reason="need --runtor option to run")
+        for item in items:
+            if "tor" in item.keywords:
+                item.add_marker(skip_tor)
+
+
+@pytest.fixture
+def temp_dir():
+    """Creates a persistent temporary directory for the CLI tests to use"""
+    global test_temp_dir
+    if not test_temp_dir:
+        test_temp_dir = tempfile.mkdtemp()
+    return test_temp_dir
+
+
+@pytest.fixture
+def temp_dir_1024(temp_dir):
+    """ Create a temporary directory that has a single file of a
+    particular size (1024 bytes).
+    """
+
+    new_temp_dir = tempfile.mkdtemp(dir=temp_dir)
+    tmp_file, tmp_file_path = tempfile.mkstemp(dir=new_temp_dir)
+    with open(tmp_file, "wb") as f:
+        f.write(b"*" * 1024)
+    return new_temp_dir
+
+
+# pytest > 2.9 only needs @pytest.fixture
+@pytest.yield_fixture
+def temp_dir_1024_delete(temp_dir):
+    """ Create a temporary directory that has a single file of a
+    particular size (1024 bytes). The temporary directory (including
+    the file inside) will be deleted after fixture usage.
+    """
+
+    with tempfile.TemporaryDirectory(dir=temp_dir) as new_temp_dir:
+        tmp_file, tmp_file_path = tempfile.mkstemp(dir=new_temp_dir)
+        with open(tmp_file, "wb") as f:
+            f.write(b"*" * 1024)
+        yield new_temp_dir
+
+
+@pytest.fixture
+def temp_file_1024(temp_dir):
+    """ Create a temporary file of a particular size (1024 bytes). """
+
+    with tempfile.NamedTemporaryFile(delete=False, dir=temp_dir) as tmp_file:
+        tmp_file.write(b"*" * 1024)
+    return tmp_file.name
+
+
+# pytest > 2.9 only needs @pytest.fixture
+@pytest.yield_fixture
+def temp_file_1024_delete(temp_dir):
+    """
+    Create a temporary file of a particular size (1024 bytes).
+    The temporary file will be deleted after fixture usage.
+    """
+
+    with tempfile.NamedTemporaryFile(dir=temp_dir, delete=False) as tmp_file:
+        tmp_file.write(b"*" * 1024)
+        tmp_file.flush()
+        tmp_file.close()
+        yield tmp_file.name
+
+
+# pytest > 2.9 only needs @pytest.fixture
+@pytest.yield_fixture(scope="session")
+def custom_zw():
+    zw = web.share_mode.ZipWriter(
+        common.Common(),
+        zip_filename=common.Common.random_string(4, 6),
+        processed_size_callback=lambda _: "custom_callback",
+    )
+    yield zw
+    zw.close()
+    os.remove(zw.zip_filename)
+
+
+# pytest > 2.9 only needs @pytest.fixture
+@pytest.yield_fixture(scope="session")
+def default_zw():
+    zw = web.share_mode.ZipWriter(common.Common())
+    yield zw
+    zw.close()
+    tmp_dir = os.path.dirname(zw.zip_filename)
+    try:
+        shutil.rmtree(tmp_dir, ignore_errors=True)
+    except:
+        pass
+
+
+@pytest.fixture
+def locale_en(monkeypatch):
+    monkeypatch.setattr("locale.getdefaultlocale", lambda: ("en_US", "UTF-8"))
+
+
+@pytest.fixture
+def locale_fr(monkeypatch):
+    monkeypatch.setattr("locale.getdefaultlocale", lambda: ("fr_FR", "UTF-8"))
+
+
+@pytest.fixture
+def locale_invalid(monkeypatch):
+    monkeypatch.setattr("locale.getdefaultlocale", lambda: ("xx_XX", "UTF-8"))
+
+
+@pytest.fixture
+def locale_ru(monkeypatch):
+    monkeypatch.setattr("locale.getdefaultlocale", lambda: ("ru_RU", "UTF-8"))
+
+
+@pytest.fixture
+def platform_darwin(monkeypatch):
+    monkeypatch.setattr("platform.system", lambda: "Darwin")
+
+
+@pytest.fixture  # (scope="session")
+def platform_linux(monkeypatch):
+    monkeypatch.setattr("platform.system", lambda: "Linux")
+
+
+@pytest.fixture
+def platform_windows(monkeypatch):
+    monkeypatch.setattr("platform.system", lambda: "Windows")
+
+
+@pytest.fixture
+def sys_argv_sys_prefix(monkeypatch):
+    monkeypatch.setattr("sys.argv", [sys.prefix])
+
+
+@pytest.fixture
+def sys_frozen(monkeypatch):
+    monkeypatch.setattr("sys.frozen", True, raising=False)
+
+
+@pytest.fixture
+def sys_meipass(monkeypatch):
+    monkeypatch.setattr("sys._MEIPASS", os.path.expanduser("~"), raising=False)
+
+
+@pytest.fixture  # (scope="session")
+def sys_onionshare_dev_mode(monkeypatch):
+    monkeypatch.setattr("sys.onionshare_dev_mode", True, raising=False)
+
+
+@pytest.fixture
+def time_time_100(monkeypatch):
+    monkeypatch.setattr("time.time", lambda: 100)
+
+
+@pytest.fixture
+def time_strftime(monkeypatch):
+    monkeypatch.setattr("time.strftime", lambda _: "Jun 06 2013 11:05:00")
+
+
+@pytest.fixture
+def common_obj():
+    return common.Common()
+
+
+@pytest.fixture
+def settings_obj(sys_onionshare_dev_mode, platform_linux):
+    _common = common.Common()
+    _common.version = "DUMMY_VERSION_1.2.3"
+    return settings.Settings(_common)

cli/tests/test_cli.py

@@ -0,0 +1,58 @@
+import os
+
+import pytest
+
+from onionshare_cli import OnionShare
+from onionshare_cli.common import Common
+from onionshare_cli.mode_settings import ModeSettings
+
+
+class MyOnion:
+    def __init__(self):
+        self.auth_string = "TestHidServAuth"
+        self.private_key = ""
+        self.scheduled_key = None
+
+    @staticmethod
+    def start_onion_service(
+        self, mode_settings_obj, await_publication=True, save_scheduled_key=False
+    ):
+        return "test_service_id.onion"
+
+
+@pytest.fixture
+def onionshare_obj():
+    common = Common()
+    return OnionShare(common, MyOnion())
+
+
+@pytest.fixture
+def mode_settings_obj():
+    common = Common()
+    return ModeSettings(common)
+
+
+class TestOnionShare:
+    def test_init(self, onionshare_obj):
+        assert onionshare_obj.hidserv_dir is None
+        assert onionshare_obj.onion_host is None
+        assert onionshare_obj.cleanup_filenames == []
+        assert onionshare_obj.local_only is False
+
+    def test_start_onion_service(self, onionshare_obj, mode_settings_obj):
+        onionshare_obj.start_onion_service(mode_settings_obj)
+        assert 17600 <= onionshare_obj.port <= 17650
+        assert onionshare_obj.onion_host == "test_service_id.onion"
+
+    def test_start_onion_service_local_only(self, onionshare_obj, mode_settings_obj):
+        onionshare_obj.local_only = True
+        onionshare_obj.start_onion_service(mode_settings_obj)
+        assert onionshare_obj.onion_host == "127.0.0.1:{}".format(onionshare_obj.port)
+
+    def test_cleanup(self, onionshare_obj, temp_dir_1024, temp_file_1024):
+        onionshare_obj.cleanup_filenames = [temp_dir_1024, temp_file_1024]
+        onionshare_obj.cleanup()
+
+        assert os.path.exists(temp_dir_1024) is False
+        assert os.path.exists(temp_dir_1024) is False
+        assert onionshare_obj.cleanup_filenames == []

cli/tests/test_cli_settings.py

@@ -0,0 +1,147 @@
+import json
+import os
+import tempfile
+import sys
+
+import pytest
+
+from onionshare_cli import common, settings
+
+
+@pytest.fixture
+def settings_obj(sys_onionshare_dev_mode, platform_linux):
+    _common = common.Common()
+    _common.version = "DUMMY_VERSION_1.2.3"
+    return settings.Settings(_common)
+
+
+class TestSettings:
+    def test_init(self, settings_obj):
+        expected_settings = {
+            "version": "DUMMY_VERSION_1.2.3",
+            "connection_type": "bundled",
+            "control_port_address": "127.0.0.1",
+            "control_port_port": 9051,
+            "socks_address": "127.0.0.1",
+            "socks_port": 9050,
+            "socket_file_path": "/var/run/tor/control",
+            "auth_type": "no_auth",
+            "auth_password": "",
+            "use_autoupdate": True,
+            "autoupdate_timestamp": None,
+            "no_bridges": True,
+            "tor_bridges_use_obfs4": False,
+            "tor_bridges_use_meek_lite_azure": False,
+            "tor_bridges_use_custom_bridges": "",
+            "persistent_tabs": [],
+        }
+        for key in settings_obj._settings:
+            # Skip locale, it will not always default to the same thing
+            if key != "locale":
+                assert settings_obj._settings[key] == settings_obj.default_settings[key]
+                assert settings_obj._settings[key] == expected_settings[key]
+
+    def test_fill_in_defaults(self, settings_obj):
+        del settings_obj._settings["version"]
+        settings_obj.fill_in_defaults()
+        assert settings_obj._settings["version"] == "DUMMY_VERSION_1.2.3"
+
+    def test_load(self, temp_dir, settings_obj):
+        custom_settings = {
+            "version": "CUSTOM_VERSION",
+            "socks_port": 9999,
+            "use_stealth": True,
+        }
+        tmp_file, tmp_file_path = tempfile.mkstemp(dir=temp_dir)
+        with open(tmp_file, "w") as f:
+            json.dump(custom_settings, f)
+        settings_obj.filename = tmp_file_path
+        settings_obj.load()
+
+        assert settings_obj._settings["version"] == "CUSTOM_VERSION"
+        assert settings_obj._settings["socks_port"] == 9999
+        assert settings_obj._settings["use_stealth"] is True
+
+        os.remove(tmp_file_path)
+        assert os.path.exists(tmp_file_path) is False
+
+    def test_save(self, monkeypatch, temp_dir, settings_obj):
+        settings_filename = "default_settings.json"
+        new_temp_dir = tempfile.mkdtemp(dir=temp_dir)
+        settings_path = os.path.join(new_temp_dir, settings_filename)
+        settings_obj.filename = settings_path
+        settings_obj.save()
+        with open(settings_path, "r") as f:
+            settings = json.load(f)
+
+        assert settings_obj._settings == settings
+
+        os.remove(settings_path)
+        assert os.path.exists(settings_path) is False
+
+    def test_get(self, settings_obj):
+        assert settings_obj.get("version") == "DUMMY_VERSION_1.2.3"
+        assert settings_obj.get("connection_type") == "bundled"
+        assert settings_obj.get("control_port_address") == "127.0.0.1"
+        assert settings_obj.get("control_port_port") == 9051
+        assert settings_obj.get("socks_address") == "127.0.0.1"
+        assert settings_obj.get("socks_port") == 9050
+        assert settings_obj.get("socket_file_path") == "/var/run/tor/control"
+        assert settings_obj.get("auth_type") == "no_auth"
+        assert settings_obj.get("auth_password") == ""
+        assert settings_obj.get("use_autoupdate") is True
+        assert settings_obj.get("autoupdate_timestamp") is None
+        assert settings_obj.get("autoupdate_timestamp") is None
+        assert settings_obj.get("no_bridges") is True
+        assert settings_obj.get("tor_bridges_use_obfs4") is False
+        assert settings_obj.get("tor_bridges_use_meek_lite_azure") is False
+        assert settings_obj.get("tor_bridges_use_custom_bridges") == ""
+
+    def test_set_version(self, settings_obj):
+        settings_obj.set("version", "CUSTOM_VERSION")
+        assert settings_obj._settings["version"] == "CUSTOM_VERSION"
+
+    def test_set_control_port_port(self, settings_obj):
+        settings_obj.set("control_port_port", 999)
+        assert settings_obj._settings["control_port_port"] == 999
+
+        settings_obj.set("control_port_port", "NON_INTEGER")
+        assert settings_obj._settings["control_port_port"] == 9051
+
+    def test_set_socks_port(self, settings_obj):
+        settings_obj.set("socks_port", 888)
+        assert settings_obj._settings["socks_port"] == 888
+
+        settings_obj.set("socks_port", "NON_INTEGER")
+        assert settings_obj._settings["socks_port"] == 9050
+
+    @pytest.mark.skipif(sys.platform != "Darwin", reason="requires Darwin")
+    def test_filename_darwin(self, monkeypatch, platform_darwin):
+        obj = settings.Settings(common.Common())
+        assert obj.filename == os.path.expanduser(
+            "~/Library/Application Support/OnionShare-testdata/onionshare.json"
+        )
+
+    @pytest.mark.skipif(sys.platform != "Linux", reason="requires Linux")
+    def test_filename_linux(self, monkeypatch, platform_linux):
+        obj = settings.Settings(common.Common())
+        assert obj.filename == os.path.expanduser(
+            "~/.config/onionshare-testdata/onionshare.json"
+        )
+
+    @pytest.mark.skipif(sys.platform != "win32", reason="requires Windows")
+    def test_filename_windows(self, monkeypatch, platform_windows):
+        obj = settings.Settings(common.Common())
+        assert obj.filename == os.path.expanduser(
+            "~\\AppData\\Roaming\\OnionShare-testdata\\onionshare.json"
+        )
+
+    def test_set_custom_bridge(self, settings_obj):
+        settings_obj.set(
+            "tor_bridges_use_custom_bridges",
+            "Bridge 45.3.20.65:9050 21300AD88890A49C429A6CB9959CFD44490A8F6E",
+        )
+        assert (
+            settings_obj._settings["tor_bridges_use_custom_bridges"]
+            == "Bridge 45.3.20.65:9050 21300AD88890A49C429A6CB9959CFD44490A8F6E"
+        )

cli/tests/test_cli_web.py

@@ -0,0 +1,231 @@
+import contextlib
+import inspect
+import io
+import os
+import random
+import re
+import socket
+import sys
+import zipfile
+import tempfile
+import base64
+
+import pytest
+from werkzeug.datastructures import Headers
+
+from onionshare_cli.common import Common
+from onionshare_cli.web import Web
+from onionshare_cli.settings import Settings
+from onionshare_cli.mode_settings import ModeSettings
+
+DEFAULT_ZW_FILENAME_REGEX = re.compile(r"^onionshare_[a-z2-7]{6}.zip$")
+RANDOM_STR_REGEX = re.compile(r"^[a-z2-7]+$")
+
+
+def web_obj(temp_dir, common_obj, mode, num_files=0):
+    """ Creates a Web object, in either share mode or receive mode, ready for testing """
+    common_obj.settings = Settings(common_obj)
+    mode_settings = ModeSettings(common_obj)
+    web = Web(common_obj, False, mode_settings, mode)
+    web.generate_password()
+    web.running = True
+
+    web.app.testing = True
+
+    # Share mode
+    if mode == "share":
+        # Add files
+        files = []
+        for _ in range(num_files):
+            with tempfile.NamedTemporaryFile(delete=False, dir=temp_dir) as tmp_file:
+                tmp_file.write(b"*" * 1024)
+                files.append(tmp_file.name)
+        web.share_mode.set_file_info(files)
+    # Receive mode
+    else:
+        pass
+
+    return web
+
+
+class TestWeb:
+    def test_share_mode(self, temp_dir, common_obj):
+        web = web_obj(temp_dir, common_obj, "share", 3)
+        assert web.mode == "share"
+        with web.app.test_client() as c:
+            # Load / without auth
+            res = c.get("/")
+            res.get_data()
+            assert res.status_code == 401
+
+            # Load / with invalid auth
+            res = c.get("/", headers=self._make_auth_headers("invalid"))
+            res.get_data()
+            assert res.status_code == 401
+
+            # Load / with valid auth
+            res = c.get("/", headers=self._make_auth_headers(web.password))
+            res.get_data()
+            assert res.status_code == 200
+
+            # Download
+            res = c.get("/download", headers=self._make_auth_headers(web.password))
+            res.get_data()
+            assert res.status_code == 200
+            assert (
+                res.mimetype == "application/zip"
+                or res.mimetype == "application/x-zip-compressed"
+            )
+
+    def test_share_mode_autostop_sharing_on(self, temp_dir, common_obj, temp_file_1024):
+        web = web_obj(temp_dir, common_obj, "share", 3)
+        web.settings.set("share", "autostop_sharing", True)
+
+        assert web.running == True
+
+        with web.app.test_client() as c:
+            # Download the first time
+            res = c.get("/download", headers=self._make_auth_headers(web.password))
+            res.get_data()
+            assert res.status_code == 200
+            assert (
+                res.mimetype == "application/zip"
+                or res.mimetype == "application/x-zip-compressed"
+            )
+
+            assert web.running == False
+
+    def test_share_mode_autostop_sharing_off(
+        self, temp_dir, common_obj, temp_file_1024
+    ):
+        web = web_obj(temp_dir, common_obj, "share", 3)
+        web.settings.set("share", "autostop_sharing", False)
+
+        assert web.running == True
+
+        with web.app.test_client() as c:
+            # Download the first time
+            res = c.get("/download", headers=self._make_auth_headers(web.password))
+            res.get_data()
+            assert res.status_code == 200
+            assert (
+                res.mimetype == "application/zip"
+                or res.mimetype == "application/x-zip-compressed"
+            )
+            assert web.running == True
+
+    def test_receive_mode(self, temp_dir, common_obj):
+        web = web_obj(temp_dir, common_obj, "receive")
+        assert web.mode == "receive"
+
+        with web.app.test_client() as c:
+            # Load / without auth
+            res = c.get("/")
+            res.get_data()
+            assert res.status_code == 401
+
+            # Load / with invalid auth
+            res = c.get("/", headers=self._make_auth_headers("invalid"))
+            res.get_data()
+            assert res.status_code == 401
+
+            # Load / with valid auth
+            res = c.get("/", headers=self._make_auth_headers(web.password))
+            res.get_data()
+            assert res.status_code == 200
+
+    def test_public_mode_on(self, temp_dir, common_obj):
+        web = web_obj(temp_dir, common_obj, "receive")
+        web.settings.set("general", "public", True)
+
+        with web.app.test_client() as c:
+            # Loading / should work without auth
+            res = c.get("/")
+            data1 = res.get_data()
+            assert res.status_code == 200
+
+    def test_public_mode_off(self, temp_dir, common_obj):
+        web = web_obj(temp_dir, common_obj, "receive")
+        web.settings.set("general", "public", False)
+
+        with web.app.test_client() as c:
+            # Load / without auth
+            res = c.get("/")
+            res.get_data()
+            assert res.status_code == 401
+
+            # But static resources should work without auth
+            res = c.get(f"{web.static_url_path}/css/style.css")
+            res.get_data()
+            assert res.status_code == 200
+
+            # Load / with valid auth
+            res = c.get("/", headers=self._make_auth_headers(web.password))
+            res.get_data()
+            assert res.status_code == 200
+
+    def _make_auth_headers(self, password):
+        auth = base64.b64encode(b"onionshare:" + password.encode()).decode()
+        h = Headers()
+        h.add("Authorization", "Basic " + auth)
+        return h
+
+
+class TestZipWriterDefault:
+    @pytest.mark.parametrize(
+        "test_input",
+        (
+            f"onionshare_{''.join(random.choice('abcdefghijklmnopqrstuvwxyz234567') for _ in range(6))}.zip"
+            for _ in range(50)
+        ),
+    )
+    def test_default_zw_filename_regex(self, test_input):
+        assert bool(DEFAULT_ZW_FILENAME_REGEX.match(test_input))
+
+    def test_zw_filename(self, default_zw):
+        zw_filename = os.path.basename(default_zw.zip_filename)
+        assert bool(DEFAULT_ZW_FILENAME_REGEX.match(zw_filename))
+
+    def test_zipfile_filename_matches_zipwriter_filename(self, default_zw):
+        assert default_zw.z.filename == default_zw.zip_filename
+
+    def test_zipfile_allow_zip64(self, default_zw):
+        assert default_zw.z._allowZip64 is True
+
+    def test_zipfile_mode(self, default_zw):
+        assert default_zw.z.mode == "w"
+
+    def test_callback(self, default_zw):
+        assert default_zw.processed_size_callback(None) is None
+
+    def test_add_file(self, default_zw, temp_file_1024_delete):
+        default_zw.add_file(temp_file_1024_delete)
+        zipfile_info = default_zw.z.getinfo(os.path.basename(temp_file_1024_delete))
+
+        assert zipfile_info.compress_type == zipfile.ZIP_DEFLATED
+        assert zipfile_info.file_size == 1024
+
+    def test_add_directory(self, temp_dir_1024_delete, default_zw):
+        previous_size = default_zw._size  # size before adding directory
+        default_zw.add_dir(temp_dir_1024_delete)
+        assert default_zw._size == previous_size + 1024
+
+
+class TestZipWriterCustom:
+    @pytest.mark.parametrize(
+        "test_input",
+        (
+            Common.random_string(
+                random.randint(2, 50), random.choice((None, random.randint(2, 50)))
+            )
+            for _ in range(50)
+        ),
+    )
+    def test_random_string_regex(self, test_input):
+        assert bool(RANDOM_STR_REGEX.match(test_input))
+
+    def test_custom_filename(self, custom_zw):
+        assert bool(RANDOM_STR_REGEX.match(custom_zw.zip_filename))
+
+    def test_custom_callback(self, custom_zw):
+        assert custom_zw.processed_size_callback(None) == "custom_callback"

desktop/BUILD.md

@@ -29,6 +29,25 @@ cd onionshare
 
 ## Linux
 
+OnionShare uses [Briefcase](https://briefcase.readthedocs.io/en/latest/).
+
+Install Briefcase dependencies from your package repositories by following [these instructions](https://docs.beeware.org/en/latest/tutorial/tutorial-0.html#install-dependencies).
+
+Now create and/or activate a virtual environment.
+
+```
+python3 -m venv venv
+. venv/bin/activate
+```
+
+While your virtual environment is active, install briefcase from pip.
+
+```
+pip install briefcase
+```
+
+
+
 ### Use newest software
 
 The recommended way to develop OnionShare is to use the latest versions of all dependencies.

desktop/LICENSE

@@ -0,0 +1,679 @@
+(Note: Third-party licenses can be found under install/licenses/.)
+
+OnionShare
+Copyright (C) 2014-2020 Micah Lee, et al. <micah@micahflee.com>
+
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+the GNU General Public License is intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights.  Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received.  You must make sure that they, too, receive
+or can get the source code.  And you must show them these terms so they
+know their rights.
+
+  Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+
+  For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software.  For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Use with the GNU Affero General Public License.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+    <program>  Copyright (C) <year>  <name of author>
+    This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU GPL, see
+<http://www.gnu.org/licenses/>.
+
+  The GNU General Public License does not permit incorporating your program
+into proprietary programs.  If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.  But first, please read
+<http://www.gnu.org/philosophy/why-not-lgpl.html>.