Use abstract domain sockets for RPC. Use epoll backend for TCP connections on Android.

fix formatting of rnode server urls

.github/ISSUE_TEMPLATE/🐛-bug-report.md

@@ -12,10 +12,14 @@ Before creating a bug report on this issue tracker, you **must** read the [Contr
 
 - The issue tracker is used by developers of this project. **Do not use it to ask general questions, or for support requests**.
 - Ideas and feature requests can be made on the [Discussions](https://github.com/markqvist/Reticulum/discussions). **Only** feature requests accepted by maintainers and developers are tracked and included on the issue tracker. **Do not post feature requests here**.
-- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), delete this section from your bug report.
+- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), **delete this section only** (*"Read the Contribution Guidelines"*) from your bug report, **and fill in all the other sections**.
 
 **Describe the Bug**
-A clear and concise description of what the bug is.
+First of all: Is this really a bug? Is it reproducible?
+
+If this is a request for help because something is not working as you expected, stop right here, and go to the [discussions](https://github.com/markqvist/Reticulum/discussions) instead, where you can post your questions and get help from other users.
+
+If this really is a bug or issue with the software, remove this section of the template, and provide **a clear and concise description of what the bug is**.
 
 **To Reproduce**
 Describe in detail how to reproduce the bug.
@@ -24,7 +28,7 @@ Describe in detail how to reproduce the bug.
 A clear and concise description of what you expected to happen.
 
 **Logs & Screenshots**
-Please include any relevant log output. If applicable, also add screenshots to help explain your problem.
+Please include any relevant log output. If applicable, also add screenshots to help explain your problem. In most cases, without any relevant log output, we will not be able to determine the cause of the bug, or reproduce it.
 
 **System Information**
 - OS and version

.gitignore

@@ -6,6 +6,7 @@ sbapp/bin
 sbapp/app_storage
 sbapp/RNS
 sbapp/LXMF
+sbapp/LXST
 sbapp/precompiled
 sbapp/*.DS_Store
 sbapp/*.pyc

README.md

@@ -1,7 +1,7 @@
 Sideband <img align="right" src="https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg"/>
 =========
 
-Sideband is an extensible LXMF messaging client, situational awareness tracker and remote control and monitoring system for Android, Linux, macOS and Windows. It allows you to communicate with other people or LXMF-compatible systems over Reticulum networks using LoRa, Packet Radio, WiFi, I2P, Encrypted QR Paper Messages, or anything else Reticulum supports.
+Sideband is an extensible LXMF messaging and LXST telephony client, situational awareness tracker and remote control and monitoring system for Android, Linux, macOS and Windows. It allows you to communicate with other people or LXMF-compatible systems over Reticulum networks using LoRa, Packet Radio, WiFi, I2P, Encrypted QR Paper Messages, or anything else Reticulum supports.
 
 ![Screenshot](https://github.com/markqvist/Sideband/raw/main/docs/screenshots/devices_small.webp)
 
@@ -13,10 +13,11 @@ This also means that Sideband operates differently than what you might be used t
 
 Sideband provides many useful and interesting functions, such as:
 
-- **Secure** and **self-sovereign** messaging using the LXMF protocol over Reticulum.
+- **Secure** and **self-sovereign** messaging and voice calls using the LXMF and LXST protocols over Reticulum.
 - **Image** and **file transfers** over all supported mediums.
 - **Audio messages** that work even over **LoRa** and **radio links**, thanks to [Codec2](https://github.com/drowe67/codec2/) and [Opus](https://github.com/xiph/opus) encoding.
 - Secure and direct P2P **telemetry and location sharing**. No third parties or servers ever have your data.
+- The telemetry system is **completely extensible** via [simple plugins](#creating-plugins).
 - Situation display on both online and **locally stored offline maps**.
 - Geospatial awareness calculations.
 - Exchanging messages through **encrypted QR-codes on paper**, or through messages embedded directly in **lxm://** links.
@@ -39,7 +40,7 @@ Sideband can run on most computing devices, but installation methods vary by dev
 
 ## On Android
 
-For your Android devices, you can install Sideband through F-Droid, by adding the [Between the Borders Repo](https://reticulum.betweentheborders.com/fdroid/repo/), or you can download an [APK on the latest release page](https://github.com/markqvist/Sideband/releases/latest). Both sources are signed with the same release keys, and can be used interchangably.
+For your Android devices, you can download an [APK on the latest release page](https://github.com/markqvist/Sideband/releases/latest).
 
 After the application is installed on your Android device, it is also possible to pull updates directly through the **Repository** section of the application.
 
@@ -47,9 +48,8 @@ After the application is installed on your Android device, it is also possible t
 
 On all Linux-based operating systems, Sideband is available as a `pipx`/`pip` package. This installation method **includes desktop integration**, so that Sideband will show up in your applications menu and launchers. Below are install steps for the most common recent Linux distros. For Debian 11, see the end of this section.
 
-**Please note!** The very latest Python release, Python 3.13 is currently **not** compatible with the Kivy framework, that Sideband uses to render its user interface. If your Linux distribution uses Python 3.13 as its default Python installation, you will need to install an earlier version as well. Using [the latest release of Python 3.12](https://www.python.org/downloads/release/python-3127/) is recommended.
-
-You will first need to install a few dependencies for audio messaging and Codec2 support to work:
+#### Basic Installation
+You will first need to install a few dependencies for voice calls, audio messaging and Codec2 support to work:
 
 ```bash
 # For Debian (12+), Ubuntu (22.04+) and derivatives
@@ -68,10 +68,6 @@ Once those are installed, install the Sideband application itself:
 ```bash
 # Finally, install Sideband using pipx:
 pipx install sbapp
-
-# If you need to specify a specific Python version,
-# use something like the following:
-pipx install sbapp --python python3.12
 ```
 
 After installation, you can now run Sideband in a number of different ways:
@@ -84,7 +80,10 @@ After installation, you can now run Sideband in a number of different ways:
 pipx ensurepath
 
 # The first time you run Sideband, you will need to do it
-# from the terminal:
+# from the terminal, for the application launcher item to
+# show up. On some distros you may also need to log out
+# and back in again, or simply reboot the machine for the
+# application entry to show up in your menu.
 sideband
 
 # At the first launch, it will add an application icon
@@ -101,6 +100,9 @@ sideband --daemon
 sideband -v
 ```
 
+If you do not already have Reticulum connectivity set up on your computer or local network, you will probably want to edit the Reticulum configuration file at `~/.reticulum/config` and [add any interfaces](https://reticulum.network/manual/interfaces.html) you need for connectivity.
+
+#### Advanced Installation
 You can also install Sideband in various alternative ways:
 
 ```bash
@@ -135,17 +137,17 @@ You can install Sideband on all Raspberry Pi models that support 64-bit operatin
 
 Aditionally, the `pycodec2` package needs to be installed manually. I have provided a pre-built version, that you can download and install with a single command, or if you don't want to trust my pre-built version, you can [build and install it from source yourself](https://github.com/gregorias/pycodec2/blob/main/DEV.md).
 
-The install instructions below assume that you are installing Sideband on 64-bit Raspberry Pi OS (based on Debian Bookworm). If you're running something else on your Pi, you might need to modify some commands slightly. To install Sideband on Raspberry Pi, follow these steps:
+The install instructions below assume that you are installing Sideband on 64-bit Raspberry Pi OS (based on Debian Bookworm or later). If you're running something else on your Pi, you might need to modify some commands slightly. To install Sideband on Raspberry Pi with full support for voice calls, audio messages and Codec2, follow these steps:
 
 ```bash
 # First of all, install the required dependencies:
 sudo apt install python3-pip python3-pyaudio python3-dev python3-cryptography build-essential libopusfile0 libsdl2-dev libavcodec-dev libavdevice-dev libavfilter-dev portaudio19-dev codec2 libcodec2-1.0 xclip xsel
 
 # If you don't want to compile pycodec2 yourself,
-# download the pre-compiled package provided here
+# download the pre-compiled package provided here:
 wget https://raw.githubusercontent.com/markqvist/Sideband/main/docs/utilities/pycodec2-3.0.1-cp311-cp311-linux_aarch64.whl
 
-# Install it:
+# And install it:
 pip install ./pycodec2-3.0.1-cp311-cp311-linux_aarch64.whl --break-system-packages
 
 # You can now install Sideband
@@ -159,6 +161,8 @@ sudo reboot
 sideband
 ```
 
+If you do not already have Reticulum connectivity set up on your computer or local network, you will probably want to edit the Reticulum configuration file at `~/.reticulum/config` and [add any interfaces](https://reticulum.network/manual/interfaces.html) you need for connectivity.
+
 ## On macOS
 
 To install Sideband on macOS, you have two options available:
@@ -185,6 +189,8 @@ pip3 install rns --break-system-packages
 
 If you do not have Python and `pip` available, [download and install it](https://www.python.org/downloads/) first.
 
+If you do not already have Reticulum connectivity set up on your computer or local network, you will probably want to edit the Reticulum configuration file at `~/.reticulum/config` and [add any interfaces](https://reticulum.network/manual/interfaces.html) you need for connectivity.
+
 #### Source Package Install
 
 For more advanced setups, including the ability to run Sideband in headless daemon mode, enable debug logging output, configuration import and export and more, you may want to install it from the source package via `pip` instead.
@@ -195,24 +201,33 @@ To install Sideband via `pip`, follow these instructions:
 
 ```bash
 # Install Sideband and dependencies on macOS using pip:
-pip3 install sbapp --user --break-system-packages
-
-# Optionally install RNS command line utilities:
-pip3 install rns
+pip3 install sbapp
 
 # Run Sideband from the terminal:
+#################################
+sideband
+# or
 python3 -m sbapp.main
 
 # Enable debug logging:
+#################################
+sideband -v
+# or
 python3 -m sbapp.main -v
 
 # Start Sideband in daemon mode:
+#################################
+sideband -d
+# or
 python3 -m sbapp.main -d
 
-# If you add your pip install location to
-# the PATH environment variable, you can
-# also run Sideband simply using:
-sideband
+# If Python and pip was installed correctly,
+# you can simply use the "sideband" command
+# directly. Otherwise, you will manually
+# need to add the pip binaries directory to
+# your PATH environment variable, or start
+# Sideband via the "python3 -m sbapp.main"
+# syntax.
 
 ```
 
@@ -229,6 +244,8 @@ Simply download the packaged Windows ZIP file from the [latest release page](htt
 
 When running Sideband for the first time, a default Reticulum configuration file will be created, if you don't already have one. If you don't have any existing Reticulum connectivity available locally, you may want to edit the file, located at `C:\Users\USERNAME\.reticulum\config` and manually add an interface that provides connectivity to a wider network. If you just want to connect over the Internet, you can add one of the public hubs on the [Reticulum Testnet](https://reticulum.network/connect.html).
 
+#### Installing Reticulum Utilities
+
 Though the ZIP file contains everything necessary to run Sideband, it is also recommended to install the Reticulum command line utilities separately, so that you can use commands like `rnstatus` and `rnsd` from the command line. This will make it easier to manage Reticulum connectivity on your system. If you do not already have Python installed on your system, [download and install it](https://www.python.org/downloads/) first.
 
 **Important!** When asked by the installer, make sure to add the Python program to your `PATH` environment variables. If you don't do this, you will not be able to use the `pip` installer, or run any of the installed commands. When Python has been installed, you can open a command prompt and install the Reticulum package via `pip`:
@@ -253,6 +270,28 @@ The Sideband application can now be launched by running the command `sideband` i
 
 Since this installation method automatically installs the `rns` and `lxmf` packages as well, you will also have access to using all the included RNS and LXMF utilities like `rnstatus`, `rnsd` and `lxmd` on your system.
 
+# Creating Plugins
+
+Sideband features a flexible and extensible plugin system, that allows you to hook all kinds of control, status reporting, command execution and telemetry collection into the LXMF messaging system. Plugins can be created as either *Telemetry*, *Command* or *Service* plugins, for different use-cases.
+
+To create plugins for Sideband, you can find a variety of [code examples](https://github.com/markqvist/Sideband/tree/main/docs/example_plugins) in this repository, that you can use as a basis for writing your own plugins. The example plugins include:
+
+- [Custom telemetry](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/telemetry.py)
+- [Getting BME280 temperature, humidity and pressure](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/bme280_telemetry.py)
+- [Basic commands](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/basic.py)
+- [Location telemetry from GPSd on Linux](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/gpsd_location.py)
+- [Location telemetry from Windows Location Provider](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/windows_location.py)
+- [Getting statistics from your LXMF propagation node](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/lxmd_telemetry.py)
+- [Viewing cameras and streams](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/view.py)
+- [Fetching an XKCD comic](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/comic.py)
+- [Creating a service plugin](https://github.com/markqvist/Sideband/blob/main/docs/example_plugins/service.py)
+
+For creating telemetry plugins, Sideband includes 20+ built-in sensor types to chose from, for representing all kinds telemetry data. If none of those fit your needs, there is a `Custom` sensor type, that can include any kind of data.
+
+Command plugins allow you to define any kind of action or command to be run when receiving command messages from other LXMF clients. In the example directory, you will find various command plugin templates, for example for viewing security cameras or webcams through Sideband.
+
+Service plugins allow you to integrate any kind of service, bridge or other system into Sideband, and have that react to events or state changes in Sideband itself.
+
 # Paper Messaging Example
 
 You can try out the paper messaging functionality by using the following QR-code. It is a paper message sent to the LXMF address `6b3362bd2c1dbf87b66a85f79a8d8c75`. To be able to decrypt and read the message, you will need to import the following base32-encoded Reticulum Identity into the app:
@@ -288,26 +327,6 @@ You can help support the continued development of open, free and private communi
 
 <br/>
 
-# Planned Features
-
-- <s>Secure and private location and telemetry sharing</s>
-- <s>Including images in messages</s>
-- <s>Sending file attachments</s>
-- <s>Offline and online maps</s>
-- <s>Paper messages</s>
-- <s>Using Sideband as a Reticulum Transport Instance</s>
-- <s>Encryption keys export and import</s>
-- <s>Plugin support for commands, services and telemetry</s>
-- <s>Sending voice messages (using Codec2 and Opus)</s>
-- <s>Adding a Linux desktop integration</s>
-- <s>Adding prebuilt Windows binaries to the releases</s>
-- <s>Adding prebuilt macOS binaries to the releases</s>
-- Adding a Nomad Net page browser
-- LXMF sneakernet functionality
-- Network visualisation and test tools
-- Better message sorting mechanism
-- A debug log viewer
-
 # License
 Unless otherwise noted, this work is licensed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
 

docs/example_plugins/bme280_telemetry.py

@@ -0,0 +1,67 @@
+# This plugin provides temperature, humidity
+# and pressure data via a BME280 sensor
+# connected over I2C. The plugin requires
+# the "smbus2" and "RPi.bme280" modules to
+# be available on your system. These can be
+# installed with:
+#
+# pip install smbus2 RPi.bme280
+
+import os
+import RNS
+from importlib.util import find_spec
+
+class BME280Plugin(SidebandTelemetryPlugin):
+    plugin_name = "telemetry_bme280"
+
+    I2C_ADDRESS = 0x76
+    I2C_BUS     = 1
+
+    # If your BME280 has an offset from the true
+    # temperature, you can compensate for this
+    # by modifying this parameter.
+    TEMPERATURE_CORRECTION = 0.0
+
+    def start(self):
+        RNS.log("BME280 telemetry plugin starting...")
+
+        if find_spec("smbus2"): import smbus2
+        else: raise OSError(f"No smbus2 module available, cannot start BME280 telemetry plugin")
+        
+        if find_spec("bme280"): import bme280
+        else: raise OSError(f"No bme280 module available, cannot start BME280 telemetry plugin")
+
+        self.sensor_connected = False
+
+        try:
+            self.bme280 = bme280
+            self.address = self.I2C_ADDRESS
+            self.bus = smbus2.SMBus(self.I2C_BUS)
+            self.calibration = self.bme280.load_calibration_params(self.bus, self.address)
+            self.sensor_connected = True
+            self.tc = self.TEMPERATURE_CORRECTION
+        
+        except Exception as e:
+            RNS.log(f"Could not connect to I2C device while starting BME280 telemetry plugin", RNS.LOG_ERROR)
+            RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
+
+        super().start()
+
+    def stop(self):
+        self.bus.close()
+        super().stop()
+
+    def update_telemetry(self, telemeter):
+        if telemeter != None:
+            if self.sensor_connected:
+                try:
+                    sample = self.bme280.sample(self.bus, self.address, self.calibration); ts = telemeter.sensors
+                    telemeter.synthesize("temperature"); ts["temperature"].data = {"c": round(sample.temperature+self.tc,1)}
+                    telemeter.synthesize("humidity");    ts["humidity"].data    = {"percent_relative": round(sample.humidity,1)}
+                    telemeter.synthesize("pressure");    ts["pressure"].data    = {"mbar": round(sample.pressure,1)}
+
+                except Exception as e:
+                    RNS.log("An error occurred while updating BME280 sensor data", RNS.LOG_ERROR)
+                    RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
+
+plugin_class = BME280Plugin

docs/example_plugins/lxmd_telemetry.py

@@ -0,0 +1,41 @@
+# This is an LXMd telemetry plugin that
+# queries a running LXMF Propagation Node
+# for status and statistics.
+
+import RNS
+
+class LXMdTelemetryPlugin(SidebandTelemetryPlugin):
+    plugin_name = "lxmd_telemetry"
+
+    def start(self):
+        # Do any initialisation work here
+        RNS.log("LXMd telemetry plugin starting...")
+
+        # And finally call start on superclass
+        super().start()
+
+    def stop(self):
+        # Do any teardown work here
+        pass
+
+        # And finally call stop on superclass
+        super().stop()
+
+    def update_telemetry(self, telemeter):
+        if telemeter != None:
+            if not "lxmf_propagation" in telemeter.sensors:
+                # Create lxmd status sensor if it is not already
+                # enabled in the running telemeter instance
+                telemeter.enable("lxmf_propagation")
+
+                # Set the identity file used to communicate with
+                # the running LXMd instance.
+                telemeter.sensors["lxmf_propagation"].set_identity("~/.lxmd/identity")
+
+                # You can also get LXMF Propagation Node stats
+                # from an LXMd instance running inside nomadnet
+                # telemeter.sensors["lxmf_propagation"].set_identity("~/.nomadnetwork/storage/identity")
+
+# Finally, tell Sideband what class in this
+# file is the actual plugin class.
+plugin_class = LXMdTelemetryPlugin

docs/example_plugins/telemetry.py

@@ -59,7 +59,8 @@ class BasicTelemetryPlugin(SidebandTelemetryPlugin):
 
             # Create fuel sensor
             telemeter.synthesize("fuel")
-            telemeter.sensors["fuel"].update_entry(capacity=75, level=61)
+            telemeter.sensors["fuel"].update_entry(capacity=75, level=61, type_label="Main")
+            telemeter.sensors["fuel"].update_entry(capacity=15, level=15, type_label="Reserve")
 
 # Finally, tell Sideband what class in this
 # file is the actual plugin class.

docs/example_plugins/view.py

@@ -310,7 +310,8 @@ class ViewCommandPlugin(SidebandCommandPlugin):
             else:
                 image_field     = self.sources[source].get_image_field(quality_preset)
                 image_timestamp = self.timestamp_str(self.sources[source].last_update)
-                message         = f"Source [b]{source}[/b] at [b]{image_timestamp}[/b]"
+                message         = "#!md\n" # Tell sideband to format this message
+                message        += f"Source [b]{source}[/b] at [b]{image_timestamp}[/b]"
 
                 if image_field != None:
                     self.image_response(message, image_field, requestor)

docs/example_plugins/windows_location.py

@@ -0,0 +1,88 @@
+# Windows Location Provider plugin example, provided by @haplo-dev
+
+import RNS
+import time
+import threading
+import asyncio
+from winsdk.windows.devices import geolocation
+
+class WindowsLocationPlugin(SidebandTelemetryPlugin):
+    plugin_name = "windows_location"
+
+    def __init__(self, sideband_core):
+        self.update_interval = 5.0
+        self.should_run = False
+
+        self.latitude = None
+        self.longitude = None
+        self.altitude = None
+        self.speed = None
+        self.bearing = None
+        self.accuracy = None
+        self.last_update = None
+
+        super().__init__(sideband_core)
+
+    def start(self):
+        RNS.log("Starting Windows Location provider plugin...")
+
+        self.should_run = True
+        update_thread = threading.Thread(target=self.update_job, daemon=True)
+        update_thread.start()
+
+        super().start()
+
+    def stop(self):
+        self.should_run = False
+        super().stop()
+
+    def update_job(self):
+        while self.should_run:
+            RNS.log("Updating location from Windows Geolocation...", RNS.LOG_DEBUG)
+            try:
+                asyncio.run(self.get_location())
+            except Exception as e:
+                RNS.log(f"Error getting location: {str(e)}", RNS.LOG_ERROR)
+
+            time.sleep(self.update_interval)
+
+    async def get_location(self):
+        geolocator = geolocation.Geolocator()
+        position = await geolocator.get_geoposition_async()
+
+        self.last_update = time.time()
+        self.latitude = position.coordinate.latitude
+        self.longitude = position.coordinate.longitude
+        self.altitude = position.coordinate.altitude
+        self.accuracy = position.coordinate.accuracy
+
+        # Note: Windows Geolocation doesn't provide speed and bearing directly
+        # You might need to calculate these from successive position updates
+        self.speed = None
+        self.bearing = None
+
+    def has_location(self):
+        return all([self.latitude, self.longitude, self.altitude, self.accuracy]) is not None
+
+    def update_telemetry(self, telemeter):
+        if self.is_running() and telemeter is not None:
+            if self.has_location():
+                RNS.log("Updating location from Windows Geolocation", RNS.LOG_DEBUG)
+                if "location" not in telemeter.sensors:
+                    telemeter.synthesize("location")
+
+                telemeter.sensors["location"].latitude = self.latitude
+                telemeter.sensors["location"].longitude = self.longitude
+                telemeter.sensors["location"].altitude = self.altitude
+                telemeter.sensors["location"].speed = self.speed
+                telemeter.sensors["location"].bearing = self.bearing
+                telemeter.sensors["location"].accuracy = self.accuracy
+                telemeter.sensors["location"].stale_time = 5
+                telemeter.sensors["location"].set_update_time(self.last_update)
+            
+            else:
+                RNS.log("No location from Windows Geolocation yet", RNS.LOG_DEBUG)
+
+# Finally, tell Sideband what class in this
+# file is the actual plugin class.
+plugin_class = WindowsLocationPlugin

docs/utilities/rns_audio_call_calc.py

@@ -1,129 +0,0 @@
-import os
-import math
-import RNS
-import RNS.vendor.umsgpack as mp
-
-def simulate(link_speed=9600, audio_slot_ms=70, codec_rate=1200, method="msgpack"):
-    # Simulated on-air link speed
-    LINK_SPEED = link_speed
-
-    # Packing method, can be "msgpack" or "protobuf"
-    PACKING_METHOD = method
-
-    # The target audio slot time
-    TARGET_MS = audio_slot_ms
-
-    # Packets needed per second for half-duplex audio
-    PACKETS_PER_SECOND = 1000/TARGET_MS
-
-    # Effective audio encoder bitrate
-    CODEC_RATE = codec_rate
-
-    # Maximum number of supported audio modes
-    MAX_ENUM = 127
-
-    # Per-packet overhead on a established link is 19
-    # bytes, 3 for header and context, 16 for link ID
-    RNS_OVERHEAD = 19
-
-    # Physical-layer overhead. For RNode, this is 1
-    # byte per RNS packet.
-    PHY_OVERHEAD = 1
-
-    # Total transport overhead
-    TRANSPORT_OVERHEAD = PHY_OVERHEAD+RNS_OVERHEAD
-
-    # Calculate parameters
-    AUDIO_LEN = int(math.ceil(CODEC_RATE/(1000/TARGET_MS)/8))
-    PER_BYTE_LATENCY_MS = 1000/(LINK_SPEED/8)
-
-    # Pack the message with msgpack to get real-
-    # world packed message size
-
-    if PACKING_METHOD == "msgpack":
-        # Calculate msgpack overhead
-        PL_LEN = len(mp.packb([MAX_ENUM, os.urandom(AUDIO_LEN)]))
-        PACKING_OVERHEAD = PL_LEN-AUDIO_LEN
-    elif PACKING_METHOD == "protobuf":
-        # For protobuf, assume the 8 bytes of stated overhead
-        PACKING_OVERHEAD = 8
-        PL_LEN = AUDIO_LEN+PACKING_OVERHEAD
-    else:
-        print("Unsupported packing method")
-        exit(1)
-
-    # Calculate required encrypted token blocks
-    BLOCKSIZE = 16
-    REQUIRED_BLOCKS = math.ceil((PL_LEN+1)/BLOCKSIZE)
-    ENCRYPTED_PAYLOAD_LEN = REQUIRED_BLOCKS*BLOCKSIZE
-    BLOCK_HEADROOM = (REQUIRED_BLOCKS*BLOCKSIZE) - PL_LEN - 1
-
-    # The complete on-air packet length
-    PACKET_LEN     = PHY_OVERHEAD+RNS_OVERHEAD+ENCRYPTED_PAYLOAD_LEN
-    PACKET_LATENCY = round(PACKET_LEN*PER_BYTE_LATENCY_MS, 1)
-
-    # TODO: This should include any additional
-    # airtime consumption such as preamble and TX-tail.
-    PACKET_AIRTIME = PACKET_LEN*PER_BYTE_LATENCY_MS
-    AIRTIME_PCT = (PACKET_AIRTIME/TARGET_MS) * 100
-
-    # Maximum amount of concurrent full-duplex
-    # calls that can coexist on the same channel
-    CONCURRENT_CALLS = math.floor(100/AIRTIME_PCT)
-
-    # Calculate latencies
-    TRANSPORT_LATENCY  = round((PHY_OVERHEAD+RNS_OVERHEAD)*PER_BYTE_LATENCY_MS, 1)
-
-    PAYLOAD_LATENCY    = round(ENCRYPTED_PAYLOAD_LEN*PER_BYTE_LATENCY_MS, 1)
-    RAW_DATA_LATENCY   = round(AUDIO_LEN*PER_BYTE_LATENCY_MS, 1)
-    PACKING_LATENCY    = round(PACKING_OVERHEAD*PER_BYTE_LATENCY_MS, 1)
-    
-    DATA_LATENCY       = round(ENCRYPTED_PAYLOAD_LEN*PER_BYTE_LATENCY_MS, 1)
-    ENCRYPTION_LATENCY = round((ENCRYPTED_PAYLOAD_LEN-PL_LEN)*PER_BYTE_LATENCY_MS, 1)
-    if ENCRYPTED_PAYLOAD_LEN-PL_LEN == 1:
-        E_OPT_STR = "(optimal)"
-    else:
-        E_OPT_STR = "(sub-optimal)"
-
-    TOTAL_LATENCY      = round(TARGET_MS+PACKET_LATENCY, 1)
-
-    print( "\n===== Simulation Parameters ===\n")
-    print(f"  Packing method       : {method}")
-    print(f"  Sampling delay       : {TARGET_MS}ms")
-    print(f"  Codec bitrate        : {CODEC_RATE} bps")
-    print(f"  Audio data           : {AUDIO_LEN} bytes")
-    print(f"  Packing overhead     : {PACKING_OVERHEAD} bytes")
-    print(f"  Payload length       : {PL_LEN} bytes")
-    print(f"  AES blocks needed    : {REQUIRED_BLOCKS}")
-    print(f"  Encrypted payload    : {ENCRYPTED_PAYLOAD_LEN} bytes")
-    print(f"  Transport overhead   : {TRANSPORT_OVERHEAD} bytes ({RNS_OVERHEAD} from RNS, {PHY_OVERHEAD} from PHY)")
-    print(f"  On-air length        : {PACKET_LEN} bytes")
-    print(f"  Packet airtime       : {PACKET_AIRTIME}ms")
-
-    print( "\n===== Results for "+RNS.prettyspeed(LINK_SPEED)+" Link Speed ===\n")
-    print(f"  Final latency        : {TOTAL_LATENCY}ms")
-    print(f"    Recording latency  : contributes {TARGET_MS}ms")
-    print(f"    Packet transport   : contributes {PACKET_LATENCY}ms")
-    print(f"      Payload          : contributes {PAYLOAD_LATENCY}ms")
-    print(f"        Audio data     : contributes {RAW_DATA_LATENCY}ms")
-    print(f"        Packing format : contributes {PACKING_LATENCY}ms")
-    print(f"        Encryption     : contributes {ENCRYPTION_LATENCY}ms {E_OPT_STR}")
-    print(f"      RNS+PHY overhead : contributes {TRANSPORT_LATENCY}ms")
-    print(f"")
-    print(f"  Half-duplex airtime  : {round(AIRTIME_PCT, 2)}% of link capacity")
-    print(f"    Concurrent calls   : {int(CONCURRENT_CALLS)}\n")
-    print(f"  Full-duplex airtime  : {round(AIRTIME_PCT*2, 2)}% of link capacity")
-    print(f"    Concurrent calls   : {int(CONCURRENT_CALLS/2)}")
-
-    if BLOCK_HEADROOM != 0:
-        print("")
-        print(f"  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
-        print(f"  Unaligned AES block! Each packet could fit")
-        print(f"  {BLOCK_HEADROOM} bytes of additional audio data")
-        print(f"  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
-
-print(  "\n= With mspack =================")
-simulate(method="msgpack")
-
-#print("\n\n= With protobuf ===============")
-#simulate(method="protobuf")

sbapp/Makefile

@@ -97,7 +97,11 @@ getrns:
 	-(rm ./RNS/__pycache__ -r)
 	(cp -rv ../../LXMF/LXMF ./;rm ./LXMF/Utilities/LXMF)
 	-(rm ./LXMF/__pycache__ -r)
+	(cp -rv ../../LXST/LXST ./;rm ./LXST/Utilities/LXST)
+	-(rm ./LXST/__pycache__ -r)
+	-(rm ./LXST/Utilities/__pycache__ -r)
 
 cleanrns:
 	-(rm ./RNS -r)
 	-(rm ./LXMF -r)
+	-(rm ./LXST -r)

sbapp/buildozer.spec

@@ -10,9 +10,9 @@ source.exclude_patterns = app_storage/*,venv/*,Makefile,./Makefil*,requirements,
 
 version.regex = __version__ = ['"](.*)['"]
 version.filename = %(source.dir)s/main.py
-android.numeric_version = 20241020
+android.numeric_version = 20250327
 
-requirements = kivy==2.3.0,libbz2,pillow==10.2.0,qrcode==7.3.1,usb4a,usbserial4a,able_recipe,libwebp,libogg,libopus,opusfile,numpy,cryptography,ffpyplayer,codec2,pycodec2,sh,pynacl,typing-extensions
+requirements = kivy==2.3.0,libbz2,pillow==10.2.0,qrcode==7.3.1,usb4a,usbserial4a,able_recipe,libwebp,libogg,libopus,opusfile,numpy,cryptography,ffpyplayer,codec2,pycodec2,sh,pynacl,typing-extensions,mistune>=3.0.2,beautifulsoup4
 
 android.gradle_dependencies =  com.android.support:support-compat:28.0.0
 #android.enable_androidx = True

sbapp/main.py

@@ -1,6 +1,6 @@
 __debug_build__ = False
 __disable_shaders__ = False
-__version__ = "1.2.0"
+__version__ = "1.5.1"
 __variant__ = ""
 
 import sys
@@ -19,13 +19,14 @@ import RNS
 import LXMF
 import time
 import os
+import re
 import pathlib
 import base64
 import threading
 import RNS.vendor.umsgpack as msgpack
 
-WINDOW_DEFAULT_WIDTH  = "494"
-WINDOW_DEFAULT_HEIGHT = "800"
+WINDOW_DEFAULT_WIDTH  = 494
+WINDOW_DEFAULT_HEIGHT = 800
 
 app_ui_scaling_path = None
 def apply_ui_scale():
@@ -175,9 +176,25 @@ if not args.daemon:
         sys.path.append(local)
 
     if not RNS.vendor.platformutils.is_android():
+        model = None
+        max_width = WINDOW_DEFAULT_WIDTH
+        max_height = WINDOW_DEFAULT_HEIGHT
+
+        try:
+            if os.path.isfile("/sys/firmware/devicetree/base/model"):
+                with open("/sys/firmware/devicetree/base/model", "r") as mf:
+                    model = mf.read()
+        except: pass
+
+        if model:
+            if model.startswith("Raspberry Pi "): max_height = 625
+
+        window_width = min(WINDOW_DEFAULT_WIDTH, max_width)
+        window_height = min(WINDOW_DEFAULT_HEIGHT, max_height)
+
         from kivy.config import Config
-        Config.set("graphics", "width", WINDOW_DEFAULT_WIDTH)
-        Config.set("graphics", "height", WINDOW_DEFAULT_HEIGHT)
+        Config.set("graphics", "width", str(window_width))
+        Config.set("graphics", "height", str(window_height))
 
 if args.daemon:
     from .sideband.core import SidebandCore
@@ -196,7 +213,7 @@ if args.daemon:
     NewConv = DaemonElement; Telemetry = DaemonElement; ObjectDetails = DaemonElement; Announces = DaemonElement;
     Messages = DaemonElement; ts_format = DaemonElement; messages_screen_kv = DaemonElement; plyer = DaemonElement; multilingual_markup = DaemonElement;
     ContentNavigationDrawer = DaemonElement; DrawerList = DaemonElement; IconListItem = DaemonElement; escape_markup = DaemonElement;
-    SoundLoader = DaemonElement; BoxLayout = DaemonElement;
+    SoundLoader = DaemonElement; BoxLayout = DaemonElement; mdconv = DaemonElement;
 
 else:
     apply_ui_scale()
@@ -238,6 +255,7 @@ else:
         from ui.conversations import Conversations, MsgSync, NewConv
         from ui.telemetry import Telemetry
         from ui.utilities import Utilities
+        from ui.voice import Voice
         from ui.objectdetails import ObjectDetails
         from ui.announces import Announces
         from ui.messages import Messages, ts_format, messages_screen_kv
@@ -266,6 +284,7 @@ else:
         from .ui.announces import Announces
         from .ui.telemetry import Telemetry
         from .ui.utilities import Utilities
+        from .ui.voice import Voice
         from .ui.objectdetails import ObjectDetails
         from .ui.messages import Messages, ts_format, messages_screen_kv
         from .ui.helpers import ContentNavigationDrawer, DrawerList, IconListItem
@@ -274,9 +293,7 @@ else:
         import sbapp.pyogg as pyogg
         from sbapp.pydub import AudioSegment
 
-        class toast:
-            def __init__(self, *kwargs):
-                pass
+        from kivymd.toast import toast
 
         from kivy.config import Config
         Config.set('input', 'mouse', 'mouse,disable_multitouch')
@@ -353,6 +370,7 @@ class SidebandApp(MDApp):
         self.settings_ready = False
         self.telemetry_ready = False
         self.utilities_ready = False
+        self.voice_ready = False
         self.connectivity_ready = False
         self.hardware_ready = False
         self.repository_ready = False
@@ -385,7 +403,8 @@ class SidebandApp(MDApp):
         self.connectivity_updater = None
         self.last_map_update = 0
         self.last_telemetry_received = 0
-        self.reposository_url = None
+        self.repository_url = None
+        self.rnode_flasher_url = None
 
 
     #################################################
@@ -1093,6 +1112,10 @@ class SidebandApp(MDApp):
                         self.hw_error_dialog.open()
                         self.hw_error_dialog.is_open = True
 
+            incoming_call = self.sideband.getstate("voice.incoming_call")
+            if incoming_call:
+                self.sideband.setstate("voice.incoming_call", None)
+                toast(f"Call from {incoming_call}", duration=7)
 
         if self.root.ids.screen_manager.current == "messages_screen":
             self.messages_view.update()
@@ -1266,13 +1289,13 @@ class SidebandApp(MDApp):
                             self.messages_view.ids.message_text.write_tab = True
                         Clock.schedule_once(tab_job, 0.15)
 
-                elif self.rec_dialog != None and self.rec_dialog_is_open:
+                elif len(modifiers) == 0 and self.rec_dialog != None and self.rec_dialog_is_open:
                         if text == " ":
                             self.msg_rec_a_rec(None)
                         elif keycode == 40:
                             self.msg_rec_a_save(None)
 
-                elif not self.rec_dialog_is_open and not self.messages_view.ids.message_text.focus and self.messages_view.ptt_enabled and keycode == 44:
+                elif len(modifiers) == 0 and not self.rec_dialog_is_open and not self.messages_view.ids.message_text.focus and self.messages_view.ptt_enabled and keycode == 44:
                     if not self.key_ptt_down:
                         self.key_ptt_down = True
                         self.message_ptt_down_action()
@@ -1367,6 +1390,15 @@ class SidebandApp(MDApp):
                     if text == "o":
                         self.objects_action()
 
+                    if text == "e":
+                        self.voice_action()
+
+                    if text == " ":
+                        self.voice_answer_action()
+
+                    if text == ".":
+                        self.voice_reject_action()
+
                     if text == "r":
                         if self.root.ids.screen_manager.current == "conversations_screen":
                             if self.include_objects:
@@ -1425,6 +1457,8 @@ class SidebandApp(MDApp):
                         self.close_sub_utilities_action()
                     elif self.root.ids.screen_manager.current == "logviewer_screen":
                         self.close_sub_utilities_action()
+                    elif self.root.ids.screen_manager.current == "voice_settings_screen":
+                        self.close_sub_voice_action()
                     else:
                         self.open_conversations(direction="right")
 
@@ -1502,6 +1536,7 @@ class SidebandApp(MDApp):
 
     def announce_now_action(self, sender=None):
         self.sideband.lxmf_announce()
+        if self.sideband.telephone: self.sideband.telephone.announce()
 
         yes_button = MDRectangleFlatButton(text="OK",font_size=dp(18))
 
@@ -1524,6 +1559,58 @@ class SidebandApp(MDApp):
 
     ### Messages (conversation) screen
     ######################################
+
+    def md_to_bbcode(self, text):
+        if not hasattr(self, "mdconv"):
+            if RNS.vendor.platformutils.is_android(): from md import mdconv
+            else: from .md import mdconv
+            self.mdconv = mdconv
+
+        converted = self.mdconv(text)
+        while converted.endswith("\n"):
+            converted = converted[:-1]
+
+        return converted
+
+    def process_bb_markup(self, text):
+        st = time.time()
+        ms = int(sp(14))
+        h1s = int(sp(20))
+        h2s = int(sp(18))
+        h3s = int(sp(16))
+        
+        if not hasattr(self, "pres"):
+            self.presz = re.compile(r"\[(?:size=\d*?)\]", re.IGNORECASE | re.MULTILINE )
+            self.pres = []
+            res = [ [r"\[(?:code|icode).*?\]", f"[font=mono][size={ms}]"],
+                    [r"\[\/(?:code|icode).*?\]", "[/size][/font]"],
+                    [r"\[(?:heading)\]", f"[b][size={h1s}]"],
+                    [r"\[(?:heading=1)*?\]", f"[b][size={h1s}]"],
+                    [r"\[(?:heading=2)*?\]", f"[b][size={h2s}]"],
+                    [r"\[(?:heading=3)*?\]", f"[b][size={h3s}]"],
+                    [r"\[(?:heading=).*?\]", f"[b][size={h3s}]"], # Match all remaining lower-level headings
+                    [r"\[\/(?:heading).*?\]", "[/size][/b]"],
+                    [r"\[(?:list).*?\]", ""],
+                    [r"\[\/(?:list).*?\]", ""],
+                    [r"\n\[(?:\*).*?\]", "\n - "],
+                    [r"\[(?:url).*?\]", ""], # Strip URLs for now
+                    [r"\[\/(?:url).*?\]", ""],
+                    [r"\[(?:img).*?\].*\[\/(?:img).*?\]", ""] # Strip images for now
+                   ]
+
+            for r in res:
+                self.pres.append([re.compile(r[0], re.IGNORECASE | re.MULTILINE ), r[1]])
+
+
+        size_matches = self.presz.findall(text)
+        for sm in size_matches:
+            text = text.replace(sm, f"{sm[:-1]}sp]")
+
+        for pr in self.pres:
+            text = pr[0].sub(pr[1], text)
+
+        return text
+
     def conversation_from_announce_action(self, context_dest):
         if self.sideband.has_conversation(context_dest):
             pass
@@ -1542,13 +1629,17 @@ class SidebandApp(MDApp):
                 self.conversation_action(item)
 
     def conversation_action(self, sender):
-        context_dest = sender.sb_uid
-        def cb(dt):
-            self.open_conversation(context_dest)
-        def cbu(dt):
-            self.conversations_view.update()
-        Clock.schedule_once(cb, 0.15)
-        Clock.schedule_once(cbu, 0.15+0.25)
+        if sender.conv_type == self.sideband.CONV_P2P:
+            context_dest = sender.sb_uid
+            def cb(dt): self.open_conversation(context_dest)
+            def cbu(dt): self.conversations_view.update()
+            Clock.schedule_once(cb, 0.15)
+            Clock.schedule_once(cbu, 0.15+0.25)
+
+        elif sender.conv_type == self.sideband.CONV_VOICE:
+            identity_hash = sender.sb_uid
+            def cb(dt): self.dial_action(identity_hash)
+            Clock.schedule_once(cb, 0.15)
 
     def open_conversation(self, context_dest, direction="left"):
         self.rec_dialog_is_open = False
@@ -1766,6 +1857,11 @@ class SidebandApp(MDApp):
         if self.root.ids.screen_manager.current == "messages_screen":
             self.object_details_action(self.messages_view, from_conv=True)
 
+    def outbound_mode_reset(self, sender=None):
+        self.outbound_mode_paper = False
+        self.outbound_mode_propagation = False
+        self.outbound_mode_command = False
+
     def message_propagation_action(self, sender):
         if self.outbound_mode_command:
             self.outbound_mode_paper = False
@@ -1795,18 +1891,10 @@ class SidebandApp(MDApp):
             tf = open(path, "rb")
             tf.close()
             self.attach_path = path
+            if self.outbound_mode_command:
+                self.outbound_mode_reset()
             
-            if RNS.vendor.platformutils.is_android():
-                toast("Attached \""+str(fbn)+"\"")
-            else:
-                ok_button = MDRectangleFlatButton(text="OK",font_size=dp(18))
-                ate_dialog = MDDialog(
-                    title="File Attached",
-                    text="The file \""+str(fbn)+"\" was attached, and will be included with the next message sent.",
-                    buttons=[ ok_button ],
-                )
-                ok_button.bind(on_release=ate_dialog.dismiss)
-                ate_dialog.open()
+            toast("Attached \""+str(fbn)+"\"")
 
         except Exception as e:
             RNS.log(f"Error while attaching \"{fbn}\": "+str(e), RNS.LOG_ERROR)
@@ -2047,6 +2135,8 @@ class SidebandApp(MDApp):
 
             self.sideband.ui_stopped_recording()
             if self.message_process_audio():
+                if self.outbound_mode_command:
+                    self.outbound_mode_reset()
                 self.message_send_action()
         Clock.schedule_once(cb_s, 0.35)
 
@@ -2194,6 +2284,8 @@ class SidebandApp(MDApp):
                 else:
                     self.message_process_audio()
 
+                if self.outbound_mode_command:
+                    self.outbound_mode_reset()
                 self.update_message_widgets()
                 toast("Added recorded audio to message")
             
@@ -2320,18 +2412,7 @@ class SidebandApp(MDApp):
             self.attach_type = None
             self.update_message_widgets()
 
-            if RNS.vendor.platformutils.get_platform() == "android":
-                toast("Attachment removed")
-            else:
-                ok_button = MDRectangleFlatButton(text="OK",font_size=dp(18))
-                ate_dialog = MDDialog(
-                    title="Attachment Removed",
-                    text="The attached resource was removed from the message",
-                    buttons=[ ok_button ],
-                )
-                ok_button.bind(on_release=ate_dialog.dismiss)
-                ate_dialog.open()
-
+            toast("Attachment removed")
 
     def shared_attachment_action(self, attachment_data):
         if not self.root.ids.screen_manager.current == "messages_screen":
@@ -2523,21 +2604,27 @@ class SidebandApp(MDApp):
         if RNS.vendor.platformutils.is_android():
             hs = dp(22)
             yes_button = MDRectangleFlatButton(text="OK",font_size=dp(18))
+            full_button = MDRectangleFlatButton(text="Full RNS Status",font_size=dp(18), theme_text_color="Custom", line_color=self.color_accept, text_color=self.color_accept)
             dialog = MDDialog(
                 title="Connectivity Status",
                 text=str(self.get_connectivity_text()),
-                buttons=[ yes_button ],
+                buttons=[full_button, yes_button],
                 # elevation=0,
             )
             def cs_updater(dt):
                 dialog.text = str(self.get_connectivity_text())
             def dl_yes(s):
-                self.connectivity_updater.cancel()
                 dialog.dismiss()
                 if self.connectivity_updater != None:
                     self.connectivity_updater.cancel()
+            def cb_rns(sender):
+                dialog.dismiss()
+                if self.connectivity_updater != None:
+                    self.connectivity_updater.cancel()
+                self.rnstatus_action()
 
             yes_button.bind(on_release=dl_yes)
+            full_button.bind(on_release=cb_rns)
             dialog.open()
 
             if self.connectivity_updater != None:
@@ -2546,9 +2633,12 @@ class SidebandApp(MDApp):
             self.connectivity_updater = Clock.schedule_interval(cs_updater, 2.0)
 
         else:
-            if not self.utilities_ready:
-                self.utilities_init()
-            self.utilities_screen.rnstatus_action()
+            self.rnstatus_action()
+
+    def rnstatus_action(self, sender=None):
+        if not self.utilities_ready:
+            self.utilities_init()
+        self.utilities_screen.rnstatus_action()
 
     def ingest_lxm_action(self, sender):
         def cb(dt):
@@ -2705,7 +2795,8 @@ class SidebandApp(MDApp):
                     n_address = dialog.d_content.ids["n_address_field"].text
                     n_name = dialog.d_content.ids["n_name_field"].text
                     n_trusted = dialog.d_content.ids["n_trusted"].active
-                    new_result = self.sideband.new_conversation(n_address, n_name, n_trusted)
+                    n_voice_only = dialog.d_content.ids["n_voice_only"].active
+                    new_result = self.sideband.new_conversation(n_address, n_name, n_trusted, n_voice_only)
 
                 except Exception as e:
                     RNS.log("Error while creating conversation: "+str(e), RNS.LOG_ERROR)
@@ -2767,11 +2858,11 @@ class SidebandApp(MDApp):
             self.information_screen.ids.information_scrollview.effect_cls = ScrollEffect
             self.information_screen.ids.information_logo.icon = self.sideband.asset_dir+"/rns_256.png"
 
-            str_comps  = " - [b]Reticulum[/b] (MIT License)\n - [b]LXMF[/b] (MIT License)\n - [b]KivyMD[/b] (MIT License)"
+            str_comps  =   " - [b]Reticulum[/b] (MIT License)\n - [b]LXMF[/b] (MIT License)\n - [b]KivyMD[/b] (MIT License)"
             str_comps += "\n - [b]Kivy[/b] (MIT License)\n - [b]Codec2[/b] (LGPL License)\n - [b]PyCodec2[/b] (BSD-3 License)"
-            str_comps += "\n - [b]PyDub[/b] (MIT License)\n - [b]PyOgg[/b] (Public Domain)"
-            str_comps += "\n - [b]GeoidHeight[/b] (LGPL License)\n - [b]Python[/b] (PSF License)"
-            str_comps += "\n\nGo to [u][ref=link]https://unsigned.io/donate[/ref][/u] to support the project.\n\nThe Sideband app is Copyright (c) 2024 Mark Qvist / unsigned.io\n\nPermission is granted to freely share and distribute binary copies of "+self.root.ids.app_version_info.text+", so long as no payment or compensation is charged for said distribution or sharing.\n\nIf you were charged or paid anything for this copy of Sideband, please report it to [b]license@unsigned.io[/b].\n\nTHIS IS EXPERIMENTAL SOFTWARE - SIDEBAND COMES WITH ABSOLUTELY NO WARRANTY - USE AT YOUR OWN RISK AND RESPONSIBILITY"
+            str_comps += "\n - [b]PyDub[/b] (MIT License)\n - [b]PyOgg[/b] (Public Domain)\n - [b]FFmpeg[/b] (GPL3 License)"
+            str_comps += "\n - [b]GeoidHeight[/b] (LGPL License)\n - [b]Paho MQTT[/b] (EPL2 License)\n - [b]Python[/b] (PSF License)"
+            str_comps += "\n\nGo to [u][ref=link]https://unsigned.io/donate[/ref][/u] to support the project.\n\nThe Sideband app is Copyright © 2025 Mark Qvist / unsigned.io\n\nPermission is granted to freely share and distribute binary copies of "+self.root.ids.app_version_info.text+", so long as no payment or compensation is charged for said distribution or sharing.\n\nIf you were charged or paid anything for this copy of Sideband, please report it to [b]license@unsigned.io[/b].\n\nTHIS IS EXPERIMENTAL SOFTWARE - SIDEBAND COMES WITH ABSOLUTELY NO WARRANTY - USE AT YOUR OWN RISK AND RESPONSIBILITY"
             info = "This is "+self.root.ids.app_version_info.text+", on RNS v"+RNS.__version__+" and LXMF v"+LXMF.__version__+".\n\nHumbly build using the following open components:\n\n"+str_comps
             self.information_screen.ids.information_info.text = info
             self.information_screen.ids.information_info.bind(on_ref_press=link_exec)
@@ -3052,6 +3143,10 @@ class SidebandApp(MDApp):
                 self.sideband.config["trusted_markup_only"] = self.settings_screen.ids.settings_trusted_markup_only.active
                 self.sideband.save_configuration()
 
+            def save_compose_in_markdown(sender=None, event=None):
+                self.sideband.config["compose_in_markdown"] = self.settings_screen.ids.settings_compose_in_markdown.active
+                self.sideband.save_configuration()
+
             def save_advanced_stats(sender=None, event=None):
                 self.sideband.config["advanced_stats"] = self.settings_screen.ids.settings_advanced_statistics.active
                 self.sideband.save_configuration()
@@ -3102,6 +3197,15 @@ class SidebandApp(MDApp):
                 self.sideband.config["hq_ptt"] = self.settings_screen.ids.settings_hq_ptt.active
                 self.sideband.save_configuration()
 
+            def save_voice_enabled(sender=None, event=None):
+                self.sideband.config["voice_enabled"] = self.settings_screen.ids.settings_voice_enabled.active
+                self.sideband.save_configuration()
+
+                if self.sideband.config["voice_enabled"] == True:
+                    self.sideband.start_voice()
+                else:
+                    self.sideband.stop_voice()
+
             def save_print_command(sender=None, event=None):
                 if not sender.focus:
                     in_cmd = self.settings_screen.ids.settings_print_command.text
@@ -3230,6 +3334,9 @@ class SidebandApp(MDApp):
             self.settings_screen.ids.settings_trusted_markup_only.active = self.sideband.config["trusted_markup_only"]
             self.settings_screen.ids.settings_trusted_markup_only.bind(active=save_trusted_markup_only)
 
+            self.settings_screen.ids.settings_compose_in_markdown.active = self.sideband.config["compose_in_markdown"]
+            self.settings_screen.ids.settings_compose_in_markdown.bind(active=save_compose_in_markdown)
+
             self.settings_screen.ids.settings_ignore_invalid_stamps.active = self.sideband.config["lxmf_ignore_invalid_stamps"]
             self.settings_screen.ids.settings_ignore_invalid_stamps.bind(active=save_lxmf_ignore_invalid_stamps)
 
@@ -3274,6 +3381,10 @@ class SidebandApp(MDApp):
             self.settings_screen.ids.settings_hq_ptt.active = self.sideband.config["hq_ptt"]
             self.settings_screen.ids.settings_hq_ptt.bind(active=save_hq_ptt)
 
+            self.settings_screen.ids.settings_voice_enabled.active = self.sideband.config["voice_enabled"]
+            self.settings_screen.ids.settings_voice_enabled.bind(active=save_voice_enabled)
+            if RNS.vendor.platformutils.is_android(): self.settings_screen.ids.settings_voice_enabled.disabled = True
+
             self.settings_screen.ids.settings_debug.active = self.sideband.config["debug"]
             self.settings_screen.ids.settings_debug.bind(active=save_debug)
 
@@ -3705,9 +3816,9 @@ class SidebandApp(MDApp):
             self.root.ids.screen_manager.transition = self.slide_transition
 
     def repository_link_action(self, sender=None, event=None):
-        if self.reposository_url != None:
+        if self.repository_url != None:
             def lj():
-                webbrowser.open(self.reposository_url)
+                webbrowser.open(self.repository_url)
             threading.Thread(target=lj, daemon=True).start()
 
     def repository_update_info(self, sender=None):
@@ -3749,27 +3860,38 @@ class SidebandApp(MDApp):
             ips = getIP()
             if ips == None or len(ips) == 0:
                 info += "The repository server is running, but the local device IP address could not be determined.\n\nYou can access the repository by pointing a browser to: https://DEVICE_IP:4444/"
-                self.reposository_url = None
+                self.repository_url = None
             else:
                 ipstr = ""
+                self.repository_url = None
                 for ip in ips:
-                    ipstr += "https://"+str(ip)+":4444/\n"
-                    self.reposository_url = ipstr
+                    ipurl = "https://" + str(ip) + ":4444/"
+                    ipstr += "[u][ref=link]"+ipurl+"[/ref][u]\n"
+                    if self.repository_url == None:
+                        self.repository_url = ipurl
+                        self.rnode_flasher_url = ipurl+"mirrors/rnode-flasher/index.html"
 
                 ms = "" if len(ips) == 1 else "es"
-                info += "The repository server is running at the following address"+ms+":\n [u][ref=link]"+ipstr+"[/ref][u]"
+                info += "The repository server is running at the following address" + ms +":\n\n"+ipstr
                 self.repository_screen.ids.repository_info.bind(on_ref_press=self.repository_link_action)
 
             def cb(dt):
                 self.repository_screen.ids.repository_enable_button.disabled = True
                 self.repository_screen.ids.repository_disable_button.disabled = False
+                if hasattr(self, "wants_flasher_launch") and self.wants_flasher_launch == True:
+                    self.wants_flasher_launch = False
+                    if self.rnode_flasher_url != None:
+                        def lj():
+                            webbrowser.open(self.rnode_flasher_url)
+                        threading.Thread(target=lj, daemon=True).start()
+
             Clock.schedule_once(cb, 0.1)
 
         else:
             self.repository_screen.ids.repository_enable_button.disabled = False
             self.repository_screen.ids.repository_disable_button.disabled = True
 
-        info += "\n"
+        info += ""
         self.repository_screen.ids.repository_info.text = info
 
     def repository_start_action(self, sender=None):
@@ -3777,7 +3899,7 @@ class SidebandApp(MDApp):
         Clock.schedule_once(self.repository_update_info, 1.0)
 
     def repository_stop_action(self, sender=None):
-        self.reposository_url = None
+        self.repository_url = None
         self.sideband.stop_webshare() 
         Clock.schedule_once(self.repository_update_info, 0.75)
 
@@ -4226,7 +4348,7 @@ class SidebandApp(MDApp):
             valid = False
         
         try:
-            valid_vals = [7.8, 10.4, 15.6, 20.8, 31.25, 41.7, 62.5, 125, 250, 500]
+            valid_vals = [7.8, 10.4, 15.6, 20.8, 31.25, 41.7, 62.5, 125, 250, 500, 203.125, 406.25, 812.5, 1625]
             val = float(self.hardware_rnode_screen.ids.hardware_rnode_bandwidth.text)
             if not val in valid_vals:
                 raise ValueError("Invalid bandwidth")
@@ -4875,7 +4997,7 @@ class SidebandApp(MDApp):
             self.bind_clipboard_actions(self.keys_screen.ids)
 
             self.keys_screen.ids.keys_scrollview.effect_cls = ScrollEffect
-            info = "Your primary encryption keys are stored in a Reticulum Identity within the Sideband app. If you want to backup this Identity for later use on this or another device, you can export it as a plain text blob, with the key data encoded in Base32 format. This will allow you to restore your address in Sideband or other LXMF clients at a later point.\n\n[b]Warning![/b] Anyone that gets access to the key data will be able to control your LXMF address, impersonate you, and read your messages. In is [b]extremely important[/b] that you keep the Identity data secure if you export it.\n\nBefore displaying or exporting your Identity data, make sure that no machine or person in your vicinity is able to see, copy or record your device screen or similar."
+            info = "Your primary encryption keys are stored in a Reticulum Identity within the Sideband app. If you want to backup this Identity for later use on this or another device, you can export it as a plain text blob, with the key data encoded in Base32 format. This will allow you to restore your address in Sideband or other LXMF clients at a later point.\n\n[b]Warning![/b] Anyone that gets access to the key data will be able to control your LXMF address, impersonate you, and read your messages. It is [b]extremely important[/b] that you keep the Identity data secure if you export it.\n\nBefore displaying or exporting your Identity data, make sure that no machine or person in your vicinity is able to see, copy or record your device screen or similar."
 
             if not RNS.vendor.platformutils.get_platform() == "android":
                 self.widget_hide(self.keys_screen.ids.keys_share)
@@ -5174,6 +5296,62 @@ class SidebandApp(MDApp):
         self.utilities_action(direction="right")
 
 
+    ### voice Screen
+    ######################################
+
+    def voice_init(self):
+        if not self.voice_ready:
+            self.voice_screen = Voice(self)
+            self.voice_ready = True
+    
+    def voice_open(self, sender=None, direction="left", no_transition=False, dial_on_complete=None):
+        if no_transition:
+            self.root.ids.screen_manager.transition = self.no_transition
+        else:
+            self.root.ids.screen_manager.transition = self.slide_transition
+            self.root.ids.screen_manager.transition.direction = direction
+
+        self.root.ids.screen_manager.current = "voice_screen"
+        self.root.ids.nav_drawer.set_state("closed")
+        self.sideband.setstate("app.displaying", self.root.ids.screen_manager.current)
+
+        if no_transition:
+            self.root.ids.screen_manager.transition = self.slide_transition
+
+        self.voice_screen.update_call_status()
+        if dial_on_complete:
+            self.voice_screen.dial_target = dial_on_complete
+            self.voice_screen.screen.ids.identity_hash.text = RNS.hexrep(dial_on_complete, delimit=False)
+            Clock.schedule_once(self.voice_screen.dial_action, 0.25)
+
+    def voice_action(self, sender=None, direction="left", dial_on_complete=None):
+        if self.voice_ready:
+            self.voice_open(direction=direction, dial_on_complete=dial_on_complete)
+        else:
+            self.loader_action(direction=direction)
+            def final(dt):
+                self.voice_init()
+                def o(dt):
+                    self.voice_open(no_transition=True, dial_on_complete=dial_on_complete)
+                Clock.schedule_once(o, ll_ot)
+            Clock.schedule_once(final, ll_ft)
+
+    def close_sub_voice_action(self, sender=None):
+        self.voice_action(direction="right")
+
+    def dial_action(self, identity_hash):
+        self.voice_action(dial_on_complete=identity_hash)
+
+    def voice_answer_action(self, sender=None):
+        if self.sideband.voice_running:
+            if self.sideband.telephone.is_ringing: self.sideband.telephone.answer()
+
+    def voice_reject_action(self, sender=None):
+        if self.sideband.voice_running:
+            if self.sideband.telephone.is_ringing or self.sideband.telephone.is_in_call:
+                self.sideband.telephone.hangup()
+                toast("Call ended")
+
     ### Telemetry Screen
     ######################################
 
@@ -5778,8 +5956,23 @@ class SidebandApp(MDApp):
             self.map_action()
             self.map_show(location)
 
-    def map_display_telemetry(self, sender=None):
-        self.object_details_action(sender)
+    def map_display_telemetry(self, sender=None, event=None):
+        alt_event = False
+        if sender != None:
+            if hasattr(sender, "last_touch"):
+                if hasattr(sender.last_touch, "button"):
+                    if sender.last_touch.button == "right":
+                        alt_event = True
+
+        if alt_event:
+            try:
+                if hasattr(sender, "source_dest"):
+                    self.sideband.request_latest_telemetry(from_addr=sender.source_dest)
+                    toast("Telemetry request sent")
+            except Exception as e:
+                RNS.log(f"Could not request telemetry update: {e}", RNS.LOG_ERROR)
+        else:
+            self.object_details_action(sender)
 
     def map_display_own_telemetry(self, sender=None):
         self.sideband.update_telemetry()
@@ -5963,7 +6156,7 @@ class SidebandApp(MDApp):
 
                 latest_viewable = None
                 if not skip:
-                    for telemetry_entry in telemetry_entries[telemetry_source]:
+                    for telemetry_entry in sorted(telemetry_entries[telemetry_source], key=lambda t: t[0], reverse=True):
                         telemetry_timestamp = telemetry_entry[0]
                         telemetry_data = telemetry_entry[1]
                         t = Telemeter.from_packed(telemetry_data)
@@ -5972,6 +6165,10 @@ class SidebandApp(MDApp):
                             if "location" in telemetry and telemetry["location"] != None and telemetry["location"]["latitude"] != None and telemetry["location"]["longitude"] != None:
                                 latest_viewable = telemetry
                                 break
+                            elif "connection_map" in telemetry:
+                                # TODO: Telemetry entries with connection map sensor types are skipped for now,
+                                # until a proper rendering mechanism is implemented
+                                break
 
                     if latest_viewable != None:
                         l = latest_viewable["location"]
@@ -6091,15 +6288,21 @@ If you use Reticulum and LXMF on hardware that does not carry any identifiers ti
  - [b]Ctrl-Shift-F[/b] add file
  - [b]Ctrl-D[/b] or [b]Ctrl-S[/b] Send message
 
- [b]Voice & PTT[/b]
+ [b]Voice & PTT Messages[/b]
  - [b]Space[/b] Start/stop recording
  - [b]Enter[/b] Save recording to message
  - With PTT enabled, hold [b]Space[/b] to talk
 
+ [b]Voice Calls[/b]
+ - [b]Ctrl-Space[/b] Answer incoming call
+ - [b]Ctrl-.[/b] Reject incoming call
+ - [b]Ctrl-.[/b] Hang up active call
+
  [b]Navigation[/b]
  - [b]Ctrl-[i]n[/i][/b] Go to conversation number [i]n[/i]
  - [b]Ctrl-R[/b] Go to Conversations
  - [b]Ctrl-O[/b] Go to Objects & Devices
+ - [b]Ctrl-E[/b] Go to Voice
  - [b]Ctrl-L[/b] Go to Announce Stream
  - [b]Ctrl-M[/b] Go to Situation Map
  - [b]Ctrl-U[/b] Go to Utilities

sbapp/md/__init__.py

@@ -0,0 +1 @@
+from .md import mdconv

sbapp/md/md.py

@@ -0,0 +1,110 @@
+import mistune
+from mistune.core import BaseRenderer
+from mistune.plugins.formatting import strikethrough, mark, superscript, subscript, insert
+from mistune.plugins.table import table, table_in_list
+from mistune.plugins.footnotes import footnotes
+from mistune.plugins.task_lists import task_lists
+from mistune.plugins.spoiler import spoiler
+from mistune.util import escape as escape_text, safe_entity
+
+def mdconv(markdown_text, domain=None, debug=False):
+    parser = mistune.create_markdown(renderer=BBRenderer(), plugins=[strikethrough, mark, superscript, subscript, insert, footnotes, task_lists, spoiler])
+    return parser(markdown_text)
+
+class BBRenderer(BaseRenderer):
+    NAME = "bbcode"
+
+    def __init__(self, escape=False):
+        super(BBRenderer, self).__init__()
+        self._escape = escape
+
+    def render_token(self, token, state):
+        func = self._get_method(token["type"])
+        attrs = token.get("attrs")
+
+        if "raw" in token: text = token["raw"]
+        elif "children" in token: text = self.render_tokens(token["children"], state)
+        else:
+            if attrs: return func(**attrs)
+            else: return func()
+        
+        if attrs: return func(text, **attrs)
+        else: return func(text)
+
+    # Simple renderers
+    def emphasis(self, text): return f"[i]{text}[/i]"
+    def strong(self, text): return f"[b]{text}[/b]"
+    def codespan(self, text): return f"[icode]{text}[/icode]"
+    def linebreak(self): return "\n"
+    def softbreak(self): return "\n"
+    def list_item(self, text): return f"• {text}\n"
+    def task_list_item(self, text, checked=False): e = "󰱒" if checked else "󰄱"; return f"{e} {text}\n"
+    def strikethrough(self, text): return f"[s]{text}[/s]"
+    def insert(self, text): return f"[u]{text}[/u]"
+    def inline_spoiler(self, text): return f"[ISPOILER]{text}[/ISPOILER]"
+    def block_spoiler(self, text): return f"[SPOILER]\n{text}\n[/SPOILER]"
+    def block_error(self, text): return f"[color=red][icode]{text}[/icode][/color]\n"
+    def block_html(self, html): return ""
+    def link(self, text, url, title=None): return f"[u]{text}[/u] ({url})"
+    def footnote_ref(self, key, index): return f"[sup][u]{index}[/u][/sup]"
+    def footnotes(self, text): return f"[b]Footnotes[/b]\n{text}"
+    def footnote_item(self, text, key, index): return f"[ANAME=footnote-{index}]{index}[/ANAME]. {text}"
+    def superscript(self, text: str) -> str: return f"[sup]{text}[/sup]"
+    def subscript(self, text): return f"[sub]{text}[/sub]"
+    def block_quote(self, text: str) -> str: return f"| [i]{text}[/i]"
+    def paragraph(self, text): return f"{text}\n\n"
+    def blank_line(self): return ""    
+    def block_text(self, text): return text
+
+    # Renderers needing some logic
+    def text(self, text):
+        if self._escape: return escape_text(text)
+        else: return text
+
+    def inline_html(self, html: str) -> str:
+        if self._escape: return escape_text(html)
+        else: return html
+
+    def heading(self, text, level, **attrs):
+        if 1 <= level <= 3: return f"[HEADING={level}]{text}[/HEADING]\n"
+        else: return f"[HEADING=3]{text}[/HEADING]\n"
+    
+    def block_code(self, code: str, **attrs) -> str:
+        special_cases = {"plaintext": None, "text": None, "txt": None}
+        if "info" in attrs:
+            lang_info = safe_entity(attrs["info"].strip())
+            lang = lang_info.split(None, 1)[0].lower()
+            bbcode_lang = special_cases.get(lang, lang)
+            if bbcode_lang: return f"[CODE={bbcode_lang}]{escape_text(code)}[/CODE]\n"
+            else: return f"[CODE]{escape_text(code)}[/CODE]\n"
+        
+        else: return f"[CODE]{escape_text(code)}[/CODE]\n"
+
+    def list(self, text, ordered, **attrs):
+        depth = 0; sln = ""; tli = ""
+        if "depth" in attrs: depth = attrs["depth"]
+        if depth != 0: sln = "\n"
+        if depth == 0: tli = "\n"
+        def remove_empty_lines(text):
+            lines = text.split("\n")
+            non_empty_lines = [line for line in lines if line.strip() != ""]
+            nli = ""; dlm = "\n"+"  "*depth
+            if depth != 0: nli = dlm
+            return nli+dlm.join(non_empty_lines)
+
+        text = remove_empty_lines(text)
+        return sln+text+"\n"+tli
+
+    # TODO: Implement various table types and other special formatting
+    def table(self, children, **attrs): return children
+    def table_head(self, children, **attrs): return children
+    def table_body(self, children, **attrs): return children
+    def table_row(self, children, **attrs): return children
+    def table_cell(self, text, align=None, head=False, **attrs): return f"{text}\n"
+    def def_list(self, text): return f"{text}\n"
+    def def_list_head(self, text): return f"{text}\n"
+    def def_list_item(self, text): return f"{text}\n"
+    def abbr(self, text, title): return text
+    def mark(self, text): return text
+    def image(self, text, url, title=None): return ""
+    def thematic_break(self): return "-------------\n"

sbapp/patches/intent-filter.xml

@@ -28,6 +28,10 @@
    <action android:name="android.intent.action.SEND" />
    <category android:name="android.intent.category.DEFAULT" />
    <data android:mimeType="image/*" />
+   <data android:mimeType="video/*" />
+   <data android:mimeType="audio/*" />
+   <data android:mimeType="text/*" />
+   <data android:mimeType="application/*" />
 </intent-filter>
 
 <meta-data android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"

sbapp/pmqtt/__init__.py

@@ -0,0 +1,5 @@
+__version__ = "2.1.1.dev0"
+
+
+class MQTTException(Exception):
+    pass

sbapp/pmqtt/enums.py

@@ -0,0 +1,113 @@
+import enum
+
+
+class MQTTErrorCode(enum.IntEnum):
+    MQTT_ERR_AGAIN = -1
+    MQTT_ERR_SUCCESS = 0
+    MQTT_ERR_NOMEM = 1
+    MQTT_ERR_PROTOCOL = 2
+    MQTT_ERR_INVAL = 3
+    MQTT_ERR_NO_CONN = 4
+    MQTT_ERR_CONN_REFUSED = 5
+    MQTT_ERR_NOT_FOUND = 6
+    MQTT_ERR_CONN_LOST = 7
+    MQTT_ERR_TLS = 8
+    MQTT_ERR_PAYLOAD_SIZE = 9
+    MQTT_ERR_NOT_SUPPORTED = 10
+    MQTT_ERR_AUTH = 11
+    MQTT_ERR_ACL_DENIED = 12
+    MQTT_ERR_UNKNOWN = 13
+    MQTT_ERR_ERRNO = 14
+    MQTT_ERR_QUEUE_SIZE = 15
+    MQTT_ERR_KEEPALIVE = 16
+
+
+class MQTTProtocolVersion(enum.IntEnum):
+    MQTTv31 = 3
+    MQTTv311 = 4
+    MQTTv5 = 5
+
+
+class CallbackAPIVersion(enum.Enum):
+    """Defined the arguments passed to all user-callback.
+
+    See each callbacks for details: `on_connect`, `on_connect_fail`, `on_disconnect`, `on_message`, `on_publish`,
+    `on_subscribe`, `on_unsubscribe`, `on_log`, `on_socket_open`, `on_socket_close`,
+    `on_socket_register_write`, `on_socket_unregister_write`
+    """
+    VERSION1 = 1
+    """The version used with paho-mqtt 1.x before introducing CallbackAPIVersion.
+
+    This version had different arguments depending if MQTTv5 or MQTTv3 was used. `Properties` & `ReasonCode` were missing
+    on some callback (apply only to MQTTv5).
+
+    This version is deprecated and will be removed in version 3.0.
+    """
+    VERSION2 = 2
+    """ This version fix some of the shortcoming of previous version.
+
+    Callback have the same signature if using MQTTv5 or MQTTv3. `ReasonCode` are used in MQTTv3.
+    """
+
+
+class MessageType(enum.IntEnum):
+    CONNECT = 0x10
+    CONNACK = 0x20
+    PUBLISH = 0x30
+    PUBACK = 0x40
+    PUBREC = 0x50
+    PUBREL = 0x60
+    PUBCOMP = 0x70
+    SUBSCRIBE = 0x80
+    SUBACK = 0x90
+    UNSUBSCRIBE = 0xA0
+    UNSUBACK = 0xB0
+    PINGREQ = 0xC0
+    PINGRESP = 0xD0
+    DISCONNECT = 0xE0
+    AUTH = 0xF0
+
+
+class LogLevel(enum.IntEnum):
+    MQTT_LOG_INFO = 0x01
+    MQTT_LOG_NOTICE = 0x02
+    MQTT_LOG_WARNING = 0x04
+    MQTT_LOG_ERR = 0x08
+    MQTT_LOG_DEBUG = 0x10
+
+
+class ConnackCode(enum.IntEnum):
+    CONNACK_ACCEPTED = 0
+    CONNACK_REFUSED_PROTOCOL_VERSION = 1
+    CONNACK_REFUSED_IDENTIFIER_REJECTED = 2
+    CONNACK_REFUSED_SERVER_UNAVAILABLE = 3
+    CONNACK_REFUSED_BAD_USERNAME_PASSWORD = 4
+    CONNACK_REFUSED_NOT_AUTHORIZED = 5
+
+
+class _ConnectionState(enum.Enum):
+    MQTT_CS_NEW = enum.auto()
+    MQTT_CS_CONNECT_ASYNC = enum.auto()
+    MQTT_CS_CONNECTING = enum.auto()
+    MQTT_CS_CONNECTED = enum.auto()
+    MQTT_CS_CONNECTION_LOST = enum.auto()
+    MQTT_CS_DISCONNECTING = enum.auto()
+    MQTT_CS_DISCONNECTED = enum.auto()
+
+
+class MessageState(enum.IntEnum):
+    MQTT_MS_INVALID = 0
+    MQTT_MS_PUBLISH = 1
+    MQTT_MS_WAIT_FOR_PUBACK = 2
+    MQTT_MS_WAIT_FOR_PUBREC = 3
+    MQTT_MS_RESEND_PUBREL = 4
+    MQTT_MS_WAIT_FOR_PUBREL = 5
+    MQTT_MS_RESEND_PUBCOMP = 6
+    MQTT_MS_WAIT_FOR_PUBCOMP = 7
+    MQTT_MS_SEND_PUBREC = 8
+    MQTT_MS_QUEUED = 9
+
+
+class PahoClientMode(enum.IntEnum):
+    MQTT_CLIENT = 0
+    MQTT_BRIDGE = 1

sbapp/pmqtt/matcher.py

@@ -0,0 +1,78 @@
+class MQTTMatcher:
+    """Intended to manage topic filters including wildcards.
+
+    Internally, MQTTMatcher use a prefix tree (trie) to store
+    values associated with filters, and has an iter_match()
+    method to iterate efficiently over all filters that match
+    some topic name."""
+
+    class Node:
+        __slots__ = '_children', '_content'
+
+        def __init__(self):
+            self._children = {}
+            self._content = None
+
+    def __init__(self):
+        self._root = self.Node()
+
+    def __setitem__(self, key, value):
+        """Add a topic filter :key to the prefix tree
+        and associate it to :value"""
+        node = self._root
+        for sym in key.split('/'):
+            node = node._children.setdefault(sym, self.Node())
+        node._content = value
+
+    def __getitem__(self, key):
+        """Retrieve the value associated with some topic filter :key"""
+        try:
+            node = self._root
+            for sym in key.split('/'):
+                node = node._children[sym]
+            if node._content is None:
+                raise KeyError(key)
+            return node._content
+        except KeyError as ke:
+            raise KeyError(key) from ke
+
+    def __delitem__(self, key):
+        """Delete the value associated with some topic filter :key"""
+        lst = []
+        try:
+            parent, node = None, self._root
+            for k in key.split('/'):
+                 parent, node = node, node._children[k]
+                 lst.append((parent, k, node))
+            # TODO
+            node._content = None
+        except KeyError as ke:
+            raise KeyError(key) from ke
+        else:  # cleanup
+            for parent, k, node in reversed(lst):
+                if node._children or node._content is not None:
+                     break
+                del parent._children[k]
+
+    def iter_match(self, topic):
+        """Return an iterator on all values associated with filters
+        that match the :topic"""
+        lst = topic.split('/')
+        normal = not topic.startswith('$')
+        def rec(node, i=0):
+            if i == len(lst):
+                if node._content is not None:
+                    yield node._content
+            else:
+                part = lst[i]
+                if part in node._children:
+                    for content in rec(node._children[part], i + 1):
+                        yield content
+                if '+' in node._children and (normal or i > 0):
+                    for content in rec(node._children['+'], i + 1):
+                        yield content
+            if '#' in node._children and (normal or i > 0):
+                content = node._children['#']._content
+                if content is not None:
+                    yield content
+        return rec(self._root)

sbapp/pmqtt/packettypes.py

@@ -0,0 +1,43 @@
+"""
+*******************************************************************
+  Copyright (c) 2017, 2019 IBM Corp.
+
+  All rights reserved. This program and the accompanying materials
+  are made available under the terms of the Eclipse Public License v2.0
+  and Eclipse Distribution License v1.0 which accompany this distribution.
+
+  The Eclipse Public License is available at
+     http://www.eclipse.org/legal/epl-v20.html
+  and the Eclipse Distribution License is available at
+    http://www.eclipse.org/org/documents/edl-v10.php.
+
+  Contributors:
+     Ian Craggs - initial implementation and/or documentation
+*******************************************************************
+"""
+
+
+class PacketTypes:
+
+    """
+    Packet types class.  Includes the AUTH packet for MQTT v5.0.
+
+    Holds constants for each packet type such as PacketTypes.PUBLISH
+    and packet name strings: PacketTypes.Names[PacketTypes.PUBLISH].
+
+    """
+
+    indexes = range(1, 16)
+
+    # Packet types
+    CONNECT, CONNACK, PUBLISH, PUBACK, PUBREC, PUBREL, \
+        PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK, \
+        PINGREQ, PINGRESP, DISCONNECT, AUTH = indexes
+
+    # Dummy packet type for properties use - will delay only applies to will
+    WILLMESSAGE = 99
+
+    Names = ( "reserved", \
+    "Connect", "Connack", "Publish", "Puback", "Pubrec", "Pubrel", \
+    "Pubcomp", "Subscribe", "Suback", "Unsubscribe", "Unsuback", \
+    "Pingreq", "Pingresp", "Disconnect", "Auth")

sbapp/pmqtt/properties.py

@@ -0,0 +1,421 @@
+# *******************************************************************
+#   Copyright (c) 2017, 2019 IBM Corp.
+#
+#   All rights reserved. This program and the accompanying materials
+#   are made available under the terms of the Eclipse Public License v2.0
+#   and Eclipse Distribution License v1.0 which accompany this distribution.
+#
+#   The Eclipse Public License is available at
+#      http://www.eclipse.org/legal/epl-v20.html
+#   and the Eclipse Distribution License is available at
+#     http://www.eclipse.org/org/documents/edl-v10.php.
+#
+#   Contributors:
+#      Ian Craggs - initial implementation and/or documentation
+# *******************************************************************
+
+import struct
+
+from .packettypes import PacketTypes
+
+
+class MQTTException(Exception):
+    pass
+
+
+class MalformedPacket(MQTTException):
+    pass
+
+
+def writeInt16(length):
+    # serialize a 16 bit integer to network format
+    return bytearray(struct.pack("!H", length))
+
+
+def readInt16(buf):
+    # deserialize a 16 bit integer from network format
+    return struct.unpack("!H", buf[:2])[0]
+
+
+def writeInt32(length):
+    # serialize a 32 bit integer to network format
+    return bytearray(struct.pack("!L", length))
+
+
+def readInt32(buf):
+    # deserialize a 32 bit integer from network format
+    return struct.unpack("!L", buf[:4])[0]
+
+
+def writeUTF(data):
+    # data could be a string, or bytes.  If string, encode into bytes with utf-8
+    if not isinstance(data, bytes):
+        data = bytes(data, "utf-8")
+    return writeInt16(len(data)) + data
+
+
+def readUTF(buffer, maxlen):
+    if maxlen >= 2:
+        length = readInt16(buffer)
+    else:
+        raise MalformedPacket("Not enough data to read string length")
+    maxlen -= 2
+    if length > maxlen:
+        raise MalformedPacket("Length delimited string too long")
+    buf = buffer[2:2+length].decode("utf-8")
+    # look for chars which are invalid for MQTT
+    for c in buf: # look for D800-DFFF in the UTF string
+        ord_c = ord(c)
+        if ord_c >= 0xD800 and ord_c <= 0xDFFF:
+            raise MalformedPacket("[MQTT-1.5.4-1] D800-DFFF found in UTF-8 data")
+        if ord_c == 0x00: # look for null in the UTF string
+            raise MalformedPacket("[MQTT-1.5.4-2] Null found in UTF-8 data")
+        if ord_c == 0xFEFF:
+            raise MalformedPacket("[MQTT-1.5.4-3] U+FEFF in UTF-8 data")
+    return buf, length+2
+
+
+def writeBytes(buffer):
+    return writeInt16(len(buffer)) + buffer
+
+
+def readBytes(buffer):
+    length = readInt16(buffer)
+    return buffer[2:2+length], length+2
+
+
+class VariableByteIntegers:  # Variable Byte Integer
+    """
+    MQTT variable byte integer helper class.  Used
+    in several places in MQTT v5.0 properties.
+
+    """
+
+    @staticmethod
+    def encode(x):
+        """
+          Convert an integer 0 <= x <= 268435455 into multi-byte format.
+          Returns the buffer converted from the integer.
+        """
+        if not 0 <= x <= 268435455:
+            raise ValueError(f"Value {x!r} must be in range 0-268435455")
+        buffer = b''
+        while 1:
+            digit = x % 128
+            x //= 128
+            if x > 0:
+                digit |= 0x80
+            buffer += bytes([digit])
+            if x == 0:
+                break
+        return buffer
+
+    @staticmethod
+    def decode(buffer):
+        """
+          Get the value of a multi-byte integer from a buffer
+          Return the value, and the number of bytes used.
+
+          [MQTT-1.5.5-1] the encoded value MUST use the minimum number of bytes necessary to represent the value
+        """
+        multiplier = 1
+        value = 0
+        bytes = 0
+        while 1:
+            bytes += 1
+            digit = buffer[0]
+            buffer = buffer[1:]
+            value += (digit & 127) * multiplier
+            if digit & 128 == 0:
+                break
+            multiplier *= 128
+        return (value, bytes)
+
+
+class Properties:
+    """MQTT v5.0 properties class.
+
+    See Properties.names for a list of accepted property names along with their numeric values.
+
+    See Properties.properties for the data type of each property.
+
+    Example of use::
+
+        publish_properties = Properties(PacketTypes.PUBLISH)
+        publish_properties.UserProperty = ("a", "2")
+        publish_properties.UserProperty = ("c", "3")
+
+    First the object is created with packet type as argument, no properties will be present at
+    this point. Then properties are added as attributes, the name of which is the string property
+    name without the spaces.
+
+    """
+
+    def __init__(self, packetType):
+        self.packetType = packetType
+        self.types = ["Byte", "Two Byte Integer", "Four Byte Integer", "Variable Byte Integer",
+                      "Binary Data", "UTF-8 Encoded String", "UTF-8 String Pair"]
+
+        self.names = {
+            "Payload Format Indicator": 1,
+            "Message Expiry Interval": 2,
+            "Content Type": 3,
+            "Response Topic": 8,
+            "Correlation Data": 9,
+            "Subscription Identifier": 11,
+            "Session Expiry Interval": 17,
+            "Assigned Client Identifier": 18,
+            "Server Keep Alive": 19,
+            "Authentication Method": 21,
+            "Authentication Data": 22,
+            "Request Problem Information": 23,
+            "Will Delay Interval": 24,
+            "Request Response Information": 25,
+            "Response Information": 26,
+            "Server Reference": 28,
+            "Reason String": 31,
+            "Receive Maximum": 33,
+            "Topic Alias Maximum": 34,
+            "Topic Alias": 35,
+            "Maximum QoS": 36,
+            "Retain Available": 37,
+            "User Property": 38,
+            "Maximum Packet Size": 39,
+            "Wildcard Subscription Available": 40,
+            "Subscription Identifier Available": 41,
+            "Shared Subscription Available": 42
+        }
+
+        self.properties = {
+            # id:  type, packets
+            # payload format indicator
+            1: (self.types.index("Byte"), [PacketTypes.PUBLISH, PacketTypes.WILLMESSAGE]),
+            2: (self.types.index("Four Byte Integer"), [PacketTypes.PUBLISH, PacketTypes.WILLMESSAGE]),
+            3: (self.types.index("UTF-8 Encoded String"), [PacketTypes.PUBLISH, PacketTypes.WILLMESSAGE]),
+            8: (self.types.index("UTF-8 Encoded String"), [PacketTypes.PUBLISH, PacketTypes.WILLMESSAGE]),
+            9: (self.types.index("Binary Data"), [PacketTypes.PUBLISH, PacketTypes.WILLMESSAGE]),
+            11: (self.types.index("Variable Byte Integer"),
+                 [PacketTypes.PUBLISH, PacketTypes.SUBSCRIBE]),
+            17: (self.types.index("Four Byte Integer"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK, PacketTypes.DISCONNECT]),
+            18: (self.types.index("UTF-8 Encoded String"), [PacketTypes.CONNACK]),
+            19: (self.types.index("Two Byte Integer"), [PacketTypes.CONNACK]),
+            21: (self.types.index("UTF-8 Encoded String"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK, PacketTypes.AUTH]),
+            22: (self.types.index("Binary Data"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK, PacketTypes.AUTH]),
+            23: (self.types.index("Byte"),
+                 [PacketTypes.CONNECT]),
+            24: (self.types.index("Four Byte Integer"), [PacketTypes.WILLMESSAGE]),
+            25: (self.types.index("Byte"), [PacketTypes.CONNECT]),
+            26: (self.types.index("UTF-8 Encoded String"), [PacketTypes.CONNACK]),
+            28: (self.types.index("UTF-8 Encoded String"),
+                 [PacketTypes.CONNACK, PacketTypes.DISCONNECT]),
+            31: (self.types.index("UTF-8 Encoded String"),
+                 [PacketTypes.CONNACK, PacketTypes.PUBACK, PacketTypes.PUBREC,
+                  PacketTypes.PUBREL, PacketTypes.PUBCOMP, PacketTypes.SUBACK,
+                  PacketTypes.UNSUBACK, PacketTypes.DISCONNECT, PacketTypes.AUTH]),
+            33: (self.types.index("Two Byte Integer"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK]),
+            34: (self.types.index("Two Byte Integer"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK]),
+            35: (self.types.index("Two Byte Integer"), [PacketTypes.PUBLISH]),
+            36: (self.types.index("Byte"), [PacketTypes.CONNACK]),
+            37: (self.types.index("Byte"), [PacketTypes.CONNACK]),
+            38: (self.types.index("UTF-8 String Pair"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK,
+                  PacketTypes.PUBLISH, PacketTypes.PUBACK,
+                  PacketTypes.PUBREC, PacketTypes.PUBREL, PacketTypes.PUBCOMP,
+                  PacketTypes.SUBSCRIBE, PacketTypes.SUBACK,
+                  PacketTypes.UNSUBSCRIBE, PacketTypes.UNSUBACK,
+                  PacketTypes.DISCONNECT, PacketTypes.AUTH, PacketTypes.WILLMESSAGE]),
+            39: (self.types.index("Four Byte Integer"),
+                 [PacketTypes.CONNECT, PacketTypes.CONNACK]),
+            40: (self.types.index("Byte"), [PacketTypes.CONNACK]),
+            41: (self.types.index("Byte"), [PacketTypes.CONNACK]),
+            42: (self.types.index("Byte"), [PacketTypes.CONNACK]),
+        }
+
+    def allowsMultiple(self, compressedName):
+        return self.getIdentFromName(compressedName) in [11, 38]
+
+    def getIdentFromName(self, compressedName):
+        # return the identifier corresponding to the property name
+        result = -1
+        for name in self.names.keys():
+            if compressedName == name.replace(' ', ''):
+                result = self.names[name]
+                break
+        return result
+
+    def __setattr__(self, name, value):
+        name = name.replace(' ', '')
+        privateVars = ["packetType", "types", "names", "properties"]
+        if name in privateVars:
+            object.__setattr__(self, name, value)
+        else:
+            # the name could have spaces in, or not.  Remove spaces before assignment
+            if name not in [aname.replace(' ', '') for aname in self.names.keys()]:
+                raise MQTTException(
+                    f"Property name must be one of {self.names.keys()}")
+            # check that this attribute applies to the packet type
+            if self.packetType not in self.properties[self.getIdentFromName(name)][1]:
+                raise MQTTException(f"Property {name} does not apply to packet type {PacketTypes.Names[self.packetType]}")
+
+            # Check for forbidden values
+            if not isinstance(value, list):
+                if name in ["ReceiveMaximum", "TopicAlias"] \
+                        and (value < 1 or value > 65535):
+
+                    raise MQTTException(f"{name} property value must be in the range 1-65535")
+                elif name in ["TopicAliasMaximum"] \
+                        and (value < 0 or value > 65535):
+
+                    raise MQTTException(f"{name} property value must be in the range 0-65535")
+                elif name in ["MaximumPacketSize", "SubscriptionIdentifier"] \
+                        and (value < 1 or value > 268435455):
+
+                    raise MQTTException(f"{name} property value must be in the range 1-268435455")
+                elif name in ["RequestResponseInformation", "RequestProblemInformation", "PayloadFormatIndicator"] \
+                        and (value != 0 and value != 1):
+
+                    raise MQTTException(
+                        f"{name} property value must be 0 or 1")
+
+            if self.allowsMultiple(name):
+                if not isinstance(value, list):
+                    value = [value]
+                if hasattr(self, name):
+                    value = object.__getattribute__(self, name) + value
+            object.__setattr__(self, name, value)
+
+    def __str__(self):
+        buffer = "["
+        first = True
+        for name in self.names.keys():
+            compressedName = name.replace(' ', '')
+            if hasattr(self, compressedName):
+                if not first:
+                    buffer += ", "
+                buffer += f"{compressedName} : {getattr(self, compressedName)}"
+                first = False
+        buffer += "]"
+        return buffer
+
+    def json(self):
+        data = {}
+        for name in self.names.keys():
+            compressedName = name.replace(' ', '')
+            if hasattr(self, compressedName):
+                val = getattr(self, compressedName)
+                if compressedName == 'CorrelationData' and isinstance(val, bytes):
+                    data[compressedName] = val.hex()
+                else:
+                    data[compressedName] = val
+        return data
+
+    def isEmpty(self):
+        rc = True
+        for name in self.names.keys():
+            compressedName = name.replace(' ', '')
+            if hasattr(self, compressedName):
+                rc = False
+                break
+        return rc
+
+    def clear(self):
+        for name in self.names.keys():
+            compressedName = name.replace(' ', '')
+            if hasattr(self, compressedName):
+                delattr(self, compressedName)
+
+    def writeProperty(self, identifier, type, value):
+        buffer = b""
+        buffer += VariableByteIntegers.encode(identifier)  # identifier
+        if type == self.types.index("Byte"):  # value
+            buffer += bytes([value])
+        elif type == self.types.index("Two Byte Integer"):
+            buffer += writeInt16(value)
+        elif type == self.types.index("Four Byte Integer"):
+            buffer += writeInt32(value)
+        elif type == self.types.index("Variable Byte Integer"):
+            buffer += VariableByteIntegers.encode(value)
+        elif type == self.types.index("Binary Data"):
+            buffer += writeBytes(value)
+        elif type == self.types.index("UTF-8 Encoded String"):
+            buffer += writeUTF(value)
+        elif type == self.types.index("UTF-8 String Pair"):
+            buffer += writeUTF(value[0]) + writeUTF(value[1])
+        return buffer
+
+    def pack(self):
+        # serialize properties into buffer for sending over network
+        buffer = b""
+        for name in self.names.keys():
+            compressedName = name.replace(' ', '')
+            if hasattr(self, compressedName):
+                identifier = self.getIdentFromName(compressedName)
+                attr_type = self.properties[identifier][0]
+                if self.allowsMultiple(compressedName):
+                    for prop in getattr(self, compressedName):
+                        buffer += self.writeProperty(identifier,
+                                                     attr_type, prop)
+                else:
+                    buffer += self.writeProperty(identifier, attr_type,
+                                                 getattr(self, compressedName))
+        return VariableByteIntegers.encode(len(buffer)) + buffer
+
+    def readProperty(self, buffer, type, propslen):
+        if type == self.types.index("Byte"):
+            value = buffer[0]
+            valuelen = 1
+        elif type == self.types.index("Two Byte Integer"):
+            value = readInt16(buffer)
+            valuelen = 2
+        elif type == self.types.index("Four Byte Integer"):
+            value = readInt32(buffer)
+            valuelen = 4
+        elif type == self.types.index("Variable Byte Integer"):
+            value, valuelen = VariableByteIntegers.decode(buffer)
+        elif type == self.types.index("Binary Data"):
+            value, valuelen = readBytes(buffer)
+        elif type == self.types.index("UTF-8 Encoded String"):
+            value, valuelen = readUTF(buffer, propslen)
+        elif type == self.types.index("UTF-8 String Pair"):
+            value, valuelen = readUTF(buffer, propslen)
+            buffer = buffer[valuelen:]  # strip the bytes used by the value
+            value1, valuelen1 = readUTF(buffer, propslen - valuelen)
+            value = (value, value1)
+            valuelen += valuelen1
+        return value, valuelen
+
+    def getNameFromIdent(self, identifier):
+        rc = None
+        for name in self.names:
+            if self.names[name] == identifier:
+                rc = name
+        return rc
+
+    def unpack(self, buffer):
+        self.clear()
+        # deserialize properties into attributes from buffer received from network
+        propslen, VBIlen = VariableByteIntegers.decode(buffer)
+        buffer = buffer[VBIlen:]  # strip the bytes used by the VBI
+        propslenleft = propslen
+        while propslenleft > 0:  # properties length is 0 if there are none
+            identifier, VBIlen2 = VariableByteIntegers.decode(
+                buffer)  # property identifier
+            buffer = buffer[VBIlen2:]  # strip the bytes used by the VBI
+            propslenleft -= VBIlen2
+            attr_type = self.properties[identifier][0]
+            value, valuelen = self.readProperty(
+                buffer, attr_type, propslenleft)
+            buffer = buffer[valuelen:]  # strip the bytes used by the value
+            propslenleft -= valuelen
+            propname = self.getNameFromIdent(identifier)
+            compressedName = propname.replace(' ', '')
+            if not self.allowsMultiple(compressedName) and hasattr(self, compressedName):
+                raise MQTTException(
+                    f"Property '{property}' must not exist more than once")
+            setattr(self, propname, value)
+        return self, propslen + VBIlen

sbapp/pmqtt/publish.py

@@ -0,0 +1,306 @@
+# Copyright (c) 2014 Roger Light <roger@atchoo.org>
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v2.0
+# and Eclipse Distribution License v1.0 which accompany this distribution.
+#
+# The Eclipse Public License is available at
+#    http://www.eclipse.org/legal/epl-v20.html
+# and the Eclipse Distribution License is available at
+#   http://www.eclipse.org/org/documents/edl-v10.php.
+#
+# Contributors:
+#    Roger Light - initial API and implementation
+
+"""
+This module provides some helper functions to allow straightforward publishing
+of messages in a one-shot manner. In other words, they are useful for the
+situation where you have a single/multiple messages you want to publish to a
+broker, then disconnect and nothing else is required.
+"""
+from __future__ import annotations
+
+import collections
+from collections.abc import Iterable
+from typing import TYPE_CHECKING, Any, List, Tuple, Union
+
+from .enums import CallbackAPIVersion, MQTTProtocolVersion
+from .properties import Properties
+from .reasoncodes import ReasonCode
+
+from .. import mqtt
+from . import client as paho
+
+if TYPE_CHECKING:
+    try:
+        from typing import NotRequired, Required, TypedDict  # type: ignore
+    except ImportError:
+        from typing_extensions import NotRequired, Required, TypedDict
+
+    try:
+        from typing import Literal
+    except ImportError:
+        from typing_extensions import Literal  # type: ignore
+
+
+
+    class AuthParameter(TypedDict, total=False):
+        username: Required[str]
+        password: NotRequired[str]
+
+
+    class TLSParameter(TypedDict, total=False):
+        ca_certs: Required[str]
+        certfile: NotRequired[str]
+        keyfile: NotRequired[str]
+        tls_version: NotRequired[int]
+        ciphers: NotRequired[str]
+        insecure: NotRequired[bool]
+
+
+    class MessageDict(TypedDict, total=False):
+        topic: Required[str]
+        payload: NotRequired[paho.PayloadType]
+        qos: NotRequired[int]
+        retain: NotRequired[bool]
+
+    MessageTuple = Tuple[str, paho.PayloadType, int, bool]
+
+    MessagesList = List[Union[MessageDict, MessageTuple]]
+
+
+def _do_publish(client: paho.Client):
+    """Internal function"""
+
+    message = client._userdata.popleft()
+
+    if isinstance(message, dict):
+        client.publish(**message)
+    elif isinstance(message, (tuple, list)):
+        client.publish(*message)
+    else:
+        raise TypeError('message must be a dict, tuple, or list')
+
+
+def _on_connect(client: paho.Client, userdata: MessagesList, flags, reason_code, properties):
+    """Internal v5 callback"""
+    if reason_code == 0:
+        if len(userdata) > 0:
+            _do_publish(client)
+    else:
+        raise mqtt.MQTTException(paho.connack_string(reason_code))
+
+
+def _on_publish(
+    client: paho.Client, userdata: collections.deque[MessagesList], mid: int, reason_codes: ReasonCode, properties: Properties,
+) -> None:
+    """Internal callback"""
+    #pylint: disable=unused-argument
+
+    if len(userdata) == 0:
+        client.disconnect()
+    else:
+        _do_publish(client)
+
+
+def multiple(
+    msgs: MessagesList,
+    hostname: str = "localhost",
+    port: int = 1883,
+    client_id: str = "",
+    keepalive: int = 60,
+    will: MessageDict | None = None,
+    auth: AuthParameter | None = None,
+    tls: TLSParameter | None = None,
+    protocol: MQTTProtocolVersion = paho.MQTTv311,
+    transport: Literal["tcp", "websockets"] = "tcp",
+    proxy_args: Any | None = None,
+) -> None:
+    """Publish multiple messages to a broker, then disconnect cleanly.
+
+    This function creates an MQTT client, connects to a broker and publishes a
+    list of messages. Once the messages have been delivered, it disconnects
+    cleanly from the broker.
+
+    :param msgs: a list of messages to publish. Each message is either a dict or a
+           tuple.
+
+           If a dict, only the topic must be present. Default values will be
+           used for any missing arguments. The dict must be of the form:
+
+           msg = {'topic':"<topic>", 'payload':"<payload>", 'qos':<qos>,
+           'retain':<retain>}
+           topic must be present and may not be empty.
+           If payload is "", None or not present then a zero length payload
+           will be published.
+           If qos is not present, the default of 0 is used.
+           If retain is not present, the default of False is used.
+
+           If a tuple, then it must be of the form:
+           ("<topic>", "<payload>", qos, retain)
+
+    :param str hostname: the address of the broker to connect to.
+               Defaults to localhost.
+
+    :param int port: the port to connect to the broker on. Defaults to 1883.
+
+    :param str client_id: the MQTT client id to use. If "" or None, the Paho library will
+                generate a client id automatically.
+
+    :param int keepalive: the keepalive timeout value for the client. Defaults to 60
+                seconds.
+
+    :param will: a dict containing will parameters for the client: will = {'topic':
+           "<topic>", 'payload':"<payload">, 'qos':<qos>, 'retain':<retain>}.
+           Topic is required, all other parameters are optional and will
+           default to None, 0 and False respectively.
+           Defaults to None, which indicates no will should be used.
+
+    :param auth: a dict containing authentication parameters for the client:
+           auth = {'username':"<username>", 'password':"<password>"}
+           Username is required, password is optional and will default to None
+           if not provided.
+           Defaults to None, which indicates no authentication is to be used.
+
+    :param tls: a dict containing TLS configuration parameters for the client:
+          dict = {'ca_certs':"<ca_certs>", 'certfile':"<certfile>",
+          'keyfile':"<keyfile>", 'tls_version':"<tls_version>",
+          'ciphers':"<ciphers">, 'insecure':"<bool>"}
+          ca_certs is required, all other parameters are optional and will
+          default to None if not provided, which results in the client using
+          the default behaviour - see the paho.mqtt.client documentation.
+          Alternatively, tls input can be an SSLContext object, which will be
+          processed using the tls_set_context method.
+          Defaults to None, which indicates that TLS should not be used.
+
+    :param str transport: set to "tcp" to use the default setting of transport which is
+          raw TCP. Set to "websockets" to use WebSockets as the transport.
+
+    :param proxy_args: a dictionary that will be given to the client.
+    """
+
+    if not isinstance(msgs, Iterable):
+        raise TypeError('msgs must be an iterable')
+    if len(msgs) == 0:
+        raise ValueError('msgs is empty')
+
+    client = paho.Client(
+        CallbackAPIVersion.VERSION2,
+        client_id=client_id,
+        userdata=collections.deque(msgs),
+        protocol=protocol,
+        transport=transport,
+    )
+
+    client.enable_logger()
+    client.on_publish = _on_publish
+    client.on_connect = _on_connect  # type: ignore
+
+    if proxy_args is not None:
+        client.proxy_set(**proxy_args)
+
+    if auth:
+        username = auth.get('username')
+        if username:
+            password = auth.get('password')
+            client.username_pw_set(username, password)
+        else:
+            raise KeyError("The 'username' key was not found, this is "
+                           "required for auth")
+
+    if will is not None:
+        client.will_set(**will)
+
+    if tls is not None:
+        if isinstance(tls, dict):
+            insecure = tls.pop('insecure', False)
+            # mypy don't get that tls no longer contains the key insecure
+            client.tls_set(**tls)  # type: ignore[misc]
+            if insecure:
+                # Must be set *after* the `client.tls_set()` call since it sets
+                # up the SSL context that `client.tls_insecure_set` alters.
+                client.tls_insecure_set(insecure)
+        else:
+            # Assume input is SSLContext object
+            client.tls_set_context(tls)
+
+    client.connect(hostname, port, keepalive)
+    client.loop_forever()
+
+
+def single(
+    topic: str,
+    payload: paho.PayloadType = None,
+    qos: int = 0,
+    retain: bool = False,
+    hostname: str = "localhost",
+    port: int = 1883,
+    client_id: str = "",
+    keepalive: int = 60,
+    will: MessageDict | None = None,
+    auth: AuthParameter | None = None,
+    tls: TLSParameter | None = None,
+    protocol: MQTTProtocolVersion = paho.MQTTv311,
+    transport: Literal["tcp", "websockets"] = "tcp",
+    proxy_args: Any | None = None,
+) -> None:
+    """Publish a single message to a broker, then disconnect cleanly.
+
+    This function creates an MQTT client, connects to a broker and publishes a
+    single message. Once the message has been delivered, it disconnects cleanly
+    from the broker.
+
+    :param str topic: the only required argument must be the topic string to which the
+            payload will be published.
+
+    :param payload: the payload to be published. If "" or None, a zero length payload
+              will be published.
+
+    :param int qos: the qos to use when publishing,  default to 0.
+
+    :param bool retain: set the message to be retained (True) or not (False).
+
+    :param str hostname: the address of the broker to connect to.
+               Defaults to localhost.
+
+    :param int port: the port to connect to the broker on. Defaults to 1883.
+
+    :param str client_id: the MQTT client id to use. If "" or None, the Paho library will
+                generate a client id automatically.
+
+    :param int keepalive: the keepalive timeout value for the client. Defaults to 60
+                seconds.
+
+    :param will: a dict containing will parameters for the client: will = {'topic':
+           "<topic>", 'payload':"<payload">, 'qos':<qos>, 'retain':<retain>}.
+           Topic is required, all other parameters are optional and will
+           default to None, 0 and False respectively.
+           Defaults to None, which indicates no will should be used.
+
+    :param auth: a dict containing authentication parameters for the client:
+           Username is required, password is optional and will default to None
+           auth = {'username':"<username>", 'password':"<password>"}
+           if not provided.
+           Defaults to None, which indicates no authentication is to be used.
+
+    :param tls: a dict containing TLS configuration parameters for the client:
+          dict = {'ca_certs':"<ca_certs>", 'certfile':"<certfile>",
+          'keyfile':"<keyfile>", 'tls_version':"<tls_version>",
+          'ciphers':"<ciphers">, 'insecure':"<bool>"}
+          ca_certs is required, all other parameters are optional and will
+          default to None if not provided, which results in the client using
+          the default behaviour - see the paho.mqtt.client documentation.
+          Defaults to None, which indicates that TLS should not be used.
+          Alternatively, tls input can be an SSLContext object, which will be
+          processed using the tls_set_context method.
+
+    :param transport: set to "tcp" to use the default setting of transport which is
+          raw TCP. Set to "websockets" to use WebSockets as the transport.
+
+    :param proxy_args: a dictionary that will be given to the client.
+    """
+
+    msg: MessageDict = {'topic':topic, 'payload':payload, 'qos':qos, 'retain':retain}
+
+    multiple([msg], hostname, port, client_id, keepalive, will, auth, tls,
+             protocol, transport, proxy_args)

sbapp/pmqtt/reasoncodes.py

@@ -0,0 +1,223 @@
+# *******************************************************************
+#   Copyright (c) 2017, 2019 IBM Corp.
+#
+#   All rights reserved. This program and the accompanying materials
+#   are made available under the terms of the Eclipse Public License v2.0
+#   and Eclipse Distribution License v1.0 which accompany this distribution.
+#
+#   The Eclipse Public License is available at
+#      http://www.eclipse.org/legal/epl-v20.html
+#   and the Eclipse Distribution License is available at
+#     http://www.eclipse.org/org/documents/edl-v10.php.
+#
+#   Contributors:
+#      Ian Craggs - initial implementation and/or documentation
+# *******************************************************************
+
+import functools
+import warnings
+from typing import Any
+
+from .packettypes import PacketTypes
+
+
+@functools.total_ordering
+class ReasonCode:
+    """MQTT version 5.0 reason codes class.
+
+    See ReasonCode.names for a list of possible numeric values along with their
+    names and the packets to which they apply.
+
+    """
+
+    def __init__(self, packetType: int, aName: str ="Success", identifier: int =-1):
+        """
+        packetType: the type of the packet, such as PacketTypes.CONNECT that
+            this reason code will be used with.  Some reason codes have different
+            names for the same identifier when used a different packet type.
+
+        aName: the String name of the reason code to be created.  Ignored
+            if the identifier is set.
+
+        identifier: an integer value of the reason code to be created.
+
+        """
+
+        self.packetType = packetType
+        self.names = {
+            0: {"Success": [PacketTypes.CONNACK, PacketTypes.PUBACK,
+                            PacketTypes.PUBREC, PacketTypes.PUBREL, PacketTypes.PUBCOMP,
+                            PacketTypes.UNSUBACK, PacketTypes.AUTH],
+                "Normal disconnection": [PacketTypes.DISCONNECT],
+                "Granted QoS 0": [PacketTypes.SUBACK]},
+            1: {"Granted QoS 1": [PacketTypes.SUBACK]},
+            2: {"Granted QoS 2": [PacketTypes.SUBACK]},
+            4: {"Disconnect with will message": [PacketTypes.DISCONNECT]},
+            16: {"No matching subscribers":
+                 [PacketTypes.PUBACK, PacketTypes.PUBREC]},
+            17: {"No subscription found": [PacketTypes.UNSUBACK]},
+            24: {"Continue authentication": [PacketTypes.AUTH]},
+            25: {"Re-authenticate": [PacketTypes.AUTH]},
+            128: {"Unspecified error": [PacketTypes.CONNACK, PacketTypes.PUBACK,
+                                        PacketTypes.PUBREC, PacketTypes.SUBACK, PacketTypes.UNSUBACK,
+                                        PacketTypes.DISCONNECT], },
+            129: {"Malformed packet":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            130: {"Protocol error":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            131: {"Implementation specific error": [PacketTypes.CONNACK,
+                                                    PacketTypes.PUBACK, PacketTypes.PUBREC, PacketTypes.SUBACK,
+                                                    PacketTypes.UNSUBACK, PacketTypes.DISCONNECT], },
+            132: {"Unsupported protocol version": [PacketTypes.CONNACK]},
+            133: {"Client identifier not valid": [PacketTypes.CONNACK]},
+            134: {"Bad user name or password": [PacketTypes.CONNACK]},
+            135: {"Not authorized": [PacketTypes.CONNACK, PacketTypes.PUBACK,
+                                     PacketTypes.PUBREC, PacketTypes.SUBACK, PacketTypes.UNSUBACK,
+                                     PacketTypes.DISCONNECT], },
+            136: {"Server unavailable": [PacketTypes.CONNACK]},
+            137: {"Server busy": [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            138: {"Banned": [PacketTypes.CONNACK]},
+            139: {"Server shutting down": [PacketTypes.DISCONNECT]},
+            140: {"Bad authentication method":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            141: {"Keep alive timeout": [PacketTypes.DISCONNECT]},
+            142: {"Session taken over": [PacketTypes.DISCONNECT]},
+            143: {"Topic filter invalid":
+                  [PacketTypes.SUBACK, PacketTypes.UNSUBACK, PacketTypes.DISCONNECT]},
+            144: {"Topic name invalid":
+                  [PacketTypes.CONNACK, PacketTypes.PUBACK,
+                   PacketTypes.PUBREC, PacketTypes.DISCONNECT]},
+            145: {"Packet identifier in use":
+                  [PacketTypes.PUBACK, PacketTypes.PUBREC,
+                   PacketTypes.SUBACK, PacketTypes.UNSUBACK]},
+            146: {"Packet identifier not found":
+                  [PacketTypes.PUBREL, PacketTypes.PUBCOMP]},
+            147: {"Receive maximum exceeded": [PacketTypes.DISCONNECT]},
+            148: {"Topic alias invalid": [PacketTypes.DISCONNECT]},
+            149: {"Packet too large": [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            150: {"Message rate too high": [PacketTypes.DISCONNECT]},
+            151: {"Quota exceeded": [PacketTypes.CONNACK, PacketTypes.PUBACK,
+                                     PacketTypes.PUBREC, PacketTypes.SUBACK, PacketTypes.DISCONNECT], },
+            152: {"Administrative action": [PacketTypes.DISCONNECT]},
+            153: {"Payload format invalid":
+                  [PacketTypes.PUBACK, PacketTypes.PUBREC, PacketTypes.DISCONNECT]},
+            154: {"Retain not supported":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            155: {"QoS not supported":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            156: {"Use another server":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            157: {"Server moved":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            158: {"Shared subscription not supported":
+                  [PacketTypes.SUBACK, PacketTypes.DISCONNECT]},
+            159: {"Connection rate exceeded":
+                  [PacketTypes.CONNACK, PacketTypes.DISCONNECT]},
+            160: {"Maximum connect time":
+                  [PacketTypes.DISCONNECT]},
+            161: {"Subscription identifiers not supported":
+                  [PacketTypes.SUBACK, PacketTypes.DISCONNECT]},
+            162: {"Wildcard subscription not supported":
+                  [PacketTypes.SUBACK, PacketTypes.DISCONNECT]},
+        }
+        if identifier == -1:
+            if packetType == PacketTypes.DISCONNECT and aName == "Success":
+                aName = "Normal disconnection"
+            self.set(aName)
+        else:
+            self.value = identifier
+            self.getName()  # check it's good
+
+    def __getName__(self, packetType, identifier):
+        """
+        Get the reason code string name for a specific identifier.
+        The name can vary by packet type for the same identifier, which
+        is why the packet type is also required.
+
+        Used when displaying the reason code.
+        """
+        if identifier not in self.names:
+            raise KeyError(identifier)
+        names = self.names[identifier]
+        namelist = [name for name in names.keys() if packetType in names[name]]
+        if len(namelist) != 1:
+            raise ValueError(f"Expected exactly one name, found {namelist!r}")
+        return namelist[0]
+
+    def getId(self, name):
+        """
+        Get the numeric id corresponding to a reason code name.
+
+        Used when setting the reason code for a packetType
+        check that only valid codes for the packet are set.
+        """
+        for code in self.names.keys():
+            if name in self.names[code].keys():
+                if self.packetType in self.names[code][name]:
+                    return code
+        raise KeyError(f"Reason code name not found: {name}")
+
+    def set(self, name):
+        self.value = self.getId(name)
+
+    def unpack(self, buffer):
+        c = buffer[0]
+        name = self.__getName__(self.packetType, c)
+        self.value = self.getId(name)
+        return 1
+
+    def getName(self):
+        """Returns the reason code name corresponding to the numeric value which is set.
+        """
+        return self.__getName__(self.packetType, self.value)
+
+    def __eq__(self, other):
+        if isinstance(other, int):
+            return self.value == other
+        if isinstance(other, str):
+            return other == str(self)
+        if isinstance(other, ReasonCode):
+            return self.value == other.value
+        return False
+
+    def __lt__(self, other):
+        if isinstance(other, int):
+            return self.value < other
+        if isinstance(other, ReasonCode):
+            return self.value < other.value
+        return NotImplemented
+
+    def __repr__(self):
+        try:
+            packet_name = PacketTypes.Names[self.packetType]
+        except IndexError:
+            packet_name = "Unknown"
+
+        return f"ReasonCode({packet_name}, {self.getName()!r})"
+
+    def __str__(self):
+        return self.getName()
+
+    def json(self):
+        return self.getName()
+
+    def pack(self):
+        return bytearray([self.value])
+
+    @property
+    def is_failure(self) -> bool:
+        return self.value >= 0x80
+
+
+class _CompatibilityIsInstance(type):
+    def __instancecheck__(self, other: Any) -> bool:
+        return isinstance(other, ReasonCode)
+
+
+class ReasonCodes(ReasonCode, metaclass=_CompatibilityIsInstance):
+    def __init__(self, *args, **kwargs):
+        warnings.warn("ReasonCodes is deprecated, use ReasonCode (singular) instead",
+            category=DeprecationWarning,
+            stacklevel=2,
+        )
+        super().__init__(*args, **kwargs)

sbapp/pmqtt/subscribe.py

@@ -0,0 +1,281 @@
+# Copyright (c) 2016 Roger Light <roger@atchoo.org>
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v2.0
+# and Eclipse Distribution License v1.0 which accompany this distribution.
+#
+# The Eclipse Public License is available at
+#    http://www.eclipse.org/legal/epl-v20.html
+# and the Eclipse Distribution License is available at
+#   http://www.eclipse.org/org/documents/edl-v10.php.
+#
+# Contributors:
+#    Roger Light - initial API and implementation
+
+"""
+This module provides some helper functions to allow straightforward subscribing
+to topics and retrieving messages. The two functions are simple(), which
+returns one or messages matching a set of topics, and callback() which allows
+you to pass a callback for processing of messages.
+"""
+
+from .. import mqtt
+from . import client as paho
+
+
+def _on_connect(client, userdata, flags, reason_code, properties):
+    """Internal callback"""
+    if reason_code != 0:
+        raise mqtt.MQTTException(paho.connack_string(reason_code))
+
+    if isinstance(userdata['topics'], list):
+        for topic in userdata['topics']:
+            client.subscribe(topic, userdata['qos'])
+    else:
+        client.subscribe(userdata['topics'], userdata['qos'])
+
+
+def _on_message_callback(client, userdata, message):
+    """Internal callback"""
+    userdata['callback'](client, userdata['userdata'], message)
+
+
+def _on_message_simple(client, userdata, message):
+    """Internal callback"""
+
+    if userdata['msg_count'] == 0:
+        return
+
+    # Don't process stale retained messages if 'retained' was false
+    if message.retain and not userdata['retained']:
+        return
+
+    userdata['msg_count'] = userdata['msg_count'] - 1
+
+    if userdata['messages'] is None and userdata['msg_count'] == 0:
+        userdata['messages'] = message
+        client.disconnect()
+        return
+
+    userdata['messages'].append(message)
+    if userdata['msg_count'] == 0:
+        client.disconnect()
+
+
+def callback(callback, topics, qos=0, userdata=None, hostname="localhost",
+             port=1883, client_id="", keepalive=60, will=None, auth=None,
+             tls=None, protocol=paho.MQTTv311, transport="tcp",
+             clean_session=True, proxy_args=None):
+    """Subscribe to a list of topics and process them in a callback function.
+
+    This function creates an MQTT client, connects to a broker and subscribes
+    to a list of topics. Incoming messages are processed by the user provided
+    callback.  This is a blocking function and will never return.
+
+    :param callback: function with the same signature as `on_message` for
+               processing the messages received.
+
+    :param topics: either a string containing a single topic to subscribe to, or a
+             list of topics to subscribe to.
+
+    :param int qos: the qos to use when subscribing. This is applied to all topics.
+
+    :param userdata: passed to the callback
+
+    :param str hostname: the address of the broker to connect to.
+               Defaults to localhost.
+
+    :param int port: the port to connect to the broker on. Defaults to 1883.
+
+    :param str client_id: the MQTT client id to use. If "" or None, the Paho library will
+                generate a client id automatically.
+
+    :param int keepalive: the keepalive timeout value for the client. Defaults to 60
+                seconds.
+
+    :param will: a dict containing will parameters for the client: will = {'topic':
+           "<topic>", 'payload':"<payload">, 'qos':<qos>, 'retain':<retain>}.
+           Topic is required, all other parameters are optional and will
+           default to None, 0 and False respectively.
+
+           Defaults to None, which indicates no will should be used.
+
+    :param auth: a dict containing authentication parameters for the client:
+           auth = {'username':"<username>", 'password':"<password>"}
+           Username is required, password is optional and will default to None
+           if not provided.
+           Defaults to None, which indicates no authentication is to be used.
+
+    :param tls: a dict containing TLS configuration parameters for the client:
+          dict = {'ca_certs':"<ca_certs>", 'certfile':"<certfile>",
+          'keyfile':"<keyfile>", 'tls_version':"<tls_version>",
+          'ciphers':"<ciphers">, 'insecure':"<bool>"}
+          ca_certs is required, all other parameters are optional and will
+          default to None if not provided, which results in the client using
+          the default behaviour - see the paho.mqtt.client documentation.
+          Alternatively, tls input can be an SSLContext object, which will be
+          processed using the tls_set_context method.
+          Defaults to None, which indicates that TLS should not be used.
+
+    :param str transport: set to "tcp" to use the default setting of transport which is
+          raw TCP. Set to "websockets" to use WebSockets as the transport.
+
+    :param clean_session: a boolean that determines the client type. If True,
+                    the broker will remove all information about this client
+                    when it disconnects. If False, the client is a persistent
+                    client and subscription information and queued messages
+                    will be retained when the client disconnects.
+                    Defaults to True.
+
+    :param proxy_args: a dictionary that will be given to the client.
+    """
+
+    if qos < 0 or qos > 2:
+        raise ValueError('qos must be in the range 0-2')
+
+    callback_userdata = {
+        'callback':callback,
+        'topics':topics,
+        'qos':qos,
+        'userdata':userdata}
+
+    client = paho.Client(
+        paho.CallbackAPIVersion.VERSION2,
+        client_id=client_id,
+        userdata=callback_userdata,
+        protocol=protocol,
+        transport=transport,
+        clean_session=clean_session,
+    )
+    client.enable_logger()
+
+    client.on_message = _on_message_callback
+    client.on_connect = _on_connect
+
+    if proxy_args is not None:
+        client.proxy_set(**proxy_args)
+
+    if auth:
+        username = auth.get('username')
+        if username:
+            password = auth.get('password')
+            client.username_pw_set(username, password)
+        else:
+            raise KeyError("The 'username' key was not found, this is "
+                           "required for auth")
+
+    if will is not None:
+        client.will_set(**will)
+
+    if tls is not None:
+        if isinstance(tls, dict):
+            insecure = tls.pop('insecure', False)
+            client.tls_set(**tls)
+            if insecure:
+                # Must be set *after* the `client.tls_set()` call since it sets
+                # up the SSL context that `client.tls_insecure_set` alters.
+                client.tls_insecure_set(insecure)
+        else:
+            # Assume input is SSLContext object
+            client.tls_set_context(tls)
+
+    client.connect(hostname, port, keepalive)
+    client.loop_forever()
+
+
+def simple(topics, qos=0, msg_count=1, retained=True, hostname="localhost",
+           port=1883, client_id="", keepalive=60, will=None, auth=None,
+           tls=None, protocol=paho.MQTTv311, transport="tcp",
+           clean_session=True, proxy_args=None):
+    """Subscribe to a list of topics and return msg_count messages.
+
+    This function creates an MQTT client, connects to a broker and subscribes
+    to a list of topics. Once "msg_count" messages have been received, it
+    disconnects cleanly from the broker and returns the messages.
+
+    :param topics: either a string containing a single topic to subscribe to, or a
+             list of topics to subscribe to.
+
+    :param int qos: the qos to use when subscribing. This is applied to all topics.
+
+    :param int msg_count: the number of messages to retrieve from the broker.
+                if msg_count == 1 then a single MQTTMessage will be returned.
+                if msg_count > 1 then a list of MQTTMessages will be returned.
+
+    :param bool retained: If set to True, retained messages will be processed the same as
+               non-retained messages. If set to False, retained messages will
+               be ignored. This means that with retained=False and msg_count=1,
+               the function will return the first message received that does
+               not have the retained flag set.
+
+    :param str hostname: the address of the broker to connect to.
+               Defaults to localhost.
+
+    :param int port: the port to connect to the broker on. Defaults to 1883.
+
+    :param str client_id: the MQTT client id to use. If "" or None, the Paho library will
+                generate a client id automatically.
+
+    :param int keepalive: the keepalive timeout value for the client. Defaults to 60
+                seconds.
+
+    :param will: a dict containing will parameters for the client: will = {'topic':
+           "<topic>", 'payload':"<payload">, 'qos':<qos>, 'retain':<retain>}.
+           Topic is required, all other parameters are optional and will
+           default to None, 0 and False respectively.
+           Defaults to None, which indicates no will should be used.
+
+    :param auth: a dict containing authentication parameters for the client:
+           auth = {'username':"<username>", 'password':"<password>"}
+           Username is required, password is optional and will default to None
+           if not provided.
+           Defaults to None, which indicates no authentication is to be used.
+
+    :param tls: a dict containing TLS configuration parameters for the client:
+          dict = {'ca_certs':"<ca_certs>", 'certfile':"<certfile>",
+          'keyfile':"<keyfile>", 'tls_version':"<tls_version>",
+          'ciphers':"<ciphers">, 'insecure':"<bool>"}
+          ca_certs is required, all other parameters are optional and will
+          default to None if not provided, which results in the client using
+          the default behaviour - see the paho.mqtt.client documentation.
+          Alternatively, tls input can be an SSLContext object, which will be
+          processed using the tls_set_context method.
+          Defaults to None, which indicates that TLS should not be used.
+
+    :param protocol: the MQTT protocol version to use. Defaults to MQTTv311.
+
+    :param transport: set to "tcp" to use the default setting of transport which is
+          raw TCP. Set to "websockets" to use WebSockets as the transport.
+
+    :param clean_session: a boolean that determines the client type. If True,
+                    the broker will remove all information about this client
+                    when it disconnects. If False, the client is a persistent
+                    client and subscription information and queued messages
+                    will be retained when the client disconnects.
+                    Defaults to True. If protocol is MQTTv50, clean_session
+                    is ignored.
+
+    :param proxy_args: a dictionary that will be given to the client.
+    """
+
+    if msg_count < 1:
+        raise ValueError('msg_count must be > 0')
+
+    # Set ourselves up to return a single message if msg_count == 1, or a list
+    # if > 1.
+    if msg_count == 1:
+        messages = None
+    else:
+        messages = []
+
+    # Ignore clean_session if protocol is MQTTv50, otherwise Client will raise
+    if protocol == paho.MQTTv5:
+        clean_session = None
+
+    userdata = {'retained':retained, 'msg_count':msg_count, 'messages':messages}
+
+    callback(_on_message_simple, topics, qos, userdata, hostname, port,
+             client_id, keepalive, will, auth, tls, protocol, transport,
+             clean_session, proxy_args)
+
+    return userdata['messages']

sbapp/pmqtt/subscribeoptions.py

@@ -0,0 +1,113 @@
+"""
+*******************************************************************
+  Copyright (c) 2017, 2019 IBM Corp.
+
+  All rights reserved. This program and the accompanying materials
+  are made available under the terms of the Eclipse Public License v2.0
+  and Eclipse Distribution License v1.0 which accompany this distribution.
+
+  The Eclipse Public License is available at
+     http://www.eclipse.org/legal/epl-v20.html
+  and the Eclipse Distribution License is available at
+    http://www.eclipse.org/org/documents/edl-v10.php.
+
+  Contributors:
+     Ian Craggs - initial implementation and/or documentation
+*******************************************************************
+"""
+
+
+
+class MQTTException(Exception):
+    pass
+
+
+class SubscribeOptions:
+    """The MQTT v5.0 subscribe options class.
+
+    The options are:
+        qos:                As in MQTT v3.1.1.
+        noLocal:            True or False. If set to True, the subscriber will not receive its own publications.
+        retainAsPublished:  True or False. If set to True, the retain flag on received publications will be as set
+                            by the publisher.
+        retainHandling:     RETAIN_SEND_ON_SUBSCRIBE, RETAIN_SEND_IF_NEW_SUB or RETAIN_DO_NOT_SEND
+                            Controls when the broker should send retained messages:
+                                - RETAIN_SEND_ON_SUBSCRIBE: on any successful subscribe request
+                                - RETAIN_SEND_IF_NEW_SUB: only if the subscribe request is new
+                                - RETAIN_DO_NOT_SEND: never send retained messages
+    """
+
+    # retain handling options
+    RETAIN_SEND_ON_SUBSCRIBE, RETAIN_SEND_IF_NEW_SUB, RETAIN_DO_NOT_SEND = range(
+        0, 3)
+
+    def __init__(
+        self,
+        qos: int = 0,
+        noLocal: bool = False,
+        retainAsPublished: bool = False,
+        retainHandling: int = RETAIN_SEND_ON_SUBSCRIBE,
+    ):
+        """
+        qos:                0, 1 or 2.  0 is the default.
+        noLocal:            True or False. False is the default and corresponds to MQTT v3.1.1 behavior.
+        retainAsPublished:  True or False. False is the default and corresponds to MQTT v3.1.1 behavior.
+        retainHandling:     RETAIN_SEND_ON_SUBSCRIBE, RETAIN_SEND_IF_NEW_SUB or RETAIN_DO_NOT_SEND
+                            RETAIN_SEND_ON_SUBSCRIBE is the default and corresponds to MQTT v3.1.1 behavior.
+        """
+        object.__setattr__(self, "names",
+                           ["QoS", "noLocal", "retainAsPublished", "retainHandling"])
+        self.QoS = qos  # bits 0,1
+        self.noLocal = noLocal  # bit 2
+        self.retainAsPublished = retainAsPublished  # bit 3
+        self.retainHandling = retainHandling  # bits 4 and 5: 0, 1 or 2
+        if self.retainHandling not in (0, 1, 2):
+            raise AssertionError(f"Retain handling should be 0, 1 or 2, not {self.retainHandling}")
+        if self.QoS not in (0, 1, 2):
+            raise AssertionError(f"QoS should be 0, 1 or 2, not {self.QoS}")
+
+    def __setattr__(self, name, value):
+        if name not in self.names:
+            raise MQTTException(
+                f"{name} Attribute name must be one of {self.names}")
+        object.__setattr__(self, name, value)
+
+    def pack(self):
+        if self.retainHandling not in (0, 1, 2):
+            raise AssertionError(f"Retain handling should be 0, 1 or 2, not {self.retainHandling}")
+        if self.QoS not in (0, 1, 2):
+            raise AssertionError(f"QoS should be 0, 1 or 2, not {self.QoS}")
+        noLocal = 1 if self.noLocal else 0
+        retainAsPublished = 1 if self.retainAsPublished else 0
+        data = [(self.retainHandling << 4) | (retainAsPublished << 3) |
+                (noLocal << 2) | self.QoS]
+        return bytes(data)
+
+    def unpack(self, buffer):
+        b0 = buffer[0]
+        self.retainHandling = ((b0 >> 4) & 0x03)
+        self.retainAsPublished = True if ((b0 >> 3) & 0x01) == 1 else False
+        self.noLocal = True if ((b0 >> 2) & 0x01) == 1 else False
+        self.QoS = (b0 & 0x03)
+        if self.retainHandling not in (0, 1, 2):
+            raise AssertionError(f"Retain handling should be 0, 1 or 2, not {self.retainHandling}")
+        if self.QoS not in (0, 1, 2):
+            raise AssertionError(f"QoS should be 0, 1 or 2, not {self.QoS}")
+        return 1
+
+    def __repr__(self):
+        return str(self)
+
+    def __str__(self):
+        return "{QoS="+str(self.QoS)+", noLocal="+str(self.noLocal) +\
+            ", retainAsPublished="+str(self.retainAsPublished) +\
+            ", retainHandling="+str(self.retainHandling)+"}"
+
+    def json(self):
+        data = {
+            "QoS": self.QoS,
+            "noLocal": self.noLocal,
+            "retainAsPublished": self.retainAsPublished,
+            "retainHandling": self.retainHandling,
+        }
+        return data

sbapp/sideband/certgen.py

@@ -47,7 +47,11 @@ def get_key(key_path, force_reload=False):
         return LOADED_KEY
     elif os.path.isfile(KEY_PATH):
         with open(KEY_PATH, "rb") as f:
-            key = load_pem_private_key(f.read(), KEY_PASSPHRASE)
+            if cryptography_major_version > 3:
+                key = load_pem_private_key(f.read(), KEY_PASSPHRASE)
+            else:
+                from cryptography.hazmat.backends import default_backend
+                key = load_pem_private_key(f.read(), KEY_PASSPHRASE, backend=default_backend())
     else:
         if cryptography_major_version > 3:
             key = ec.generate_private_key(curve=ec.SECP256R1())
@@ -87,6 +91,7 @@ def gen_cert(cert_path, key):
     cb = cb.not_valid_before(datetime.datetime.now(datetime.timezone.utc)+datetime.timedelta(days=-14))
     cb = cb.not_valid_after(datetime.datetime.now(datetime.timezone.utc)+datetime.timedelta(days=3652))
     cb = cb.add_extension(x509.SubjectAlternativeName([x509.DNSName("localhost")]), critical=False)
+
     if cryptography_major_version > 3:
         cert = cb.sign(key, hashes.SHA256())
     else:

sbapp/sideband/core.py

@@ -7,6 +7,8 @@ import struct
 import sqlite3
 import random
 import shlex
+import re
+import gc
 
 import RNS.vendor.umsgpack as msgpack
 import RNS.Interfaces.Interface as Interface
@@ -20,6 +22,7 @@ from collections import deque
 from .res import sideband_fb_data
 from .sense import Telemeter, Commands
 from .plugins import SidebandCommandPlugin, SidebandServicePlugin, SidebandTelemetryPlugin
+from .mqtt import MQTT
 
 if RNS.vendor.platformutils.get_platform() == "android":
     import plyer
@@ -64,11 +67,16 @@ class PropagationNodeDetector():
                         # age = 0
                         pass
 
-                    link_stats = {"rssi": self.owner_app.sideband.reticulum.get_packet_rssi(announce_packet_hash),
-                                 "snr": self.owner_app.sideband.reticulum.get_packet_snr(announce_packet_hash),
-                                 "q": self.owner_app.sideband.reticulum.get_packet_q(announce_packet_hash)}
+                    if self.owner_app != None:
+                        stat_endpoint = self.owner_app.sideband
+                    else:
+                        stat_endpoint = self.owner
 
-                    RNS.log("Detected active propagation node "+RNS.prettyhexrep(destination_hash)+" emission "+str(age)+" seconds ago, "+str(hops)+" hops away")
+                    link_stats = {"rssi": stat_endpoint.reticulum.get_packet_rssi(announce_packet_hash),
+                                 "snr": stat_endpoint.reticulum.get_packet_snr(announce_packet_hash),
+                                 "q": stat_endpoint.reticulum.get_packet_q(announce_packet_hash)}
+
+                    RNS.log("Detected active propagation node "+RNS.prettyhexrep(destination_hash)+" emission "+str(age)+" seconds ago, "+str(hops)+" hops away", RNS.LOG_EXTREME)
                     self.owner.log_announce(destination_hash, app_data, dest_type=PropagationNodeDetector.aspect_filter, link_stats=link_stats)
 
                     if self.owner.config["lxmf_propagation_node"] == None:
@@ -84,10 +92,10 @@ class PropagationNodeDetector():
                         pass
 
                 else:
-                    RNS.log(f"Received malformed propagation node announce from {RNS.prettyhexrep(destination_hash)} with data: {app_data}", RNS.LOG_DEBUG)
+                    RNS.log(f"Received malformed propagation node announce from {RNS.prettyhexrep(destination_hash)} with data: {app_data}", RNS.LOG_EXTREME)
 
             else:
-                RNS.log(f"Received malformed propagation node announce from {RNS.prettyhexrep(destination_hash)} with data: {app_data}", RNS.LOG_DEBUG)
+                RNS.log(f"Received malformed propagation node announce from {RNS.prettyhexrep(destination_hash)} with data: {app_data}", RNS.LOG_EXTREME)
 
         except Exception as e:
             RNS.log("Error while processing received propagation node announce: "+str(e))
@@ -100,14 +108,17 @@ class SidebandCore():
     CONV_P2P       = 0x01
     CONV_GROUP     = 0x02
     CONV_BROADCAST = 0x03
+    CONV_VOICE     = 0x04
 
     MAX_ANNOUNCES  = 24
 
     SERVICE_JOB_INTERVAL   = 1
     PERIODIC_JOBS_INTERVAL = 60
     PERIODIC_SYNC_RETRY = 360
+    TELEMETRY_KEEP     = 60*60*24*7
     TELEMETRY_INTERVAL = 60
     SERVICE_TELEMETRY_INTERVAL = 300
+    TELEMETRY_CLEAN_INTERVAL = 3600
 
     IF_CHANGE_ANNOUNCE_MIN_INTERVAL = 3.5  # In seconds
     AUTO_ANNOUNCE_RANDOM_MIN        = 90   # In minutes
@@ -122,19 +133,20 @@ class SidebandCore():
         # Add the announce to the directory announce
         # stream logger
 
-        link_stats = {"rssi": self.reticulum.get_packet_rssi(announce_packet_hash),
-                     "snr": self.reticulum.get_packet_snr(announce_packet_hash),
-                     "q": self.reticulum.get_packet_q(announce_packet_hash)}
+        if self.reticulum != None:
+            link_stats = {"rssi": self.reticulum.get_packet_rssi(announce_packet_hash),
+                         "snr": self.reticulum.get_packet_snr(announce_packet_hash),
+                         "q": self.reticulum.get_packet_q(announce_packet_hash)}
 
-        # This reformats the new v0.5.0 announce data back to the expected format
-        # for Sidebands database and other handling functions.
-        dn = LXMF.display_name_from_app_data(app_data)
-        sc = LXMF.stamp_cost_from_app_data(app_data)
-        app_data = b""
-        if dn != None:
-            app_data = dn.encode("utf-8")
+            # This reformats the new v0.5.0 announce data back to the expected format
+            # for Sidebands database and other handling functions.
+            dn = LXMF.display_name_from_app_data(app_data)
+            sc = LXMF.stamp_cost_from_app_data(app_data)
+            app_data = b""
+            if dn != None:
+                app_data = dn.encode("utf-8")
 
-        self.log_announce(destination_hash, app_data, dest_type=SidebandCore.aspect_filter, stamp_cost=sc, link_stats=link_stats)
+            self.log_announce(destination_hash, app_data, dest_type=SidebandCore.aspect_filter, stamp_cost=sc, link_stats=link_stats)
 
     def __init__(self, owner_app, config_path = None, is_service=False, is_client=False, android_app_dir=None, verbose=False, owner_service=None, service_context=None, is_daemon=False, load_config_only=False):
         self.is_service = is_service
@@ -157,6 +169,8 @@ class SidebandCore():
         self.owner_app = owner_app
         self.reticulum = None
         self.webshare_server = None
+        self.voice_running = False
+        self.telephone = None
         self.telemeter = None
         self.telemetry_running = False
         self.latest_telemetry = None
@@ -166,6 +180,8 @@ class SidebandCore():
         self.pending_telemetry_send_try = 0
         self.pending_telemetry_send_maxtries = 2
         self.telemetry_send_blocked_until = 0
+        self.telemetry_clean_interval = self.TELEMETRY_CLEAN_INTERVAL
+        self.last_telemetry_clean = 0
         self.pending_telemetry_request = False
         self.telemetry_request_max_history = 7*24*60*60
         self.live_tracked_objects = {}
@@ -181,6 +197,7 @@ class SidebandCore():
         self.allow_service_dispatch = True
         self.version_str = ""
         self.config_template = rns_config
+        self.default_config_template = rns_config
 
         if config_path == None:
             self.app_dir     = plyer.storagepath.get_home_dir()+"/.config/sideband"
@@ -250,6 +267,9 @@ class SidebandCore():
 
         self.webshare_ssl_key_path  = self.app_dir+"/app_storage/ssl_key.pem"
         self.webshare_ssl_cert_path = self.app_dir+"/app_storage/ssl_cert.pem"
+
+        self.mqtt = None
+        self.mqtt_handle_lock = threading.Lock()
         
         self.first_run     = True
         self.saving_configuration = False
@@ -449,6 +469,7 @@ class SidebandCore():
         self.config["eink_mode"] = True
         self.config["lxm_limit_1mb"] = True
         self.config["trusted_markup_only"] = False
+        self.config["compose_in_markdown"] = False
 
         # Connectivity
         self.config["connect_transport"] = False
@@ -515,6 +536,12 @@ class SidebandCore():
         self.config["telemetry_send_to_trusted"] = False
         self.config["telemetry_send_to_collector"] = False
 
+        # Voice
+        self.config["voice_enabled"] = False
+        self.config["voice_output"] = None
+        self.config["voice_input"] = None
+        self.config["voice_ringer"] = None
+
         if not os.path.isfile(self.db_path):
             self.__db_init()
         else:
@@ -548,7 +575,7 @@ class SidebandCore():
         RNS.log("Loading Sideband identity...", RNS.LOG_DEBUG)
         self.identity = RNS.Identity.from_file(self.identity_path)
 
-        self.rpc_addr = ("127.0.0.1", 48165)
+        self.rpc_addr = f"\0sideband/rpc"
         self.rpc_key  = RNS.Identity.full_hash(self.identity.get_private_key())
 
         RNS.log("Loading Sideband configuration... "+str(self.config_path), RNS.LOG_DEBUG)
@@ -593,6 +620,8 @@ class SidebandCore():
             self.config["hq_ptt"] = False
         if not "trusted_markup_only" in self.config:
             self.config["trusted_markup_only"] = False
+        if not "compose_in_markdown" in self.config:
+            self.config["compose_in_markdown"] = False
 
         if not "input_language" in self.config:
             self.config["input_language"] = None
@@ -714,6 +743,18 @@ class SidebandCore():
             self.config["telemetry_request_interval"] = 43200
         if not "telemetry_collector_enabled" in self.config:
             self.config["telemetry_collector_enabled"] = False
+        if not "telemetry_to_mqtt" in self.config:
+            self.config["telemetry_to_mqtt"] = False
+        if not "telemetry_mqtt_host" in self.config:
+            self.config["telemetry_mqtt_host"] = None
+        if not "telemetry_mqtt_port" in self.config:
+            self.config["telemetry_mqtt_port"] = None
+        if not "telemetry_mqtt_user" in self.config:
+            self.config["telemetry_mqtt_user"] = None
+        if not "telemetry_mqtt_pass" in self.config:
+            self.config["telemetry_mqtt_pass"] = None
+        if not "telemetry_mqtt_validate_ssl" in self.config:
+            self.config["telemetry_mqtt_validate_ssl"] = False
 
         if not "telemetry_icon" in self.config:
             self.config["telemetry_icon"] = SidebandCore.DEFAULT_APPEARANCE[0]
@@ -765,6 +806,8 @@ class SidebandCore():
             self.config["telemetry_s_acceleration"] = False
         if not "telemetry_s_proximity" in self.config:
             self.config["telemetry_s_proximity"] = False
+        if not "telemetry_s_rns_transport" in self.config:
+            self.config["telemetry_s_rns_transport"] = False
         if not "telemetry_s_fixed_location" in self.config:
             self.config["telemetry_s_fixed_location"] = False
         if not "telemetry_s_fixed_latlon" in self.config:
@@ -805,6 +848,17 @@ class SidebandCore():
         if not "map_storage_file" in self.config:
             self.config["map_storage_file"] = None
 
+        if not "voice_enabled" in self.config:
+            self.config["voice_enabled"] = False
+        if not "voice_output" in self.config:
+            self.config["voice_output"] = None
+        if not "voice_input" in self.config:
+            self.config["voice_input"] = None
+        if not "voice_ringer" in self.config:
+            self.config["voice_ringer"] = None
+        if not "voice_trusted_only" in self.config:
+            self.config["voice_trusted_only"] = False
+
         # Make sure we have a database
         if not os.path.isfile(self.db_path):
             self.__db_init()
@@ -961,13 +1015,16 @@ class SidebandCore():
                     notifications_permitted = True
 
                 if notifications_permitted:
-                    if RNS.vendor.platformutils.get_platform() == "android":
-                        if self.is_service:
-                            self.owner_service.android_notification(title, content, group=group, context_id=context_id)
+                    try:
+                        if RNS.vendor.platformutils.get_platform() == "android":
+                            if self.is_service:
+                                self.owner_service.android_notification(title, content, group=group, context_id=context_id)
+                            else:
+                                plyer.notification.notify(title, content, notification_icon=self.notification_icon, context_override=None)
                         else:
-                            plyer.notification.notify(title, content, notification_icon=self.notification_icon, context_override=None)
-                    else:
-                        plyer.notification.notify(title, content, app_icon=self.icon_32)
+                            plyer.notification.notify(title, content, app_icon=self.icon_32)
+                    except Exception as e:
+                        RNS.log("An error occurred while posting a notification to the operating system: {e}", RNS.LOG_ERROR)
 
     def log_announce(self, dest, app_data, dest_type, stamp_cost=None, link_stats=None):
         try:
@@ -975,7 +1032,7 @@ class SidebandCore():
                 app_data = b""
             if type(app_data) != bytes:
                 app_data = msgpack.packb([app_data, stamp_cost])
-            RNS.log("Received "+str(dest_type)+" announce for "+RNS.prettyhexrep(dest)+" with data: "+str(app_data), RNS.LOG_DEBUG)
+            RNS.log("Received "+str(dest_type)+" announce for "+RNS.prettyhexrep(dest), RNS.LOG_DEBUG)
             self._db_save_announce(dest, app_data, dest_type, link_stats)
             self.setstate("app.flags.new_announces", True)
 
@@ -1022,6 +1079,20 @@ class SidebandCore():
             RNS.log("Error while checking trust for "+RNS.prettyhexrep(context_dest)+": "+str(e), RNS.LOG_ERROR)
             return False
 
+    def voice_is_trusted(self, identity_hash):
+        context_dest = identity_hash
+        try:
+            lxmf_destination_hash = RNS.Destination.hash_from_name_and_identity("lxmf.delivery", identity_hash)
+            existing_voice = self._db_conversation(context_dest)
+            existing_lxmf  = self._db_conversation(lxmf_destination_hash)
+            if existing_lxmf: trust = existing_lxmf["trust"]
+            else: trust = existing_voice["trust"]
+            return trust == 1
+
+        except Exception as e:
+            RNS.log("Could not decode a valid peer name from data: "+str(e), RNS.LOG_DEBUG)
+            return False
+
     def is_object(self, context_dest, conv_data = None):
         try:
             if conv_data == None:
@@ -1181,6 +1252,22 @@ class SidebandCore():
             RNS.log("Could not decode a valid peer name from data: "+str(e), RNS.LOG_DEBUG)
             return RNS.prettyhexrep(context_dest)
 
+    def voice_display_name(self, identity_hash):
+        context_dest = identity_hash
+        if context_dest == self.lxmf_destination.hash:
+            return self.config["display_name"]
+
+        try:
+            lxmf_destination_hash = RNS.Destination.hash_from_name_and_identity("lxmf.delivery", identity_hash)
+            existing_voice = self._db_conversation(context_dest)
+            existing_lxmf  = self._db_conversation(lxmf_destination_hash)
+            if existing_lxmf: return self.peer_display_name(lxmf_destination_hash)
+            else: return self.peer_display_name(identity_hash)
+
+        except Exception as e:
+            RNS.log("Could not decode a valid peer name from data: "+str(e), RNS.LOG_DEBUG)
+            return RNS.prettyhexrep(context_dest)
+
     def clear_conversation(self, context_dest):
         self._db_clear_conversation(context_dest)
 
@@ -1703,7 +1790,7 @@ class SidebandCore():
         try:
             with self.rpc_lock:
                 if self.rpc_connection == None:
-                    self.rpc_connection = multiprocessing.connection.Client(self.rpc_addr, authkey=self.rpc_key)
+                    self.rpc_connection = multiprocessing.connection.Client(self.rpc_addr, family="AF_UNIX", authkey=self.rpc_key)
                 self.rpc_connection.send(request)
                 response = self.rpc_connection.recv()
                 return response
@@ -1806,15 +1893,81 @@ class SidebandCore():
                     RNS.log(ed, RNS.LOG_DEBUG)
                     return None
 
+    def _get_destination_mtu(self, destination_hash):
+        try:
+            mr = self.message_router
+            oh = destination_hash
+            ol = None
+            if oh in mr.direct_links:
+                ol = mr.direct_links[oh]
+            elif oh in mr.backchannel_links:
+                ol = mr.backchannel_links[oh]
+
+            if ol != None:
+                return ol.get_mtu()
+
+            return None
+
+        except Exception as e:
+            RNS.trace_exception(e)
+            return None
+
+    def get_destination_mtu(self, destination_hash):
+        if not RNS.vendor.platformutils.is_android():
+            return self._get_destination_mtu(destination_hash)
+        else:
+            if self.is_service:
+                return self._get_destination_mtu(destination_hash)
+            else:
+                try:
+                    return self.service_rpc_request({"get_destination_mtu": destination_hash})
+                except Exception as e:
+                    ed = "Error while getting destination link MTU over RPC: "+str(e)
+                    RNS.log(ed, RNS.LOG_DEBUG)
+                    return None
+
+    def _get_destination_edr(self, destination_hash):
+        try:
+            mr = self.message_router
+            oh = destination_hash
+            ol = None
+            if oh in mr.direct_links:
+                ol = mr.direct_links[oh]
+            elif oh in mr.backchannel_links:
+                ol = mr.backchannel_links[oh]
+
+            if ol != None:
+                return ol.get_expected_rate()
+
+            return None
+
+        except Exception as e:
+            RNS.trace_exception(e)
+            return None
+
+    def get_destination_edr(self, destination_hash):
+        if not RNS.vendor.platformutils.is_android():
+            return self._get_destination_edr(destination_hash)
+        else:
+            if self.is_service:
+                return self._get_destination_edr(destination_hash)
+            else:
+                try:
+                    return self.service_rpc_request({"get_destination_edr": destination_hash})
+                except Exception as e:
+                    ed = "Error while getting destination link EIFR over RPC: "+str(e)
+                    RNS.log(ed, RNS.LOG_DEBUG)
+                    return None
+
     def __start_rpc_listener(self):
         try:
             RNS.log("Starting RPC listener", RNS.LOG_DEBUG)
-            self.rpc_listener = multiprocessing.connection.Listener(self.rpc_addr, authkey=self.rpc_key)
+            self.rpc_listener = multiprocessing.connection.Listener(self.rpc_addr, family="AF_UNIX", authkey=self.rpc_key)
             thread = threading.Thread(target=self.__rpc_loop)
             thread.daemon = True
             thread.start()
         except Exception as e:
-            RNS.log("Could not start RPC listener on "+str(self.rpc_addr)+". Terminating now. Clear up anything using the port and try again.", RNS.LOG_ERROR)
+            RNS.log("Could not start RPC listener on @"+str(self.rpc_addr[1:])+". Terminating now. Clear up anything using the port and try again.", RNS.LOG_ERROR)
             RNS.panic()
 
     def __rpc_loop(self):
@@ -1850,6 +2003,10 @@ class SidebandCore():
                                     connection.send(self._get_plugins_info())
                                 elif "get_destination_establishment_rate" in call:
                                     connection.send(self._get_destination_establishment_rate(call["get_destination_establishment_rate"]))
+                                elif "get_destination_mtu" in call:
+                                    connection.send(self._get_destination_mtu(call["get_destination_mtu"]))
+                                elif "get_destination_edr" in call:
+                                    connection.send(self._get_destination_edr(call["get_destination_edr"]))
                                 elif "send_message" in call:
                                     args = call["send_message"]
                                     send_result = self.send_message(
@@ -1862,6 +2019,10 @@ class SidebandCore():
                                         image=args["image"],
                                         audio=args["audio"])
                                     connection.send(send_result)
+                                elif "cancel_message" in call:
+                                    args = call["cancel_message"]
+                                    cancel_result = self.cancel_message(args["message_id"])
+                                    connection.send(cancel_result)
                                 elif "send_command" in call:
                                     args = call["send_command"]
                                     send_result = self.send_command(
@@ -1982,6 +2143,7 @@ class SidebandCore():
         dbc = db.cursor()
 
         dbc.execute("CREATE TABLE IF NOT EXISTS telemetry (id INTEGER PRIMARY KEY, dest_context BLOB, ts INTEGER, data BLOB)")
+        dbc.execute("CREATE INDEX IF NOT EXISTS idx_telemetry_ts ON telemetry(ts)")
         db.commit()
 
     def _db_upgradetables(self):
@@ -2252,6 +2414,11 @@ class SidebandCore():
 
                 self.setstate("app.flags.last_telemetry", time.time())
 
+                if self.config["telemetry_to_mqtt"] == True:
+                    def mqtt_job():
+                        self.mqtt_handle_telemetry(context_dest, telemetry)
+                    threading.Thread(target=mqtt_job, daemon=True).start()
+
                 return telemetry
 
             except Exception as e:
@@ -2518,6 +2685,7 @@ class SidebandCore():
                         "last_rx": last_rx,
                         "last_tx": last_tx,
                         "last_activity": last_activity,
+                        "type": entry[4],
                         "trust": entry[5],
                         "data": data,
                     }
@@ -2669,6 +2837,27 @@ class SidebandCore():
 
         self.__event_conversations_changed()
 
+    def _db_create_voice_object(self, identity_hash, name = None, trust = False):
+        RNS.log("Creating voice object for "+RNS.prettyhexrep(identity_hash), RNS.LOG_DEBUG)
+        with self.db_lock:
+            db = self.__db_connect()
+            dbc = db.cursor()
+
+            def_name = "".encode("utf-8")
+            query = "INSERT INTO conv (dest_context, last_tx, last_rx, unread, type, trust, name, data) values (?, ?, ?, ?, ?, ?, ?, ?)"
+            data = (identity_hash, 0, time.time(), 0, SidebandCore.CONV_VOICE, 0, def_name, msgpack.packb(None))
+
+            dbc.execute(query, data)
+            db.commit()
+
+        if trust:
+            self._db_conversation_set_trusted(identity_hash, True)
+
+        if name != None and name != "":
+            self._db_conversation_set_name(identity_hash, name)
+
+        self.__event_conversations_changed()
+
     def _db_delete_message(self, msg_hash):
         RNS.log("Deleting message "+RNS.prettyhexrep(msg_hash))
         with self.db_lock:
@@ -2680,7 +2869,7 @@ class SidebandCore():
             db.commit()
 
     def _db_clean_messages(self):
-        RNS.log("Purging stale messages... "+str(self.db_path))
+        RNS.log("Purging stale messages... ", RNS.LOG_DEBUG)
         with self.db_lock:
             db = self.__db_connect()
             dbc = db.cursor()
@@ -2689,6 +2878,20 @@ class SidebandCore():
             dbc.execute(query, {"outbound_state": LXMF.LXMessage.OUTBOUND, "sending_state": LXMF.LXMessage.SENDING})
             db.commit()
 
+    def _db_clean_telemetry(self):
+        RNS.log("Cleaning telemetry... ", RNS.LOG_DEBUG)
+        clean_time = time.time()-self.TELEMETRY_KEEP
+        with self.db_lock:
+            db = self.__db_connect()
+            dbc = db.cursor()
+
+            query = f"delete from telemetry where (ts < {clean_time});"
+            dbc.execute(query, {"outbound_state": LXMF.LXMessage.OUTBOUND, "sending_state": LXMF.LXMessage.SENDING})
+            db.commit()
+
+            self.last_telemetry_clean = time.time()
+
+
     def _db_message_set_state(self, lxm_hash, state, is_retry=False, ratchet_id=None, originator_stamp=None):
         msg_extras = None
         if ratchet_id != None:
@@ -3068,6 +3271,39 @@ class SidebandCore():
         self.update_telemeter_config()
         self.setstate("app.flags.last_telemetry", time.time())
 
+    def mqtt_handle_telemetry(self, context_dest, telemetry):
+        with self.mqtt_handle_lock:
+            # TODO: Remove debug
+            if hasattr(self, "last_mqtt_recycle") and time.time() > self.last_mqtt_recycle + 60*4:
+                # RNS.log("Recycling MQTT handler", RNS.LOG_DEBUG)
+                self.mqtt.stop()
+                self.mqtt.client = None
+                self.mqtt = None
+                gc.collect()
+
+            if self.mqtt == None:
+                self.mqtt = MQTT()
+                self.last_mqtt_recycle = time.time()
+
+            self.mqtt.set_server(self.config["telemetry_mqtt_host"], self.config["telemetry_mqtt_port"])
+            self.mqtt.set_auth(self.config["telemetry_mqtt_user"], self.config["telemetry_mqtt_pass"])
+            self.mqtt.handle(context_dest, telemetry)
+
+            # TODO: Remove debug
+            # if not hasattr(self, "memtr"):
+            #     from pympler import muppy
+            #     from pympler import summary
+            #     import resource
+            #     self.res = resource
+            #     self.ms = summary; self.mp = muppy
+            #     self.memtr = self.ms.summarize(self.mp.get_objects())
+            #     RNS.log(f"RSS: {RNS.prettysize(self.res.getrusage(self.res.RUSAGE_SELF).ru_maxrss*1000)}")
+            # else:
+            #     memsum = self.ms.summarize(self.mp.get_objects())
+            #     memdiff = self.ms.get_diff(self.memtr, memsum)
+            #     self.ms.print_(memdiff)
+            #     RNS.log(f"RSS: {RNS.prettysize(self.res.getrusage(self.res.RUSAGE_SELF).ru_maxrss*1000)}")
+
     def update_telemetry(self):
         try:
             try:
@@ -3133,7 +3369,9 @@ class SidebandCore():
                 else:
                     self.telemeter = Telemeter(android_context=self.service_context, service=True, location_provider=self.owner_service)
 
-            sensors = ["location", "information", "battery", "pressure", "temperature", "humidity", "magnetic_field", "ambient_light", "gravity", "angular_velocity", "acceleration", "proximity"]
+            sensors = ["location", "information", "battery", "pressure", "temperature", "humidity", "magnetic_field",
+                       "ambient_light", "gravity", "angular_velocity", "acceleration", "proximity", "rns_transport"]
+
             for sensor in sensors:
                 if self.config["telemetry_s_"+sensor]:
                     self.telemeter.enable(sensor)
@@ -3149,7 +3387,7 @@ class SidebandCore():
                     else:
                         self.telemeter.disable(sensor)
 
-            for telemetry_plugin in self.active_telemetry_plugins:
+            for telemetry_plugin in self.active_telemetry_plugins.copy():
                 try:
                     plugin = self.active_telemetry_plugins[telemetry_plugin]
                     plugin.update_telemetry(self.telemeter)
@@ -3181,6 +3419,10 @@ class SidebandCore():
         if self.config["telemetry_enabled"] == True:
             self.update_telemeter_config()
             if self.telemeter != None:
+                if self.config["telemetry_to_mqtt"]:
+                    def mqtt_job():
+                        self.mqtt_handle_telemetry(self.lxmf_destination.hash, self.telemeter.packed())
+                    threading.Thread(target=mqtt_job, daemon=True).start()
                 return self.telemeter.read_all()
             else:
                 return {}
@@ -3265,6 +3507,7 @@ class SidebandCore():
                                     if self.config["start_announce"] == True:
                                         time.sleep(12)
                                         self.lxmf_announce(attached_interface=self.interface_local)
+                                        if self.telephone: self.telephone.announce(attached_interface=self.interface_local)
                                 threading.Thread(target=job, daemon=True).start()
 
                     if hasattr(self, "interface_rnode") and self.interface_rnode != None:
@@ -3352,6 +3595,7 @@ class SidebandCore():
                             aif = announce_attached_interface
                             time.sleep(delay)
                             self.lxmf_announce(attached_interface=aif)
+                            if self.telephone: self.telephone.announce(attached_interface=aif)
                         return x
 
                     threading.Thread(target=gen_announce_job(announce_delay, announce_attached_interface), daemon=True).start()
@@ -3474,6 +3718,9 @@ class SidebandCore():
                                     self.setpersistent("lxmf.syncretrying", False)
 
                 if self.config["telemetry_enabled"]:
+                    if time.time()-self.last_telemetry_clean > self.telemetry_clean_interval:
+                        self._db_clean_telemetry()
+
                     if self.config["telemetry_send_to_collector"]:
                         if self.config["telemetry_collector"] != None and self.config["telemetry_collector"] != self.lxmf_destination.hash:
                             try:
@@ -3563,6 +3810,7 @@ class SidebandCore():
                 def da():
                     time.sleep(8)
                     self.lxmf_announce()
+                    if self.telephone: self.telephone.announce()
                     self.last_if_change_announce = time.time()
                 threading.Thread(target=da, daemon=True).start()
 
@@ -3570,8 +3818,8 @@ class SidebandCore():
             self.periodic_thread.start()
 
         if self.is_standalone or self.is_client:
-            if self.config["telemetry_enabled"]:
-                self.run_telemetry()
+            if self.config["telemetry_enabled"]: self.run_telemetry()
+            if self.config["voice_enabled"]: self.start_voice()
         elif self.is_service:
             self.run_service_telemetry()
 
@@ -3867,13 +4115,13 @@ class SidebandCore():
                                     ifac_size = None
 
                                 interface_config = {
-                                    "name": "TCPClientInterface",
+                                    "name": "TCP Client",
                                     "target_host": tcp_host,
                                     "target_port": tcp_port,
                                     "kiss_framing": False,
                                     "i2p_tunneled": False,
                                 }
-                                tcpinterface = RNS.Interfaces.TCPInterface.TCPClientInterface(RNS.Transport, interface_config)
+                                tcpinterface = RNS.Interfaces.BackboneInterface.BackboneClientInterface(RNS.Transport, interface_config)
                                 tcpinterface.OUT = True
 
                                 if RNS.Reticulum.transport_enabled():
@@ -4267,6 +4515,21 @@ class SidebandCore():
                 RNS.log("An error occurred while getting message transfer stamp cost: "+str(e), RNS.LOG_ERROR)
                 return None
 
+    def _service_cancel_message(self, message_id):
+        if not RNS.vendor.platformutils.is_android():
+            return False
+        else:
+            if self.is_client:
+                try:
+                    return self.service_rpc_request({"cancel_message": {"message_id": message_id }})
+                
+                except Exception as e:
+                    RNS.log("Error while cancelling message over RPC: "+str(e), RNS.LOG_DEBUG)
+                    RNS.trace_exception(e)
+                    return False
+            else:
+                return False
+
     def _service_send_message(self, content, destination_hash, propagation, skip_fields=False, no_display=False, attachment = None, image = None, audio = None):
         if not RNS.vendor.platformutils.is_android():
             return False
@@ -4310,6 +4573,26 @@ class SidebandCore():
             else:
                 return False
 
+    def cancel_message(self, message_id):
+        if self.allow_service_dispatch and self.is_client:
+            try:
+                return self._service_cancel_message(message_id)
+
+            except Exception as e:
+                RNS.log("Error while cancelling message: "+str(e), RNS.LOG_ERROR)
+                RNS.trace_exception(e)
+                return False
+
+        else:
+            try:
+                self.message_router.cancel_outbound(message_id)
+                return True
+
+            except Exception as e:
+                RNS.log("Error while cancelling message: "+str(e), RNS.LOG_ERROR)
+                RNS.trace_exception(e)
+                return False
+
     def send_message(self, content, destination_hash, propagation, skip_fields=False, no_display=False, attachment = None, image = None, audio = None):
         if self.allow_service_dispatch and self.is_client:
             try:
@@ -4349,6 +4632,14 @@ class SidebandCore():
                     fields[LXMF.FIELD_IMAGE] = image
                 if audio != None:
                     fields[LXMF.FIELD_AUDIO] = audio
+                md_sig = "#!md\n"
+                if content.startswith(md_sig):
+                    content = content[len(md_sig):]
+                    fields[LXMF.FIELD_RENDERER] = LXMF.RENDERER_MARKDOWN
+                elif self.config["compose_in_markdown"]:
+                    fields[LXMF.FIELD_RENDERER] = LXMF.RENDERER_MARKDOWN
+                elif self.has_bb_markup(content):
+                    fields[LXMF.FIELD_RENDERER] = LXMF.RENDERER_BBCODE
 
                 lxm = LXMF.LXMessage(dest, source, content, title="", desired_method=desired_method, fields = fields, include_ticket=self.is_trusted(destination_hash))
                 
@@ -4435,7 +4726,7 @@ class SidebandCore():
                 RNS.log("Error while sending message: "+str(e), RNS.LOG_ERROR)
                 return False
 
-    def new_conversation(self, dest_str, name = "", trusted = False):
+    def new_conversation(self, dest_str, name = "", trusted = False, voice_only = False):
         if len(dest_str) != RNS.Reticulum.TRUNCATED_HASHLENGTH//8*2:
             return False
 
@@ -4445,7 +4736,8 @@ class SidebandCore():
                 RNS.log("Cannot create conversation with own LXMF address", RNS.LOG_ERROR)
                 return False
             else:
-                self._db_create_conversation(addr_b, name, trusted)
+                if not voice_only: self._db_create_conversation(addr_b, name, trusted)
+                else:              self._db_create_voice_object(addr_b, name, trusted)
 
         except Exception as e:
             RNS.log("Error while creating conversation: "+str(e), RNS.LOG_ERROR)
@@ -4487,6 +4779,19 @@ class SidebandCore():
 
         self.setstate("lxm_uri_ingest.result", response)
 
+    def strip_bb_markup(self, text):
+        if not hasattr(self, "smr") or self.smr == None:
+            self.smr = re.compile(r"\[\/?(?:b|i|u|url|quote|code|img|color|size)*?.*?\]",re.IGNORECASE | re.MULTILINE )
+        return self.smr.sub("", text)
+
+    def has_bb_markup(self, text):
+        if not hasattr(self, "smr") or self.smr == None:
+            self.smr = re.compile(r"\[\/?(?:b|i|u|url|quote|code|img|color|size)*?.*?\]",re.IGNORECASE | re.MULTILINE )
+        if self.smr.match(text):
+            return True
+        else:
+            return False
+
     def lxm_ingest(self, message, originator = False):
         should_notify = False
         is_trusted = False
@@ -4569,7 +4874,7 @@ class SidebandCore():
         if should_notify:
             nlen = 128
             text = message.content.decode("utf-8")
-            notification_content = text[:nlen]
+            notification_content = self.strip_bb_markup(text[:nlen])
             if len(text) > nlen:
                 notification_content += " [...]"
 
@@ -4676,6 +4981,7 @@ class SidebandCore():
 
     def start(self):
         self._db_clean_messages()
+        self._db_clean_telemetry()
         self.__start_jobs_immediate()
 
         thread = threading.Thread(target=self.__start_jobs_deferred)
@@ -4683,7 +4989,7 @@ class SidebandCore():
         thread.start()
 
         self.setstate("core.started", True)
-        RNS.log("Sideband Core "+str(self)+" "+str(self.version_str)+" started")
+        RNS.log("Sideband Core "+str(self)+" "+str(self.version_str)+"started")
 
     def stop_webshare(self):
         if self.webshare_server != None:
@@ -4901,6 +5207,10 @@ class SidebandCore():
             RNS.log("Error while handling commands: "+str(e), RNS.LOG_ERROR)
 
     def create_telemetry_collector_response(self, to_addr, timebase, is_authorized_telemetry_request=False):
+        if self.getstate(f"telemetry.{RNS.hexrep(to_addr, delimit=False)}.update_sending") == True:
+            RNS.log("Not sending new telemetry collector response, since an earlier transfer is already in progress", RNS.LOG_DEBUG)
+            return "in_progress"
+
         added_sources = {}
         sources = self.list_telemetry(after=timebase)
         only_latest = self.config["telemetry_requests_only_send_latest"]
@@ -4977,6 +5287,40 @@ class SidebandCore():
             if not self.reticulum.is_connected_to_shared_instance:
                 RNS.Transport.detach_interfaces()
 
+    def start_voice(self):
+        try:
+            if not self.voice_running:
+                RNS.log("Starting voice service", RNS.LOG_DEBUG)
+                self.voice_running = True
+                from .voice import ReticulumTelephone
+                self.telephone = ReticulumTelephone(self.identity, owner=self, speaker=self.config["voice_output"], microphone=self.config["voice_input"], ringer=self.config["voice_ringer"])
+                ringtone_path = os.path.join(self.asset_dir, "audio", "notifications", "soft1.opus")
+                self.telephone.set_ringtone(ringtone_path)
+
+        except Exception as e:
+            self.voice_running = False
+            RNS.log(f"An error occurred while starting voice services, the contained exception was: {e}", RNS.LOG_ERROR)
+            RNS.trace_exception(e)
+
+    def stop_voice(self):
+        try:
+            if self.voice_running:
+                RNS.log("Stopping voice service", RNS.LOG_DEBUG)
+                if self.telephone:
+                    self.telephone.stop()
+                    del self.telephone
+
+                self.telephone = None
+                self.voice_running = False
+
+        except Exception as e:
+            RNS.log(f"An error occurred while stopping voice services, the contained exception was: {e}", RNS.LOG_ERROR)
+            RNS.trace_exception(e)
+
+    def incoming_call(self, remote_identity):
+        display_name = self.voice_display_name(remote_identity.hash)
+        self.setstate("voice.incoming_call", display_name)
+
 rns_config = """# This template is used to generate a
 # running configuration for Sideband's
 # internal RNS instance. Incorrect changes
@@ -5012,6 +5356,6 @@ rns_config = """# This template is used to generate a
 
 # No additional interfaces are currently
 # defined, but you can use this section
-# to do so.
+# to add additional custom interfaces.
 [interfaces]
 """

sbapp/sideband/mqtt.py

@@ -0,0 +1,132 @@
+import RNS
+import time
+import threading
+from collections import deque
+from .sense import Telemeter, Commands
+
+if RNS.vendor.platformutils.get_platform() == "android":
+    import pmqtt.client as mqtt
+else:
+    from sbapp.pmqtt import client as mqtt
+
+class MQTT():
+    QUEUE_MAXLEN = 65536
+    SCHEDULER_SLEEP = 1
+
+    def __init__(self):
+        self.client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
+        self.host = None
+        self.port = None
+        self.run = False
+        self.is_connected = False
+        self.queue_lock = threading.Lock()
+        self.waiting_msgs = deque(maxlen=MQTT.QUEUE_MAXLEN)
+        self.waiting_telemetry = set()
+        self.client.on_connect_fail = self.connect_failed
+        self.client.on_disconnect = self.disconnected
+        self.start()
+
+    def start(self):
+        self.run = True
+        threading.Thread(target=self.jobs, daemon=True).start()
+        RNS.log("Started MQTT scheduler", RNS.LOG_DEBUG)
+
+    def stop(self):
+        RNS.log("Stopping MQTT scheduler", RNS.LOG_DEBUG)
+        self.run = False
+
+    def jobs(self):
+        while self.run:
+            try:
+                if len(self.waiting_msgs) > 0:
+                    RNS.log(f"Processing {len(self.waiting_msgs)} MQTT messages", RNS.LOG_DEBUG)
+                    if self.process_queue():
+                        RNS.log("All MQTT messages processed", RNS.LOG_DEBUG)
+
+            except Exception as e:
+                RNS.log(f"An error occurred while running MQTT scheduler jobs: {e}", RNS.LOG_ERROR)
+                RNS.trace_exception(e)
+
+            time.sleep(MQTT.SCHEDULER_SLEEP)
+
+        try: self.disconnect()
+        except Exception as e: RNS.log(f"An error occurred while disconnecting MQTT server: {e}", RNS.LOG_ERROR)
+
+        RNS.log("Stopped MQTT scheduler", RNS.LOG_DEBUG)
+
+    def connect_failed(self, client, userdata):
+        RNS.log(f"Connection to MQTT server failed", RNS.LOG_DEBUG)
+        self.is_connected = False
+
+    def disconnected(self, client, userdata, disconnect_flags, reason_code, properties):
+        RNS.log(f"Disconnected from MQTT server, reason code: {reason_code}", RNS.LOG_EXTREME)
+        self.is_connected = False
+
+    def set_server(self, host, port):
+        try:
+            port = int(port)
+        except:
+            port = None
+
+        self.host = host
+        self.port = port or 1883
+
+    def set_auth(self, username, password):
+        self.client.username_pw_set(username, password)
+
+    def connect(self):
+        RNS.log(f"Connecting MQTT server {self.host}:{self.port}", RNS.LOG_DEBUG) # TODO: Remove debug
+        cs = self.client.connect(self.host, self.port)
+        self.client.loop_start()
+
+    def disconnect(self):
+        RNS.log("Disconnecting from MQTT server", RNS.LOG_EXTREME) # TODO: Remove debug
+        self.client.disconnect()
+        self.client.loop_stop()
+        self.is_connected = False
+
+    def post_message(self, topic, data):
+        mqtt_msg = self.client.publish(topic, data, qos=1)
+        self.waiting_telemetry.add(mqtt_msg)
+
+    def process_queue(self):
+        with self.queue_lock:
+            try:
+                self.connect()
+            except Exception as e:
+                RNS.log(f"An error occurred while connecting to MQTT server: {e}", RNS.LOG_ERROR)
+                return False
+
+            try:
+                while len(self.waiting_msgs) > 0:
+                    topic, data = self.waiting_msgs.pop()
+                    self.post_message(topic, data)
+            except Exception as e:
+                RNS.log(f"An error occurred while publishing MQTT messages: {e}", RNS.LOG_ERROR)
+                RNS.trace_exception(e)
+
+            try:
+                for msg in self.waiting_telemetry:
+                    msg.wait_for_publish()
+                self.waiting_telemetry.clear()
+
+            except Exception as e:
+                RNS.log(f"An error occurred while publishing MQTT messages: {e}", RNS.LOG_ERROR)
+                RNS.trace_exception(e)
+
+            self.disconnect()
+            return True
+
+    def handle(self, context_dest, telemetry):
+        remote_telemeter = Telemeter.from_packed(telemetry)
+        read_telemetry = remote_telemeter.read_all()
+        telemetry_timestamp = read_telemetry["time"]["utc"]
+        root_path = f"lxmf/telemetry/{RNS.hexrep(context_dest, delimit=False)}"
+        for sensor in remote_telemeter.sensors:
+            s = remote_telemeter.sensors[sensor]
+            topics = s.render_mqtt()
+            if topics != None:
+                for topic in topics:
+                    topic_path = f"{root_path}/{topic}"
+                    data = topics[topic]
+                    self.waiting_msgs.append((topic_path, data))

sbapp/sideband/voice.py

@@ -0,0 +1,171 @@
+import RNS
+import os
+import sys
+import time
+
+from LXST._version import __version__
+from LXST.Primitives.Telephony import Telephone
+from RNS.vendor.configobj import ConfigObj
+
+class ReticulumTelephone():
+    STATE_AVAILABLE  = 0x00
+    STATE_CONNECTING = 0x01
+    STATE_RINGING    = 0x02
+    STATE_IN_CALL    = 0x03
+
+    HW_SLEEP_TIMEOUT = 15
+    HW_STATE_IDLE    = 0x00
+    HW_STATE_DIAL    = 0x01
+    HW_STATE_SLEEP   = 0xFF
+
+    RING_TIME        = 30
+    WAIT_TIME        = 60
+    PATH_TIME        = 10
+
+    def __init__(self, identity, owner = None, service = False, speaker=None, microphone=None, ringer=None):
+        self.identity          = identity
+        self.service           = service
+        self.owner             = owner
+        self.config            = None
+        self.should_run        = False
+        self.telephone         = None
+        self.state             = self.STATE_AVAILABLE
+        self.hw_state          = self.HW_STATE_IDLE
+        self.hw_last_event     = time.time()
+        self.hw_input          = ""
+        self.direction         = None
+        self.last_input        = None
+        self.first_run         = False
+        self.ringtone_path     = None
+        self.speaker_device    = speaker
+        self.microphone_device = microphone
+        self.ringer_device     = ringer
+        self.phonebook         = {}
+        self.aliases           = {}
+        self.names             = {}
+        
+        self.telephone  = Telephone(self.identity, ring_time=self.RING_TIME, wait_time=self.WAIT_TIME)
+        self.telephone.set_ringing_callback(self.ringing)
+        self.telephone.set_established_callback(self.call_established)
+        self.telephone.set_ended_callback(self.call_ended)
+        self.telephone.set_speaker(self.speaker_device)
+        self.telephone.set_microphone(self.microphone_device)
+        self.telephone.set_ringer(self.ringer_device)
+        self.telephone.set_allowed(self.__is_allowed)
+        RNS.log(f"{self} initialised", RNS.LOG_DEBUG)
+
+    def set_ringtone(self, ringtone_path):
+        if os.path.isfile(ringtone_path):
+            self.ringtone_path = ringtone_path
+            self.telephone.set_ringtone(self.ringtone_path)
+
+    def set_speaker(self, device):
+        self.speaker_device = device
+        self.telephone.set_speaker(self.speaker_device)
+
+    def set_microphone(self, device):
+        self.microphone_device = device
+        self.telephone.set_microphone(self.microphone_device)
+
+    def set_ringer(self, device):
+        self.ringer_device = device
+        self.telephone.set_ringer(self.ringer_device)
+
+    def announce(self, attached_interface=None):
+        self.telephone.announce(attached_interface=attached_interface)
+
+    @property
+    def is_available(self):
+        return self.state == self.STATE_AVAILABLE
+
+    @property
+    def is_in_call(self):
+        return self.state == self.STATE_IN_CALL
+
+    @property
+    def is_ringing(self):
+        return self.state == self.STATE_RINGING
+
+    @property
+    def call_is_connecting(self):
+        return self.state == self.STATE_CONNECTING
+
+    @property
+    def hw_is_idle(self):
+        return self.hw_state == self.HW_STATE_IDLE
+
+    @property
+    def hw_is_dialing(self):
+        return self.hw_state == self.HW_STATE_DIAL
+
+    def start(self):
+        if not self.should_run:
+            self.should_run = True
+            self.run()
+
+    def stop(self):
+        self.should_run = False
+        self.telephone.teardown()
+        self.telephone = None
+
+    def hangup(self): self.telephone.hangup()
+    def answer(self): self.telephone.answer(self.caller)
+    def set_busy(self, busy): self.telephone.set_busy(busy)
+
+    def dial(self, identity_hash):
+        self.last_dialled_identity_hash = identity_hash
+        destination_hash = RNS.Destination.hash_from_name_and_identity("lxst.telephony", identity_hash)
+        if RNS.Transport.has_path(destination_hash):
+            call_hops = RNS.Transport.hops_to(destination_hash)
+            cs = "" if call_hops == 1 else "s"
+            RNS.log(f"Connecting call over {call_hops} hop{cs}...", RNS.LOG_DEBUG)
+            identity = RNS.Identity.recall(destination_hash)
+            self.call(identity)
+        else:
+            return "no_path"
+
+    def redial(self, args=None):
+        if self.last_dialled_identity_hash: self.dial(self.last_dialled_identity_hash)
+
+    def call(self, remote_identity):
+        RNS.log(f"Calling {RNS.prettyhexrep(remote_identity.hash)}...", RNS.LOG_DEBUG)
+        self.state = self.STATE_CONNECTING
+        self.caller = remote_identity
+        self.direction = "to"
+        self.telephone.call(self.caller)
+
+    def ringing(self, remote_identity):
+        if self.hw_state == self.HW_STATE_SLEEP: self.hw_state = self.HW_STATE_IDLE
+        self.state = self.STATE_RINGING
+        self.caller  = remote_identity
+        self.direction = "from" if self.direction == None else "to"
+        RNS.log(f"Incoming call from {RNS.prettyhexrep(self.caller.hash)}", RNS.LOG_DEBUG)
+        if self.owner:
+            self.owner.incoming_call(remote_identity)
+
+    def call_ended(self, remote_identity):
+        if self.is_in_call or self.is_ringing or self.call_is_connecting:
+            if self.is_in_call:         RNS.log(f"Call with {RNS.prettyhexrep(self.caller.hash)} ended\n", RNS.LOG_DEBUG)
+            if self.is_ringing:         RNS.log(f"Call {self.direction} {RNS.prettyhexrep(self.caller.hash)} was not answered\n", RNS.LOG_DEBUG)
+            if self.call_is_connecting: RNS.log(f"Call to {RNS.prettyhexrep(self.caller.hash)} could not be connected\n", RNS.LOG_DEBUG)
+            self.direction = None
+            self.state = self.STATE_AVAILABLE
+
+    def call_established(self, remote_identity):
+        if self.call_is_connecting or self.is_ringing:
+            self.state = self.STATE_IN_CALL
+            RNS.log(f"Call established with {RNS.prettyhexrep(self.caller.hash)}", RNS.LOG_DEBUG)
+
+    def __is_allowed(self, identity_hash):
+        if self.owner.config["voice_trusted_only"]:
+            return self.owner.voice_is_trusted(identity_hash)
+        else: return True
+
+    def __spin(self, until=None, msg=None, timeout=None):
+        if msg: RNS.log(msg, RNS.LOG_DEBUG)
+        if timeout != None: timeout = time.time()+timeout
+        while (timeout == None or time.time()<timeout) and not until(): time.sleep(0.1)
+        if timeout != None and time.time() > timeout:
+            return False
+        else:
+            return True

sbapp/ui/conversations.py

@@ -6,6 +6,7 @@ from kivy.uix.boxlayout import BoxLayout
 from kivy.properties import StringProperty, BooleanProperty
 from kivymd.uix.list import MDList, IconLeftWidget, IconRightWidget, OneLineAvatarIconListItem
 from kivymd.uix.menu import MDDropdownMenu
+from kivymd.toast import toast
 from kivy.uix.gridlayout import GridLayout
 from kivy.uix.boxlayout import BoxLayout
 from kivy.clock import Clock
@@ -53,6 +54,7 @@ class Conversations():
             self.app.root.ids.screen_manager.add_widget(self.screen)
         
         self.conversation_dropdown = None
+        self.voice_dropdown = None
         self.delete_dialog = None
         self.clear_dialog = None
         self.clear_telemetry_dialog = None
@@ -91,6 +93,7 @@ class Conversations():
         self.app.sideband.setstate("wants.viewupdate.conversations", False)
 
     def trust_icon(self, conv):
+        conv_type = conv["type"]
         context_dest = conv["dest"]
         unread = conv["unread"]
         appearance = self.app.sideband.peer_appearance(context_dest, conv=conv)
@@ -106,25 +109,28 @@ class Conversations():
                 trust_icon = appearance[0] or da[0];
         
         else:
-            if self.app.sideband.requests_allowed_from(context_dest):
-                if unread:
-                    if is_trusted:
-                        trust_icon = "email-seal"
-                    else:
-                        trust_icon = "email"
-                else:
-                    trust_icon = "account-lock-open"
+            if conv_type == self.app.sideband.CONV_VOICE:
+                trust_icon = "phone"
             else:
-                if is_trusted:
+                if self.app.sideband.requests_allowed_from(context_dest):
                     if unread:
-                        trust_icon = "email-seal"
+                        if is_trusted:
+                            trust_icon = "email-seal"
+                        else:
+                            trust_icon = "email"
                     else:
-                        trust_icon = "account-check"
+                        trust_icon = "account-lock-open"
                 else:
-                    if unread:
-                        trust_icon = "email"
+                    if is_trusted:
+                        if unread:
+                            trust_icon = "email-seal"
+                        else:
+                            trust_icon = "account-check"
                     else:
-                        trust_icon = "account-question"
+                        if unread:
+                            trust_icon = "email"
+                        else:
+                            trust_icon = "account-question"
 
         return trust_icon
 
@@ -166,6 +172,7 @@ class Conversations():
 
         iconl._default_icon_pad = dp(ic_p)
         iconl.icon_size = dp(ic_s)
+        iconl.conv_type = conv["type"]
 
         return iconl
 
@@ -187,6 +194,7 @@ class Conversations():
             
         for conv in self.context_dests:
             context_dest = conv["dest"]
+            conv_type = conv["type"]
             unread = conv["unread"]
             last_activity = conv["last_activity"]
 
@@ -203,6 +211,7 @@ class Conversations():
                 item.sb_uid = context_dest
                 item.sb_unread = unread
                 iconl.sb_uid = context_dest
+                item.conv_type = conv_type
 
                 def gen_edit(item):
                     def x():
@@ -283,7 +292,7 @@ class Conversations():
 
                             yes_button.bind(on_release=dl_yes)
                             no_button.bind(on_release=dl_no)
-                            item.dmenu.dismiss()
+                            self.voice_dropdown.dismiss(); self.conversation_dropdown.dismiss()
                             dialog.open()
                             RNS.log("Generated edit dialog in "+str(RNS.prettytime(time.time()-t_s)), RNS.LOG_DEBUG)
 
@@ -312,7 +321,7 @@ class Conversations():
                             yes_button.bind(on_release=dl_yes)
                             no_button.bind(on_release=dl_no)
 
-                        item.dmenu.dismiss()
+                        self.voice_dropdown.dismiss(); self.conversation_dropdown.dismiss()
                         self.clear_dialog.open()
                     return x
 
@@ -336,7 +345,7 @@ class Conversations():
                             yes_button.bind(on_release=dl_yes)
                             no_button.bind(on_release=dl_no)
 
-                        item.dmenu.dismiss()
+                        self.voice_dropdown.dismiss(); self.conversation_dropdown.dismiss()
                         self.clear_telemetry_dialog.open()
                     return x
 
@@ -362,27 +371,61 @@ class Conversations():
                             yes_button.bind(on_release=dl_yes)
                             no_button.bind(on_release=dl_no)
 
-                        item.dmenu.dismiss()
+                        self.voice_dropdown.dismiss(); self.conversation_dropdown.dismiss()
                         self.delete_dialog.open()
                     return x
 
-                # def gen_move_to(item):
-                #     def x():
-                #         item.dmenu.dismiss()
-                #         self.app.sideband.conversation_set_object(self.conversation_dropdown.context_dest, not self.app.sideband.is_object(self.conversation_dropdown.context_dest))
-                #         self.app.conversations_view.update()
-                #     return x
-
                 def gen_copy_addr(item):
                     def x():
                         Clipboard.copy(RNS.hexrep(self.conversation_dropdown.context_dest, delimit=False))
-                        item.dmenu.dismiss()
+                        self.voice_dropdown.dismiss(); self.conversation_dropdown.dismiss()
+                    return x
+
+                def gen_call(item):
+                    def x():
+                        identity = RNS.Identity.recall(self.conversation_dropdown.context_dest)
+                        if identity: self.app.dial_action(identity.hash)
+                        else: toast("Can't call, identity unknown")
+                        self.voice_dropdown.dismiss(); self.conversation_dropdown.dismiss()
                     return x
 
                 item.iconr = IconRightWidget(icon="dots-vertical");
 
+                if self.voice_dropdown == None:
+                    dmi_h = 40
+                    dmv_items = [
+                        {
+                            "viewclass": "OneLineListItem",
+                            "text": "Edit",
+                            "height": dp(dmi_h),
+                            "on_release": gen_edit(item)
+                        },
+                        {
+                            "text": "Copy Identity Hash",
+                            "viewclass": "OneLineListItem",
+                            "height": dp(dmi_h),
+                            "on_release": gen_copy_addr(item)
+                        },
+                        {
+                            "text": "Delete",
+                            "viewclass": "OneLineListItem",
+                            "height": dp(dmi_h),
+                            "on_release": gen_del(item)
+                        }
+                    ]
+
+                    self.voice_dropdown = MDDropdownMenu(
+                        caller=item.iconr,
+                        items=dmv_items,
+                        position="auto",
+                        width=dp(256),
+                        elevation=0,
+                        radius=dp(3),
+                    )
+                    self.voice_dropdown.effect_cls = ScrollEffect
+                    self.voice_dropdown.md_bg_color = self.app.color_hover
+
                 if self.conversation_dropdown == None:
-                    obj_str = "conversations" if is_object else "objects"
                     dmi_h = 40
                     dm_items = [
                         {
@@ -391,18 +434,18 @@ class Conversations():
                             "height": dp(dmi_h),
                             "on_release": gen_edit(item)
                         },
+                        {
+                            "viewclass": "OneLineListItem",
+                            "text": "Call",
+                            "height": dp(dmi_h),
+                            "on_release": gen_call(item)
+                        },
                         {
                             "text": "Copy Address",
                             "viewclass": "OneLineListItem",
                             "height": dp(dmi_h),
                             "on_release": gen_copy_addr(item)
                         },
-                        # {
-                        #     "text": "Move to objects",
-                        #     "viewclass": "OneLineListItem",
-                        #     "height": dp(dmi_h),
-                        #     "on_release": gen_move_to(item)
-                        # },
                         {
                             "text": "Clear Messages",
                             "viewclass": "OneLineListItem",
@@ -434,11 +477,15 @@ class Conversations():
                     self.conversation_dropdown.effect_cls = ScrollEffect
                     self.conversation_dropdown.md_bg_color = self.app.color_hover
 
-                item.dmenu = self.conversation_dropdown
+                if conv_type == self.app.sideband.CONV_VOICE:
+                    item.dmenu = self.voice_dropdown
+                else:
+                    item.dmenu = self.conversation_dropdown
 
                 def callback_factory(ref, dest):
                     def x(sender):
                         self.conversation_dropdown.context_dest = dest
+                        self.voice_dropdown.context_dest = dest
                         ref.dmenu.caller = ref.iconr
                         ref.dmenu.open()
                     return x
@@ -448,6 +495,7 @@ class Conversations():
                 item.add_widget(item.iconr)
 
                 item.trusted = self.app.sideband.is_trusted(context_dest, conv_data=existing_conv)
+                item.conv_type = conv_type
                 
                 self.added_item_dests.append(context_dest)
                 self.list.add_widget(item)
@@ -519,7 +567,7 @@ Builder.load_string("""
     orientation: "vertical"
     spacing: "24dp"
     size_hint_y: None
-    height: dp(250)
+    height: dp(260)
 
     MDTextField:
         id: n_address_field
@@ -540,7 +588,7 @@ Builder.load_string("""
         orientation: "horizontal"
         size_hint_y: None
         padding: [0,0,dp(8),dp(24)]
-        height: dp(48)
+        height: dp(24)
         MDLabel:
             id: "trusted_switch_label"
             text: "Trusted"
@@ -551,6 +599,21 @@ Builder.load_string("""
             pos_hint: {"center_y": 0.3}
             active: False
 
+    MDBoxLayout:
+        orientation: "horizontal"
+        size_hint_y: None
+        padding: [0,0,dp(8),dp(24)]
+        height: dp(24)
+        MDLabel:
+            id: "trusted_switch_label"
+            text: "Voice Only"
+            font_style: "H6"
+
+        MDSwitch:
+            id: n_voice_only
+            pos_hint: {"center_y": 0.3}
+            active: False
+
 <ConvSettings>
     orientation: "vertical"
     spacing: "16dp"

sbapp/ui/helpers.py

@@ -20,11 +20,13 @@ color_delivered = "Blue"
 color_paper = "Indigo"
 color_propagated = "Indigo"
 color_failed = "Red"
+color_cancelled = "Red"
 color_unknown = "Gray"
 intensity_msgs_dark = "800"
 intensity_msgs_light = "500"
 intensity_play_dark = "600"
 intensity_play_light = "300"
+intensity_cancelled = "900"
 
 
 intensity_msgs_dark_alt = "800"
@@ -38,6 +40,7 @@ color_paper_alt = "DeepPurple"
 color_playing_alt = "Amber"
 color_failed_alt = "Red"
 color_unknown_alt = "Gray"
+color_cancelled_alt = "Red"
 
 class ContentNavigationDrawer(Screen):
     pass
@@ -120,10 +123,12 @@ def sig_icon_for_q(q):
         return "󰣸"
     elif q > 50:
         return "󰣶"
-    elif q > 30:
+    elif q > 20:
         return "󰣴"
-    elif q > 10:
+    elif q > 5:
         return "󰣾"
+    else:
+        return "󰣽"
 
 persistent_fonts = ["nf", "term"]
 nf_mapped = "nf"

sbapp/ui/layouts.py

@@ -96,6 +96,16 @@ MDNavigationLayout:
                             IconLeftWidget:
                                 icon: "account-voice"
                                 on_release: root.ids.screen_manager.app.announces_action(self)
+                                
+
+                        OneLineIconListItem:
+                            text: "Voice"
+                            on_release: root.ids.screen_manager.app.voice_action(self)
+                            # _no_ripple_effect: True
+                        
+                            IconLeftWidget:
+                                icon: "phone-in-talk"
+                                on_release: root.ids.screen_manager.app.voice_action(self)
 
                                                        
                         # OneLineIconListItem:
@@ -1655,6 +1665,22 @@ MDScreen:
                         disabled: False
                         active: False
 
+                MDBoxLayout:
+                    orientation: "horizontal"
+                    size_hint_y: None
+                    padding: [0,0,dp(24),dp(0)]
+                    height: dp(48)
+                    
+                    MDLabel:
+                        text: "Compose messages in markdown"
+                        font_style: "H6"
+
+                    MDSwitch:
+                        id: settings_compose_in_markdown
+                        pos_hint: {"center_y": 0.3}
+                        disabled: False
+                        active: False
+
                 MDBoxLayout:
                     orientation: "horizontal"
                     size_hint_y: None
@@ -1774,7 +1800,7 @@ MDScreen:
                     height: dp(48)
                     
                     MDLabel:
-                        text: "Use high-quality voice for PTT"
+                        text: "High-quality codec for LXMF PTT"
                         font_style: "H6"
 
                     MDSwitch:
@@ -1783,6 +1809,22 @@ MDScreen:
                         disabled: False
                         active: False
 
+                MDBoxLayout:
+                    orientation: "horizontal"
+                    size_hint_y: None
+                    padding: [0,0,dp(24),dp(0)]
+                    height: dp(48)
+                    
+                    MDLabel:
+                        text: "Enable voice calls"
+                        font_style: "H6"
+
+                    MDSwitch:
+                        id: settings_voice_enabled
+                        pos_hint: {"center_y": 0.3}
+                        disabled: False
+                        active: False
+
                 # MDBoxLayout:
                 #     orientation: "horizontal"
                 #     size_hint_y: None

sbapp/ui/messages.py

@@ -34,14 +34,14 @@ if RNS.vendor.platformutils.get_platform() == "android":
     import plyer
     from sideband.sense import Telemeter, Commands
     from ui.helpers import ts_format, file_ts_format, mdc
-    from ui.helpers import color_playing, color_received, color_delivered, color_propagated, color_paper, color_failed, color_unknown, intensity_msgs_dark, intensity_msgs_light, intensity_play_dark, intensity_play_light
-    from ui.helpers import color_received_alt, color_received_alt_light, color_delivered_alt, color_propagated_alt, color_paper_alt, color_failed_alt, color_unknown_alt, color_playing_alt, intensity_msgs_dark_alt, intensity_msgs_light_alt, intensity_delivered_alt_dark
+    from ui.helpers import color_playing, color_received, color_delivered, color_propagated, color_paper, color_failed, color_unknown, intensity_msgs_dark, intensity_msgs_light, intensity_play_dark, intensity_play_light, color_cancelled, intensity_cancelled
+    from ui.helpers import color_received_alt, color_received_alt_light, color_delivered_alt, color_propagated_alt, color_paper_alt, color_failed_alt, color_unknown_alt, color_playing_alt, intensity_msgs_dark_alt, intensity_msgs_light_alt, intensity_delivered_alt_dark, color_cancelled_alt
 else:
     import sbapp.plyer as plyer
     from sbapp.sideband.sense import Telemeter, Commands
     from .helpers import ts_format, file_ts_format, mdc
-    from .helpers import color_playing, color_received, color_delivered, color_propagated, color_paper, color_failed, color_unknown, intensity_msgs_dark, intensity_msgs_light, intensity_play_dark, intensity_play_light
-    from .helpers import color_received_alt, color_received_alt_light, color_delivered_alt, color_propagated_alt, color_paper_alt, color_failed_alt, color_unknown_alt, color_playing_alt, intensity_msgs_dark_alt, intensity_msgs_light_alt, intensity_delivered_alt_dark
+    from .helpers import color_playing, color_received, color_delivered, color_propagated, color_paper, color_failed, color_unknown, intensity_msgs_dark, intensity_msgs_light, intensity_play_dark, intensity_play_light, color_cancelled, intensity_cancelled
+    from .helpers import color_received_alt, color_received_alt_light, color_delivered_alt, color_propagated_alt, color_paper_alt, color_failed_alt, color_unknown_alt, color_playing_alt, intensity_msgs_dark_alt, intensity_msgs_light_alt, intensity_delivered_alt_dark, color_cancelled_alt
 
 if RNS.vendor.platformutils.is_darwin():
     from PIL import Image as PilImage
@@ -110,8 +110,6 @@ class Messages():
         msg = self.app.sideband.message(lxm_hash)
         if msg:
             close_button = MDRectangleFlatButton(text="Close", font_size=dp(18))
-            # d_items = [ ]
-            # d_items.append(DialogItem(IconLeftWidget(icon="postage-stamp"), text="[size="+str(ss)+"]Stamp[/size]"))
 
             d_text = ""
 
@@ -213,6 +211,7 @@ class Messages():
             c_paper      = color_paper_alt
             c_unknown    = color_unknown_alt
             c_failed     = color_failed_alt
+            c_cancelled  = color_cancelled_alt
         else:
             c_delivered  = color_delivered
             c_received   = color_received
@@ -221,6 +220,7 @@ class Messages():
             c_paper      = color_paper
             c_unknown    = color_unknown
             c_failed     = color_failed
+            c_cancelled  = color_cancelled
 
         for new_message in self.app.sideband.list_messages(self.context_dest, after=self.latest_message_timestamp,limit=limit):
             self.new_messages.append(new_message)
@@ -301,6 +301,17 @@ class Messages():
                         delivery_syms += " 📦"
                     delivery_syms = multilingual_markup(delivery_syms.encode("utf-8")).decode("utf-8")
 
+                    if msg["state"] > LXMF.LXMessage.SENT:
+                        if hasattr(w, "dmenu"):
+                            if hasattr(w.dmenu, "items"):
+                                remove_item = None
+                                for item in w.dmenu.items:
+                                    if item["text"] == "Cancel message":
+                                        remove_item = item
+                                        break
+                                if remove_item != None:
+                                    w.dmenu.items.remove(remove_item)
+
                     if msg["state"] == LXMF.LXMessage.OUTBOUND or msg["state"] == LXMF.LXMessage.SENDING or msg["state"] == LXMF.LXMessage.SENT:
                         w.md_bg_color = msg_color = mdc(c_unknown, intensity_msgs)
                         txstr = time.strftime(ts_format, time.localtime(msg["sent"]))
@@ -334,6 +345,12 @@ class Messages():
                             w.heading += f"\n[b]Audio Message[/b] ({alstr})"
                         m["state"] = msg["state"]
 
+                    att_heading_str = ""
+                    if hasattr(w, "has_attachment") and w.has_attachment:
+                        att_heading_str = "\n[b]Attachments[/b] "
+                        for attachment in w.attachments_field:
+                            att_heading_str += str(attachment[0])+", "
+                        att_heading_str = att_heading_str[:-2]
 
                     if msg["state"] == LXMF.LXMessage.DELIVERED:
                         w.md_bg_color = msg_color = mdc(c_delivered, intensity_delivered)
@@ -369,7 +386,7 @@ class Messages():
                         m["state"] = msg["state"]
 
                     if msg["state"] == LXMF.LXMessage.FAILED:
-                        w.md_bg_color = msg_color = mdc(c_failed, intensity_msgs)
+                        w.md_bg_color = msg_color = mdc(c_failed, intensity_cancelled)
                         txstr = time.strftime(ts_format, time.localtime(msg["sent"]))
                         titlestr = ""
                         if msg["title"]:
@@ -381,6 +398,34 @@ class Messages():
                             w.heading += f"\n[b]Audio Message[/b] ({alstr})"
                         w.dmenu.items.append(w.dmenu.retry_item)
 
+                    if msg["state"] == LXMF.LXMessage.CANCELLED:
+                        w.md_bg_color = msg_color = mdc(c_cancelled, intensity_cancelled)
+                        txstr = time.strftime(ts_format, time.localtime(msg["sent"]))
+                        titlestr = ""
+                        if msg["title"]:
+                            titlestr = "[b]Title[/b] "+msg["title"].decode("utf-8")+"\n"
+                        w.heading = titlestr+"[b]Sent[/b] "+txstr+"\n[b]State[/b] Cancelled"
+                        m["state"] = msg["state"]
+                        if w.has_audio:
+                            alstr = RNS.prettysize(w.audio_size)
+                            w.heading += f"\n[b]Audio Message[/b] ({alstr})"
+                        w.dmenu.items.append(w.dmenu.retry_item)
+
+                    if msg["state"] == LXMF.LXMessage.REJECTED:
+                        w.md_bg_color = msg_color = mdc(c_cancelled, intensity_cancelled)
+                        txstr = time.strftime(ts_format, time.localtime(msg["sent"]))
+                        titlestr = ""
+                        if msg["title"]:
+                            titlestr = "[b]Title[/b] "+msg["title"].decode("utf-8")+"\n"
+                        w.heading = titlestr+"[b]Sent[/b] "+txstr+"\n[b]State[/b] Rejected"
+                        m["state"] = msg["state"]
+                        if w.has_audio:
+                            alstr = RNS.prettysize(w.audio_size)
+                            w.heading += f"\n[b]Audio Message[/b] ({alstr})"
+                        w.dmenu.items.append(w.dmenu.retry_item)
+
+                    w.heading += att_heading_str
+
 
     def hide_widget(self, wid, dohide=True):
         if hasattr(wid, 'saved_attrs'):
@@ -427,6 +472,7 @@ class Messages():
             c_paper      = color_paper_alt
             c_unknown    = color_unknown_alt
             c_failed     = color_failed_alt
+            c_cancelled  = color_cancelled_alt
         else:
             c_delivered  = color_delivered
             c_received   = color_received
@@ -435,6 +481,7 @@ class Messages():
             c_paper      = color_paper
             c_unknown    = color_unknown
             c_failed     = color_failed
+            c_cancelled  = color_cancelled
 
         self.ids.message_text.font_name = self.app.input_font
 
@@ -443,13 +490,27 @@ class Messages():
 
         for m in self.new_messages:
             if not m["hash"] in self.added_item_hashes:
+                renderer = None
+                message_source = m["content"]
+                if "lxm" in m and m["lxm"] and m["lxm"].fields != None and LXMF.FIELD_RENDERER in m["lxm"].fields:
+                    renderer = m["lxm"].fields[LXMF.FIELD_RENDERER]
+
                 try:
                     if self.app.sideband.config["trusted_markup_only"] and not self.is_trusted:
                         message_input = str( escape_markup(m["content"].decode("utf-8")) ).encode("utf-8")
                     else:
                         message_input = m["content"]
+                        if renderer == LXMF.RENDERER_MARKDOWN:
+                            message_input = self.app.md_to_bbcode(message_input.decode("utf-8")).encode("utf-8")
+                            message_input = self.app.process_bb_markup(message_input.decode("utf-8")).encode("utf-8")
+                        elif renderer == LXMF.RENDERER_BBCODE:
+                            message_input = self.app.process_bb_markup(message_input.decode("utf-8")).encode("utf-8")
+                        else:
+                            message_input = str(escape_markup(m["content"].decode("utf-8"))).encode("utf-8")
+
                 except Exception as e:
                     RNS.log(f"Message content could not be decoded: {e}", RNS.LOG_DEBUG)
+                    RNS.trace_exception(e)
                     message_input = b""
 
                 if message_input.strip() == b"":
@@ -602,9 +663,17 @@ class Messages():
                         heading_str = titlestr+"[b]Created[/b] "+txstr+"\n[b]State[/b] Paper Message"
 
                     elif m["state"] == LXMF.LXMessage.FAILED:
-                        msg_color = mdc(c_failed, intensity_msgs)
+                        msg_color = mdc(c_failed, intensity_cancelled)
                         heading_str = titlestr+"[b]Sent[/b] "+txstr+"\n[b]State[/b] Failed"
 
+                    elif m["state"] == LXMF.LXMessage.CANCELLED:
+                        msg_color = mdc(c_cancelled, intensity_cancelled)
+                        heading_str = titlestr+"[b]Sent[/b] "+txstr+"\n[b]State[/b] Cancelled"
+
+                    elif m["state"] == LXMF.LXMessage.REJECTED:
+                        msg_color = mdc(c_cancelled, intensity_cancelled)
+                        heading_str = titlestr+"[b]Sent[/b] "+txstr+"\n[b]State[/b] Rejected"
+
                     elif m["state"] == LXMF.LXMessage.OUTBOUND or m["state"] == LXMF.LXMessage.SENDING:
                         msg_color = mdc(c_unknown, intensity_msgs)
                         heading_str = titlestr+"[b]Sent[/b] "+txstr+"\n[b]State[/b] Sending                          "
@@ -618,9 +687,6 @@ class Messages():
                     heading_str = titlestr
                     if phy_stats_str != "" and self.app.sideband.config["advanced_stats"]:
                         heading_str += phy_stats_str+"\n"
-                    # TODO: Remove
-                    # if stamp_valid:
-                    #     txstr += f" [b]Stamp[/b] value is {stamp_value} "
 
                     heading_str += "[b]Sent[/b] "+txstr+delivery_syms
                     heading_str += "\n[b]Received[/b] "+rxstr
@@ -658,19 +724,29 @@ class Messages():
 
                 if has_attachment:
                     item.attachments_field = attachments_field
+                    item.has_attachment = True
+                else:
+                    item.has_attachment = False
 
                 if has_audio:
                     def play_audio(sender):
-                        self.app.play_audio_field(sender.audio_field)
-                        stored_color = sender.md_bg_color
-                        if sender.lsource == self.app.sideband.lxmf_destination.hash:
-                            sender.md_bg_color = mdc(c_delivered, intensity_play)
-                        else:
-                            sender.md_bg_color = mdc(c_received, intensity_play)
+                        touch_event = None; block_play = False
+                        if sender and hasattr(sender, "last_touch"): touch_event = sender.last_touch
+                        if touch_event and hasattr(touch_event, "dpos"):
+                            delta = abs(touch_event.dpos[0]) + abs(touch_event.dpos[1])
+                            if delta >= 2.0: block_play = True
 
-                        def cb(dt):
-                            sender.md_bg_color = stored_color
-                        Clock.schedule_once(cb, 0.25)
+                        if not block_play:
+                            self.app.play_audio_field(sender.audio_field)
+                            stored_color = sender.md_bg_color
+                            if sender.lsource == self.app.sideband.lxmf_destination.hash:
+                                sender.md_bg_color = mdc(c_delivered, intensity_play)
+                            else:
+                                sender.md_bg_color = mdc(c_received, intensity_play)
+
+                            def cb(dt):
+                                sender.md_bg_color = stored_color
+                            Clock.schedule_once(cb, 0.25)
 
                     item.has_audio = True
                     item.audio_size = len(audio_field[1])
@@ -798,6 +874,13 @@ class Messages():
 
                     return x
 
+                def gen_cancel(mhash, item):
+                    def x():
+                        self.app.sideband.cancel_message(mhash)
+                        item.dmenu.dismiss()
+
+                    return x
+
                 def gen_save_image(item):
                     if RNS.vendor.platformutils.is_android():
                         def x():
@@ -1080,7 +1163,7 @@ class Messages():
                                 "viewclass": "OneLineListItem",
                                 "text": "Copy message text",
                                 "height": dp(40),
-                                "on_release": gen_copy(message_input.decode("utf-8"), item)
+                                "on_release": gen_copy(message_source.decode("utf-8"), item)
                             },
                             {
                                 "text": "Delete",
@@ -1114,7 +1197,7 @@ class Messages():
                                 "viewclass": "OneLineListItem",
                                 "text": "Copy message text",
                                 "height": dp(40),
-                                "on_release": gen_copy(message_input.decode("utf-8"), item)
+                                "on_release": gen_copy(message_source.decode("utf-8"), item)
                             },
                             {
                                 "text": "Delete",
@@ -1132,7 +1215,7 @@ class Messages():
                                 "viewclass": "OneLineListItem",
                                 "text": "Copy",
                                 "height": dp(40),
-                                "on_release": gen_copy(message_input.decode("utf-8"), item)
+                                "on_release": gen_copy(message_source.decode("utf-8"), item)
                             },
                             {
                                 "text": "Delete",
@@ -1149,7 +1232,7 @@ class Messages():
                                     "viewclass": "OneLineListItem",
                                     "text": "Copy",
                                     "height": dp(40),
-                                    "on_release": gen_copy(message_input.decode("utf-8"), item)
+                                    "on_release": gen_copy(message_source.decode("utf-8"), item)
                                 },
                                 {
                                     "viewclass": "OneLineListItem",
@@ -1172,7 +1255,7 @@ class Messages():
                                     "viewclass": "OneLineListItem",
                                     "text": "Copy",
                                     "height": dp(40),
-                                    "on_release": gen_copy(message_input.decode("utf-8"), item)
+                                    "on_release": gen_copy(message_source.decode("utf-8"), item)
                                 },
                                 {
                                     "text": "Delete",
@@ -1197,6 +1280,14 @@ class Messages():
                                 "on_release": gen_save_attachment(item)
                             }
                             dm_items.append(extra_item)
+                        if m["source"] == self.app.sideband.lxmf_destination.hash and m["state"] <= LXMF.LXMessage.SENT:
+                            extra_item = {
+                                "viewclass": "OneLineListItem",
+                                "text": "Cancel message",
+                                "height": dp(40),
+                                "on_release": gen_cancel(m["hash"], item)
+                            }
+                            dm_items.append(extra_item)
 
                 item.dmenu = MDDropdownMenu(
                     caller=item.ids.msg_submenu,

sbapp/ui/objectdetails.py

@@ -767,6 +767,16 @@ class RVDetails(MDRecycleView):
                             threading.Thread(target=lj, daemon=True).start()
 
                         release_function = select
+                    
+                    elif name == "Reticulum Transport":
+                        te = "enabled" if s["values"]["transport_enabled"] else "disabled"
+                        formatted_values = f"Reticulum Transport [b]{te}[/b]"
+                    
+                    elif name == "LXMF Propagation":
+                        tp = str(s["values"]["total_peers"])
+                        ap = str(s["values"]["active_peers"])
+                        formatted_values = f"Peered with [b]{tp}[/b] LXMF Propagation Nodes, [b]{ap}[/b] available"
+                    
                     else:
                         formatted_values = f"{name}"
                         for vn in s["values"]:
@@ -812,17 +822,23 @@ class RVDetails(MDRecycleView):
                 if nhi and nhi != "None":
                     self.entries.append({"icon": "routes", "text": f"Current path on [b]{nhi}[/b]", "on_release": pass_job})
 
-                try:
-                    ler = self.delegate.app.sideband.get_destination_establishment_rate(self.delegate.object_hash)
-                    if ler:
-                        lers = RNS.prettyspeed(ler, "b")
-                        self.entries.append({"icon": "lock-check-outline", "text": f"Direct link established, LER is [b]{lers}[/b]", "on_release": pass_job})
-                except Exception as e:
-                    RNS.trace_exception(e)
-
                 if nh != RNS.Transport.PATHFINDER_M:
                     hs = "hop" if nh == 1 else "hops"
                     self.entries.append({"icon": "atom-variant", "text": f"Network distance is [b]{nh} {hs}[/b]", "on_release": pass_job})
+
+                try:
+                    ler = self.delegate.app.sideband.get_destination_establishment_rate(self.delegate.object_hash)
+                    mtu = self.delegate.app.sideband.get_destination_mtu(self.delegate.object_hash) or RNS.Reticulum.MTU
+                    edr = self.delegate.app.sideband.get_destination_edr(self.delegate.object_hash)
+                    if ler:
+                        lers = RNS.prettyspeed(ler, "b")
+                        mtus = RNS.prettysize(mtu)
+                        edrs = f"{RNS.prettyspeed(edr)}" if edr != None else ""
+                        self.entries.append({"icon": "lock-check-outline", "text": f"Link established, LER is [b]{lers}[/b], MTU is [b]{mtus}[/b]", "on_release": pass_job})
+                        if edr: self.entries.append({"icon": "approximately-equal", "text": f"Expected data rate is [b]{edrs}[/b]", "on_release": pass_job})
+                except Exception as e:
+                    RNS.trace_exception(e)
+
             except Exception as e:
                 RNS.trace_exception(e)
 

sbapp/ui/telemetry.py

@@ -44,6 +44,30 @@ class Telemetry():
         else:
             self.screen.ids.telemetry_collector.text = RNS.hexrep(self.app.sideband.config["telemetry_collector"], delimit=False)
 
+        self.screen.ids.telemetry_mqtt_host.bind(focus=self.telemetry_save)
+        if self.app.sideband.config["telemetry_mqtt_host"] == None:
+            self.screen.ids.telemetry_mqtt_host.text = ""
+        else:
+            self.screen.ids.telemetry_mqtt_host.text = self.app.sideband.config["telemetry_mqtt_host"]
+
+        self.screen.ids.telemetry_mqtt_port.bind(focus=self.telemetry_save)
+        if self.app.sideband.config["telemetry_mqtt_port"] == None:
+            self.screen.ids.telemetry_mqtt_port.text = ""
+        else:
+            self.screen.ids.telemetry_mqtt_port.text = self.app.sideband.config["telemetry_mqtt_port"]
+
+        self.screen.ids.telemetry_mqtt_user.bind(focus=self.telemetry_save)
+        if self.app.sideband.config["telemetry_mqtt_user"] == None:
+            self.screen.ids.telemetry_mqtt_user.text = ""
+        else:
+            self.screen.ids.telemetry_mqtt_user.text = self.app.sideband.config["telemetry_mqtt_user"]
+
+        self.screen.ids.telemetry_mqtt_pass.bind(focus=self.telemetry_save)
+        if self.app.sideband.config["telemetry_mqtt_pass"] == None:
+            self.screen.ids.telemetry_mqtt_pass.text = ""
+        else:
+            self.screen.ids.telemetry_mqtt_pass.text = self.app.sideband.config["telemetry_mqtt_pass"]
+
         self.screen.ids.telemetry_icon_preview.icon_color  = self.app.sideband.config["telemetry_fg"]
         self.screen.ids.telemetry_icon_preview.md_bg_color = self.app.sideband.config["telemetry_bg"]
         self.screen.ids.telemetry_icon_preview.icon = self.app.sideband.config["telemetry_icon"]
@@ -83,6 +107,9 @@ class Telemetry():
         
         self.screen.ids.telemetry_allow_requests_from_anyone.active = self.app.sideband.config["telemetry_allow_requests_from_anyone"]
         self.screen.ids.telemetry_allow_requests_from_anyone.bind(active=self.telemetry_save)
+
+        self.screen.ids.telemetry_to_mqtt.active = self.app.sideband.config["telemetry_to_mqtt"]
+        self.screen.ids.telemetry_to_mqtt.bind(active=self.telemetry_save)
         
         
         self.screen.ids.telemetry_scrollview.effect_cls = ScrollEffect
@@ -259,6 +286,11 @@ class Telemetry():
         self.app.sideband.config["telemetry_allow_requests_from_trusted"] = self.screen.ids.telemetry_allow_requests_from_trusted.active
         self.app.sideband.config["telemetry_allow_requests_from_anyone"] = self.screen.ids.telemetry_allow_requests_from_anyone.active
         self.app.sideband.config["telemetry_collector_enabled"] = self.screen.ids.telemetry_collector_enabled.active
+        self.app.sideband.config["telemetry_to_mqtt"] = self.screen.ids.telemetry_to_mqtt.active
+        self.app.sideband.config["telemetry_mqtt_host"] = self.screen.ids.telemetry_mqtt_host.text
+        self.app.sideband.config["telemetry_mqtt_port"] = self.screen.ids.telemetry_mqtt_port.text
+        self.app.sideband.config["telemetry_mqtt_user"] = self.screen.ids.telemetry_mqtt_user.text
+        self.app.sideband.config["telemetry_mqtt_pass"] = self.screen.ids.telemetry_mqtt_pass.text
         
         self.app.sideband.save_configuration()
         if run_telemetry_update:
@@ -366,6 +398,8 @@ class Telemetry():
         self.sensors_screen.ids.telemetry_s_accelerometer.bind(active=self.sensors_save)
         self.sensors_screen.ids.telemetry_s_proximity.active = self.app.sideband.config["telemetry_s_proximity"]
         self.sensors_screen.ids.telemetry_s_proximity.bind(active=self.sensors_save)
+        self.sensors_screen.ids.telemetry_s_rns_transport.active = self.app.sideband.config["telemetry_s_rns_transport"]
+        self.sensors_screen.ids.telemetry_s_rns_transport.bind(active=self.sensors_save)
         self.sensors_screen.ids.telemetry_s_information.active = self.app.sideband.config["telemetry_s_information"]
         self.sensors_screen.ids.telemetry_s_information.bind(active=self.sensors_save)
         self.sensors_screen.ids.telemetry_s_information_text.text = str(self.app.sideband.config["telemetry_s_information_text"])
@@ -434,6 +468,7 @@ class Telemetry():
         self.app.sideband.config["telemetry_s_angular_velocity"] = self.sensors_screen.ids.telemetry_s_gyroscope.active
         self.app.sideband.config["telemetry_s_acceleration"] = self.sensors_screen.ids.telemetry_s_accelerometer.active
         self.app.sideband.config["telemetry_s_proximity"] = self.sensors_screen.ids.telemetry_s_proximity.active
+        self.app.sideband.config["telemetry_s_rns_transport"] = self.sensors_screen.ids.telemetry_s_rns_transport.active
 
         if self.app.sideband.config["telemetry_s_information"] != self.sensors_screen.ids.telemetry_s_information.active:
             run_telemetry_update = True
@@ -880,6 +915,90 @@ MDScreen:
                             on_release: root.delegate.telemetry_bg_color(self)
                             disabled: False
 
+                MDLabel:
+                    text: "MQTT Configuration"
+                    font_style: "H6"
+
+                MDLabel:
+                    id: telemetry_info6
+                    markup: True
+                    text: "\\nFor integration with other systems, you can configure Sideband to send all known telemetry data to an MQTT server in real-time as it is received or generated.\\n"
+                    size_hint_y: None
+                    text_size: self.width, None
+                    height: self.texture_size[1]
+
+                MDBoxLayout:
+                    orientation: "horizontal"
+                    size_hint_y: None
+                    padding: [0,0,dp(24),dp(0)]
+                    height: dp(48)
+                    
+                    MDLabel:
+                        text: "Send telemetry to MQTT"
+                        font_style: "H6"
+
+                    MDSwitch:
+                        id: telemetry_to_mqtt
+                        pos_hint: {"center_y": 0.3}
+                        active: False
+
+                MDBoxLayout:
+                    orientation: "vertical"
+                    spacing: dp(24)
+                    size_hint_y: None
+                    padding: [dp(0),dp(0),dp(0),dp(0)]
+                    #height: dp(232)
+                    height: self.minimum_height
+
+                    MDTextField:
+                        id: telemetry_mqtt_host
+                        hint_text: "Server Hostname"
+                        text: ""
+                        font_size: dp(24)
+
+                MDBoxLayout:
+                    orientation: "vertical"
+                    spacing: dp(24)
+                    size_hint_y: None
+                    padding: [dp(0),dp(0),dp(0),dp(0)]
+                    #height: dp(232)
+                    height: self.minimum_height
+
+                    MDTextField:
+                        id: telemetry_mqtt_port
+                        hint_text: "Server Port"
+                        text: ""
+                        font_size: dp(24)
+
+                MDBoxLayout:
+                    orientation: "vertical"
+                    spacing: dp(24)
+                    size_hint_y: None
+                    padding: [dp(0),dp(0),dp(0),dp(0)]
+                    #height: dp(232)
+                    height: self.minimum_height
+
+                    MDTextField:
+                        id: telemetry_mqtt_user
+                        hint_text: "Username"
+                        text: ""
+                        font_size: dp(24)
+
+                MDBoxLayout:
+                    orientation: "vertical"
+                    spacing: dp(24)
+                    size_hint_y: None
+                    padding: [dp(0),dp(0),dp(0),dp(60)]
+                    #height: dp(232)
+                    height: self.minimum_height
+
+                    MDTextField:
+                        id: telemetry_mqtt_pass
+                        password: True
+                        hint_text: "Password"
+                        text: ""
+                        font_size: dp(24)
+
                 MDLabel:
                     text: "Advanced Configuration"
                     font_style: "H6"
@@ -1211,6 +1330,21 @@ MDScreen:
                         pos_hint: {"center_y": 0.3}
                         active: False
 
+                MDBoxLayout:
+                    orientation: "horizontal"
+                    size_hint_y: None
+                    padding: [0,0,dp(24),dp(0)]
+                    height: dp(48)
+                    
+                    MDLabel:
+                        text: "Reticulum Transport Stats"
+                        font_style: "H6"
+
+                    MDSwitch:
+                        id: telemetry_s_rns_transport
+                        pos_hint: {"center_y": 0.3}
+                        active: False
+
                 MDBoxLayout:
                     orientation: "horizontal"
                     size_hint_y: None

sbapp/ui/utilities.py

@@ -40,7 +40,7 @@ class Utilities():
             self.screen.delegate = self
             self.app.root.ids.screen_manager.add_widget(self.screen)
         
-        self.screen.ids.telemetry_scrollview.effect_cls = ScrollEffect
+        self.screen.ids.utilities_scrollview.effect_cls = ScrollEffect
         info  = "This section contains various utilities and diagnostics tools, "
         info += "that can be helpful while using Sideband and Reticulum."
         
@@ -64,6 +64,7 @@ class Utilities():
         )
         def dl_yes(s):
             dialog.dismiss()
+            self.app.wants_flasher_launch = True
             self.app.sideband.start_webshare()
             def cb(dt):
                 self.app.repository_action()
@@ -135,13 +136,24 @@ class Utilities():
     def update_advanced(self, sender=None):
         if RNS.vendor.platformutils.is_android():
             ct = self.app.sideband.config["config_template"]
+            if ct == None:
+                ct = self.app.sideband.default_config_template
             self.advanced_screen.ids.config_template.text = f"[font=RobotoMono-Regular][size={int(dp(12))}]{ct}[/size][/font]"
         else:
             self.advanced_screen.ids.config_template.text = f"[font=RobotoMono-Regular][size={int(dp(12))}]On this platform, Reticulum configuration is managed by the system. You can change the configuration by editing the file located at:\n\n{self.app.sideband.reticulum.configpath}[/size][/font]"
 
+    def reset_config(self, sender=None):
+        if RNS.vendor.platformutils.is_android():
+            self.app.sideband.config["config_template"] = None
+            self.app.sideband.save_configuration()
+            self.update_advanced()
+
     def copy_config(self, sender=None):
         if RNS.vendor.platformutils.is_android():
-            Clipboard.copy(self.app.sideband.config_template)
+            if self.app.sideband.config["config_template"]:
+                Clipboard.copy(self.app.sideband.config["config_template"])
+            else:
+                Clipboard.copy(self.app.sideband.default_config_template)
 
     def paste_config(self, sender=None):
         if RNS.vendor.platformutils.is_android():
@@ -208,7 +220,7 @@ MDScreen:
                 ]
 
         ScrollView:
-            id: telemetry_scrollview
+            id: utilities_scrollview
 
             MDBoxLayout:
                 orientation: "vertical"
@@ -409,7 +421,7 @@ MDScreen:
                     padding: [dp(0), dp(14), dp(0), dp(24)]
 
                     MDRectangleFlatIconButton:
-                        id: telemetry_button
+                        id: conf_copy_button
                         icon: "content-copy"
                         text: "Copy Configuration"
                         padding: [dp(0), dp(14), dp(0), dp(14)]
@@ -420,7 +432,7 @@ MDScreen:
                         disabled: False
 
                     MDRectangleFlatIconButton:
-                        id: coordinates_button
+                        id: conf_paste_button
                         icon: "download"
                         text: "Paste Configuration"
                         padding: [dp(0), dp(14), dp(0), dp(14)]
@@ -430,6 +442,23 @@ MDScreen:
                         on_release: root.delegate.paste_config(self)
                         disabled: False
 
+                MDBoxLayout:
+                    orientation: "horizontal"
+                    spacing: dp(24)
+                    size_hint_y: None
+                    height: self.minimum_height
+                    padding: [dp(0), dp(0), dp(0), dp(24)]
+
+                    MDRectangleFlatIconButton:
+                        id: conf_reset_button
+                        icon: "cog-counterclockwise"
+                        text: "Reset Configuration"
+                        padding: [dp(0), dp(14), dp(0), dp(14)]
+                        icon_size: dp(24)
+                        font_size: dp(16)
+                        size_hint: [1.0, None]
+                        on_release: root.delegate.reset_config(self)
+                        disabled: False
 
                 MDLabel:
                     id: config_template

sbapp/ui/voice.py

@@ -0,0 +1,481 @@
+import time
+import RNS
+
+from typing import Union
+from kivy.metrics import dp,sp
+from kivy.lang.builder import Builder
+from kivy.core.clipboard import Clipboard
+from kivy.utils import escape_markup
+from kivymd.uix.recycleview import MDRecycleView
+from kivymd.uix.list import OneLineIconListItem
+from kivymd.uix.pickers import MDColorPicker
+from kivymd.uix.button import MDRectangleFlatButton
+from kivymd.uix.button import MDRectangleFlatIconButton
+from kivymd.uix.dialog import MDDialog
+from kivymd.icon_definitions import md_icons
+from kivymd.toast import toast
+from kivy.properties import StringProperty, BooleanProperty
+from kivy.effects.scroll import ScrollEffect
+from kivy.clock import Clock
+from sideband.sense import Telemeter
+import threading
+from datetime import datetime
+
+if RNS.vendor.platformutils.get_platform() == "android":
+    from ui.helpers import ts_format
+    from android.permissions import request_permissions, check_permission
+else:
+    from .helpers import ts_format
+
+class Voice():
+    def __init__(self, app):
+        self.app = app
+        self.screen = None
+        self.settings_screen = None
+        self.dial_target = None
+        self.ui_updater = None
+        self.path_requesting = None
+        self.output_devices = []
+        self.input_devices = []
+        self.listed_output_devices = []
+        self.listed_input_devices = []
+        self.listed_ringer_devices = []
+    
+        if not self.app.root.ids.screen_manager.has_screen("voice_screen"):
+            self.screen = Builder.load_string(layout_voice_screen)
+            self.screen.app = self.app
+            self.screen.delegate = self
+            self.app.root.ids.screen_manager.add_widget(self.screen)
+        
+        self.screen.ids.voice_scrollview.effect_cls = ScrollEffect
+
+    def update_call_status(self, dt=None):
+        if self.app.root.ids.screen_manager.current == "voice_screen":
+            if self.ui_updater == None: self.ui_updater = Clock.schedule_interval(self.update_call_status, 0.5)
+        else:
+            if self.ui_updater:
+                self.ui_updater.cancel()
+                self.ui_updater = None
+
+        db = self.screen.ids.dial_button
+        ih = self.screen.ids.identity_hash
+        if self.app.sideband.voice_running:
+            telephone = self.app.sideband.telephone
+            if self.path_requesting:
+                db.disabled = True
+                ih.disabled = True
+
+            else:
+                if telephone.is_available:
+                    ih.disabled = False
+                    self.target_input_action(ih)
+                else:
+                    ih.disabled = True
+
+                if telephone.is_in_call or telephone.call_is_connecting:
+                    ih.disabled = True
+                    db.disabled = False
+                    db.text = "Hang up"
+                    db.icon = "phone-hangup"
+
+                elif telephone.is_ringing:
+                    ih.disabled = True
+                    db.disabled = False
+                    db.text = "Answer"
+                    db.icon = "phone-ring"
+                    if telephone.caller: ih.text = RNS.hexrep(telephone.caller.hash, delimit=False)
+
+        else:
+            db.disabled = True; db.text = "Voice calls disabled"
+            ih.disabled = True
+
+    def target_valid(self):
+        if self.app.sideband.voice_running:
+            db = self.screen.ids.dial_button
+            db.disabled = False; db.text = "Call"
+            db.icon = "phone-outgoing"
+
+    def target_invalid(self):
+        if self.app.sideband.voice_running:
+            db = self.screen.ids.dial_button
+            db.disabled = True; db.text = "Call"
+            db.icon = "phone-outgoing"
+
+    def target_input_action(self, sender):
+        if sender:
+            target_hash = sender.text
+            if len(target_hash) == RNS.Reticulum.TRUNCATED_HASHLENGTH//8*2:
+                try:
+                    identity_hash = bytes.fromhex(target_hash)
+                    self.dial_target = identity_hash
+                    self.target_valid()
+
+                except Exception as e: self.target_invalid()
+            else: self.target_invalid()
+
+    def request_path(self, destination_hash):
+        if not self.path_requesting:
+            self.app.sideband.telephone.set_busy(True)
+            toast("Requesting path...")
+            self.screen.ids.dial_button.disabled = True
+            self.path_requesting = destination_hash
+            RNS.Transport.request_path(destination_hash)
+            threading.Thread(target=self._path_wait_job, daemon=True).start()
+        
+        else:
+            toast("Waiting for path request answer...")
+
+    def _path_wait_job(self):
+        timeout = time.time()+self.app.sideband.telephone.PATH_TIME
+        while not RNS.Transport.has_path(self.path_requesting) and time.time() < timeout:
+            time.sleep(0.25)
+
+        self.app.sideband.telephone.set_busy(False)
+        if RNS.Transport.has_path(self.path_requesting):
+            RNS.log(f"Calling {RNS.prettyhexrep(self.dial_target)}...", RNS.LOG_DEBUG)
+            self.app.sideband.telephone.dial(self.dial_target)
+            Clock.schedule_once(self.update_call_status, 0.1)
+
+        else:
+            Clock.schedule_once(self._path_request_failed, 0.05)
+            Clock.schedule_once(self.update_call_status, 0.1)
+
+        self.path_requesting = None
+        self.update_call_status()
+
+    def _path_request_failed(self, dt):
+        toast("Path request timed out")
+
+    def dial_action(self, sender=None):
+        if self.app.sideband.voice_running:
+            if self.app.sideband.telephone.is_available:
+
+                destination_hash = RNS.Destination.hash_from_name_and_identity("lxst.telephony", self.dial_target)
+                if not RNS.Transport.has_path(destination_hash):
+                    self.request_path(destination_hash)
+
+                else:
+                    RNS.log(f"Calling {RNS.prettyhexrep(self.dial_target)}...", RNS.LOG_DEBUG)
+                    self.app.sideband.telephone.dial(self.dial_target)
+                    self.update_call_status()
+
+            elif self.app.sideband.telephone.is_in_call or self.app.sideband.telephone.call_is_connecting:
+                RNS.log(f"Hanging up", RNS.LOG_DEBUG)
+                self.app.sideband.telephone.hangup()
+                self.update_call_status()
+
+            elif self.app.sideband.telephone.is_ringing:
+                RNS.log(f"Answering", RNS.LOG_DEBUG)
+                self.app.sideband.telephone.answer()
+                self.update_call_status()
+
+
+    ### settings screen
+    ######################################
+
+    def settings_action(self, sender=None):
+        if not self.app.root.ids.screen_manager.has_screen("voice_settings_screen"):
+            self.voice_settings_screen = Builder.load_string(layout_voice_settings_screen)
+            self.voice_settings_screen.app = self.app
+            self.voice_settings_screen.delegate = self
+            self.app.root.ids.screen_manager.add_widget(self.voice_settings_screen)
+
+        self.app.root.ids.screen_manager.transition.direction = "left"
+        self.app.root.ids.screen_manager.current = "voice_settings_screen"
+        self.voice_settings_screen.ids.voice_settings_scrollview.effect_cls = ScrollEffect
+        self.app.sideband.setstate("app.displaying", self.app.root.ids.screen_manager.current)
+        
+        self.update_settings_screen()
+
+    def update_devices(self):
+        import LXST
+        self.output_devices = []; self.input_devices = []
+        for device in LXST.Sources.Backend().soundcard.all_speakers():  self.output_devices.append(device.name)
+        for device in LXST.Sinks.Backend().soundcard.all_microphones(): self.input_devices.append(device.name)
+        if self.app.sideband.config["voice_output"] != None:
+            if not self.app.sideband.config["voice_output"] in self.output_devices: self.output_devices.append(self.app.sideband.config["voice_output"])
+        if self.app.sideband.config["voice_input"] != None:
+            if not self.app.sideband.config["voice_input"] in self.input_devices: self.input_devices.append(self.app.sideband.config["voice_input"])
+        if self.app.sideband.config["voice_ringer"] != None:
+            if not self.app.sideband.config["voice_ringer"] in self.output_devices: self.output_devices.append(self.app.sideband.config["voice_ringer"])
+
+    def update_settings_screen(self, sender=None):
+        self.voice_settings_screen.ids.voice_trusted_only.active = self.app.sideband.config["voice_trusted_only"]
+        self.voice_settings_screen.ids.voice_trusted_only.bind(active=self.settings_save_action)
+
+        bp = 6; ml = 45; fs = 16; ics = 14
+        self.update_devices()
+
+        # Output devices
+        if not "system_default" in self.listed_output_devices:
+            default_output_button = MDRectangleFlatIconButton(text="System Default", font_size=dp(fs), icon_size=dp(ics), on_release=self.output_device_action)
+            default_output_button.device = None; default_output_button.size_hint = [1.0, None]
+            if self.app.sideband.config["voice_output"] == None: default_output_button.icon = "check"
+            self.voice_settings_screen.ids.output_devices.add_widget(default_output_button)
+            self.listed_output_devices.append("system_default")
+
+        for device in self.output_devices:
+            if not device in self.listed_output_devices:
+                label = device if len(device) < ml else device[:ml-3]+"..."
+                device_button = MDRectangleFlatIconButton(text=label, font_size=dp(fs), icon_size=dp(ics), on_release=self.output_device_action)
+                device_button.padding = [dp(bp), dp(bp), dp(bp), dp(bp)]; device_button.size_hint = [1.0, None]
+                if self.app.sideband.config["voice_output"] == device: device_button.icon = "check"
+                device_button.device = device
+                self.voice_settings_screen.ids.output_devices.add_widget(device_button)
+                self.listed_output_devices.append(device)
+
+        # Input devices
+        if not "system_default" in self.listed_input_devices:
+            default_input_button = MDRectangleFlatIconButton(text="System Default", font_size=dp(fs), icon_size=dp(ics), on_release=self.input_device_action)
+            default_input_button.device = None; default_input_button.size_hint = [1.0, None]
+            if self.app.sideband.config["voice_output"] == None: default_input_button.icon = "check"
+            self.voice_settings_screen.ids.input_devices.add_widget(default_input_button)
+            self.listed_input_devices.append("system_default")
+
+        for device in self.input_devices:
+            if not device in self.listed_input_devices:
+                label = device if len(device) < ml else device[:ml-3]+"..."
+                device_button = MDRectangleFlatIconButton(text=label, font_size=dp(fs), icon_size=dp(ics), on_release=self.input_device_action)
+                device_button.padding = [dp(bp), dp(bp), dp(bp), dp(bp)]; device_button.size_hint = [1.0, None]
+                if self.app.sideband.config["voice_input"] == device: device_button.icon = "check"
+                device_button.device = device
+                self.voice_settings_screen.ids.input_devices.add_widget(device_button)
+                self.listed_input_devices.append(device)
+
+        # Ringer devices
+        if not "system_default" in self.listed_ringer_devices:
+            default_ringer_button = MDRectangleFlatIconButton(text="System Default", font_size=dp(fs), icon_size=dp(ics), on_release=self.ringer_device_action)
+            default_ringer_button.device = None; default_ringer_button.size_hint = [1.0, None]
+            if self.app.sideband.config["voice_ringer"] == None: default_ringer_button.icon = "check"
+            self.voice_settings_screen.ids.ringer_devices.add_widget(default_ringer_button)
+            self.listed_ringer_devices.append("system_default")
+
+        for device in self.output_devices:
+            if not device in self.listed_ringer_devices:
+                label = device if len(device) < ml else device[:ml-3]+"..."
+                device_button = MDRectangleFlatIconButton(text=label, font_size=dp(fs), icon_size=dp(ics), on_release=self.ringer_device_action)
+                device_button.padding = [dp(bp), dp(bp), dp(bp), dp(bp)]; device_button.size_hint = [1.0, None]
+                if self.app.sideband.config["voice_ringer"] == device: device_button.icon = "check"
+                device_button.device = device
+                self.voice_settings_screen.ids.ringer_devices.add_widget(device_button)
+                self.listed_ringer_devices.append(device)
+
+    def settings_save_action(self, sender=None, event=None):
+        self.app.sideband.config["voice_trusted_only"] = self.voice_settings_screen.ids.voice_trusted_only.active
+        self.app.sideband.save_configuration()
+
+    def output_device_action(self, sender=None):
+        self.app.sideband.config["voice_output"] = sender.device
+        self.app.sideband.save_configuration()
+        for w in self.voice_settings_screen.ids.output_devices.children: w.icon = ""
+        sender.icon = "check"
+        if self.app.sideband.telephone:
+            self.app.sideband.telephone.set_speaker(self.app.sideband.config["voice_output"])
+
+    def input_device_action(self, sender=None):
+        self.app.sideband.config["voice_input"] = sender.device
+        self.app.sideband.save_configuration()
+        for w in self.voice_settings_screen.ids.input_devices.children: w.icon = ""
+        sender.icon = "check"
+        if self.app.sideband.telephone:
+            self.app.sideband.telephone.set_microphone(self.app.sideband.config["voice_input"])
+
+    def ringer_device_action(self, sender=None):
+        self.app.sideband.config["voice_ringer"] = sender.device
+        self.app.sideband.save_configuration()
+        for w in self.voice_settings_screen.ids.ringer_devices.children: w.icon = ""
+        sender.icon = "check"
+        if self.app.sideband.telephone:
+            self.app.sideband.telephone.set_ringer(self.app.sideband.config["voice_ringer"])
+
+
+layout_voice_screen = """
+MDScreen:
+    name: "voice_screen"
+    
+    BoxLayout:
+        orientation: "vertical"
+
+        MDTopAppBar:
+            title: "Voice"
+            anchor_title: "left"
+            elevation: 0
+            left_action_items:
+                [['menu', lambda x: root.app.nav_drawer.set_state("open")]]
+            right_action_items:
+                [
+                ['wrench-cog', lambda x: root.delegate.settings_action(self)],
+                ['close', lambda x: root.app.close_any_action(self)],
+                ]
+
+        ScrollView:
+            id: voice_scrollview
+
+            MDBoxLayout:
+                orientation: "vertical"
+                size_hint_y: None
+                height: self.minimum_height
+                padding: [dp(28), dp(32), dp(28), dp(16)]
+
+                MDBoxLayout:
+                    orientation: "vertical"
+                    # spacing: "24dp"
+                    size_hint_y: None
+                    height: self.minimum_height
+                    padding: [dp(0), dp(12), dp(0), dp(0)]
+
+                    MDTextField:
+                        id: identity_hash
+                        hint_text: "Identity hash"
+                        mode: "rectangle"
+                        # size_hint: [1.0, None]
+                        pos_hint: {"center_x": .5}
+                        max_text_length: 32
+                        on_text: root.delegate.target_input_action(self)
+
+                MDBoxLayout:
+                    orientation: "vertical"
+                    spacing: "24dp"
+                    size_hint_y: None
+                    height: self.minimum_height
+                    padding: [dp(0), dp(35), dp(0), dp(35)]
+
+                    MDRectangleFlatIconButton:
+                        id: dial_button
+                        icon: "phone-outgoing"
+                        text: "Call"
+                        padding: [dp(0), dp(14), dp(0), dp(14)]
+                        icon_size: dp(24)
+                        font_size: dp(16)
+                        size_hint: [1.0, None]
+                        on_release: root.delegate.dial_action(self)
+                        disabled: True
+"""
+
+layout_voice_settings_screen = """
+MDScreen:
+    name: "voice_settings_screen"
+    
+    BoxLayout:
+        orientation: "vertical"
+
+        MDTopAppBar:
+            id: top_bar
+            title: "Voice Configuration"
+            anchor_title: "left"
+            elevation: 0
+            left_action_items:
+                [['menu', lambda x: root.app.nav_drawer.set_state("open")]]
+            right_action_items:
+                [
+                ['close', lambda x: root.app.close_sub_voice_action(self)],
+                ]
+
+        MDScrollView:
+            id: voice_settings_scrollview
+            size_hint_x: 1
+            size_hint_y: None
+            size: [root.width, root.height-root.ids.top_bar.height]
+            do_scroll_x: False
+            do_scroll_y: True
+
+            MDBoxLayout:
+                orientation: "vertical"
+                size_hint_y: None
+                height: self.minimum_height
+                padding: [dp(28), dp(48), dp(28), dp(16)]
+
+                MDLabel:
+                    text: "Call Handling"
+                    font_style: "H6"
+                    height: self.texture_size[1]
+                    padding: [dp(0), dp(0), dp(0), dp(12)]
+
+                MDLabel:
+                    id: voice_settings_info
+                    markup: True
+                    text: "You can block calls from all other callers than contacts marked as trusted, by enabling the following option."
+                    size_hint_y: None
+                    text_size: self.width, None
+                    height: self.texture_size[1]
+                    padding: [dp(0), dp(16), dp(0), dp(16)]
+
+                MDBoxLayout:
+                    orientation: "horizontal"
+                    padding: [0,0,dp(24),0]
+                    size_hint_y: None
+                    height: dp(48)
+                    
+                    MDLabel:
+                        text: "Block non-trusted callers"
+                        font_style: "H6"
+
+                    MDSwitch:
+                        id: voice_trusted_only
+                        pos_hint: {"center_y": 0.3}
+                        active: False
+
+                MDLabel:
+                    text: "Audio Devices"
+                    font_style: "H6"
+                    padding: [dp(0), dp(96), dp(0), dp(12)]
+
+                MDLabel:
+                    id: voice_settings_info
+                    markup: True
+                    text: "You can configure which audio devices Sideband will use for voice calls, by selecting either the system default device, or specific audio devices available."
+                    size_hint_y: None
+                    text_size: self.width, None
+                    height: self.texture_size[1]
+                    padding: [dp(0), dp(64), dp(0), dp(32)]
+
+                MDLabel:
+                    text: "[b]Output[/b]"
+                    font_size: dp(18)
+                    markup: True
+
+                MDBoxLayout:
+                    id: output_devices
+                    orientation: "vertical"
+                    spacing: "12dp"
+                    size_hint_y: None
+                    height: self.minimum_height
+                    padding: [dp(0), dp(24), dp(0), dp(48)]
+
+                    # MDRectangleFlatIconButton:
+                    #     id: output_default_button
+                    #     text: "System Default"
+                    #     padding: [dp(0), dp(14), dp(0), dp(14)]
+                    #     icon_size: dp(24)
+                    #     font_size: dp(16)
+                    #     size_hint: [1.0, None]
+                    #     on_release: root.delegate.output_device_action(self)
+                    #     disabled: False
+
+                MDLabel:
+                    text: "[b]Input[/b]"
+                    font_size: dp(18)
+                    markup: True
+
+                MDBoxLayout:
+                    id: input_devices
+                    orientation: "vertical"
+                    spacing: "12dp"
+                    size_hint_y: None
+                    height: self.minimum_height
+                    padding: [dp(0), dp(24), dp(0), dp(48)]
+
+                MDLabel:
+                    text: "[b]Ringer[/b]"
+                    font_size: dp(18)
+                    markup: True
+
+                MDBoxLayout:
+                    id: ringer_devices
+                    orientation: "vertical"
+                    spacing: "12dp"
+                    size_hint_y: None
+                    height: self.minimum_height
+                    padding: [dp(0), dp(24), dp(0), dp(48)]
+
+"""

setup.py

@@ -114,8 +114,8 @@ setuptools.setup(
         ]
     },
     install_requires=[
-        "rns>=0.8.8",
-        "lxmf>=0.5.8",
+        "rns>=0.9.4",
+        "lxmf>=0.6.3",
         "kivy>=2.3.0",
         "pillow>=10.2.0",
         "qrcode",
@@ -123,10 +123,14 @@ setuptools.setup(
         "ffpyplayer",
         "sh",
         "numpy<=1.26.4",
+        "lxst>=0.2.7",
+        "mistune>=3.0.2",
+        "beautifulsoup4",
         "pycodec2;sys.platform!='Windows' and sys.platform!='win32' and sys.platform!='darwin'",
         "pyaudio;sys.platform=='linux'",
         "pyobjus;sys.platform=='darwin'",
         "pyogg;sys.platform=='Windows' and sys.platform!='win32'",
+        "audioop-lts>=0.2.1;python_version>='3.13'"
     ],
     python_requires='>=3.7',
 )

sideband.spec

@@ -7,7 +7,7 @@ a = Analysis(
     pathex=[],
     binaries=[],
     datas=[],
-    hiddenimports=[],
+    hiddenimports=["mistune", "bs4"],
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],
@@ -36,6 +36,7 @@ def extra_datas(mydir):
 a.datas += extra_datas('sbapp')
 a.datas += extra_datas('RNS')
 a.datas += extra_datas('LXMF')
+a.datas += extra_datas('LXST')
 
 exe = EXE(
     pyz,