Bug fixing and improvements

lxmf_bridge_matrix/README.md

@@ -1,2 +1,213 @@
 # lxmf_bridge_matrix
+This program provides an interface between LXMF and Matrix. It serves as a single message endpoint and not to transfer the LXMF/Reticlum traffic 1:1 to Matrix. It serves the purpose of providing an endpoint in the Reticulum network for matrix rooms. Through this all LXMF capable applications can communicate with it via messages.
+
+The main functionality is the connection of a matrix room with a lxmf distribution group. Therefore, this program is optimized for the distribution group. But it could also act as a normal message endpoint.
+
 For more information, see the configuration options (at the end of the program files). Everything else is briefly documented there. After the first start this configuration will be created as default config in the corresponding file.
+
+
+### Features
+- Optimized for compatibility with the distributon group
+- Compatible with all LXMF applications (NomadNet, Sideband, ...)
+- Compatible with all Matrix rooms
+- Configurable routing table to connect more than one room/group
+
+
+## Examples of use
+
+### 
+
+### General info how the messages are transported
+All messages between client<->server are transported as single 1:1 messages in the LXMF/Reticulum network.
+Accordingly, encryption takes place between these end points.
+If a direct delivery of the message does not work, it is sent to a propagation node. There it is stored temporarily and can be retrieved by the client later.
+
+As these are normal LXMF messages, any LXMF capable application can be used to communicate with it.
+
+
+## Current Status
+It should currently be considered beta software and still work in progress.
+
+All core features are implemented and functioning, but additions will probably occur as real-world use is explored.
+
+There may be errors or the compatibility after an update is no longer guaranteed.
+
+The full documentation is not yet available. Due to lack of time I can also not say when this will be further processed.
+
+
+## Screenshots / Usage examples
+<img src="../docs/screenshots/lxmf_bridge_matrix_01.png" width="200px">
+
+
+## Installation manual
+
+### Install:
+- Install all required prerequisites. (Default Reticulum installation. Only necessary if reticulum is not yet installed.)
+  ```bash
+  apt update
+  apt upgrade
+  
+  apt install python3-pip
+  
+  pip install pip --upgrade
+  reboot
+  
+  pip3 install rns
+  pip3 install pyserial netifaces
+  
+  pip3 install lxmf
+  ```
+- Install all required prerequisites.
+  ```bash
+  apt-get install libolm-dev
+  pip3 install matrix-nio[e2e]
+  ```
+- Change the Reticulum configuration to suit your needs and use-case.
+  ```bash
+  nano /.reticulum/config
+  ```
+- Download the [file](lxmf_bridge_matrix.py) from this repository.
+  ```bash
+  wget https://raw.githubusercontent.com/SebastianObi/LXMF-Tools/main/lxmf_bridge_matrix/lxmf_bridge_matrix.py
+  ```
+- Make it executable with the following command
+  ```bash
+  chmod +x lxmf_bridge_matrix.py
+  ```
+
+### Start:
+- Start it
+  ```bash
+  ./lxmf_bridge_matrix.py
+  ```
+- After the first start edit the configuration file to suit your needs and use-case. The file location is displayed.
+- Example minimal configuration (override of the default config `config.cfg`). These are the most relevant settings that need to be adjusted. All other settings are in `config.cfg`
+  ```bash
+  nano /root/.lxmf_bridge_matrix/config.cfg.owr
+  ```
+  ```bash
+  ```
+- Start it again. Finished!
+  ```bash
+  ./lxmf_bridge_matrix.py
+  ```
+
+
+### Run as a system service/deamon:
+- Create a service file.
+  ```bash
+  nano /etc/systemd/system/lxmf_bridge_matrix.service
+  ```
+- Copy and edit the following content to your own needs.
+  ```bash
+  [Unit]
+  Description=LXMF Bridge Matrix Daemon
+  After=multi-user.target
+  [Service]
+  # ExecStartPre=/bin/sleep 10
+  Type=simple
+  Restart=always
+  RestartSec=3
+  User=root
+  ExecStart=/root/lxmf_bridge_matrix.py
+  [Install]
+  WantedBy=multi-user.target
+  ```
+- Enable the service.
+  ```bash
+  systemctl enable lxmf_bridge_matrix
+  ```
+- Start the service.
+  ```bash
+  systemctl start lxmf_bridge_matrix
+  ```
+
+
+### Start/Stop service:
+  ```bash
+  systemctl start lxmf_bridge_matrix
+  systemctl stop lxmf_bridge_matrix
+  ```
+
+
+### Enable/Disable service:
+  ```bash
+  systemctl enable lxmf_bridge_matrix
+  systemctl disable lxmf_bridge_matrix
+  ```
+
+
+### Run several instances (To copy the same application):
+- Run the program with a different configuration path.
+  ```bash
+  ./lxmf_bridge_matrix.py -p /root/.lxmf_bridge_matrix_2nd
+  ./lxmf_bridge_matrix.py -p /root/.lxmf_bridge_matrix_3nd
+  ```
+- After the first start edit the configuration file to suit your needs and use-case. The file location is displayed.
+
+
+### First usage:
+- With a manual start via the console, the own LXMF address is displayed:
+  ```
+  [] ...............................................................................
+  [] LXMF - Address: <801f48d54bc71cb3e0886944832aaf8d>
+  [] ...............................................................................`
+  ```
+- This address is also annouced at startup in the default setting.
+- Now the software can be used.
+
+
+### Startup parameters:
+```bash
+usage: lxmf_bridge_matrix.py [-h] [-p PATH] [-pr PATH_RNS] [-pl PATH_LOG] [-l LOGLEVEL] [-s] [--exampleconfig]
+                                  [--exampleconfigoverride]
+
+LXMF Bridge Matrix
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -p PATH, --path PATH  Path to alternative config directory
+  -pr PATH_RNS, --path_rns PATH_RNS
+                        Path to alternative Reticulum config directory
+  -pl PATH_LOG, --path_log PATH_LOG
+                        Path to alternative log directory
+  -l LOGLEVEL, --loglevel LOGLEVEL
+  -s, --service         Running as a service and should log to file
+  --exampleconfig       Print verbose configuration example to stdout and exit
+  --exampleconfigoverride
+                        Print verbose configuration example to stdout and exit
+```
+
+
+### Config/data files:
+- config.cfg
+  
+  This is the default config file.
+
+- config.cfg.owr
+  
+  This is the user configuration file to override the default configuration file.
+  All settings made here have precedence.
+  This file can be used to clearly summarize all settings that deviate from the default.
+  This also has the advantage that all changed settings can be kept when updating the program.
+
+
+## Configuration manual (Examples)
+The configurations shown here are only a part of the total configuration.
+It only serves to show the configuration that is necessary and adapted for the respective function.
+All configurations must be made in the file `config.cfg.owr`.
+All possible settings can be seen in the default configuration file `config.cfg`.
+
+
+## Admin manual
+This guide applies to all admins. Here are briefly explained the administative possibilities.
+
+
+## User manual
+This guide applies to users or admins. Here are briefly explained the normal possibilities of the software.
+
+
+## FAQ
+
+### How do I start with the software?
+You should read the `Installation manual` section. There everything is explained briefly. Just work through everything from top to bottom :)

lxmf_bridge_mqtt/README.md

@@ -18,7 +18,7 @@ All messages between client<->server are transported as single 1:1 messages in t
 Accordingly, encryption takes place between these end points.
 If a direct delivery of the message does not work, it is sent to a propagation node. There it is stored temporarily and can be retrieved by the client later.
 
-As these are normal LXMF messages, any LXMF capable application can be used to communicate with the group.
+As these are normal LXMF messages, any LXMF capable application can be used to communicate with it.
 
 
 ## Current Status
@@ -53,6 +53,10 @@ The full documentation is not yet available. Due to lack of time I can also not
   
   pip3 install lxmf
   ```
+- Install all required prerequisites.
+  ```bash
+  pip3 install paho-mqtt
+  ```
 - Change the Reticulum configuration to suit your needs and use-case.
   ```bash
   nano /.reticulum/config
@@ -150,10 +154,10 @@ The full documentation is not yet available. Due to lack of time I can also not
 
 ### Startup parameters:
 ```bash
-usage: lxmf_distribution_group_minimal.py [-h] [-p PATH] [-pr PATH_RNS] [-pl PATH_LOG] [-l LOGLEVEL] [-s] [--exampleconfig]
+usage: lxmf_bridge_mqtt.py [-h] [-p PATH] [-pr PATH_RNS] [-pl PATH_LOG] [-l LOGLEVEL] [-s] [--exampleconfig]
-                                  [--exampleconfigoverride] [--exampledata]
+                                  [--exampleconfigoverride]
 
-LXMF Distribution Group - Server-Side group functions for LXMF based apps
+LXMF Bridge MQTT
 
 optional arguments:
   -h, --help            show this help message and exit
@@ -167,7 +171,6 @@ optional arguments:
   --exampleconfig       Print verbose configuration example to stdout and exit
   --exampleconfigoverride
                         Print verbose configuration example to stdout and exit
-  --exampledata         Print verbose configuration example to stdout and exit
 ```
 
 

lxmf_bridge_mqtt/lxmf_bridge_mqtt.py

@@ -291,13 +291,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -306,7 +306,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -337,10 +337,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -635,23 +636,28 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
 
-            if not CONFIG["main"].getboolean("power") or not CONFIG["router"].getboolean("lxmf_announce_to_mqtt"):
+        if len(app_data) == 0:
-                log("LXMF - Routing disabled", LOG_DEBUG)
+            return
-                return
 
-            if CONFIG.has_option("allowed", "any") or CONFIG.has_option("allowed", "all") or CONFIG.has_option("allowed", "anybody") or CONFIG.has_option("allowed", RNS.hexrep(destination_hash, False)) or CONFIG.has_option("allowed", RNS.prettyhexrep(destination_hash)):
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
-                message_out = json.dumps({
-                    "source": RNS.hexrep(destination_hash, False),
-                    "data": app_data.decode("utf-8")
-                })
 
-                MQTT_CONNECTION.publish(CONFIG["mqtt"]["topic_announce"], message_out)
+        if not CONFIG["main"].getboolean("power") or not CONFIG["router"].getboolean("lxmf_announce_to_mqtt"):
-            else:
+            log("LXMF - Routing disabled", LOG_DEBUG)
-                log("LXMF - Source " + RNS.prettyhexrep(message.source_hash) + " not allowed", LOG_DEBUG)
+            return
-                return
+
+        if CONFIG.has_option("allowed", "any") or CONFIG.has_option("allowed", "all") or CONFIG.has_option("allowed", "anybody") or CONFIG.has_option("allowed", RNS.hexrep(destination_hash, False)) or CONFIG.has_option("allowed", RNS.prettyhexrep(destination_hash)):
+            message_out = json.dumps({
+                "source": RNS.hexrep(destination_hash, False),
+                "data": app_data.decode("utf-8")
+            })
+
+            MQTT_CONNECTION.publish(CONFIG["mqtt"]["topic_announce"], message_out)
+        else:
+            log("LXMF - Source " + RNS.prettyhexrep(message.source_hash) + " not allowed", LOG_DEBUG)
+            return
 
 
 
@@ -667,7 +673,36 @@ def lxmf_message_received_callback(message):
         return
 
     if CONFIG.has_option("allowed", "any") or CONFIG.has_option("allowed", "all") or CONFIG.has_option("allowed", "anybody") or CONFIG.has_option("allowed", RNS.hexrep(message.source_hash, False)) or CONFIG.has_option("allowed", RNS.prettyhexrep(message.source_hash)):
+
+        title = message.title.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "lxmf_to_mqtt_deny_title")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
         content = message.content.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "lxmf_to_mqtt_deny_content")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
+        if message.fields:
+            denys = config_get(CONFIG, "message", "lxmf_to_mqtt_deny_fields")
+            if denys != "":
+                denys = denys.split(",")
+                if "*" in denys:
+                    return
+                for deny in denys:
+                    if deny in message.fields:
+                        return
 
         length = config_getint(CONFIG, "message", "lxmf_to_mqtt_length_min", 0)
         if length> 0:
@@ -1074,6 +1109,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1514,6 +1562,13 @@ state_to_mqtt = True
 #### Message settings ####
 [message]
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+lxmf_to_mqtt_deny_title = 
+lxmf_to_mqtt_deny_content = 
+lxmf_to_mqtt_deny_fields = 
+
 # Text is added.
 lxmf_to_mqtt_prefix = 
 lxmf_to_mqtt_suffix = 
@@ -1531,6 +1586,13 @@ lxmf_to_mqtt_length_min = 0 #0=any length
 lxmf_to_mqtt_length_max = 0 #0=any length
 
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+mqtt_to_lxmf_deny_title = 
+mqtt_to_lxmf_deny_content = 
+mqtt_to_lxmf_deny_fields = 
+
 # Text is added.
 mqtt_to_lxmf_prefix = 
 mqtt_to_lxmf_suffix = 

lxmf_chatbot/lxmf_chatbot.py

@@ -289,13 +289,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -304,7 +304,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -335,10 +335,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -633,8 +634,13 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
+
+        if len(app_data) == 0:
+            return
+
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
 
 
 
@@ -647,7 +653,35 @@ def lxmf_message_received_callback(message):
 
     if CONFIG.has_option("allowed", "any") or CONFIG.has_option("allowed", "all") or CONFIG.has_option("allowed", "anybody") or CONFIG.has_option("allowed", RNS.hexrep(message.source_hash, False)) or CONFIG.has_option("allowed", RNS.prettyhexrep(message.source_hash)):
 
+        title = message.title.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "deny_title")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
         content = message.content.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "deny_content")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
+        if message.fields:
+            denys = config_get(CONFIG, "message", "deny_fields")
+            if denys != "":
+                denys = denys.split(",")
+                if "*" in denys:
+                    return
+                for deny in denys:
+                    if deny in message.fields:
+                        return
 
         length = config_getint(CONFIG, "message", "receive_length_min", 0)
         if length> 0:
@@ -894,6 +928,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1249,6 +1296,13 @@ signature_validated = Yes
 #### Message settings ####
 [message]
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+deny_title = 
+deny_content = 
+deny_fields = 
+
 # Text is added.
 receive_prefix = 
 receive_suffix = 

lxmf_cmd/lxmf_cmd.py

@@ -287,13 +287,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -302,7 +302,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -333,10 +333,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -631,8 +632,13 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
+
+        if len(app_data) == 0:
+            return
+
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
 
 
 
@@ -644,7 +650,36 @@ def lxmf_message_received_callback(message):
         return
 
     if CONFIG.has_option("allowed", "any") or CONFIG.has_option("allowed", "all") or CONFIG.has_option("allowed", "anybody") or CONFIG.has_option("allowed", RNS.hexrep(message.source_hash, False)) or CONFIG.has_option("allowed", RNS.prettyhexrep(message.source_hash)):
+
+        title = message.title.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "deny_title")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
         content = message.content.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "deny_content")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
+        if message.fields:
+            denys = config_get(CONFIG, "message", "deny_fields")
+            if denys != "":
+                denys = denys.split(",")
+                if "*" in denys:
+                    return
+                for deny in denys:
+                    if deny in message.fields:
+                        return
 
         length = config_getint(CONFIG, "message", "receive_length_min", 0)
         if length> 0:
@@ -929,6 +964,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1273,6 +1321,13 @@ signature_validated = Yes
 #### Message settings ####
 [message]
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+deny_title = 
+deny_content = 
+deny_fields = 
+
 # Text is added.
 receive_prefix = 
 receive_suffix = 

lxmf_distribution_group/Examples/Channel_for_Communicator_Software/config.cfg.owr

@@ -43,6 +43,8 @@ enabled = False
 
 #### Message settings ####
 [message]
+deny_fields = ts
+
 send_title_prefix = 
 send_prefix = 
 
@@ -63,8 +65,8 @@ enabled = True
 
 #### User rights assignment ####
 [rights]
-admin = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,send_local,anonymous,join
+admin = receive_local,send_local,anonymous,join
-mod = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,send_local,anonymous,join
+mod = receive_local,send_local,anonymous,join
 user = receive_local,join
 guest = receive_local,join
 wait = join
@@ -72,8 +74,8 @@ wait = join
 
 #### User cmd assignment ####
 [cmds]
-admin = update,leave,invite,kick,block,unblock,allow,deny
+admin = update,update_all,leave,invite,kick,block,unblock
-mod = update,leave,invite,kick,block,unblock,allow,deny
+mod = update,update_all,leave,invite,kick,block,unblock
 user = update,leave
 guest = update,leave
 wait = update,leave

lxmf_distribution_group/Examples/Group_for_Communicator_Software/config.cfg.owr

@@ -43,6 +43,8 @@ enabled = False
 
 #### Message settings ####
 [message]
+deny_fields = ts
+
 send_title_prefix = !source_name! <!source_address!>
 send_prefix = 
 
@@ -63,17 +65,17 @@ enabled = True
 
 #### User rights assignment ####
 [rights]
-admin = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,send_local,join
+admin = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,send_local,join
-mod = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,send_local,join
+mod = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,send_local,join
-user = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,send_local,join
+user = receive_local,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,send_local,join
 guest = receive_local,join
 wait = join,leave
 
 
 #### User cmd assignment ####
 [cmds]
-admin = update,leave,invite,kick,block,unblock,allow,deny
+admin = update,update_all,leave,invite,kick,block,unblock
-mod = update,leave,invite,kick,block,unblock,allow,deny
+mod = update,update_all,leave,invite,kick,block,unblock
 user = update,leave
 guest = update,leave
 wait = update,leave

lxmf_distribution_group/README.md

@@ -64,8 +64,6 @@ The full documentation is not yet available. Due to lack of time I can also not
 ## Development Roadmap
 - Planned, but not yet scheduled
   - Propagation Node fallback
-  - Propagation Node auto discover
-  - Propagation Node auto select
   - Parameters for backup/restore configuration and data
   - Parameters for backup/restore identity
   - Cluster bridges/repeater
@@ -149,6 +147,9 @@ The full documentation is not yet available. Due to lack of time I can also not
   # Propagation node address/hash.
   propagation_node = ca2762fe5283873719aececfb9e18835
   
+  # Set propagation node automatically.
+  propagation_node_auto = True
+  
   # Try to deliver a message via the LXMF propagation network,
   # if a direct delivery to the recipient is not possible.
   try_propagation_on_fail = Yes
@@ -452,6 +453,7 @@ Not yet implemented
   [lxmf]
   desired_method = direct #direct/propagated
   propagation_node = ca2762fe5283873719aececfb9e18835
+  propagation_node_auto = True
   try_propagation_on_fail = Yes
   ```
 
@@ -461,6 +463,7 @@ Not yet implemented
   ```
   [lxmf]
   propagation_node = ca2762fe5283873719aececfb9e18835
+  propagation_node_auto = True
   sync_startup = Yes
   sync_startup_delay = 30 #Seconds
   sync_periodic = Yes

lxmf_distribution_group/lxmf_distribution_group.py

@@ -298,13 +298,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -313,7 +313,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -344,10 +344,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -801,52 +802,64 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
 
-            global DATA
+        if len(app_data) == 0:
+            return
 
-            lng_key = "-" + CONFIG["main"]["lng"]
+        try:
+            app_data_dict = umsgpack.unpackb(app_data)
+            if isinstance(app_data_dict, dict) and "c" in app_data_dict:
+                app_data = app_data_dict["c"]
+        except:
+            pass
 
-            sections = []
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
-            for (key, val) in CONFIG.items("rights"):
-                if DATA.has_section(key):
-                    sections.append(key)
 
-            if CONFIG["main"].getboolean("auto_name_def") or CONFIG["main"].getboolean("auto_name_change"):
+        global DATA
-                source_hash = RNS.hexrep(destination_hash, False)
-                for section in DATA.sections():
-                    for (key, val) in DATA.items(section):
-                        if key == source_hash:
-                            if (val == "" and CONFIG["main"].getboolean("auto_name_def")) or (val != "" and CONFIG["main"].getboolean("auto_name_change")):
-                                value = app_data.decode("utf-8").strip()
-                                if value != DATA[section][key]:
-                                    if DATA[section][key] == "":
-                                        content_type = "name_def"
-                                        content_add = " " + value
-                                    else:
-                                        content_type = "name_change"
-                                        content_add = " " + DATA[section][key] + " -> " + value
 
-                                    DATA[section][key] = value
+        lng_key = "-" + CONFIG["main"]["lng"]
 
-                                    content_group = config_get(CONFIG, "interface_messages", "member_"+content_type, "", lng_key)
+        sections = []
-                                    if content_group != "":
+        for (key, val) in CONFIG.items("rights"):
-                                        fields = fields_generate(lng_key, h=destination_hash ,n=value, tpl=content_type)
+            if DATA.has_section(key):
-                                        content_group = replace(content_group, source_hash, value, "", lng_key)
+                sections.append(key)
-                                        content_group = content_group + content_add
-                                        for section in sections:
-                                            if "receive_auto_"+content_type in config_get(CONFIG, "rights", section).split(","):
-                                                for (key, val) in DATA.items(section):
-                                                    if key != source_hash:
-                                                        LXMF_CONNECTION.send(key, content_group, "", fields, None, "interface_send")
 
-                                    if CONFIG["main"].getboolean("auto_save_data"):
+        if CONFIG["main"].getboolean("auto_name_def") or CONFIG["main"].getboolean("auto_name_change"):
-                                        DATA.remove_option("main", "unsaved")
+            source_hash = RNS.hexrep(destination_hash, False)
-                                        if not data_save(PATH + "/data.cfg"):
+            for section in DATA.sections():
-                                            DATA["main"]["unsaved"] = "True"
+                for (key, val) in DATA.items(section):
-                                    else:
+                    if key == source_hash:
+                        if (val == "" and CONFIG["main"].getboolean("auto_name_def")) or (val != "" and CONFIG["main"].getboolean("auto_name_change")):
+                            value = app_data.decode("utf-8").strip()
+                            if value != DATA[section][key]:
+                                if DATA[section][key] == "":
+                                    content_type = "name_def"
+                                    content_add = " " + value
+                                else:
+                                    content_type = "name_change"
+                                    content_add = " " + DATA[section][key] + " -> " + value
+
+                                DATA[section][key] = value
+
+                                content_group = config_get(CONFIG, "interface_messages", "member_"+content_type, "", lng_key)
+                                if content_group != "":
+                                    fields = fields_generate(lng_key, h=destination_hash ,n=value, tpl=content_type)
+                                    content_group = replace(content_group, source_hash, value, "", lng_key)
+                                    content_group = content_group + content_add
+                                    for section in sections:
+                                        if "receive_auto_"+content_type in config_get(CONFIG, "rights", section).split(","):
+                                            for (key, val) in DATA.items(section):
+                                                if key != source_hash:
+                                                    LXMF_CONNECTION.send(key, content_group, "", fields, None, "interface_send")
+
+                                if CONFIG["main"].getboolean("auto_save_data"):
+                                    DATA.remove_option("main", "unsaved")
+                                    if not data_save(PATH + "/data.cfg"):
                                         DATA["main"]["unsaved"] = "True"
+                                else:
+                                    DATA["main"]["unsaved"] = "True"
 
 
 
@@ -857,19 +870,44 @@ def lxmf_message_received_callback(message):
         log("LXMF - Source " + RNS.prettyhexrep(message.source_hash) + " have no valid signature", LOG_DEBUG)
         return
 
-    content = message.content.decode('utf-8')
+    title = message.title.decode('utf-8').strip()
-    content = content.strip()
+    denys = config_get(CONFIG, "message", "deny_title")
+    if denys != "":
+        denys = denys.split(",")
+        if "*" in denys:
+            return
+        for deny in denys:
+            if deny in title:
+                return
+
+    content = message.content.decode('utf-8').strip()
+    denys = config_get(CONFIG, "message", "deny_content")
+    if denys != "":
+        denys = denys.split(",")
+        if "*" in denys:
+            return
+        for deny in denys:
+            if deny in title:
+                return
+
+    if message.fields:
+        denys = config_get(CONFIG, "message", "deny_fields")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in message.fields:
+                    return
+
+    if not CONFIG["message"].getboolean("title"):
+        title = ""
+
     if CONFIG["message"].getboolean("fields") and message.fields:
         pass
     elif content == "":
         return
 
-    if CONFIG["message"].getboolean("title"):
-        title = message.title.decode('utf-8')
-        title = title.strip()
-    else:
-        title = ""
-
     fields = message.fields
 
     lng_key = "-" + CONFIG["main"]["lng"]
@@ -1007,7 +1045,13 @@ def lxmf_message_received_callback(message):
         if DATA.has_section(source_right) and source_right != "main":
             if CONFIG["main"].getboolean("auto_name_add"):
                 app_data = RNS.Identity.recall_app_data(message.source_hash)
-                if app_data != None:
+                if app_data != None and len(app_data) > 0:
+                    try:
+                        app_data_dict = umsgpack.unpackb(app_data)
+                        if isinstance(app_data_dict, dict) and "c" in app_data_dict:
+                            app_data = app_data_dict["c"]
+                    except:
+                        pass
                     source_name = app_data.decode('utf-8')
             DATA[source_right][source_hash] = source_name
             DATA.remove_option("main", "unsaved")
@@ -1220,8 +1264,9 @@ def lxmf_message_received_callback(message):
         else:
             fields = {}
         if CONFIG["main"].getboolean("fields_message"):
-            fields["hash"] = message.hash
+            if not "hash" in fields:
-            if not "anonymous" in source_rights:
+                fields["hash"] = message.hash
+            if not "anonymous" in source_rights and "src" not in fields:
                 fields["src"] = {}
                 fields["src"]["h"] = message.source_hash
                 fields["src"]["n"] = source_name
@@ -1270,8 +1315,9 @@ def lxmf_message_received_callback(message):
         if CONFIG["main"].getboolean("fields_message"):
             if CONFIG["lxmf"]["destination_type_conv"] != "":
                 fields["type"] = CONFIG["lxmf"].getint("destination_type_conv")
-            fields["hash"] = message.hash
+            if not "hash" in fields:
-            if not "anonymous" in source_rights:
+                fields["hash"] = message.hash
+            if not "anonymous" in source_rights and "src" not in fields:
                 fields["src"] = {}
                 fields["src"]["h"] = message.source_hash
                 fields["src"]["n"] = source_name
@@ -1344,8 +1390,9 @@ def lxmf_message_received_callback(message):
             if CONFIG["main"].getboolean("fields_message"):
                 if CONFIG["lxmf"]["destination_type_conv"] != "":
                     fields["type"] = CONFIG["lxmf"].getint("destination_type_conv")
-                fields["hash"] = message.hash
+                if not "hash" in fields:
-                if not "anonymous" in source_rights:
+                    fields["hash"] = message.hash
+                if not "anonymous" in source_rights and "src" not in fields:
                     fields["src"] = {}
                     fields["src"]["h"] = message.source_hash
                     fields["src"]["n"] = source_name
@@ -1519,6 +1566,18 @@ def interface(cmd, source_hash, source_name, source_right, source_rights, lng_ke
             content = config_get(CONFIG, "interface_menu", "update_error", "", lng_key)
 
 
+    # "/update_all" command.
+    elif (cmd == "update_all") and "update_all" in source_rights:
+        try:
+            content = config_get(CONFIG, "interface_menu", "update_all_ok", "", lng_key)
+            for section in sections:
+                for (key, val) in DATA.items(section):
+                    LXMF_CONNECTION.send(key, content, "", fields_generate(lng_key, m=True, d=True, r=True, cmd=section, config=section, tpl="update"), None, "interface_send")
+            content = ""
+        except:
+            content = config_get(CONFIG, "interface_menu", "update_all_error", "", lng_key)
+
+
     # "/join" command.
     elif (cmd == "join" or cmd == "subscribe") and "join" in source_rights:
         try:
@@ -2922,7 +2981,7 @@ def fields_generate(lng_key, fields=None, h=None, n=None, m=False, d=False, r=Fa
             for config in configs:
                 if config != "":
                     key, value = config.split("=", 1)
-                    fields["data"]["config"][key] = val_to_bool(value, fallback_true=value, fallback_false=value)
+                    fields["data"]["config"][key] = val_to_val(value)
 
     if tpl:
         fields["tpl"] = tpl
@@ -3548,6 +3607,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -4001,7 +4073,7 @@ periodic_save_statistic_interval = 30 #Minutes
 # As an alternative to defining the nickname manually, it can be used automatically from the announce.
 auto_name_add = True
 auto_name_def = True
-auto_name_change = False
+auto_name_change = True
 
 # Transport extended data in the announce and fields variable.
 # This is needed for the integration of advanced client apps.
@@ -4018,7 +4090,7 @@ fields_message = False
 # to be compatibel with other LXMF programs.
 destination_name = lxmf
 destination_type = delivery
-destination_type_conv = #4=Group, 6=Channel
+destination_type_conv = #4=Group, 6=Channel (Only for use with Communicator-Software.)
 
 # The name will be visible to other peers
 # on the network, and included in announces.
@@ -4073,7 +4145,7 @@ sync_periodic_interval = 360 #Minutes
 # download x messages at a time. You can change
 # this number, or set the option to 0 to disable
 # the limit, and download everything every time.
-sync_limit = 8
+sync_limit = 0
 
 # Allow only messages with valid signature.
 signature_validated = No
@@ -4177,6 +4249,13 @@ heartbeat_timeout = 15 #Minutes
 
 ## Each message received (message and command) ##
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+deny_title = 
+deny_content = 
+deny_fields = 
+
 # Text is added.
 receive_title_prefix = 
 receive_prefix = 
@@ -4304,8 +4383,8 @@ user = True
 # Delimiter for different rights: ,
 [rights]
 
-admin = interface,receive_local,receive_cluster,receive_cluster_pin_add,receive_cluster_loop,receive_cluster_join,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,receive_deny,receive_description,receive_rules,receive_pin_add,receive_pin_remove,receive_name_def,receive_name_change,receive_auto_name_def,receive_auto_name_change,reply_signature,reply_cluster_enabled,reply_cluster_right,reply_interface_enabled,reply_interface_right,reply_local_enabled,reply_local_right,reply_block,reply_length_min,reply_length_max,send_local,send_cluster,help,update,join,leave,name,address,info,pin,pin_add,pin_remove,cluster_pin_add,description,rules,readme,time,version,groups,members,admins,moderators,users,guests,search,activitys,statistic,statistic_min,statistic_full,statistic_cluster,statistic_router,statistic_local,statistic_interface,statistic_self,statistic_user,status,delivery,enable_local,enable_cluster,auto_add_user,auto_add_user_type,auto_add_cluster,auto_add_router,invite_user,invite_user_type,allow_user,allow_user_type,deny_user,deny_user_type,description_set,rules_set,announce,sync,show_run,show,add,del,move,rename,invite,kick,block,unblock,allow,deny,load,save,reload,reset,unsaved
+admin = interface,receive_local,receive_cluster,receive_cluster_pin_add,receive_cluster_loop,receive_cluster_join,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,receive_deny,receive_description,receive_rules,receive_pin_add,receive_pin_remove,receive_name_def,receive_name_change,receive_auto_name_def,receive_auto_name_change,reply_signature,reply_cluster_enabled,reply_cluster_right,reply_interface_enabled,reply_interface_right,reply_local_enabled,reply_local_right,reply_block,reply_length_min,reply_length_max,send_local,send_cluster,help,update,update_all,join,leave,name,address,info,pin,pin_add,pin_remove,cluster_pin_add,description,rules,readme,time,version,groups,members,admins,moderators,users,guests,search,activitys,statistic,statistic_min,statistic_full,statistic_cluster,statistic_router,statistic_local,statistic_interface,statistic_self,statistic_user,status,delivery,enable_local,enable_cluster,auto_add_user,auto_add_user_type,auto_add_cluster,auto_add_router,invite_user,invite_user_type,allow_user,allow_user_type,deny_user,deny_user_type,description_set,rules_set,announce,sync,show_run,show,add,del,move,rename,invite,kick,block,unblock,allow,deny,load,save,reload,reset,unsaved
-mod = interface,receive_local,receive_cluster,receive_cluster_pin_add,receive_cluster_loop,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,receive_deny,receive_description,receive_rules,receive_pin_add,reply_signature,reply_cluster_enabled,reply_cluster_right,reply_interface_enabled,reply_interface_right,reply_local_enabled,reply_local_right,reply_block,reply_length_min,reply_length_max,send_local,send_cluster,help,update,join,leave,name,address,info,pin,pin_add,pin_remove,cluster_pin_add,description,rules,readme,time,version,groups,members,admins,moderators,users,guests,search,activitys,statistic,statistic_min,statistic_cluster,statistic_router,statistic_local,statistic_self,delivery,show,add,del,move,rename,invite,kick,block,unblock,allow,deny
+mod = interface,receive_local,receive_cluster,receive_cluster_pin_add,receive_cluster_loop,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,receive_deny,receive_description,receive_rules,receive_pin_add,reply_signature,reply_cluster_enabled,reply_cluster_right,reply_interface_enabled,reply_interface_right,reply_local_enabled,reply_local_right,reply_block,reply_length_min,reply_length_max,send_local,send_cluster,help,update,update_all,join,leave,name,address,info,pin,pin_add,pin_remove,cluster_pin_add,description,rules,readme,time,version,groups,members,admins,moderators,users,guests,search,activitys,statistic,statistic_min,statistic_cluster,statistic_router,statistic_local,statistic_self,delivery,show,add,del,move,rename,invite,kick,block,unblock,allow,deny
 user = interface,receive_local,receive_cluster,receive_cluster_pin_add,receive_cluster_loop,receive_join,receive_leave,receive_invite,receive_kick,receive_block,receive_unblock,receive_allow,receive_description,receive_rules,receive_pin_add,reply_signature,reply_cluster_enabled,reply_cluster_right,reply_interface_enabled,reply_interface_right,reply_local_enabled,reply_local_right,reply_block,reply_length_min,reply_length_max,send_local,send_cluster,help,update,join,leave,name,address,info,pin,pin_add,pin_remove,cluster_pin_add,description,rules,readme,time,version,groups,members,admins,moderators,users,guests,search,activitys,statistic,statistic_min,statistic_cluster,statistic_router,statistic_local,statistic_self,delivery,invite
 guest = interface,receive_local,receive_cluster,receive_cluster_loop,update,join,leave
 wait = interface,update,join,leave
@@ -4319,8 +4398,8 @@ wait = interface,update,join,leave
 # Delimiter for different cmds: ,
 [cmds]
 
-admin = leave,invite,kick,block,unblock,allow,deny
+admin = update,update_all,leave,invite,kick,block,unblock,allow,deny
-mod = leave,invite,kick,block,unblock,allow,deny
+mod = update,update_all,leave,invite,kick,block,unblock,allow,deny
 user = leave,invite
 guest = leave
 wait = leave
@@ -4652,6 +4731,12 @@ update_ok-de = OK: Daten aktualisiert.
 update_error = ERROR: Updating data.
 update_error-de = FEHLER: Daten aktualisieren.
 
+# "/update_all" command.
+update_all_ok = OK: Data updated.
+update_all_ok-de = OK: Daten aktualisiert.
+update_all_error = ERROR: Updating data.
+update_all_error-de = FEHLER: Daten aktualisieren.
+
 # "/join" command.
 join_error = ERROR: While joining group.
 join_error-de = FEHLER: Beim Beitritt in die Gruppe.

lxmf_distribution_group_minimal/README.md

@@ -99,6 +99,9 @@ The full documentation is not yet available. Due to lack of time I can also not
   # Propagation node address/hash.
   propagation_node = ca2762fe5283873719aececfb9e18835
   
+  # Set propagation node automatically.
+  propagation_node_auto = True
+  
   # Try to deliver a message via the LXMF propagation network,
   # if a direct delivery to the recipient is not possible.
   try_propagation_on_fail = Yes

lxmf_distribution_group_minimal/lxmf_distribution_group_minimal.py

@@ -285,13 +285,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -300,7 +300,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -331,10 +331,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -629,8 +630,20 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
+
+        if len(app_data) == 0:
+            return
+
+        try:
+            app_data_dict = umsgpack.unpackb(app_data)
+            if isinstance(app_data_dict, dict) and "c" in app_data_dict:
+                app_data = app_data_dict["c"]
+        except:
+            pass
+
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
 
 
 
@@ -641,21 +654,45 @@ def lxmf_message_received_callback(message):
         log("LXMF - Source " + RNS.prettyhexrep(message.source_hash) + " have no valid signature", LOG_DEBUG)
         return
 
-    content = message.content.decode('utf-8')
+    title = message.title.decode('utf-8').strip()
-    content = content.strip()
+    denys = config_get(CONFIG, "message", "deny_title")
-    if content == "":
+    if denys != "":
-        return
+        denys = denys.split(",")
+        if "*" in denys:
+            return
+        for deny in denys:
+            if deny in title:
+                return
 
-    if CONFIG["message"].getboolean("title"):
+    content = message.content.decode('utf-8').strip()
-        title = message.title.decode('utf-8')
+    denys = config_get(CONFIG, "message", "deny_content")
-        title = title.strip()
+    if denys != "":
-    else:
+        denys = denys.split(",")
+        if "*" in denys:
+            return
+        for deny in denys:
+            if deny in title:
+                return
+
+    if message.fields:
+        denys = config_get(CONFIG, "message", "deny_fields")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in message.fields:
+                    return
+
+    if not CONFIG["message"].getboolean("title"):
         title = ""
 
-    if CONFIG["message"].getboolean("fields"):
+    if CONFIG["message"].getboolean("fields") and message.fields:
-        fields = message.fields
+        pass
-    else:
+    elif content == "":
-        fields = None
+        return
+
+    fields = message.fields
 
     source_hash = RNS.hexrep(message.source_hash, False)
     source_name = ""
@@ -1022,6 +1059,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1382,7 +1432,7 @@ sync_periodic_interval = 360 #Minutes
 # download x messages at a time. You can change
 # this number, or set the option to 0 to disable
 # the limit, and download everything every time.
-sync_limit = 8
+sync_limit = 0
 
 # Allow only messages with valid signature.
 signature_validated = No
@@ -1394,6 +1444,13 @@ signature_validated = No
 [message]
 ## Each message received (message and command) ##
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+deny_title = 
+deny_content = 
+deny_fields = 
+
 # Text is added.
 receive_prefix = 
 receive_suffix = 

lxmf_echo/lxmf_echo.py

@@ -283,13 +283,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -298,7 +298,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -329,10 +329,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -627,8 +628,13 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
+
+        if len(app_data) == 0:
+            return
+
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
 
 
 
@@ -641,7 +647,35 @@ def lxmf_message_received_callback(message):
 
     if CONFIG.has_option("allowed", "any") or CONFIG.has_option("allowed", "all") or CONFIG.has_option("allowed", "anybody") or CONFIG.has_option("allowed", RNS.hexrep(message.source_hash, False)) or CONFIG.has_option("allowed", RNS.prettyhexrep(message.source_hash)):
 
+        title = message.title.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "deny_title")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
         content = message.content.decode('utf-8').strip()
+        denys = config_get(CONFIG, "message", "deny_content")
+        if denys != "":
+            denys = denys.split(",")
+            if "*" in denys:
+                return
+            for deny in denys:
+                if deny in title:
+                    return
+
+        if message.fields:
+            denys = config_get(CONFIG, "message", "deny_fields")
+            if denys != "":
+                denys = denys.split(",")
+                if "*" in denys:
+                    return
+                for deny in denys:
+                    if deny in message.fields:
+                        return
 
         length = config_getint(CONFIG, "message", "receive_length_min", 0)
         if length> 0:
@@ -689,14 +723,19 @@ def lxmf_message_received_callback(message):
 
         content = content_prefix + content + content_suffix
 
-        if CONFIG["message"].getboolean("title"):
+        if not CONFIG["message"].getboolean("title"):
-            title = message.title.decode('utf-8')
-            title = title.strip()
-        else:
             title = ""
 
         if CONFIG["message"].getboolean("fields"):
             fields = message.fields
+            if fields:
+                search = config_get(CONFIG, "message", "fields_remove").split(",")
+                delete = []
+                for field in fields:
+                    if field in search:
+                        delete.append(field)
+                for field in delete:
+                    del fields[field]
         else:
             fields = None
 
@@ -897,6 +936,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1241,6 +1293,13 @@ signature_validated = Yes
 #### Message settings ####
 [message]
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+deny_title = 
+deny_content = 
+deny_fields = 
+
 # Text is added.
 receive_prefix = 
 receive_suffix = 
@@ -1278,6 +1337,9 @@ send_length_max = 0 #0=any length
 title = Yes
 fields = Yes
 
+# Comma-separated list with fields which will be removed.
+fields_remove = 
+
 
 
 

lxmf_ping/lxmf_ping.py

@@ -284,13 +284,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -299,7 +299,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -330,10 +330,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -680,6 +681,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 

lxmf_provisioning/lxmf_provisioning.py

@@ -35,6 +35,7 @@
 import sys
 import os
 import time
+from datetime import datetime, timezone
 import argparse
 
 #### Config ####
@@ -300,13 +301,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -315,7 +316,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -346,10 +347,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -644,8 +646,13 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
+
+        if len(app_data) == 0:
+            return
+
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
 
 
 
@@ -721,7 +728,7 @@ def jobs_in():
             CACHE_DEL = []
             db = None
             try:
-                db = psycopg2.connect(user=CONFIG["database"]["user"], password=CONFIG["database"]["password"], host=CONFIG["database"]["host"], port=CONFIG["database"]["port"], database=CONFIG["database"]["database"])
+                db = psycopg2.connect(user=CONFIG["database"]["user"], password=CONFIG["database"]["password"], host=CONFIG["database"]["host"], port=CONFIG["database"]["port"], database=CONFIG["database"]["database"], client_encoding=CONFIG["database"]["encoding"])
                 dbc = db.cursor()
 
                 for key in CACHE["in"]:
@@ -737,7 +744,7 @@ def jobs_in():
                             result = dbc.fetchall()
                             if len(result) == 0:
                                 user_id = str(uuid.uuid4())
-                                dbc.execute("INSERT INTO members (member_user_id, member_email, member_password, member_dob, member_sex, member_introduction, member_country, member_state, member_city, member_occupation, member_skills, member_tasks, member_wallet_address, member_accept_rules, member_language, member_locale, member_status) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, '0')", (
+                                dbc.execute("INSERT INTO members (member_user_id, member_email, member_password, member_dob, member_sex, member_introduction, member_country, member_state, member_city, member_occupation, member_skills, member_tasks, member_wallet_address, member_accept_rules, member_language, member_locale, member_ts_add, member_status) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, '0')", (
                                     user_id,
                                     data["email"],
                                     data["password"],
@@ -753,7 +760,8 @@ def jobs_in():
                                     data["wallet_address"],
                                     data["accept_rules"],
                                     data["language"],
-                                    data["language"]
+                                    data["language"],
+                                    datetime.now(timezone.utc)
                                     )
                                 )
                                 if CONFIG["features"].getboolean("account_add_auth"):
@@ -790,7 +798,7 @@ def jobs_in():
                             result = dbc.fetchall()
                             if len(result) == 0:
                                 user_id = str(uuid.uuid4())
-                                dbc.execute("INSERT INTO members (member_user_id, member_email, member_password, member_dob, member_sex, member_introduction, member_country, member_state, member_city, member_occupation, member_skills, member_tasks, member_wallet_address, member_accept_rules, member_language, member_locale, member_status) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, '0')", (
+                                dbc.execute("INSERT INTO members (member_user_id, member_email, member_password, member_dob, member_sex, member_introduction, member_country, member_state, member_city, member_occupation, member_skills, member_tasks, member_wallet_address, member_accept_rules, member_language, member_locale, member_ts_add, member_status) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, '0')", (
                                     user_id,
                                     data["email"],
                                     data["password"],
@@ -806,7 +814,8 @@ def jobs_in():
                                     data["wallet_address"],
                                     data["accept_rules"],
                                     data["language"],
-                                    data["language"]
+                                    data["language"],
+                                    datetime.now(timezone.utc)
                                     )
                                 )
                                 if CONFIG["features"].getboolean("account_add_auth"):
@@ -820,6 +829,35 @@ def jobs_in():
                                     CACHE_CHANGE = True
                             elif len(result) == 1:
                                 user_id = result[0][0]
+                                dbc.execute("UPDATE members SET member_email = %s, member_password = %s, member_dob = %s, member_sex = %s, member_introduction = %s, member_country = %s, member_state = %s, member_city = %s, member_occupation = %s, member_skills = %s, member_tasks = %s, member_wallet_address = %s, member_accept_rules = %s, member_language = %s, member_locale = %s, member_ts_edit = %s WHERE member_user_id = %s", (
+                                    data["email"],
+                                    data["password"],
+                                    data["dob"],
+                                    data["sex"],
+                                    data["introduction"],
+                                    data["country"],
+                                    data["state"],
+                                    data["city"],
+                                    data["occupation"],
+                                    data["skills"],
+                                    data["tasks"],
+                                    data["wallet_address"],
+                                    data["accept_rules"],
+                                    data["language"],
+                                    data["language"],
+                                    datetime.now(timezone.utc),
+                                    user_id
+                                    )
+                                )
+                                if CONFIG["features"].getboolean("account_edit_auth"):
+                                    fields = {}
+                                    if CONFIG["lxmf"]["destination_type_conv"] != "":
+                                        fields["type"] = CONFIG["lxmf"].getint("destination_type_conv")
+                                    fields["prov"] = {}
+                                    fields["prov"]["auth_state"] = CONFIG["features"].getint("account_edit_auth_state")
+                                    fields["prov"]["auth_role"] = CONFIG["features"].getint("account_edit_auth_role")
+                                    CACHE["out"][str(uuid.uuid4())] = {"hash_destination": data["hash_destination"], "content": "", "title": "", "fields": fields}
+                                    CACHE_CHANGE = True
                             else:
                                 continue
 
@@ -1188,6 +1226,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1363,11 +1414,19 @@ def setup(path=None, path_rns=None, path_log=None, loglevel=None, service=False)
         path = PATH
 
     announce_data = {}
-    if CONFIG["features"].getboolean("announce_versions"):
+    if CONFIG["features"].getboolean("announce_data"):
         section = "data"
         if CONFIG.has_section(section):
             for (key, val) in CONFIG.items(section):
-                announce_data[key] = val
+                if "=" in val or ";" in val:
+                    announce_data[key] = {}
+                    keys = val.split(";")
+                    for val in keys:
+                        val = val.split("=")
+                        if len(val) == 2:
+                            announce_data[key][val[0]] = val_to_val(val[1])
+                else:
+                    announce_data[key] = val
 
     LXMF_CONNECTION = lxmf_connection(
         storage_path=path,
@@ -1469,7 +1528,7 @@ announce_periodic = Yes
 announce_periodic_interval = 15 #Minutes
 
 [features]
-announce_versions = True
+announce_data = True
 account_add = True
 account_edit = True
 account_del = True
@@ -1482,12 +1541,11 @@ interval_out = 60 #Seconds
 
 [data]
 v_s = 0.0.0 #Version software
-v_c = 2022-01-01 00:00 #Version config
+v_c = 0.0.0 #Version config
-v_d = 2022-01-01 00:00 #Version data
-v_a = 2022-01-01 00:00 #Version auth
 u_s = #URL Software
 i_s = #Info Software
 cmd = #CMD
+config = #Config
 '''
 
 
@@ -1516,7 +1574,7 @@ name = LXMF Provisioning Server
 # to be compatibel with other LXMF programs.
 destination_name = lxmf
 destination_type = provisioning
-destination_type_conv = 11
+destination_type_conv = 174
 
 # The name will be visible to other peers
 # on the network, and included in announces.
@@ -1586,14 +1644,14 @@ port = 5432
 user = postgres
 password = password
 database = database
-
+encoding = utf8
 
 
 
 #### Features enabled/disabled ####
 [features]
 
-announce_versions = True
+announce_data = True
 
 account_add = True
 account_add_auth = False
@@ -1629,12 +1687,11 @@ interval_out = 60 #Seconds
 [data]
 
 v_s = 0.0.0 #Version software
-v_c = 2022-01-01 00:00 #Version config
+v_c = 0.0.0 #Version config
-v_d = 2022-01-01 00:00 #Version data
-v_a = 2022-01-01 00:00 #Version auth
 u_s = #URL Software
 i_s = #Info Software
 cmd = #CMD
+config = #Config
 '''
 
 

lxmf_terminal/lxmf_terminal.py

@@ -384,13 +384,13 @@ class lxmf_connection:
 
             if len(destination) != ((RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2):
                 log("LXMF - Destination length is invalid", LOG_ERROR)
-                return
+                return None
 
             try:
                 destination = bytes.fromhex(destination)
             except Exception as e:
                 log("LXMF - Destination is invalid", LOG_ERROR)
-                return
+                return None
 
         if destination_name == None:
             destination_name = self.destination_name
@@ -399,7 +399,7 @@ class lxmf_connection:
 
         destination_identity = RNS.Identity.recall(destination)
         destination = RNS.Destination(destination_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, destination_name, destination_type)
-        self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
+        return self.send_message(destination, self.destination, content, title, fields, timestamp, app_data)
 
 
     def send_message(self, destination, source, content="", title="", fields=None, timestamp=None, app_data=""):
@@ -430,10 +430,11 @@ class lxmf_connection:
         try:
             self.message_router.handle_outbound(message)
             time.sleep(self.send_delay)
+            return message.hash
         except Exception as e:
             log("LXMF - Could not send message " + str(message), LOG_ERROR)
             log("LXMF - The contained exception was: " + str(e), LOG_ERROR)
-            return
+            return None
 
 
     def message_notification(self, message):
@@ -728,8 +729,13 @@ class lxmf_announce_callback:
 
     @staticmethod
     def received_announce(destination_hash, announced_identity, app_data):
-        if app_data != None:
+        if app_data == None:
-            log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
+            return
+
+        if len(app_data) == 0:
+            return
+
+        log("LXMF - Received an announce from " + RNS.prettyhexrep(destination_hash) + ": " + app_data.decode("utf-8"), LOG_INFO)
 
 
 
@@ -746,7 +752,36 @@ def lxmf_message_received_callback(message):
 
         if TERMINAL:
             SESSION["source"] = message.source_hash
-            content = message.content.decode('utf-8')
+
+            title = message.title.decode('utf-8').strip()
+            denys = config_get(CONFIG, "message", "deny_title")
+            if denys != "":
+                denys = denys.split(",")
+                if "*" in denys:
+                    return
+                for deny in denys:
+                    if deny in title:
+                        return
+
+            content = message.content.decode('utf-8').strip()
+            denys = config_get(CONFIG, "message", "deny_content")
+            if denys != "":
+                denys = denys.split(",")
+                if "*" in denys:
+                    return
+                for deny in denys:
+                    if deny in title:
+                        return
+
+            if message.fields:
+                denys = config_get(CONFIG, "message", "deny_fields")
+                if denys != "":
+                    denys = denys.split(",")
+                    if "*" in denys:
+                        return
+                    for deny in denys:
+                        if deny in message.fields:
+                            return
 
             length = config_getint(CONFIG, "message", "receive_length_min", 0)
             if length> 0:
@@ -1064,6 +1099,19 @@ def val_to_bool(val, fallback_true=True, fallback_false=False):
         return fallback_false
 
 
+def val_to_val(val):
+    if val.isdigit():
+        return int(val)
+    elif val.isnumeric():
+        return float(val)
+    elif val.lower() == "true":
+        return True
+    elif val.lower() == "false":
+        return False
+    else:
+        return val
+
+
 ##############################################################################################################
 # Log
 
@@ -1429,6 +1477,13 @@ interval = 5 #Seconds
 #### Message settings ####
 [message]
 
+# Deny message if the title/content/fields contains the following content.
+# Comma-separated list with text or field keys.
+# *=any
+deny_title = 
+deny_content = 
+deny_fields = 
+
 # Text is added.
 receive_prefix = 
 receive_suffix =