Remove service name from aspects #12; minor tweaks

- Remove service name from RNS destination aspects. Service name
  now selects a suffix for the identity file and should only be
  supplied on the listener. The initiator only needs the destination
  hash of the listener to connect.
- Show a spinner during link establishment on tty sessions
- Attempt to catch and beautify exceptions on initiator
This commit is contained in:
Aaron Heise 2023-02-22 19:58:03 -06:00
parent bd12efd7cf
commit a07ce53bf9
No known key found for this signature in database
GPG Key ID: 6BA54088C41DE8BF
7 changed files with 199 additions and 110 deletions

View File

@ -26,6 +26,14 @@ There will sometimes be breaking changes in the protocol between
releases. Use at your own peril!
## Recent Changes
### v0.0.12
- Remove service name from RNS destination aspects. Service name
now selects a suffix for the identity file and should only be
supplied on the listener. The initiator only needs the destination
hash of the listener to connect.
- Show a spinner during link establishment on tty sessions
- Attempt to catch and beautify exceptions on initiator
### v0.0.11
- Event loop bursting improves throughput and CPU utilization on
both listener and initiator.
@ -40,41 +48,6 @@ releases. Use at your own peril!
- Switch to a new packet-based protocol
- Bug fixes and dependency updates
### v0.0.8
- Improved test suite exposed several issues with the handling of
command line arguments which are now fixed
- Fixed a race condition that would cause remote characters to be
lost intermittently when running remote commands that finish
immediately.
- Added automated testing that actually spins up a random listener
and initiator in a private Reticulum network and passes data
between them, uncovering more issues which are now fixed.
- Fixed (hopefully) an issue where `rnsh` doesn't know what
version it is.
### v0.0.7
Added `-A` command line option. This listener option causes the
remote command line to be appended to the arguments list of the
launched program. This allows the listener to jail connections
to a particular executable while still allowing parameters.
### v0.0.6
Minor improvements in transport efficiency
### v0.0.5
#### Remote command line and pipe compatibility
Command line options have changed somewhat to allow the initiator
to supply a command line. This allows `rnsh` to function similarly
to SSH. You can pipe into or out of `rnsh` to send input through
remote commands or remote command output through other commands.
This behavior can be blocked on the listener with the `-C` option.
When the initiator does not supply a command, the listener uses
a default command specified on its command line. If a default
command is not specified, the listener falls back to the shell
of the user it is running under.
## Quickstart
Tested (thus far) on Python 3.11 macOS 13.1 ARM64. Should
@ -98,9 +71,8 @@ rnsh -l -p
# On initiator
rnsh -p
```
Note: if you are using a non-default identity or service name, be
sure to supply these options with `-p` as the identity and
destination hashes will change depending on these settings.
Note: service name no longer is supplied on initiator. The destination
hash encapsulates this information.
#### Listener
- Listening for default service name ("default").
@ -123,20 +95,20 @@ rnsh a5f72aefc2cb3cdba648f73f77c4e887
## Options
```
Usage:
rnsh [--config <configdir>] [-i <identityfile>] [-s <service_name>] [-l] -p
rnsh -l [--config <configfile>] [-i <identityfile>] [-s <service_name>]
[-v... | -q...] [-b <period>] (-n | -a <identity_hash> [-a <identity_hash>] ...)
[-A | -C] [[--] <program> [<arg> ...]]
rnsh [--config <configfile>] [-i <identityfile>] [-s <service_name>]
[-v... | -q...] [-N] [-m] [-w <timeout>] <destination_hash>
[[--] <program> [<arg> ...]]
rnsh -l [-c <configdir>] [-i <identityfile> | -s <service_name>] [-v... | -q...] -p
rnsh -l [-c <configdir>] [-i <identityfile> | -s <service_name>] [-v... | -q...]
[-b <period>] (-n | -a <identity_hash> [-a <identity_hash>] ...) [-A | -C]
[[--] <program> [<arg> ...]]
rnsh [-c <configdir>] [-i <identityfile>] [-v... | -q...] -p
rnsh [-c <configdir>] [-i <identityfile>] [-v... | -q...] [-N] [-m] [-w <timeout>]
<destination_hash> [[--] <program> [<arg> ...]]
rnsh -h
rnsh --version
Options:
--config DIR Alternate Reticulum config directory to use
-c DIR --config DIR Alternate Reticulum config directory to use
-i FILE --identity FILE Specific identity file to use
-s NAME --service NAME Listen on/connect to specific service name if not default
-s NAME --service NAME Service name for identity file if not default
-p --print-identity Print identity information and exit
-l --listen Listen (server) mode. If supplied, <program> <arg>...will
be used as the command line when the initiator does not
@ -171,14 +143,30 @@ with an RNS identity, and a service name. Together, RNS makes
these into a destination hash that can be used to connect to
your listener.
Multiple listeners can use the same identity. As long as
they are given different service names. They will have
different destination hashes and not conflict.
Each listener must use a unique identity. The `-s` parameter
can be used to specify a service name, which creates a unique
identity file.
Listeners must be configured with a command line to run (at
least at this time). The identity hash string is set in the
environment variable RNS_REMOTE_IDENTITY for use in child
programs.
Listeners can be configured with a command line to run on
connect. Initiators can supply a command line as well. There
are several different options for the way the command line
is handled:
- `-C` no initiator command line is allowed; the connection will
be terminated if one is supplied.
- `-A` initiator-supplied command line is appended to listener-
configured command line
- With neither of these options, the listener will use the first
valid command line from this list:
1. Initiator-supplied command line
2. Listener command line argument
3. Default shell of user listener is running under
If the `-n` option is not set on the listener, the initiator
is required to identify before starting a command. The program
will be started with the initiator's identity hash string is set
in the environment variable `RNS_REMOTE_IDENTITY`.
Listeners are set up using the `-l` flag.

View File

@ -1,6 +1,6 @@
[tool.poetry]
name = "rnsh"
version = "0.0.11"
version = "0.0.12"
description = "Shell over Reticulum"
authors = ["acehoss <acehoss@acehoss.net>"]
license = "MIT"

View File

@ -18,20 +18,20 @@ def _split_array_at(arr: [_T], at: _T) -> ([_T], [_T]):
usage = \
'''
Usage:
rnsh [--config <configdir>] [-i <identityfile>] [-s <service_name>] [-l] -p
rnsh -l [--config <configfile>] [-i <identityfile>] [-s <service_name>]
[-v... | -q...] [-b <period>] (-n | -a <identity_hash> [-a <identity_hash>] ...)
[-A | -C] [[--] <program> [<arg> ...]]
rnsh [--config <configfile>] [-i <identityfile>] [-s <service_name>]
[-v... | -q...] [-N] [-m] [-w <timeout>] <destination_hash>
[[--] <program> [<arg> ...]]
rnsh -l [-c <configdir>] [-i <identityfile> | -s <service_name>] [-v... | -q...] -p
rnsh -l [-c <configdir>] [-i <identityfile> | -s <service_name>] [-v... | -q...]
[-b <period>] (-n | -a <identity_hash> [-a <identity_hash>] ...) [-A | -C]
[[--] <program> [<arg> ...]]
rnsh [-c <configdir>] [-i <identityfile>] [-v... | -q...] -p
rnsh [-c <configdir>] [-i <identityfile>] [-v... | -q...] [-N] [-m] [-w <timeout>]
<destination_hash> [[--] <program> [<arg> ...]]
rnsh -h
rnsh --version
Options:
--config DIR Alternate Reticulum config directory to use
-c DIR --config DIR Alternate Reticulum config directory to use
-i FILE --identity FILE Specific identity file to use
-s NAME --service NAME Listen on/connect to specific service name if not default
-s NAME --service NAME Service name for identity file if not default
-p --print-identity Print identity information and exit
-l --listen Listen (server) mode. If supplied, <program> <arg>...will
be used as the command line when the initiator does not
@ -59,6 +59,7 @@ Options:
-h --help Show this help
'''
DEFAULT_SERVICE_NAME = "default"
class Args:
def __init__(self, argv: [str]):
@ -75,9 +76,11 @@ class Args:
args = docopt.docopt(usage, argv=self.docopts_argv[1:], version=f"rnsh {rnsh.__version__}")
# json.dump(args, sys.stdout)
self.service_name = args.get("--service", None) or "default"
self.listen = args.get("--listen", None) or False
self.service_name = args.get("--service", None)
if self.listen and (self.service_name is None or len(self.service_name) > 0):
self.service_name = DEFAULT_SERVICE_NAME
self.identity = args.get("--identity", None)
self.config = args.get("--config", None)
self.print_identity = args.get("--print-identity", None) or False

View File

@ -93,11 +93,17 @@ def _sigint_handler(sig, frame):
signal.signal(signal.SIGINT, _sigint_handler)
def _prepare_identity(identity_path):
def _sanitize_service_name(service_name:str) -> str:
return re.sub(r'\W+', '', service_name)
def _prepare_identity(identity_path, service_name: str = None):
global _identity
log = _get_logger("_prepare_identity")
service_name = _sanitize_service_name(service_name or "")
if identity_path is None:
identity_path = RNS.Reticulum.identitypath + "/" + APP_NAME
identity_path = RNS.Reticulum.identitypath + "/" + APP_NAME + \
(f".{service_name}" if service_name and len(service_name) > 0 else "")
if os.path.isfile(identity_path):
_identity = RNS.Identity.from_file(identity_path)
@ -111,26 +117,32 @@ def _prepare_identity(identity_path):
def _print_identity(configdir, identitypath, service_name, include_destination: bool):
global _reticulum
_reticulum = RNS.Reticulum(configdir=configdir, loglevel=RNS.LOG_INFO)
_prepare_identity(identitypath)
destination = RNS.Destination(_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, service_name)
if service_name and len(service_name) > 0:
print(f"Using service name \"{service_name}\"")
_prepare_identity(identitypath, service_name)
destination = RNS.Destination(_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME)
print("Identity : " + str(_identity))
if include_destination:
print("Listening on : " + RNS.prettyhexrep(destination.hash))
exit(0)
async def _listen(configdir, command, identitypath=None, service_name="default", verbosity=0, quietness=0, allowed=None,
async def _listen(configdir, command, identitypath=None, service_name=None, verbosity=0, quietness=0, allowed=None,
disable_auth=None, announce_period=900, no_remote_command=True, remote_cmd_as_args=False):
global _identity, _allow_all, _allowed_identity_hashes, _reticulum, _cmd, _destination, _no_remote_command
global _remote_cmd_as_args
log = _get_logger("_listen")
if service_name is None or len(service_name) == 0:
service_name = "default"
log.info(f"Using service name {service_name}")
targetloglevel = RNS.LOG_INFO + verbosity - quietness
_reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
rnslogging.RnsHandler.set_log_level_with_rns_level(targetloglevel)
_prepare_identity(identitypath)
_destination = RNS.Destination(_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, service_name)
_prepare_identity(identitypath, service_name)
_destination = RNS.Destination(_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME)
_cmd = command
if _cmd is None or len(_cmd) == 0:
@ -218,12 +230,33 @@ async def _listen(configdir, command, identitypath=None, service_name="default",
await asyncio.sleep(0.01)
async def _spin(until: callable = None, timeout: float | None = None) -> bool:
async def _spin_tty(until=None, msg=None, timeout=None):
i = 0
syms = "⢄⢂⢁⡁⡈⡐⡠"
if timeout != None:
timeout = time.time()+timeout
print(msg+" ", end=" ")
while (timeout == None or time.time()<timeout) and not until():
await asyncio.sleep(0.1)
print(("\b\b"+syms[i]+" "), end="")
sys.stdout.flush()
i = (i+1)%len(syms)
print("\r"+" "*len(msg)+" \r", end="")
if timeout != None and time.time() > timeout:
return False
else:
return True
async def _spin_pipe(until: callable = None, msg=None, timeout: float | None = None) -> bool:
if timeout is not None:
timeout += time.time()
while (timeout is None or time.time() < timeout) and not until():
if await _check_finished(0.01):
if await _check_finished(0.1):
raise asyncio.CancelledError()
if timeout is not None and time.time() > timeout:
return False
@ -231,6 +264,13 @@ async def _spin(until: callable = None, timeout: float | None = None) -> bool:
return True
async def _spin(until: callable = None, msg=None, timeout: float | None = None) -> bool:
if os.isatty(1):
return await _spin_tty(until, msg, timeout)
else:
return await _spin_pipe(until, msg, timeout)
_link: RNS.Link | None = None
_remote_exec_grace = 2.0
_new_data: asyncio.Event | None = None
@ -266,7 +306,7 @@ class RemoteExecutionError(Exception):
async def _initiate_link(configdir, identitypath=None, verbosity=0, quietness=0, noid=False, destination=None,
service_name="default", timeout=RNS.Transport.PATH_REQUEST_TIMEOUT):
timeout=RNS.Transport.PATH_REQUEST_TIMEOUT):
global _identity, _reticulum, _link, _destination, _remote_exec_grace, _tr, _new_data
log = _get_logger("_initiate_link")
@ -291,7 +331,8 @@ async def _initiate_link(configdir, identitypath=None, verbosity=0, quietness=0,
if not RNS.Transport.has_path(destination_hash):
RNS.Transport.request_path(destination_hash)
log.info(f"Requesting path...")
if not await _spin(until=lambda: RNS.Transport.has_path(destination_hash), timeout=timeout):
if not await _spin(until=lambda: RNS.Transport.has_path(destination_hash), msg="Requesting path...",
timeout=timeout):
raise RemoteExecutionError("Path not found")
if _destination is None:
@ -300,8 +341,7 @@ async def _initiate_link(configdir, identitypath=None, verbosity=0, quietness=0,
listener_identity,
RNS.Destination.OUT,
RNS.Destination.SINGLE,
APP_NAME,
service_name
APP_NAME
)
if _link is None or _link.status == RNS.Link.PENDING:
@ -312,7 +352,8 @@ async def _initiate_link(configdir, identitypath=None, verbosity=0, quietness=0,
_link.set_link_closed_callback(_client_link_closed)
log.info(f"Establishing link...")
if not await _spin(until=lambda: _link.status == RNS.Link.ACTIVE, timeout=timeout):
if not await _spin(until=lambda: _link.status == RNS.Link.ACTIVE, msg="Establishing link...",
timeout=timeout):
raise RemoteExecutionError("Could not establish link with " + RNS.prettyhexrep(destination_hash))
log.debug("Have link")
@ -323,8 +364,16 @@ async def _initiate_link(configdir, identitypath=None, verbosity=0, quietness=0,
_link.set_packet_callback(_client_packet_handler)
async def _handle_error(errmsg: protocol.Message):
if isinstance(errmsg, protocol.ErrorMessage):
with contextlib.suppress(Exception):
if _link and _link.status == RNS.Link.ACTIVE:
_link.teardown()
await asyncio.sleep(0.1)
raise RemoteExecutionError(f"Remote error: {errmsg.msg}")
async def _initiate(configdir: str, identitypath: str, verbosity: int, quietness: int, noid: bool, destination: str,
service_name: str, timeout: float, command: [str] | None = None):
timeout: float, command: [str] | None = None):
global _new_data, _finished, _tr, _cmd, _pre_input
log = _get_logger("_initiate")
loop = asyncio.get_running_loop()
@ -339,7 +388,6 @@ async def _initiate(configdir: str, identitypath: str, verbosity: int, quietness
quietness=quietness,
noid=noid,
destination=destination,
service_name=service_name,
timeout=timeout,
)
@ -360,6 +408,7 @@ async def _initiate(configdir: str, identitypath: str, verbosity: int, quietness
try:
vp = _pq.get(timeout=max(outlet.rtt * 20, 5))
vm = messenger.receive(vp)
await _handle_error(vm)
if not isinstance(vm, protocol.VersionInfoMessage):
raise Exception("Invalid message received")
log.debug(f"Server version info: sw {vm.sw_version} prot {vm.protocol_version}")
@ -433,6 +482,7 @@ async def _initiate(configdir: str, identitypath: str, verbosity: int, quietness
try:
packet = _pq.get(timeout=sleeper.next_sleep_time() if not processed else 0.0005)
message = messenger.receive(packet)
await _handle_error(message)
processed = True
if isinstance(message, protocol.StreamDataMessage):
if message.stream_id == protocol.StreamDataMessage.STREAM_ID_STDOUT:
@ -534,22 +584,25 @@ async def _rnsh_cli_main():
remote_cmd_as_args=args.remote_cmd_as_args)
return 0
if args.destination is not None and args.service_name is not None:
return_code = await _initiate(
configdir=args.config,
identitypath=args.identity,
verbosity=args.verbose,
quietness=args.quiet,
noid=args.no_id,
destination=args.destination,
service_name=args.service_name,
timeout=args.timeout,
command=args.command_line
)
return return_code if args.mirror else 0
if args.destination is not None:
try:
return_code = await _initiate(
configdir=args.config,
identitypath=args.identity,
verbosity=args.verbose,
quietness=args.quiet,
noid=args.no_id,
destination=args.destination,
timeout=args.timeout,
command=args.command_line
)
return return_code if args.mirror else 0
except Exception as ex:
print(f"{ex}")
return 255;
else:
print("")
print(args.usage)
print(rnsh.args.usage)
print("")
return 1

View File

@ -354,7 +354,7 @@ class ListenerSession:
message = self.messenger.receive(raw)
self._handle_message(message)
except Exception as ex:
self._protocol_error("unusable packet")
self._protocol_error(f"error receiving packet: {ex}")
class RNSOutlet(LSOutletBase):

View File

@ -41,10 +41,9 @@ def test_program_initiate_no_args():
def test_program_initiate_dash_args():
docopt_threw = False
try:
args = rnsh.args.Args(shlex.split("rnsh --config ~/Projects/rnsh/testconfig -s test -vvvvvvv a5f72aefc2cb3cdba648f73f77c4e887 -- -l"))
args = rnsh.args.Args(shlex.split("rnsh --config ~/Projects/rnsh/testconfig -vvvvvvv a5f72aefc2cb3cdba648f73f77c4e887 -- -l"))
assert not args.listen
assert args.config == "~/Projects/rnsh/testconfig"
assert args.service_name == "test"
assert args.verbose == 7
assert args.destination == "a5f72aefc2cb3cdba648f73f77c4e887"
assert args.command_line == ["-l"]
@ -53,6 +52,21 @@ def test_program_initiate_dash_args():
assert not docopt_threw
def test_program_listen_dash_args():
docopt_threw = False
try:
args = rnsh.args.Args(shlex.split("rnsh -l --config ~/Projects/rnsh/testconfig -n -C -- /bin/pwd"))
assert args.listen
assert args.config == "~/Projects/rnsh/testconfig"
assert args.destination is None
assert args.no_auth
assert args.no_remote_cmd
assert args.command_line == ["/bin/pwd"]
except docopt.DocoptExit:
docopt_threw = True
assert not docopt_threw
def test_program_listen_config_print():
docopt_threw = False
try:

View File

@ -56,16 +56,18 @@ async def test_rnsh_listen_start_stop():
assert not wrapper.process.running
async def get_id_and_dest(td: str) -> tuple[str, str]:
with tests.helpers.SubprocessReader(name="getid", argv=shlex.split(f"poetry run -- rnsh -l --config \"{td}\" -p")) as wrapper:
async def get_listener_id_and_dest(td: str) -> tuple[str, str]:
with tests.helpers.SubprocessReader(name="getid", argv=shlex.split(f"poetry run -- rnsh -l -c \"{td}\" -p")) as wrapper:
wrapper.start()
await asyncio.sleep(0.1)
assert wrapper.process.running
# wait for process to start up
await tests.helpers.wait_for_condition_async(lambda: not wrapper.process.running, 5)
assert not wrapper.process.running
await asyncio.sleep(2)
# read the output
text = wrapper.read().decode("utf-8").replace("\r", "").replace("\n", "")
assert text.index("Using service name \"default\"") is not None
assert text.index("Identity") is not None
match = re.search(r"<([a-f0-9]{32})>[^<]+<([a-f0-9]{32})>", text)
assert match is not None
@ -77,29 +79,58 @@ async def get_id_and_dest(td: str) -> tuple[str, str]:
return ih, dh
async def get_initiator_id(td: str) -> str:
with tests.helpers.SubprocessReader(name="getid", argv=shlex.split(f"poetry run -- rnsh -c \"{td}\" -p")) as wrapper:
wrapper.start()
await asyncio.sleep(0.1)
assert wrapper.process.running
# wait for process to start up
await tests.helpers.wait_for_condition_async(lambda: not wrapper.process.running, 5)
assert not wrapper.process.running
# read the output
text = wrapper.read().decode("utf-8").replace("\r", "").replace("\n", "")
assert text.index("Identity") is not None
match = re.search(r"<([a-f0-9]{32})>", text)
assert match is not None
ih = match.group(1)
assert len(ih) == 32
await asyncio.sleep(0.1)
return ih
@pytest.mark.skip_ci
@pytest.mark.asyncio
async def test_rnsh_get_id_and_dest() -> [int]:
async def test_rnsh_get_listener_id_and_dest() -> [int]:
with tests.helpers.tempdir() as td:
ih, dh = await get_id_and_dest(td)
ih, dh = await get_listener_id_and_dest(td)
assert len(ih) == 32
assert len(dh) == 32
@pytest.mark.skip_ci
@pytest.mark.asyncio
async def test_rnsh_get_initiator_id() -> [int]:
with tests.helpers.tempdir() as td:
ih = await get_initiator_id(td)
assert len(ih) == 32
async def do_connected_test(listener_args: str, initiator_args: str, test: callable):
with tests.helpers.tempdir() as td:
ih, dh = await get_id_and_dest(td)
ih, dh = await get_listener_id_and_dest(td)
iih = await get_initiator_id(td)
assert len(ih) == 32
assert len(dh) == 32
with tests.helpers.SubprocessReader(name="listener", argv=shlex.split(f"poetry run -- rnsh -l --config \"{td}\" {listener_args}")) as listener, \
tests.helpers.SubprocessReader(name="initiator", argv=shlex.split(f"poetry run -- rnsh --config \"{td}\" {dh} {initiator_args}")) as initiator:
assert len(iih) == 32
with tests.helpers.SubprocessReader(name="listener", argv=shlex.split(f"poetry run -- rnsh -l -c \"{td}\" {listener_args}")) as listener, \
tests.helpers.SubprocessReader(name="initiator", argv=shlex.split(f"poetry run -- rnsh -c \"{td}\" {dh} {initiator_args}")) as initiator:
# listener startup
listener.start()
await asyncio.sleep(0.1)
assert listener.process.running
# wait for process to start up
await asyncio.sleep(3)
await asyncio.sleep(5)
# read the output
text = listener.read().decode("utf-8")
assert text.index(dh) is not None
@ -108,7 +139,7 @@ async def do_connected_test(listener_args: str, initiator_args: str, test: calla
initiator.start()
assert initiator.process.running
await test(td, ih, dh, listener, initiator)
await test(td, ih, dh, iih, listener, initiator)
# expect test to shut down initiator
assert not initiator.process.running
@ -127,13 +158,13 @@ async def do_connected_test(listener_args: str, initiator_args: str, test: calla
async def test_rnsh_get_echo_through():
cwd = os.getcwd()
async def test(td: str, ih: str, dh: str, listener: tests.helpers.SubprocessReader,
async def test(td: str, ih: str, dh: str, iih: str, listener: tests.helpers.SubprocessReader,
initiator: tests.helpers.SubprocessReader):
start_time = time.time()
while initiator.return_code is None and time.time() - start_time < 3:
await asyncio.sleep(0.1)
text = initiator.read().decode("utf-8").replace("\r", "").replace("\n", "")
assert text == cwd
assert text[len(text)-len(cwd):] == cwd
await do_connected_test("-n -C -- /bin/pwd", "", test)