diff --git a/developer/services/qrexec3.md b/developer/services/qrexec3.md index 442d88f9..38350b1f 100644 --- a/developer/services/qrexec3.md +++ b/developer/services/qrexec3.md @@ -63,7 +63,6 @@ Thus, it is usually more convenient to use `qvm-run`. There can be almost arbitrary number of `qrexec-client` processes for a domain (so, connected to the same `qrexec-daemon`, same domain) -- their data is multiplexed independently. Number of available vchan channels is the limiting factor here, it depends on the underlying hypervisor. - ## Qubes RPC services Some tasks (like inter-vm file copy) share the same RPC-like structure: a process in one VM (say, file sender) needs to invoke and send/receive data to some process in other VM (say, file receiver). @@ -81,7 +80,6 @@ Thanks to the framework, RPC programs are very simple -- both RPC client and ser The framework does all the inner work to connect these processes to each other via `qrexec-daemon` and `qrexec-agent`. Additionally, disposable VMs are tightly integrated -- RPC to a DisposableVM is identical to RPC to a normal domain, all one needs is to pass `$dispvm` as the remote domain name. - ## Qubes RPC administration @@ -234,7 +232,6 @@ In order to remove such authorization, issue this command from a dom0 terminal ( and then remove the first line(s) (before the first `##` comment) which are the "Yes to All" results. - ### Qubes RPC example We will show the necessary files to create an RPC call that adds two integers on the target and returns back the result to the invoker. @@ -311,71 +308,49 @@ And no separate client script will be used. But when invoked with other argument or from different VM, it should be denied. -# Qubes RPC internals # +# Qubes RPC internals -(*This is about the implementation of qrexec v3. -For the implementation of qrexec v2, see [here](/doc/qrexec2/#qubes-rpc-internals).*) +(*This is about the implementation of qrexec v3. For the implementation of qrexec v2, see [here](/doc/qrexec2/#qubes-rpc-internals).*) Qrexec framework consists of a number of processes communicating with each other using common IPC protocol (described in detail below). Components residing in the same domain (`qrexec-client-vm` to `qrexec-agent`, `qrexec-client` to `qrexec-daemon`) use pipes as the underlying transport medium, while components in separate domains (`qrexec-daemon` to `qrexec-agent`, data channel between `qrexec-agent`s) use vchan link. Because of [vchan limitation](https://github.com/qubesos/qubes-issues/issues/951), it is not possible to establish qrexec connection back to the source domain. - ## Dom0 tools implementation - * `/usr/lib/qubes/qrexec-daemon`: One instance is required for every active - domain. Responsible for: - * Handling execution and service requests from **dom0** (source: - `qrexec-client`). - * Handling service requests from the associated domain (source: - `qrexec-client-vm`, then `qrexec-agent`). + * `/usr/lib/qubes/qrexec-daemon`: One instance is required for every active domain. Responsible for: + * Handling execution and service requests from **dom0** (source: `qrexec-client`). + * Handling service requests from the associated domain (source: `qrexec-client-vm`, then `qrexec-agent`). * Command line: `qrexec-daemon domain-id domain-name [default user]` * `domain-id`: Numeric Qubes ID assigned to the associated domain. * `domain-name`: Associated domain name. - * `default user`: Optional. If passed, `qrexec-daemon` uses this user as - default for all execution requests that don't specify one. - * `/usr/lib/qubes/qrexec-policy`: Internal program used to evaluate the - RPC policy and deciding whether a RPC call should be allowed. - * `/usr/lib/qubes/qrexec-client`: Used to pass execution and service requests - to `qrexec-daemon`. Command line parameters: - * `-d target-domain-name`: Specifies the target for the execution/service - request. - * `-l local-program`: Optional. If present, `local-program` is executed - and its stdout/stdin are used when sending/receiving data to/from the - remote peer. - * `-e`: Optional. If present, stdout/stdin are not connected to the remote - peer. Only process creation status code is received. - * `-c `: used for connecting - a VM-VM service request by `qrexec-policy`. Details described below in - the service example. - * `cmdline`: Command line to pass to `qrexec-daemon` as the - execution/service request. Service request format is described below in - the service example. + * `default user`: Optional. If passed, `qrexec-daemon` uses this user as default for all execution requests that don't specify one. + * `/usr/lib/qubes/qrexec-policy`: Internal program used to evaluate the RPC policy and deciding whether a RPC call should be allowed. + * `/usr/lib/qubes/qrexec-client`: Used to pass execution and service requests to `qrexec-daemon`. Command line parameters: + * `-d target-domain-name`: Specifies the target for the execution/service request. + * `-l local-program`: Optional. If present, `local-program` is executed and its stdout/stdin are used when sending/receiving data to/from the remote peer. + * `-e`: Optional. If present, stdout/stdin are not connected to the remote peer. Only process creation status code is received. + * `-c `: used for connecting a VM-VM service request by `qrexec-policy`. Details described below in the service example. + * `cmdline`: Command line to pass to `qrexec-daemon` as the execution/service request. Service request format is described below in the service example. **Note:** None of the above tools are designed to be used by users directly. - ## VM tools implementation * `qrexec-agent`: One instance runs in each active domain. Responsible for: - * Handling service requests from `qrexec-client-vm` and passing them to - connected `qrexec-daemon` in dom0. + * Handling service requests from `qrexec-client-vm` and passing them to connected `qrexec-daemon` in dom0. * Executing associated `qrexec-daemon` execution/service requests. * Command line parameters: none. - * `qrexec-client-vm`: Runs in an active domain. Used to pass service requests - to `qrexec-agent`. + * `qrexec-client-vm`: Runs in an active domain. Used to pass service requests to `qrexec-agent`. * Command line: `qrexec-client-vm target-domain-name service-name local-program [local program arguments]` - * `target-domain-name`: Target domain for the service request. Source is - the current domain. + * `target-domain-name`: Target domain for the service request. Source is the current domain. * `service-name`: Requested service name. - * `local-program`: `local-program` is executed locally and its stdin/stdout - are connected to the remote service endpoint. - + * `local-program`: `local-program` is executed locally and its stdin/stdout are connected to the remote service endpoint. ## Qrexec protocol details -Qrexec protocol is message-based. All messages share a common header followed -by an optional data packet. +Qrexec protocol is message-based. +All messages share a common header followed by an optional data packet. /* uniform for all peers, data type depends on message type */ struct msg_header { @@ -395,28 +370,19 @@ If either side does not support this version, the connection is closed. Details of all possible use cases and the messages involved are described below. - ### dom0: request execution of `some_command` in domX and pass stdin/stdout -- **dom0**: `qrexec-client` is invoked in **dom0** as follows: +- **dom0**: `qrexec-client` is invoked in **dom0** as follows: - `qrexec-client -d domX [-l local_program] user:some_command` + qrexec-client -d domX [-l local_program] user:some_command` - - `user` may be substituted with the literal `DEFAULT`. In that case, - default Qubes user will be used to execute `some_command`. - -- **dom0**: `qrexec-client` sets `QREXEC_REMOTE_DOMAIN` environment variable -to **domX**. -- **dom0**: If `local_program` is set, `qrexec-client` executes it and uses -that child's stdin/stdout in place of its own when exchanging data with -`qrexec-agent` later. + `user` may be substituted with the literal `DEFAULT`. In that case, default Qubes user will be used to execute `some_command`. +- **dom0**: `qrexec-client` sets `QREXEC_REMOTE_DOMAIN` environment variable to **domX**. +- **dom0**: If `local_program` is set, `qrexec-client` executes it and uses that child's stdin/stdout in place of its own when exchanging data with `qrexec-agent` later. - **dom0**: `qrexec-client` connects to **domX**'s `qrexec-daemon`. -- **dom0**: `qrexec-daemon` sends `MSG_HELLO` header followed by `peer_info` -to `qrexec-client`. -- **dom0**: `qrexec-client` replies with `MSG_HELLO` header followed by -`peer_info` to `qrexec-daemon`. -- **dom0**: `qrexec-client` sends `MSG_EXEC_CMDLINE` header followed by -`exec_params` to `qrexec-daemon`. +- **dom0**: `qrexec-daemon` sends `MSG_HELLO` header followed by `peer_info` to `qrexec-client`. +- **dom0**: `qrexec-client` replies with `MSG_HELLO` header followed by `peer_info` to `qrexec-daemon`. +- **dom0**: `qrexec-client` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to `qrexec-daemon`. /* variable size */ struct exec_params { @@ -427,90 +393,50 @@ to `qrexec-client`. In this case, `connect_domain` and `connect_port` are set to 0. -- **dom0**: `qrexec-daemon` replies to `qrexec-client` with -`MSG_EXEC_CMDLINE` header followed by `exec_params`, but with empty `cmdline` -field. `connect_domain` is set to Qubes ID of **domX** and `connect_port` -is set to a vchan port allocated by `qrexec-daemon`. -- **dom0**: `qrexec-daemon` sends `MSG_EXEC_CMDLINE` header followed -by `exec_params` to the associated **domX** `qrexec-agent` over -vchan. `connect_domain` is set to 0 (**dom0**), `connect_port` is the same -as sent to `qrexec-client`. `cmdline` is unchanged except that the literal -`DEFAULT` is replaced with actual user name, if present. +- **dom0**: `qrexec-daemon` replies to `qrexec-client` with `MSG_EXEC_CMDLINE` header followed by `exec_params`, but with empty `cmdline` field. `connect_domain` is set to Qubes ID of **domX** and `connect_port` is set to a vchan port allocated by `qrexec-daemon`. +- **dom0**: `qrexec-daemon` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to the associated **domX** `qrexec-agent` over vchan. `connect_domain` is set to 0 (**dom0**), `connect_port` is the same as sent to `qrexec-client`. `cmdline` is unchanged except that the literal `DEFAULT` is replaced with actual user name, if present. - **dom0**: `qrexec-client` disconnects from `qrexec-daemon`. -- **dom0**: `qrexec-client` starts a vchan server using the details received -from `qrexec-daemon` and waits for connection from **domX**'s `qrexec-agent`. -- **domX**: `qrexec-agent` receives `MSG_EXEC_CMDLINE` header followed by -`exec_params` from `qrexec-daemon` over vchan. -- **domX**: `qrexec-agent` connects to `qrexec-client` over vchan using the -details from `exec_params`. -- **domX**: `qrexec-agent` executes `some_command` in **domX** and connects -the child's stdin/stdout to the data vchan. If the process creation fails, -`qrexec-agent` sends `MSG_DATA_EXIT_CODE` to `qrexec-client` followed by -the status code (**int**) and disconnects from the data vchan. -- Data read from `some_command`'s stdout is sent to the data vchan using -`MSG_DATA_STDOUT` by `qrexec-agent`. `qrexec-client` passes data received as -`MSG_DATA_STDOUT` to its own stdout (or to `local_program`'s stdin if used). -- `qrexec-client` sends data read from local stdin (or `local_program`'s -stdout if used) to `qrexec-agent` over the data vchan using -`MSG_DATA_STDIN`. `qrexec-agent` passes data received as `MSG_DATA_STDIN` -to `some_command`'s stdin. -- `MSG_DATA_STDOUT` or `MSG_DATA_STDIN` with data `len` field set to 0 in -`msg_header` is an EOF marker. Peer receiving such message should close the -associated input/output pipe. -- When `some_command` terminates, **domX**'s `qrexec-agent` sends -`MSG_DATA_EXIT_CODE` header to `qrexec-client` followed by the exit code -(**int**). `qrexec-agent` then disconnects from the data vchan. - +- **dom0**: `qrexec-client` starts a vchan server using the details received from `qrexec-daemon` and waits for connection from **domX**'s `qrexec-agent`. +- **domX**: `qrexec-agent` receives `MSG_EXEC_CMDLINE` header followed by `exec_params` from `qrexec-daemon` over vchan. +- **domX**: `qrexec-agent` connects to `qrexec-client` over vchan using the details from `exec_params`. +- **domX**: `qrexec-agent` executes `some_command` in **domX** and connects the child's stdin/stdout to the data vchan. If the process creation fails, `qrexec-agent` sends `MSG_DATA_EXIT_CODE` to `qrexec-client` followed by the status code (**int**) and disconnects from the data vchan. +- Data read from `some_command`'s stdout is sent to the data vchan using `MSG_DATA_STDOUT` by `qrexec-agent`. `qrexec-client` passes data received as `MSG_DATA_STDOUT` to its own stdout (or to `local_program`'s stdin if used). +- `qrexec-client` sends data read from local stdin (or `local_program`'s stdout if used) to `qrexec-agent` over the data vchan using `MSG_DATA_STDIN`. `qrexec-agent` passes data received as `MSG_DATA_STDIN` to `some_command`'s stdin. +- `MSG_DATA_STDOUT` or `MSG_DATA_STDIN` with data `len` field set to 0 in `msg_header` is an EOF marker. Peer receiving such message should close the associated input/output pipe. +- When `some_command` terminates, **domX**'s `qrexec-agent` sends `MSG_DATA_EXIT_CODE` header to `qrexec-client` followed by the exit code (**int**). `qrexec-agent` then disconnects from the data vchan. ### domY: invoke execution of qubes service `qubes.SomeRpc` in domX and pass stdin/stdout -- **domY**: `qrexec-client-vm` is invoked as follows: +- **domY**: `qrexec-client-vm` is invoked as follows: - `qrexec-client-vm domX qubes.SomeRpc local_program [params]` + qrexec-client-vm domX qubes.SomeRpc local_program [params] -- **domY**: `qrexec-client-vm` connects to `qrexec-agent` (via local -socket/named pipe). -- **domY**: `qrexec-client-vm` sends `trigger_service_params` data to -`qrexec-agent` (without filling the `request_id` field): +- **domY**: `qrexec-client-vm` connects to `qrexec-agent` (via local socket/named pipe). +- **domY**: `qrexec-client-vm` sends `trigger_service_params` data to `qrexec-agent` (without filling the `request_id` field): struct trigger_service_params { char service_name[64]; char target_domain[32]; struct service_params request_id; /* service request id */ }; - + struct service_params { char ident[32]; }; -- **domY**: `qrexec-agent` allocates a locally-unique (for this domain) -`request_id` (let's say `13`) and fills it in the `trigger_service_params` -struct received from `qrexec-client-vm`. -- **domY**: `qrexec-agent` sends `MSG_TRIGGER_SERVICE` header followed by -`trigger_service_params` to `qrexec-daemon` in **dom0** via vchan. -- **dom0**: **domY**'s `qrexec-daemon` executes `qrexec-policy`: `qrexec-policy -domY_id domY domX qubes.SomeRpc 13`. -- **dom0**: `qrexec-policy` evaluates if the RPC should be allowed or -denied. If the action is allowed it returns `0`, if the action is denied it -returns `1`. +- **domY**: `qrexec-agent` allocates a locally-unique (for this domain) `request_id` (let's say `13`) and fills it in the `trigger_service_params` struct received from `qrexec-client-vm`. +- **domY**: `qrexec-agent` sends `MSG_TRIGGER_SERVICE` header followed by `trigger_service_params` to `qrexec-daemon` in **dom0** via vchan. +- **dom0**: **domY**'s `qrexec-daemon` executes `qrexec-policy`: `qrexec-policy domY_id domY domX qubes.SomeRpc 13`. +- **dom0**: `qrexec-policy` evaluates if the RPC should be allowed or denied. If the action is allowed it returns `0`, if the action is denied it returns `1`. - **dom0**: **domY**'s `qrexec-daemon` checks the exit code of `qrexec-policy`. - - If `qrexec-policy` returned **not** `0`: **domY**'s `qrexec-daemon` - sends `MSG_SERVICE_REFUSED` header followed by `service_params` to - **domY**'s `qrexec-agent`. `service_params.ident` is identical to the one - received. **domY**'s `qrexec-agent` disconnects its `qrexec-client-vm` - and RPC processing is finished. + - If `qrexec-policy` returned **not** `0`: **domY**'s `qrexec-daemon` sends `MSG_SERVICE_REFUSED` header followed by `service_params` to **domY**'s `qrexec-agent`. `service_params.ident` is identical to the one received. **domY**'s `qrexec-agent` disconnects its `qrexec-client-vm` and RPC processing is finished. - If `qrexec-policy` returned `0`, RPC processing continues. -- **dom0**: if `qrexec-policy` allowed the RPC, it executed `qrexec-client --d domX -c 13,domY,domY_id user:QUBESRPC qubes.SomeRpc domY`. -- **dom0**: `qrexec-client` sets `QREXEC_REMOTE_DOMAIN` environment variable -to **domX**. +- **dom0**: if `qrexec-policy` allowed the RPC, it executed `qrexec-client -d domX -c 13,domY,domY_id user:QUBESRPC qubes.SomeRpc domY`. +- **dom0**: `qrexec-client` sets `QREXEC_REMOTE_DOMAIN` environment variable to **domX**. - **dom0**: `qrexec-client` connects to **domX**'s `qrexec-daemon`. -- **dom0**: **domX**'s `qrexec-daemon` sends `MSG_HELLO` header followed by -`peer_info` to `qrexec-client`. -- **dom0**: `qrexec-client` replies with `MSG_HELLO` header followed by -`peer_info` to **domX**'s`qrexec-daemon`. -- **dom0**: `qrexec-client` sends `MSG_EXEC_CMDLINE` header followed by -`exec_params` to **domX**'s`qrexec-daemon` +- **dom0**: **domX**'s `qrexec-daemon` sends `MSG_HELLO` header followed by `peer_info` to `qrexec-client`. +- **dom0**: `qrexec-client` replies with `MSG_HELLO` header followed by `peer_info` to **domX**'s`qrexec-daemon`. +- **dom0**: `qrexec-client` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to **domX**'s`qrexec-daemon` /* variable size */ struct exec_params { @@ -519,35 +445,15 @@ to **domX**. char cmdline[0]; /* command line to execute, size = msg_header.len - sizeof(struct exec_params) */ }; - In this case, `connect_domain` is set to id of **domY** (from the `-c` - parameter) and `connect_port` is set to 0. `cmdline` field contains the - RPC to execute, in this case `user:QUBESRPC qubes.SomeRpc domY`. + In this case, `connect_domain` is set to id of **domY** (from the `-c` parameter) and `connect_port` is set to 0. `cmdline` field contains the RPC to execute, in this case `user:QUBESRPC qubes.SomeRpc domY`. -- **dom0**: **domX**'s `qrexec-daemon` replies to `qrexec-client` with -`MSG_EXEC_CMDLINE` header followed by `exec_params`, but with empty `cmdline` -field. `connect_domain` is set to Qubes ID of **domX** and `connect_port` -is set to a vchan port allocated by **domX**'s `qrexec-daemon`. -- **dom0**: **domX**'s `qrexec-daemon` sends `MSG_EXEC_CMDLINE` header -followed by `exec_params` to **domX**'s `qrexec-agent`. `connect_domain` -and `connect_port` fields are the same as in the step above. `cmdline` is -set to the one received from `qrexec-client`, in this case `user:QUBESRPC -qubes.SomeRpc domY`. -- **dom0**: `qrexec-client` disconnects from **domX**'s `qrexec-daemon` -after receiving connection details. -- **dom0**: `qrexec-client` connects to **domY**'s `qrexec-daemon` and -exchanges `MSG_HELLO` as usual. -- **dom0**: `qrexec-client` sends `MSG_SERVICE_CONNECT` header followed by -`exec_params` to **domY**'s `qrexec-daemon`. `connect_domain` is set to ID -of **domX** (received from **domX**'s `qrexec-daemon`) and `connect_port` is -the one received as well. `cmdline` is set to request ID (`13` in this case). -- **dom0**: **domY**'s `qrexec-daemon` sends `MSG_SERVICE_CONNECT` header -followed by `exec_params` to **domY**'s `qrexec-agent`. Data fields are -unchanged from the step above. -- **domY**: `qrexec-agent` starts a vchan server on the port received in -the step above. It acts as a `qrexec-client` in this case because this is -a VM-VM connection. -- **domX**: `qrexec-agent` connects to the vchan server of **domY**'s -`qrexec-agent` (connection details were received before from **domX**'s -`qrexec-daemon`). +- **dom0**: **domX**'s `qrexec-daemon` replies to `qrexec-client` with `MSG_EXEC_CMDLINE` header followed by `exec_params`, but with empty `cmdline` field. `connect_domain` is set to Qubes ID of **domX** and `connect_port` is set to a vchan port allocated by **domX**'s `qrexec-daemon`. +- **dom0**: **domX**'s `qrexec-daemon` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to **domX**'s `qrexec-agent`. `connect_domain` and `connect_port` fields are the same as in the step above. `cmdline` is set to the one received from `qrexec-client`, in this case `user:QUBESRPC qubes.SomeRpc domY`. +- **dom0**: `qrexec-client` disconnects from **domX**'s `qrexec-daemon` after receiving connection details. +- **dom0**: `qrexec-client` connects to **domY**'s `qrexec-daemon` and exchanges `MSG_HELLO` as usual. +- **dom0**: `qrexec-client` sends `MSG_SERVICE_CONNECT` header followed by `exec_params` to **domY**'s `qrexec-daemon`. `connect_domain` is set to ID of **domX** (received from **domX**'s `qrexec-daemon`) and `connect_port` is the one received as well. `cmdline` is set to request ID (`13` in this case). +- **dom0**: **domY**'s `qrexec-daemon` sends `MSG_SERVICE_CONNECT` header followed by `exec_params` to **domY**'s `qrexec-agent`. Data fields are unchanged from the step above. +- **domY**: `qrexec-agent` starts a vchan server on the port received in the step above. It acts as a `qrexec-client` in this case because this is a VM-VM connection. +- **domX**: `qrexec-agent` connects to the vchan server of **domY**'s `qrexec-agent` (connection details were received before from **domX**'s `qrexec-daemon`). - After that, connection follows the flow of the previous example (dom0-VM).