qubes-mirage-firewall/CHANGES.md

### 0.8.2 (2022-10-12)

- Advise to use 32 MB memory, which is sufficient (#150, @palainp)
- Improve documentation (#150, @palainp)
- Remove unneeded memory management code and log messages (#150, @palainp)
- Use mirage-nat 3.0.0, remove global mutable state (#151, @hannesm)

### 0.8.1 (2022-09-14)

- support qrexec protocol version 3 (@reynir @palainp in mirage-qubes 0.9.3)
- remove special DNS rule (which used to be required for Qubes 3, issue #63, fix #142, @hannesm)
- use DNS servers from QubesDB instead of hardcoded ones for evaluation of the DNS rule (#142 @hannesm)
- remove the GUI code (not needed in Qubes 4.1 anymore, issue #62, fix #144, @palainp)
- trigger GC slightly earlier (at < 50% free space, issue #143, fix #147, @palainp)

### 0.8.0

The major change is to use PVH instead of PV. The effort was in solo5 (https://github.com/solo5/solo5) which since 0.6.6 supports Xen and PVH (developed by @mato, with some fixes (multiboot, mem size computed uniformly, not skipping first token of command line arguments) by @marmarek, @xaki23, @palainp, and @hannesm).

Another user-visible change is that the DNS resolver is read from QubesDB /qubes-primary-dns instead of using a hardcoded IP address (@palainp and @hannesm).

Also, the qrexec version negotiation has been implemented (in mirage-qubes by @reynir).

Thanks to @palainp and @winux138 keeping track of memory allocation has been improved, and also memory can be freed now.

This release uses the latest mirage release (4.2.1). It can be built with a Fedora 35 container. It uses OCaml 4.14.0.

Thanks to @talex5 for lots of code cleanups, reviews, and merges. Also thanks to @xaki23 for early and detailed feedback. Testing was done by @Tommytran732 and @Szewcson. Thanks to @burghardt for documentation improvements.

### 0.7.1

Bugfixes:

- More robust parsing of IP address in Xenstore, which may contain both IPv4 and IPv6 addresses (@linse, #103, reported by @grote)

- Avoid stack overflow with many connections in the NAT table (@linse and @hannesm, reported by @talex5 in #105, fixed by mirage-nat 2.2.2 release)

### 0.7

This version adapts qubes-mirage-firewall with
- dynamic rulesets via QubesDB (as defined in Qubes 4.0), and
- adds support for DNS hostnames in rules, using the pf-qubes library for parsing.

The DNS client is provided by DNS (>= 4.2.0) which uses a cache for name lookups. Not every packet will lead to a DNS lookup if DNS rules are in place.

A test unikernel is available in the test subdirectory.

This project was done by @linse and @yomimono in summer 2019, see PR #96.

Additional changes and bugfixes:

- Support Mirage 3.7 and mirage-nat 2.0.0 (@hannesm, #89).
  The main improvement is fragmentation and reassembly support.

- Use the smaller OCurrent images as the base for building the Docker images (@talex5, #80).
  - Before: 1 GB (ocaml/opam2:debian-10-ocaml-4.08)
  - Now: 309 MB (ocurrent/opam:alpine-3.10-ocaml-4.08)

- Removed unreachable `Lwt.catch` (@hannesm, #90).

Documentation:

- Add note that AppVM used to build from source may need a private image larger than the default 2048MB (@marmot1791, #83).

- README: create the symlink-redirected docker dir (@xaki23, #75). Otherwise, installing the docker package removes the dangling symlink.

- Note that mirage-firewall cannot be used as UpdateVM (@talex5, #68).

- Fix ln(1) call in build instructions (@jaseg, #69). The arguments were backwards.

Keeping up with upstream changes:

- Support mirage-3.7 via qubes-builder (@xaki23, #91).

- Remove unused `Clock` argument to `Uplink` (@talex5, #90).

- Rename things for newer mirage-xen versions (@xaki23, #80).

- Adjust to ipaddr-4.0.0 renaming `_bytes` to `_octets` (@xaki23, #75).

- Use OCaml 4.08.0 for qubes-builder builds (was 4.07.1) (@xaki23, #75).

- Remove netchannel pin as 1.11.0 is now released (@talex5, #72).

- Remove cmdliner pin as 1.0.4 is now released (@talex5, #71).


### 0.6

Changes to rules language:

- Allow naming hosts (@talex5, #54).
  Previously, we passed in the interface, from which it was possible (but a
  little difficult) to extract the IP address and compare with some predefined
  ones. Now, we allow the user to list IP addresses and named tags for them,
  which can be matched on easily.

- Add some types to the rules (@talex5, #54).
  Before, we inferred the types from `rules.ml` and then the compiler checked that
  it was consistent with what `firewall.ml` expected. If it wasn't then it
  reported the problem as being with `firewall.ml`, which could be confusing to
  users.

- Give exact types for `Packet.src` (@talex5, #54).
  Before, the packet passed to `rules.ml` could have any host as its `src`.
  Now, `from_client` knows that `src` must be a `Client`,
  and `from_netvm` knows that `src` is `External` or `NetVM`.

- Combine `Client_gateway` and `Firewall_uplink` (@talex5, #64).
  Before, we used `Client_gateway` for the IP address of the firewall on the client network
  and `Firewall_uplink` for its address on the uplink network.
  However, Qubes 4 uses the same IP address for both, so we can't separate these any longer,
  and there doesn't seem to be any advantage to keeping them separate anyway.

Bug fixes:

- Upgrade to latest mirage-nat to fix ICMP (@yomimono, @linse, #55).
  Now ping and traceroute should work. Reported by @xaki23.

- Respond to ARP requests for `*.*.*.1` (@talex5, #61).
  This is a work-around to get DHCP working with HVM domains.
  Reported by @cgchinicz.
  See: https://github.com/QubesOS/qubes-issues/issues/5022

- Force backend MAC to `fe:ff:ff:ff:ff:ff` to fix HVM clients (@talex5, #61).
  Xen appears to configure the same MAC address for both the frontend and
  backend in XenStore. This works if the client uses just a simple ethernet
  device, but fails if it connects via a bridge. HVM domains have an associated
  stub domain running qemu, which provides an emulated network device. The stub
  domain uses a bridge to connect qemu's interface with eth0, and this didn't
  work. Force the use of the fixed version of mirage-net-xen, which no longer
  uses XenStore to get the backend MAC, and provides a new function to get the
  frontend one.

- Wait if dom0 is slow to set the network configuration (@talex5, #60).
  Sometimes we boot before dom0 has put the network settings in QubesDB.
  If that happens, log a message, wait until the database changes, and retry.

Reproducible builds:

- Add patch to cmdliner for reproducible build (@talex5, #52).
  See https://github.com/dbuenzli/cmdliner/pull/106

- Use source date in .tar.bz2 archive (@talex5, #49).
  All files are now added using the date the `build-with-docker` script was last changed.
  Since this includes the hash of the result, it should be up-to-date.
  This ensures that rebuilding the archive doesn't change it in any way.
  Reported by Holger Levsen.

Documentation changes:

- Added example rules showing how to block access to an external service or
  allow SSH between AppVMs (@talex5, #54). Requested at
  https://groups.google.com/d/msg/qubes-users/BnL0nZGpJOE/61HOBg1rCgAJ.

- Add overview of the main components of the firewall in the README (@talex5, #54).

- Link to security advisories from README (@talex5, #58).

- Clarify how to build from source (@talex5, #51).

- Remove Qubes 3 instructions (@talex5, #48).
  See https://www.qubes-os.org/news/2019/03/28/qubes-3-2-has-reached-eol/

### 0.5

- Update to the latest mirage-net-xen, mirage-nat and tcpip libraries (@yomimono, @talex5, #45, #47).
  In iperf benchmarks between a client VM and sys-net, this more than doubled the reported bandwidth!

- Don't wait for the Qubes GUI daemon to connect before attaching client VMs (@talex5, #38).
  If the firewall is restarted while AppVMs are connected, qubesd tries to
  reconnect them before starting the GUI agent. However, the firewall was
  waiting for the GUI agent to connect before handling the connections. This
  led to a 10s delay on restart for each client VM. Reported by @xaki23.

- Add stub makefile for qubes-builder (@xaki23, #37).

- Update build instructions for latest Fedora (@talex5, #36). `yum` no longer exists.
  Also, show how to create a symlink for `/var/lib/docker` on build VMs that aren't standalone.
  Reported by @xaki23.

- Add installation instructions for Qubes 4 (@yomimono, @reynir, @talex5, #27).

- Use `Ethernet_wire.sizeof_ethernet` instead of a magic `14` (@hannesm, #46).

### 0.4

- Add support for HVM guests (needed for Qubes 4).

- Add support for disposable VMs.

- Drop frames if an interface's queue gets too long.

- Show the packet when failing to add a NAT rule. The previous message was
  just: `WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet`

### 0.3

- Add support for NAT of ICMP queries (e.g. pings) and errors (e.g. "Host unreachable").
  Before, these packets would be dropped.

- Use an LRU cache to avoid running out of memory and needing to reset the table.
  Should avoid any more out-of-memory bugs.

- Pass around parsed packets rather than raw ethernet frames.

- Pin Docker base image to a specific hash. Requested by Joanna Rutkowska.

- Update for Mirage 3.

- Remove non-Docker build instructions. Fedora 24 doesn't work with opam
  (because the current binary release of aspcud's clasp binary segfaults, which
  opam reports as `External solver failed with inconsistent return value.`).

### 0.2

Build:

- Add option to build with Docker. This fixes opam-repository to a known commit
  for reproducible builds. It also displays the actual and expected SHA hashes
  after building.

Bug fixes:

- Updated README: the build also requires "patch". Reported by William Waites.
- Monitor set of client interfaces, not client domains. Qubes does not remove
  the client directory itself when the domain exits. This prevented clients
  from reconnecting. This may also make it possible to connect clients to the
  firewall via multiple interfaces, although this doesn't seem useful.
- Handle errors writing to client. mirage-net-xen would report `Netback_shutdown`
  if we tried to write to a client after it had disconnected. Now we just log
  this and continue.
- Ensure that old client has quit before adding new one. Not sure if this can
  happen, but it removes a TODO from the code.
- Allow clients to have any IP address. We previously assumed that Qubes would
  always give clients IP addresses on a particular network. However, it is not
  required to do this and in fact uses a different network for disposable VMs.
  With this change:
  - We no longer reject clients with unknown IP addresses.
  - The `Unknown_client` classification is gone; we have no way to tell the
    difference between a client that isn't connected and an external address.
  - We now consider every client to be on a point-to-point link and do not
    answer ARP requests on behalf of other clients. Clients should assume their
    netmask is `255.255.255.255` (and ignore `/qubes-netmask`). This allows
    disposable VMs to connect to the firewall but for some reason they don't
    process any frames we send them (we get their ARP requests but they don't
    get our replies). Taking eth0 down in the disp VM, then bringing it back up
    (and re-adding the routes) allows it to work.
- Cope with writing a frame failing. If a client disconnects suddenly then we
  may get an error trying to map its grant to send the frame.
- Survive death of our GUId connection to dom0. We don't need the GUI anyway.
- Handle `Out_of_memory` adding NAT entries. Because hash tables resize in big
  steps, this can happen even if we have a fair chunk of free memory.
- Calculate checksums even for `Accept` action. If packet has been NAT'd then we
  certainly need to recalculate the checksum, but even for direct pass-through
  it might have been received with an invalid checksum due to checksum offload.
  For now, recalculate full checksum in all cases.
- Log correct destination for redirected packets. Before, we always said it was
  going to "NetVM".
- If we can't find a free port, reset the NAT table.
- Reset NAT table if memory gets low.

Other changes:

- Report current memory use to XenStore.
- Reduce logging verbosity.
- Avoid using `Lwt.join` on listening threads.
  `Lwt.join` only reports an error if _both_ threads fail.
- Keep track of transmit queue lengths. Log if we have to wait to send a frame.
- Use mirage-logs library for log reporter.
- Respond to `WaitForSession` commands (we're always ready!).
- Log `SetDateTime` messages from dom0 (we still don't actually update our clock,
  though).

Updates for upstream library changes:

- Updates for mirage 2.9.0.
  - Use new name for uplink device (`0`, not `tap0`).
  - Don't configure logging - mirage does that for us now.
- Remove tcpip pin. The 2.7.0 release has the checksum feature we need.
- Remove mirage-xen pin. mirage-xen 2.4.0 has been released with the required
  features (also fixes indentation problem reported by @cfcs).
- Add ncurses-dev to required yum packages. The ocamlfind package has started
  listing this as a required dependency for some reason, although it appears
  not to need it. Reported by cyrinux.
- Add work-around for Qubes passing Linux kernel arguments. With the new
  Functoria release of Mirage, these unrecognised arguments prevented the
  unikernel from booting. See: https://github.com/mirage/mirage/issues/493
- Remove mirage-logs pin. Now available from the main repository.
- Remove mirage-qubes pin.
  mirage-qubes 0.2 has been released, and supports the latests Logs API.
- Remove mirage-net-xen pin.
  Version 1.5 has now been released, and includes netback support.
- Update to new Logs API.
- Remove pin for mirage-clock-xen. New version has been released now.

### 0.1

Initial release.