Add comprehensive Docker container documentation

- Document container architecture with diagrams - Include build, run, and configuration instructions - Provide examples for common operations and debugging - Document volume management and data persistence - Explain networking and security considerations - Add examples for development and integration
2025-04-19 15:25:54 -04:00 · 2025-04-14 07:15:06 +00:00 · 2025-04-14 07:15:06 +00:00 · 431c6065b6
commit 431c6065b6
parent c579ce0005
1 changed files with 479 additions and 0 deletions
--- a/doc/config/veilid-docker-container.md
+++ b/doc/config/veilid-docker-container.md
@ -0,0 +1,479 @@
+# Veilid Server Docker Container
+
+This document provides comprehensive information about the Veilid Server Docker container, including its architecture, build process, runtime behavior, and operational considerations.
+
+## Table of Contents
+
+- [Overview](#overview)
+- [Container Architecture](#container-architecture)
+- [Build Process](#build-process)
+- [Running the Container](#running-the-container)
+- [Configuration](#configuration)
+- [Data Persistence](#data-persistence)
+- [Networking](#networking)
+- [Healthcheck](#healthcheck)
+- [Security Considerations](#security-considerations)
+- [Debugging](#debugging)
+- [Development Guidelines](#development-guidelines)
+
+## Overview
+
+The Veilid Server Docker container provides a containerized deployment of the Veilid Server, which operates as a node in the Veilid peer-to-peer network. The container includes both the `veilid-server` daemon and the `veilid-cli` client tool.
+
+### Key Features
+
+- Multi-stage build for optimized image size
+- Non-root operation for enhanced security
+- Proper permission handling for all components
+- Persistent volume support for data and configuration
+- Comprehensive health monitoring
+- Configurable through environment variables or mounted configuration files
+
+## Container Architecture
+
+The container follows a multi-stage build approach to minimize the final image size.
+
+```mermaid
+flowchart TD
+    A[Builder Stage] --> |Binary Artifacts| B[Runtime Stage]
+    B --> C[Running Container]
+    
+    subgraph "Builder Stage"
+    A1[Rust Compiler]
+    A2[Build Dependencies]
+    A3[Source Code]
+    end
+    
+    subgraph "Runtime Stage"
+    B1[Minimal OS Base]
+    B2[Runtime Dependencies]
+    B3[Binary Executables]
+    B4[Configuration]
+    B5[User/Permissions]
+    end
+    
+    subgraph "Container Runtime"
+    C1[veilid-server Process]
+    C2[Persistent Volumes]
+    C3[Network Interfaces]
+    C4[Health Monitoring]
+    end
+```
+
+### Component Breakdown
+
+| Component | Description |
+|-----------|-------------|
+| veilid-server | Main server daemon responsible for participating in the Veilid network |
+| veilid-cli | Command-line client tool for interacting with the server |
+| Configuration | YAML configuration controlling server behavior |
+| Protected Store | Storage for encrypted sensitive data |
+| Table Store | Key-value storage for application data |
+| Block Store | Storage for larger data blocks |
+| IPC Directory | Inter-process communication socket directory |
+
+## Build Process
+
+### Prerequisites
+
+- Docker installed and configured
+- Internet access for downloading dependencies
+
+### Building the Image
+
+```bash
+# Basic build
+docker build -t veilid-server -f Dockerfile.server .
+
+# Build with specific tag
+docker build -t veilid-server:v0.4.1 -f Dockerfile.server .
+
+# Build with build args (if needed)
+docker build --build-arg RUST_VERSION=1.81.0 -t veilid-server -f Dockerfile.server .
+```
+
+### Multi-Stage Build Details
+
+The build process consists of two primary stages:
+
+1. **Builder Stage**
+   - Uses `rust:1.81.0-bullseye` as the base image
+   - Installs build dependencies (build-essential, libssl-dev, etc.)
+   - Copies source code and builds binaries
+   - Compiles both veilid-server and veilid-cli
+
+2. **Runtime Stage**
+   - Uses `debian:bullseye-slim` as the minimal base image
+   - Copies only necessary binaries from the builder stage
+   - Installs minimal runtime dependencies
+   - Sets up user permissions and directories
+   - Configures the runtime environment
+
+## Running the Container
+
+### Basic Usage
+
+```bash
+# Run with default configuration
+docker run -d --name veilid-server \
+  -p 5150:5150/tcp \
+  -p 5150:5150/udp \
+  -p 5959:5959/tcp \
+  veilid-server
+```
+
+### Advanced Usage
+
+```bash
+# Run with custom configuration and persistent volumes
+docker run -d --name veilid-server \
+  -p 5150:5150/tcp \
+  -p 5150:5150/udp \
+  -p 5959:5959/tcp \
+  -v /path/to/config:/etc/veilid-server \
+  -v /path/to/data/protected:/var/db/veilid-server/protected_store \
+  -v /path/to/data/table:/var/db/veilid-server/table_store \
+  -v /path/to/data/block:/var/db/veilid-server/block_store \
+  -v /path/to/logs:/var/log/veilid \
+  veilid-server
+```
+
+### Docker Compose Example
+
+```yaml
+version: '3.8'
+
+services:
+  veilid-server:
+    image: veilid-server:latest
+    container_name: veilid-server
+    restart: unless-stopped
+    ports:
+      - "5150:5150/tcp"
+      - "5150:5150/udp"
+      - "5959:5959/tcp"
+    volumes:
+      - veilid-config:/etc/veilid-server
+      - veilid-protected:/var/db/veilid-server/protected_store
+      - veilid-table:/var/db/veilid-server/table_store
+      - veilid-block:/var/db/veilid-server/block_store
+      - veilid-logs:/var/log/veilid
+    healthcheck:
+      test: ["CMD", "HOME=/home/veilid", "XDG_CONFIG_HOME=/home/veilid/.config", "XDG_CACHE_HOME=/home/veilid/.cache", "XDG_DATA_HOME=/home/veilid/.local/share", "XDG_RUNTIME_DIR=/var/run/user/999", "VEILID_CLI_CONFIG_DIR=/home/veilid/.veilid-cli", "veilid-cli", "-e", "nodeid", "|", "grep", "-q", "VLD0:"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 60s
+
+volumes:
+  veilid-config:
+  veilid-protected:
+  veilid-table:
+  veilid-block:
+  veilid-logs:
+```
+
+## Configuration
+
+The container uses a YAML configuration file located at `/etc/veilid-server/veilid-server.conf`. This configuration can be customized by mounting a custom configuration file or by setting environment variables.
+
+### Default Docker Configuration
+
+The default configuration optimized for Docker environments is provided below:
+
+```yaml
+daemon:
+  enabled: false
+client_api:
+  ipc_enabled: true
+  ipc_directory: '/var/db/veilid-server/ipc'
+  network_enabled: true
+  listen_address: '0.0.0.0:5959'
+auto_attach: true
+logging:
+  system:
+    enabled: false
+    level: info
+    ignore_log_targets: []
+  terminal:
+    enabled: true
+    level: info
+    ignore_log_targets: []
+  file:
+    enabled: true
+    path: '/var/log/veilid/veilid-server.log'
+    append: true
+    level: info
+    ignore_log_targets: []
+  api:
+    enabled: true
+    level: info
+    ignore_log_targets: []
+testing:
+  subnode_index: 0
+core:
+  protected_store:
+    allow_insecure_fallback: true
+    always_use_insecure_storage: true
+    directory: '/var/db/veilid-server/protected_store'
+    delete: false
+  table_store:
+    directory: '/var/db/veilid-server/table_store'
+    delete: false
+  block_store:
+    directory: '/var/db/veilid-server/block_store'
+    delete: false
+  network:
+    protocol:
+      udp:
+        listen_address: ':5150'
+      tcp:
+        listen_address: ':5150'
+      ws:
+        listen_address: ':5150'
+      wss:
+        listen_address: ':5150'
+```
+
+### Key Configuration Options
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `client_api.network_enabled` | Enable network API access | true |
+| `client_api.listen_address` | API listen address | 0.0.0.0:5959 |
+| `auto_attach` | Automatically connect to the Veilid network | true |
+| `logging.terminal.enabled` | Enable terminal logging | true |
+| `logging.file.enabled` | Enable file logging | true |
+| `logging.file.path` | Log file path | /var/log/veilid/veilid-server.log |
+| `core.network.protocol.*.listen_address` | Protocol listen address | :5150 |
+
+### Environment Variables
+
+The container respects the following environment variables:
+
+| Variable | Description |
+|----------|-------------|
+| `HOME` | Home directory for the veilid user | 
+| `XDG_CONFIG_HOME` | Config directory |
+| `XDG_CACHE_HOME` | Cache directory |
+| `XDG_DATA_HOME` | Data directory |
+| `XDG_RUNTIME_DIR` | Runtime directory |
+| `VEILID_CLI_CONFIG_DIR` | veilid-cli configuration directory |
+
+## Data Persistence
+
+The container defines several volumes for data persistence:
+
+| Volume Path | Description |
+|-------------|-------------|
+| `/var/db/veilid-server/protected_store` | Protected storage (encrypted sensitive data) |
+| `/var/db/veilid-server/table_store` | Table storage (key-value data) |
+| `/var/db/veilid-server/block_store` | Block storage (larger data objects) |
+| `/var/log/veilid` | Log files |
+| `/etc/veilid-server` | Configuration files |
+
+### Backup and Restore
+
+To back up the Veilid Server data:
+
+```bash
+# Create backup of all volumes
+docker run --rm --volumes-from veilid-server \
+  -v $(pwd):/backup \
+  debian:bullseye-slim \
+  tar czf /backup/veilid-data-$(date +%Y%m%d).tar.gz \
+  /var/db/veilid-server /var/log/veilid /etc/veilid-server
+```
+
+To restore from a backup:
+
+```bash
+# Restore from backup
+docker run --rm --volumes-from veilid-server \
+  -v $(pwd):/backup \
+  debian:bullseye-slim \
+  tar xzf /backup/veilid-data-20250414.tar.gz -C /
+```
+
+## Networking
+
+The container exposes the following ports:
+
+| Port | Protocol | Description |
+|------|----------|-------------|
+| 5150 | TCP/UDP | Veilid P2P network communications |
+| 5959 | TCP | Client API endpoint |
+
+### Network Configuration
+
+The Veilid Server participates in the P2P network through multiple protocols:
+
+```mermaid
+graph LR
+    A[Veilid Server Container] -- UDP/5150 --> B[UDP P2P Traffic]
+    A -- TCP/5150 --> C[TCP P2P Traffic]
+    A -- WebSocket/5150 --> D[WS P2P Traffic]
+    A -- WebSocket Secure/5150 --> E[WSS P2P Traffic]
+    F[Client Applications] -- TCP/5959 --> A
+```
+
+## Healthcheck
+
+The container includes a built-in healthcheck that verifies the server is operational by executing the `nodeid` command via the veilid-cli tool:
+
+```bash
+HOME=/home/veilid \
+    XDG_CONFIG_HOME=/home/veilid/.config \
+    XDG_CACHE_HOME=/home/veilid/.cache \
+    XDG_DATA_HOME=/home/veilid/.local/share \
+    XDG_RUNTIME_DIR=/var/run/user/$(id -u veilid) \
+    VEILID_CLI_CONFIG_DIR=/home/veilid/.veilid-cli \
+    veilid-cli -e nodeid | grep -q "VLD0:" || exit 1
+```
+
+### Healthcheck Configuration
+
+| Parameter | Value | Description |
+|-----------|-------|-------------|
+| Interval | 30s | Time between health checks |
+| Timeout | 10s | Maximum time for healthcheck to complete |
+| Start Period | 60s | Grace period for initialization |
+| Retries | 3 | Number of consecutive failures before unhealthy |
+
+### Monitoring Health Status
+
+```bash
+# Check container health status
+docker inspect --format='{{.State.Health.Status}}' veilid-server
+
+# View health check logs
+docker inspect --format='{{range .State.Health.Log}}{{.Output}}{{end}}' veilid-server
+```
+
+## Security Considerations
+
+### Running as Non-Root User
+
+The container executes as the non-root `veilid` user (UID/GID varies by system but is typically 999) for enhanced security. This limits the potential impact of container escape vulnerabilities.
+
+### Filesystem Permissions
+
+The container sets appropriate permissions on all directories:
+- Config directory: 750 (rwxr-x---)
+- Data directories: 750 (rwxr-x---)
+- IPC directory: 1777 (rwxrwxrwt)
+
+### Network Security
+
+By default, the client API listens on all interfaces (0.0.0.0:5959). In production deployments, consider restricting this to specific interfaces or using network policies to limit access.
+
+## Debugging
+
+### Inspecting Logs
+
+```bash
+# View container logs
+docker logs veilid-server
+
+# Follow logs in real-time
+docker logs -f veilid-server
+
+# Access specific log file
+docker exec veilid-server cat /var/log/veilid/veilid-server.log
+```
+
+### Interactive Debugging
+
+```bash
+# Run interactive shell in container
+docker exec -it veilid-server bash
+
+# Use veilid-cli for debugging
+docker exec -it veilid-server veilid-cli -e help
+docker exec -it veilid-server veilid-cli -e nodeinfo
+```
+
+### Common Debugging Commands
+
+| Command | Description |
+|---------|-------------|
+| `veilid-cli -e nodeid` | Display the node ID |
+| `veilid-cli -e nodeinfo` | Display detailed node information |
+| `veilid-cli -e buckets` | Show routing table bucket statistics |
+| `veilid-cli -e dialinfo` | Show dial information for routing domains |
+| `veilid-cli -e config` | Display current node configuration |
+| `veilid-cli -e uptime` | Display node uptime |
+
+## Development Guidelines
+
+### Dockerfile Modifications
+
+When modifying the Dockerfile:
+
+1. Always maintain the multi-stage build approach
+2. Minimize the number of layers to reduce image size
+3. Keep runtime dependencies to a minimum
+4. Maintain proper permissions for the veilid user
+5. Test changes thoroughly before committing
+
+### Testing Modifications
+
+```bash
+# Build test image
+docker build -t veilid-server:test -f Dockerfile.server .
+
+# Run test container
+docker run -d --name veilid-server-test -p 5150:5150 -p 5959:5959 veilid-server:test
+
+# Verify container health
+docker inspect --format='{{.State.Health.Status}}' veilid-server-test
+
+# Test client connectivity
+docker exec veilid-server-test veilid-cli -e nodeid
+```
+
+### CI/CD Integration
+
+For automated builds and testing, consider the following pipeline steps:
+
+1. Build the container image
+2. Run automated tests to verify functionality
+3. Check container security with tools like Docker Scout or Trivy
+4. Push to registry only if all tests pass
+
+Example GitHub Actions workflow:
+
+```yaml
+name: Build Veilid Server Container
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - 'Dockerfile.server'
+      - 'docker-config/**'
+  pull_request:
+    branches: [ main ]
+    paths:
+      - 'Dockerfile.server'
+      - 'docker-config/**'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      
+      - name: Build image
+        run: docker build -t veilid-server:test -f Dockerfile.server .
+        
+      - name: Run container
+        run: |
+          docker run -d --name veilid-server-test -p 5150:5150 -p 5959:5959 veilid-server:test
+          sleep 60 # Allow for startup
+          
+      - name: Test container
+        run: |
+          docker inspect --format='{{.State.Health.Status}}' veilid-server-test
+          docker exec veilid-server-test veilid-cli -e nodeid
+```