Git-Mirrors/constellation
1
0
Fork 0
mirror of https://github.com/edgelesssys/constellation.git synced 2024-09-18 23:36:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

AB#2369: Use contributing.md as ToC for dev docs.

Browse Source 
* Structure content into typical sections and
split into separate files.
* Also document how to locally create measurements

Signed-off-by: Otto Bittner <cobittner@posteo.net>
This commit is contained in:
Otto Bittner 2022-09-06 14:49:13 +02:00
parent 1c63b65af2
commit ef26917c5e
6 changed files with 261 additions and 201 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

115
.github/docs/conventions.md vendored Normal file
View File

@ -0,0 +1,115 @@
# Process conventions
## Pull request process
Submissions should remain focused in scope and avoid containing unrelated commits.
For pull requests, we employ the following workflow:
1. Fork the repository to your own GitHub account
2. Create a branch locally with a descriptive name
3. Commit changes to the branch
4. Write your code according to our development guidelines
5. Push changes to your fork
6. Clean up your commit history
7. Open a PR in our repository and summarize the changes in the description
## Reporting issues and bugs, asking questions
This project uses the GitHub issue tracker. Please check the existing issues before submitting to avoid duplicates.
To report a security issue, contact security@edgeless.systems.
Your bug report should cover the following points:
* A quick summary and/or background of the issue
* Steps to reproduce (be specific, e.g., provide sample code)
* What you expected would happen
* What actually happens
* Further notes:
* Thoughts on possible causes
* Tested workarounds or fixes
## Major changes and feature requests
You should discuss larger changes and feature requests with the maintainers. Please open an issue describing your plans.
[Run CI e2e tests](/.github/docs/README.md)
# Code conventions
## General
Adhere to the style and best practices described in [Effective Go](https://golang.org/doc/effective_go.html). Read [Common Review Comments](https://github.com/golang/go/wiki/CodeReviewComments) for further information.
## Linting
This projects uses [golangci-lint](https://golangci-lint.run/) for linting.
You can [install golangci-lint](https://golangci-lint.run/usage/install/#linux-and-windows) locally,
but there is also a CI action to ensure compliance.
To locally run all configured linters, execute
```sh
golangci-lint run ./...
```
It is also recommended to use golangci-lint (and [gofumpt](https://github.com/mvdan/gofumpt) as formatter) in your IDE, by adding the recommended VS Code Settings or by [configuring it yourself](https://golangci-lint.run/usage/integrations/#editor-integration)
## Nested Go modules
As this project contains nested Go modules, it is recommended to create a local Go workspace, so your IDE can lint multiple modules at once.
```go
go 1.18
use (
.
./hack
./operators/constellation-node-operator
)
```
You can find an introduction in the [Go workspace tutorial](https://go.dev/doc/tutorial/workspaces).
If you have changed dependencies within a module and have run `go mod tidy`, you can use `go work sync` to sync versions of the same dependency of the different modules.
## Recommended VS Code Settings
The following can be added to your personal `settings.json`, but it is recommended to add it to
the `<REPOSITORY>/.vscode/settings.json` repo, so the settings will only affect this repository.
```jsonc
// Use gofumpt as formatter.
"gopls": {
"formatting.gofumpt": true,
},
// Use golangci-lint as linter. Make sure you've installed it.
"go.lintTool":"golangci-lint",
"go.lintFlags": ["--fast"],
// You can easily show Go test coverage by running a package test.
"go.coverageOptions": "showUncoveredCodeOnly",
// Executing unit tests with race detection.
// You can add preferences like "-v" or "-count=1"
"go.testFlags": ["-race"],
// Enable language features for files with build tags.
// Attention! This leads to integration test being executed when
// running a package test within a package containing integration
// tests.
"go.buildTags": "integration",
```
Additionally, we use the [Redhat YAML formatter](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) to have uniform formatting in our `.yaml` files.
## Naming convention
### Network
IP addresses:
* ip: numeric IP address
* host: either IP address or hostname
* endpoint: host+port
### Keys
* key: symmetric key
* pubKey: public key
* privKey: private key

101
.github/docs/development.md vendored Normal file
View File

@ -0,0 +1,101 @@
# Build
The following are instructions for building all components in the constellation repository, except for images. A manual on how to build images locally can be found in the [image README](/image/README.md).
Prerequisites:
* 20 GB disk space
* [Go 1.18](https://go.dev/doc/install). Can be installed with these commands:
```sh
wget https://go.dev/dl/go1.18.linux-amd64.tar.gz && sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.18.linux-amd64.tar.gz && export PATH=$PATH:/usr/local/go/bin
```
* [Docker](https://docs.docker.com/engine/install/). Can be installed with these commands on Ubuntu 22.04: `sudo apt update && sudo apt install docker.io`. As the build spawns docker containers your user account either needs to be in the `docker` group (Add with `sudo usermod -a -G docker $USER`) or you have to run builds with `sudo`. When using `sudo` remember that your root user might (depending on your distro and local config) not have the go binary in it's PATH. The current PATH can be forwarded to the root env with `sudo env PATH=$PATH <cmd>`.
* Packages on Ubuntu:
```sh
sudo apt install build-essential cmake libssl-dev pkg-config libcryptsetup12 libcryptsetup-dev
```
* Packages on Fedora:
```sh
sudo dnf install @development-tools pkg-config cmake openssl-devel cryptsetup-libs cryptsetup-devel
```
```sh
mkdir build
cd build
cmake ..
make -j`nproc`
```
# Test
You can run all integration and unitttests like this:
```sh
ctest -j `nproc`
```
You can limit the execution of tests to specific targets with e.g. `ctest -R unit-*` to only execute unit tests.
Some of the tests rely on libvirt and won't work if you don't have a virtualization capable CPU. You can find instructions on setting up libvirt in our [terraform README](/terraform/libvirt/README.md).
# Deploy
> :warning: Debug images are not safe to use in production environments. :warning:
The debug images will open an additional **unsecured** port (4000) which accepts any binary to be run on the target machine. **Make sure that those machines are not exposed to the internet.**
## Cloud
To familiarize yourself with debugd and learn how to deploy a cluster using it, read [this](/debugd/README.md) manual.
If you want to deploy a cluster for production, please refer to our user documentation [here](https://docs.edgeless.systems/constellation/getting-started/first-steps#create-a-cluster).
## Locally
In case you want to have quicker iteration cycles during development you might want to setup a local cluster.
You can do this by utilizing our terraform setup.
Instructions on how to set it up can be found in it's [README](/terraform/libvirt/README.md).
# Verification
In order to verify your cluster we describe a [verification workflow](https://constellation-docs.edgeless.systems/constellation/workflows/verify) in our official docs.
Apart from that you can also reproduce some of the measurements described in the [docs](https://constellation-docs.edgeless.systems/constellation/architecture/attestation#runtime-measurements) locally.
To do so you have to create a cluster locally as described in the [previous section](/.github/docs/development.md#locally).
However, you don't have to go through the full manual, you can stop after running `terraform apply`.
If `terraform apply` succeeded you will find two files in your current working directory: `control-plane-0_pcrs.json` and `control-plane-0_pcrs.yaml`.
You can compare the values of PCR 8 and 9 to the ones you are seeing in your `constellation-conf.yaml`.
The PCR values depend on the image you specify in `constellation_coreos_image` in your `terraform.tfvars`.
So if you want to verify a cluster deployed with a release images you will have to download the images first.
After collecting the measurements you can put them into your `constellation-conf.yaml` under the `measurements` key in order to enforce them.
# Image export
To download an image you will have to export it first.
Below you find general instructions on how to do this for GCP and Azure.
You can find values for `<image_path>` in the `version_manifest.json` that is part of each constellation release.
## GCP
In order to download an image you will have to export it to a bucket you have access to:
- "Owner" permissions on the project
- "Storage Admin" permissions on the bucket
- Export with:
```bash
gcloud compute images export --image=<image_path> --destination-uri=<bucket_uri> --export-format=qcow2 --project=<image_project>
```
- Click on "Download" on the created object
## Azure
To download an image from Azure you will have to create a disk from the image and generate a download link for that disk:
```bash
#!/bin/bash
VERSION=0.0.1
TARGET_DISK=export-${VERSION}
az disk create -g <resource_group> -l <target_region> -n $TARGET_DISK --hyper-v-generation V2 --os-type Linux --sku standard_lrs --security-type TrustedLaunch --gallery-image-reference <image_path>
az disk grant-access -n $TARGET_DISK -g constellation-images --access-level Read --duration-in-seconds 3600 | jq -r .accessSas
```
Depending on you internet connection you might have to modify the duration value.
The duration value specifies for how long the link is usable.

27
.github/docs/layout.md vendored Normal file
View File

@ -0,0 +1,27 @@
# Repository Layout
Core components:
* [access_manager](/access_manager): Contains the access-manager pod used to persist SSH users based on a K8s ConfigMap
* [cli](/cli): The CLI is used to manage a Constellation cluster
* [bootstrapper](/bootstrapper): The bootstrapper is a node agent whose most important task is to bootstrap a node
* [image](/image): Build files for the Constellation disk image
* [kms](/kms): Constellation's key management client and server
* [csi](/csi): Package used by CSI plugins to create and mount encrypted block devices
* [disk-mapper](/disk-mapper): Contains the disk-mapper that maps the encrypted node data disk during boot
Development components:
* [3rdparty](/3rdparty): Contains the third party dependencies used by Constellation
* [conformance](/conformance): Kubernetes conformance tests
* [debugd](/debugd): Debug daemon and client
* [hack](/hack): Development tools
* [proto](/proto): Proto files generator
* [terraform](/terraform): Infrastructure management using terraform (instead of `constellation create/destroy`)
* [libvirt](/terraform/libvirt): Deploy local cluster using terraform, libvirt and QEMU
Additional repositories:
* [constellation-fedora-coreos-config](https://github.com/edgelesssys/constellation-fedora-coreos-config): CoreOS build configuration with changes for Constellation
* [constellation-azuredisk-csi-driver](https://github.com/edgelesssys/constellation-azuredisk-csi-driver): Azure CSI driver with encryption on node
* [constellation-gcp-compute-persistent-disk-csi-driver](https://github.com/edgelesssys/constellation-gcp-compute-persistent-disk-csi-driver): GCP CSI driver with encryption on node

208
CONTRIBUTING.md
View File

@ -1,4 +1,4 @@
## First steps
# First steps
Thank you for getting involved! Before you start, please familiarize yourself with the [documentation](https://docs.edgeless.systems/constellation).
@ -12,202 +12,10 @@ If you want to support our development:
Constellation is licensed under the [AGPLv3](LICENSE). When contributing, you also need to agree to our [Contributor License Agreement](https://cla-assistant.io/edgelesssys/constellation).
## Development guidelines
Adhere to the style and best practices described in [Effective Go](https://golang.org/doc/effective_go.html). Read [Common Review Comments](https://github.com/golang/go/wiki/CodeReviewComments) for further information.
## Pull request process
Submissions should remain focused in scope and avoid containing unrelated commits.
For pull requests, we employ the following workflow:
1. Fork the repository to your own GitHub account
2. Create a branch locally with a descriptive name
3. Commit changes to the branch
4. Write your code according to our development guidelines
5. Push changes to your fork
6. Clean up your commit history
7. Open a PR in our repository and summarize the changes in the description
## Reporting issues and bugs, asking questions
This project uses the GitHub issue tracker. Please check the existing issues before submitting to avoid duplicates.
To report a security issue, contact security@edgeless.systems.
Your bug report should cover the following points:
* A quick summary and/or background of the issue
* Steps to reproduce (be specific, e.g., provide sample code)
* What you expected would happen
* What actually happens
* Further notes:
* Thoughts on possible causes
* Tested workarounds or fixes
## Major changes and feature requests
You should discuss larger changes and feature requests with the maintainers. Please open an issue describing your plans.
[Run CI e2e tests](/.github/docs/README.md)
## Repository Layout
Core components:
* [access_manager](access_manager): Contains the access-manager pod used to persist SSH users based on a K8s ConfigMap
* [cli](cli): The CLI is used to manage a Constellation cluster
* [bootstrapper](bootstrapper): The bootstrapper is a node agent whose most important task is to bootstrap a node
* [image](image): Build files for the Constellation disk image
* [kms](kms): Constellation's key management client and server
* [csi](csi): Package used by CSI plugins to create and mount encrypted block devices
* [disk-mapper](disk-mapper): Contains the disk-mapper that maps the encrypted node data disk during boot
Development components:
* [3rdparty](3rdparty): Contains the third party dependencies used by Constellation
* [conformance](conformance): Kubernetes conformance tests
* [debugd](debugd): Debug daemon and client
* [hack](hack): Development tools
* [proto](proto): Proto files generator
* [terraform](terraform): Infrastructure management using terraform (instead of `constellation create/destroy`)
* [libvirt](terraform/libvirt): Deploy local cluster using terraform, libvirt and QEMU
Additional repositories:
* [constellation-fedora-coreos-config](https://github.com/edgelesssys/constellation-fedora-coreos-config): CoreOS build configuration with changes for Constellation
* [constellation-azuredisk-csi-driver](https://github.com/edgelesssys/constellation-azuredisk-csi-driver): Azure CSI driver with encryption on node
* [constellation-gcp-compute-persistent-disk-csi-driver](https://github.com/edgelesssys/constellation-gcp-compute-persistent-disk-csi-driver): GCP CSI driver with encryption on node
## Build
Prerequisites:
* [Go 1.18](https://go.dev/doc/install)
* [Docker](https://docs.docker.com/engine/install/)
* Packages on Ubuntu:
```sh
sudo apt install build-essential cmake libssl-dev pkg-config libcryptsetup12 libcryptsetup-dev
```
* Packages on Fedora:
```sh
sudo dnf install @development-tools pkg-config cmake openssl-devel cryptsetup-libs cryptsetup-devel
```
```sh
mkdir build
cd build
cmake ..
make -j`nproc`
```
## Testing
You can run all integration and unitttests like this:
```sh
ctest -j `nproc`
```
### Debug Images
> :warning: These images are not safe to use in production environments. :warning:
As described in [debugd](/debugd/README.md), it is possible to use a CoreOS image targeted at dev environments. This image allows to upload any [bootstrapper](/bootstrapper/README.md) using [cdbg](/debugd/cdbg).
To enable the upload, an additional **unsecured** port (4000) is opened which accepts any binary to be run on target machine. **Make sure that this machine is not exposed to the internet.**
## Cloud credentials
Using the CLI requires the user to make authorized API calls to the CSP API. See the [docs](https://docs.edgeless.systems/constellation/getting-started/install#set-up-cloud-credentials) for configuration.
## Deploying a locally compiled bootstrapper binary
By default, `constellation create ...` will spawn cloud provider instances with a pre-baked bootstrapper binary.
For testing, you can use the constellation debug daemon (debugd) to upload your local bootstrapper binary to running instances and to obtain SSH access.
[Follow this introduction on how to install and setup `cdbg`](debugd/README.md)
## Development Guides
* [Upgrading Kubernetes](/docs/upgrade-kubernetes.md)
* [Manual local image testing](/docs/local-image-testing.md)
## Deployment Guides
* [Onboarding Customers](/docs/onboarding-customers.md)
## Linting
This projects uses [golangci-lint](https://golangci-lint.run/) for linting.
You can [install golangci-lint](https://golangci-lint.run/usage/install/#linux-and-windows) locally,
but there is also a CI action to ensure compliance.
To locally run all configured linters, execute
```sh
golangci-lint run ./...
```
It is also recommended to use golangci-lint (and [gofumpt](https://github.com/mvdan/gofumpt) as formatter) in your IDE, by adding the recommended VS Code Settings or by [configuring it yourself](https://golangci-lint.run/usage/integrations/#editor-integration)
## Nested Go modules
As this project contains nested Go modules, it is recommended to create a local Go workspace, so your IDE can lint multiple modules at once.
```go
go 1.18
use (
.
./hack
./operators/constellation-node-operator
)
```
You can find an introduction in the [Go workspace tutorial](https://go.dev/doc/tutorial/workspaces).
If you have changed dependencies within a module and have run `go mod tidy`, you can use `go work sync` to sync versions of the same dependency of the different modules.
## Recommended VS Code Settings
The following can be added to your personal `settings.json`, but it is recommended to add it to
the `<REPOSITORY>/.vscode/settings.json` repo, so the settings will only affect this repository.
```jsonc
// Use gofumpt as formatter.
"gopls": {
"formatting.gofumpt": true,
},
// Use golangci-lint as linter. Make sure you've installed it.
"go.lintTool":"golangci-lint",
"go.lintFlags": ["--fast"],
// You can easily show Go test coverage by running a package test.
"go.coverageOptions": "showUncoveredCodeOnly",
// Executing unit tests with race detection.
// You can add preferences like "-v" or "-count=1"
"go.testFlags": ["-race"],
// Enable language features for files with build tags.
// Attention! This leads to integration test being executed when
// running a package test within a package containing integration
// tests.
"go.buildTags": "integration",
```
## Naming convention
### Network
IP addresses:
* ip: numeric IP address
* host: either IP address or hostname
* endpoint: host+port
### Keys
* key: symmetric key
* pubKey: public key
* privKey: private key
# Developer docs
We have some documentation, guidelines and conventions that you can go through to familiarize yourself with the project and get started hacking on it.
* Building, testing, deploying: [local development](/.github/docs/development.md)
* [Code conventions](/.github/docs/conventions.md#code-conventions)
* [Process conventions](/.github/docs/conventions.md#process-conventions)
* [Repository layout](/.github/docs/layout.md#repository-layout)
* [Upgrading Kubernetes](/.github/docs/upgrade-kubernetes.md)

8
debugd/README.md
View File

@ -1,5 +1,12 @@
# debug daemon (debugd)
Debugd is a tool we built to allow for shorter iteration cycles during development.
The debugd gets embedded into coreOS images at the place where the bootstrapper normally sits.
Therefore, when a debug image is started, the debugd starts executing instead of the bootstrapper.
The debugd will then wait for a request from the `cdbg` tool to upload a bootstrapper binary.
Once the upload is finished debugd will start the bootstrapper.
Subsequently you can initialize your cluster with `constellation init` as usual.
## Build cdbg
```shell
@ -9,6 +16,7 @@ make cdbg
```
## debugd & cdbg usage
Before continuing, remeber to [set up](https://docs.edgeless.systems/constellation/getting-started/install#set-up-cloud-credentials) your cloud credentials for the CLI to work.
With `cdbg` and `yq` installed in your path:

3
terraform/libvirt/README.md
View File

@ -4,7 +4,7 @@
Prerequisite:
- [qcow2 constellation image](/image/)
- [qcow2 constellation image](/image/README.md)
- [setup](#setup-libvirt--terraform)
- [qemu-metadata-api](/hack/qemu-metadata-api/README.md)
@ -13,6 +13,7 @@ See [variables.tf](./variables.tf) for a description of all available variables.
```tfvars
constellation_coreos_image="/path/to/image.qcow2"
# optional other vars, uncomment and change as needed
# metadata_api_log_dir="path/to/folder/for/metadata"
# control_plane_count=3
# worker_count=2
# vcpus=2