s3proxy: add new page to documentation (v2.12)

This commit is contained in:
Malte Poll 2023-10-10 18:11:30 +02:00 committed by Malte Poll
parent 07249b1288
commit c4a3e40882
7 changed files with 146 additions and 3 deletions

View File

@ -28,7 +28,7 @@ You can add the s3proxy to your Constellation cluster as follows:
kubectl apply -f deployment-s3proxy.yaml
```
If you want to run a demo application, check out the [Filestash with s3proxy](../getting-started/examples/filstash-s3proxy.md) example.
If you want to run a demo application, check out the [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example.
## Technical details
@ -54,4 +54,4 @@ The necessary deployment modifications are to add DNS redirection and a trusted
DNS redirection can be defined for each pod, allowing you to use s3proxy for one application without changing other applications in the same cluster.
Adding a trusted TLS certificate is necessary as clients communicate with s3proxy via HTTPS.
To have your client application trust s3proxy's TLS certificate, the certificate has to be added to the client's certificate trust store.
The [Filestash with s3proxy](../getting-started/examples/filstash-s3proxy.md) example shows how to do this.
The [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example shows how to do this.

View File

@ -122,7 +122,7 @@ const sidebars = {
{
type: 'doc',
label: 'Filestash with s3proxy',
id: 'getting-started/examples/filstash-s3proxy'
id: 'getting-started/examples/filestash-s3proxy'
},
]
},

View File

@ -55,3 +55,8 @@ For encryption Constellation uses AES in XTS-Plain64. The key size is 512 bit.
To interact with the dm-integrity kernel module, Constellation uses [libcryptsetup](https://gitlab.com/cryptsetup/cryptsetup/).
When enabled, the used data integrity algorithm is [HMAC](https://datatracker.ietf.org/doc/html/rfc2104) with SHA256 as the hash function.
The tag size is 32 Bytes.
## Encrypted S3 object storage
Constellation comes with a service that you can use to transparently retrofit client-side encryption to existing applications that use S3 (AWS or compatible) for storage.
To learn more, check out the [s3proxy documentation](../workflows/s3proxy.md).

View File

@ -0,0 +1,71 @@
# Deploying Filestash
Filestash is a web frontend for different storage backends, including S3.
It's a useful application to showcase s3proxy in action.
1. Deploy s3proxy as described in [Deployment](../../workflows/s3proxy.md#deployment).
2. Create a deployment file for Filestash with one pod:
```sh
cat << EOF > "deployment-filestash.yaml"
apiVersion: apps/v1
kind: Deployment
metadata:
name: filestash
spec:
replicas: 1
selector:
matchLabels:
app: filestash
template:
metadata:
labels:
app: filestash
spec:
imagePullSecrets:
- name: regcred
hostAliases:
- ip: $(kubectl get svc s3proxy-service -o=jsonpath='{.spec.clusterIP}')
hostnames:
- "s3.eu-west-1.amazonaws.com"
containers:
- name: filestash
image: machines/filestash:latest
ports:
- containerPort: 8334
volumeMounts:
- name: ca-cert
mountPath: /etc/ssl/certs/kube-ca.crt
subPath: kube-ca.crt
volumes:
- name: ca-cert
secret:
secretName: s3proxy-tls
items:
- key: ca.crt
path: kube-ca.crt
EOF
```
The pod spec includes the `hostAliases` key, which adds an entry to the pod's `/etc/hosts`.
The entry forwards all requests for `s3.eu-west-1.amazonaws.com` to the Kubernetes service `s3proxy-service`.
If you followed the s3proxy [Deployment](../../workflows/s3proxy.md#deployment) guide, this service points to a s3proxy pod.
To use other regions than `eu-west-1`, add more entries to `hostAliases` for all regions you require.
Use the same IP for those entries. For example to add `us-east-1` add:
```yaml
- ip: $(kubectl get svc s3proxy-service -o=jsonpath='{.spec.clusterIP}')
hostnames:
- "s3.us-east-1.amazonaws.com"
```
The spec also includes a volume mount for the TLS certificate and adds it to the pod's certificate trust store.
The volume is called `ca-cert`.
The key `ca.crt` of that volume is mounted to `/etc/ssl/certs/kube-ca.crt`, which is the default certificate trust store location for that container's OpenSSL library.
Not adding the CA certificate will result in TLS authentication errors.
3. Apply the file: `kubectl apply -f deployment-filestash.yaml`
Afterward, you can use a port forward to access the Filestash pod:
`kubectl port-forward pod/$(kubectl get pod --selector='app=filestash' -o=jsonpath='{.items[*].metadata.name}') 8334:8334`

View File

@ -0,0 +1,57 @@
# Install s3proxy
Constellation includes a transparent client-side encryption proxy for [AWS S3](https://aws.amazon.com/de/s3/) and compatible stores.
s3proxy encrypts objects before sending them to S3 and automatically decrypts them on retrieval, without requiring changes to your application.
With s3proxy, you can use S3 for storage in a confidential way without having to trust the storage provider.
## Limitations
Currently, s3proxy has the following limitations:
- Only `PutObject` and `GetObject` requests are encrypted/decrypted by s3proxy.
By default, s3proxy will block requests that may expose unencrypted data to S3 (e.g. UploadPart).
The `allow-multipart` flag disables request blocking for evaluation purposes.
- Using the [Range](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestSyntax) header on `GetObject` is currently not supported and will result in an error.
These limitations will be removed with future iterations of s3proxy.
If you want to use s3proxy but these limitations stop you from doing so, consider [opening an issue](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&projects=&template=feature_request.yml).
## Deployment
You can add the s3proxy to your Constellation cluster as follows:
1. Download the deployment manifest:
```bash
wget https://raw.githubusercontent.com/edgelesssys/constellation/main/s3proxy/deploy/deployment-s3proxy.yaml
```
2. Replace the values named `replaceme` in `deployment-s3proxy.yaml` with valid AWS credentials. These credentials are used by s3proxy to access your S3 buckets.
3. Deploy s3proxy:
```bash
kubectl apply -f deployment-s3proxy.yaml
```
If you want to run a demo application, check out the [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example.
## Technical details
### Encryption
s3proxy relies on Google's [Tink Cryptographic Library](https://developers.google.com/tink) to implement cryptographic operations securely.
The used cryptographic primitives are [NIST SP 800 38f](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-38F.pdf) for key wrapping and [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)-[GCM](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Galois/counter_(GCM)) with 256 bit keys for data encryption.
s3proxy uses [envelope encryption](https://cloud.google.com/kms/docs/envelope-encryption) to encrypt objects.
This means s3proxy uses a key encryption key (KEK) issued by the [KeyService](../architecture/microservices.md#keyservice) to encrypt data encryption keys (DEKs).
Each S3 object is encrypted with its own DEK.
The encrypted DEK is then saved as metadata of the encrypted object.
This enables key rotation of the KEK without re-encrypting the data in S3.
The approach also allows access to objects from different locations, as long as each location has access to the KEK.
### Traffic interception
To use s3proxy, you have to redirect your outbound S3 traffic to s3proxy.
This can either be done by modifying your client application or by changing the deployment of your application.
The necessary deployment modifications are to add DNS redirection and a trusted TLS certificate to the client's trust store.
DNS redirection can be defined for each pod, allowing you to use s3proxy for one application without changing other applications in the same cluster.
Adding a trusted TLS certificate is necessary as clients communicate with s3proxy via HTTPS.
To have your client application trust s3proxy's TLS certificate, the certificate has to be added to the client's certificate trust store.
The [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example shows how to do this.

View File

@ -103,6 +103,11 @@
"type": "doc",
"label": "Horizontal Pod Autoscaling",
"id": "getting-started/examples/horizontal-scaling"
},
{
"type": "doc",
"label": "Filestash with s3proxy",
"id": "getting-started/examples/filestash-s3proxy"
}
]
}
@ -150,6 +155,11 @@
"label": "Install cert-manager",
"id": "workflows/cert-manager"
},
{
"type": "doc",
"label": "Install s3proxy",
"id": "workflows/s3proxy"
},
{
"type": "doc",
"label": "Terminate your cluster",