operators: infrastructure autodiscovery (#1958)

* helm: configure GCP cloud controller manager to search in all zones of a region

See also: d716fdd452/providers/gce/gce.go (L376-L380)

* operators: add nodeGroupName to ScalingGroup CRD

NodeGroupName is the human friendly name of the node group that will be exposed to customers via the Constellation config in the future.

* operators: support simple executor / scheduler to reconcile on non-k8s resources

* operators: add new return type for ListScalingGroups to support arbitrary node groups

* operators: ListScalingGroups should return additionally created node groups on AWS

* operators: ListScalingGroups should return additionally created node groups on Azure

* operators: ListScalingGroups should return additionally created node groups on GCP

* operators: ListScalingGroups should return additionally created node groups on unsupported CSPs

* operators: implement external scaling group reconciler

This controller scans the cloud provider infrastructure and changes k8s resources accordingly.
It creates ScaleSet resources when new node groups are created and deletes them if the node groups are removed.

* operators: no longer create scale sets when the operator starts

In the future, scale sets are created dynamically.

* operators: watch for node join/leave events using a controller

* operators: deploy new controllers

* docs: update auto scaling documentation with support for node groups
This commit is contained in:
Malte Poll 2023-07-05 07:27:34 +02:00 committed by Adrian Stobbe
parent 10a540c290
commit 388ff011a3
36 changed files with 1836 additions and 232 deletions

View file

@ -7,6 +7,8 @@ SPDX-License-Identifier: AGPL-3.0-only
package v1alpha1
import (
"strings"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
@ -14,6 +16,8 @@ const (
// ConditionOutdated is used to signal outdated scaling groups.
ConditionOutdated = "Outdated"
// UnknownRole is used to signal unknown scaling group roles.
UnknownRole NodeRole = ""
// WorkerRole is used to signal worker scaling groups.
WorkerRole NodeRole = "Worker"
// ControlPlaneRole is used to signal control plane scaling groups.
@ -28,6 +32,8 @@ type ScalingGroupSpec struct {
GroupID string `json:"groupId,omitempty"`
// AutoscalerGroupName is name that is expected by the autoscaler.
AutoscalerGroupName string `json:"autoscalerGroupName,omitempty"`
// NodeGroupName is the human friendly name of the node group as defined in the Constellation configuration.
NodeGroupName string `json:"nodeGroupName,omitempty"`
// Autoscaling specifies wether the scaling group should automatically scale using the cluster-autoscaler.
Autoscaling bool `json:"autoscaling,omitempty"`
// Min is the minimum number of nodes in the scaling group (used by cluster-autoscaler).
@ -42,6 +48,18 @@ type ScalingGroupSpec struct {
// +kubebuilder:validation:Enum=Worker;ControlPlane
type NodeRole string
// NodeRoleFromString returns the NodeRole for the given string.
func NodeRoleFromString(s string) NodeRole {
switch strings.ToLower(s) {
case "controlplane", "control-plane":
return ControlPlaneRole
case "worker":
return WorkerRole
default:
return UnknownRole
}
}
// ScalingGroupStatus defines the observed state of ScalingGroup.
type ScalingGroupStatus struct {
// ImageReference is the image currently used for newly created nodes in this scaling group.