Skip to content

Commit

Permalink
docs: add cluster autoscaler flavor
Browse files Browse the repository at this point in the history
  • Loading branch information
cbang-akamai committed Apr 18, 2024
1 parent 01b99bd commit 0f13cea
Show file tree
Hide file tree
Showing 3 changed files with 107 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- [Dual-stack (kubeadm)](./topics/flavors/dual-stack.md)
- [Etcd-disk (kubeadm)](./topics/flavors/etcd-disk.md)
- [ClusterClass kubeadm](./topics/flavors/clusterclass-kubeadm.md)
- [Cluster Autoscaler (kubeadm)](./topics/flavors/cluster-autoscaler.md)
- [k3s](./topics/flavors/k3s.md)
- [rke2](./topics/flavors/rke2.md)
- [Etcd](./topics/etcd.md)
Expand All @@ -19,6 +20,7 @@
- [OS Disk](./topics/disks/os-disk.md)
- [Data Disks](./topics/disks/data-disks.md)
- [Machine Health Checks](./topics/health-checking.md)
- [Autoscaling](./topics/autoscaling.md)
- [Development](./developers/development.md)
- [Releasing](./developers/releasing.md)
- [Reference](./reference/reference.md)
62 changes: 62 additions & 0 deletions docs/src/topics/autoscaling.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Auto-scaling

This guide covers auto-scaling for CAPL clusters. The recommended tool for auto-scaling on Cluster API is [Cluster
Autoscaler](https://www.github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#cluster-autoscaler).

## Flavor

The auto-scaling feature is provided by an add-on as part of the [Cluster Autoscaler
flavor](./flavors/cluster-autoscaler.md).

## Configuration

By default, the Cluster Autoscaler add-on [runs in the management cluster, managing an external workload
cluster](https://cluster-api.sigs.k8s.io/tasks/automated-machine-management/autoscaling#autoscaler-running-in-management-cluster-using-service-account-credentials-with-separate-workload-cluster).

```
+------------+ +----------+
| mgmt | | workload |
| ---------- | kubeconfig | |
| autoscaler +------------>| |
+------------+ +----------+
```

A separate Cluster Autoscaler is deployed for each workload cluster, configured to only monitor node groups for the
specific namespace and cluster name combination.

## Role-based Access Control (RBAC)

### Management Cluster

Due to constraints with the Kubernetes RBAC system (i.e. [roles cannot be subdivided beyond
namespace-granularity](https://www.github.com/kubernetes/kubernetes/issues/56582)), the Cluster Autoscaler add-on is
deployed on the management cluster to prevent leaking Cluster API data between workload clusters.

### Workload Cluster

Currently, the Cluster Autoscaler reuses the `${CLUSTER_NAME}-kubeconfig` Secret generated by the bootstrap provider to
interact with the workload cluster. The kubeconfig contents must be stored in a key named `value`. Due to this, all
Cluster Autoscaler actions in the workload cluster are performed as the `cluster-admin` role.

## Scale Down

> Cluster Autoscaler decreases the size of the cluster when some nodes are consistently unneeded for a significant
> amount of time. A node is unneeded when it has low utilization and all of its important pods can be moved elsewhere.
By default, Cluster Autoscaler scales down a node after it is marked as unneeded for 10 minutes. This can be adjusted
with the [`--scale-down-unneeded-time`
setting](https://www.github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-modify-cluster-autoscaler-reaction-time).

### Kubernetes Cloud Controller Manager for Linode (CCM)

The [Kubernetes Cloud Controller Manager for
Linode](https://www.github.com/linode/linode-cloud-controller-manager?tab=readme-ov-file#the-purpose-of-the-ccm) is
deployed on workload clusters and reconciles Kubernetes Node objects with their backing Linode infrastructure. When
scaling down a node group, the Cluster Autoscaler also deletes the Kubernetes Node object on the workload cluster. This
step preempts the Node-deletion in Kubernetes triggered by the CCM.

## Additional Resources

- [Autoscaling - The Cluster API Book](https://cluster-api.sigs.k8s.io/tasks/automated-machine-management/autoscaling)
- [Cluster Autoscaler
FAQ](https://www.github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#cluster-autoscaler)
43 changes: 43 additions & 0 deletions docs/src/topics/flavors/cluster-autoscaler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Cluster Autoscaler

This flavor adds auto-scaling via [Cluster
Autoscaler](https://www.github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#cluster-autoscaler).

## Specification

| Control Plane | CNI | Default OS | Installs ClusterClass | IPv4 | IPv6 |
|---------------|--------|--------------|-----------------------|------|------|
| Kubeadm | Cilium | Ubuntu 22.04 | No | Yes | No |

## Prerequisites

[Quickstart](../getting-started.md) completed

## Usage

1. Set up autoscaling environment variables
> We recommend using Cluster Autoscaler with the Kubernetes control plane
> ... version for which it was meant.
>
> -- <cite>[Releases · kubernetes/autoscaler](https://www.github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#releases)</cite>
```sh
export CLUSTER_AUTOSCALER_VERSION=v1.29.0
# Optional: If specified, these values must be explicitly quoted!
export WORKER_MACHINE_MIN='"1"'
export WORKER_MACHINE_MAX='"10"'
```

2. Generate cluster yaml

```sh
clusterctl generate cluster test-cluster \
--infrastructure linode:0.0.0 \
--flavor cluster-autoscaler > test-cluster.yaml
```

3. Apply cluster yaml

```sh
kubectl apply -f test-cluster.yaml
```

0 comments on commit 0f13cea

Please sign in to comment.