Skip to content

Latest commit

 

History

History
43 lines (36 loc) · 3.39 KB

README.md

File metadata and controls

43 lines (36 loc) · 3.39 KB
Raw

License: GPL v3 Kubernetes Docker dependabot auto-merging

homelab

This is a mono repository for my homelab infrastructure. I keep it production-ready within reason for a one-man hobby operation.

Network Diagram

This diagram was generated using the mingrammer diagrams lib for Python from the code in the Jupyter Notebook at diagrams/homelab.ipynb

homelab

Cluster Hardware

  • The configuration I use to deploy Talos Kubernetes Linux onto my home lab is in cluster/bootstrap/talos/
  • My cluster contains 3 control plane nodes a total of 28 CPUs, 64 GB of RAM, and 4GB of VRAM
  • Talos Linux Kubernetes cluster with 3 nodes called bigrig, tiffrig and oppenheimer
  • bigrig is my old gaming machine and the only machine with an Nvidia GPU
  • tiffrig is nearly identical to my bigrig. I built to be my wife's gaming machine originally, but is using an older Nvidia card that is no longer supported by CUDA
  • oppenheimer is an Intel NUC (I'm choosing to pronounce it "nuke"). They're great cheap low power servers with upgradeable RAM and SSD. I plan to buy more NUCs and finish the Manhattan Project team.

Cluster bootstrapping

  • The cluster/bootstrap/talos/install.sh contains commented code blocks to generate secrets and config, apply patches (see cluster/bootstrap/talos/install-patches) and deploy nodes
  • Talos system extension images are generated using https://factory.talos.dev and installed using the upgrade commands in cluster/bootstrap/talos/extensions/extensions.sh
  • Once system images have been installed, the install.sh script in each subfolder contains additional patches and upgrade commands specific to each extension.

GitOps with FluxCD

  • You can enable and disable apps by ignoring their project folders the .sourceignore file
  • Create a cluster/bootstrap/fluxcd/fluxcd.env file and define GITHUB_REPO, GITHUB_USER and GITHUB_TOKEN
  • Run the cluster/bootstrap/fluxcd/install.sh
  • This will also install Sealed Secrets and Weave Flux UI
  • Use the scripts scripts/generate-selfsigned.sh will generate a new selfsigned cert and key, create a kubernetes secret and encrypt it with sealed secrets. You can remove the last line of the script if you want to store the certs somewhere before removing.
  • If there is an install.sh file in the root of the project folder, run it. It will generate necessary secrets before deployment
  • Check the status of apps or the watch-projects kustomization in the Weave UI or with flux get -n flux-system kustomization watch-projects or flux get -n <namespace> helmrelease <appname>

Cluster Addons

  • Storage
    • Longhorn in cluster/addons/longhorn
    • Rook Ceph Operator in cluster/addons/rook-ceph
    • NFS in cluster/addons/nfs
    • External Snapshotter cluster/addons/external-snapshotter
  • Ingress
    • Traefik in cluster/addons/traefik
  • GPU
    • Nvidia K8s Device Plugin in cluster/addons/nvidia-device-plugin
    • Nvidia GPU Operator in cluster/addons/nvidia-gpu-operator