--data-dir doesn't change kubelet location

[mehmooda]

Environmental Info:
k3s version v1.21.3+k3s1 (1d1f220)
go version go1.16.6

Node(s) CPU architecture, OS, and Version:
Linux west 5.13.7-200.fc34.x86_64 #1 SMP Sat Jul 31 14:10:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration:
1 node test cluster at the moment

Describe the bug:
Using --data-dir only moves /var/lib/rancher/k3s it doesn't move /var/lib/kubelet. This means that emptyDir still uses the root filesystem for its backing data.

Steps To Reproduce:

• Installed K3s: with INSTALL_K3S_SKIP_START=true
• Edit systemd service and add --data-dir=/mnt/...
• systemctl daemon-reload; systemctl start k3s
• /var/lib/kubelet is created and is used for emptyDir volumes

Expected behavior:
/var/lib/kubelet should be a directory under data-dir

Actual behavior:
data-dir is ignored for kubelet

[brandond]

This is by design. Many components (CNI/CSI plugins, daemonsets, etc) only function properly if the default kubelet path is used. Previous releases of k3s nested /var/lib/kubelet within the k3 data-dir, but this caused so many issues that we had to move it back to the default.

[ChristianCiach]

I was surprised by this, too. Please see this and following comments for how to move the kubelet root directory and further information about this topic: #1358 (comment)

As mentioned by @brandond, this may have surprising side effects. Use at your own risk!