feat(mergerfs): Add extension

[samip5]

This will fix #396

[frezbo]

missing documentation, would need some docs on how to test this, so that tests maybe added in talos side

[smira]

And I think we should build from source (it shouldn't be crazy given that it's C++).

[samip5]

I'm also little confused as to why it would try to create /var/mnt, instead of bind mounting it..

ID       ext-mergerfs
STATE    Failed
HEALTH   ?
EVENTS   [Failed]: Failed to create runner: mkdir /var/mnt: read-only file system (3m45s ago)
         [Preparing]: Creating service runner (3m45s ago)
         [Preparing]: Running pre state (3m46s ago)
         [Waiting]: Waiting for extension service config (4m11s ago)
         [Waiting]: Waiting for service "containerd" to be "up", extension service config (4m13s ago)
         [Waiting]: Waiting for service "containerd" to be registered, extension service config (4m16s ago)
         [Waiting]: Waiting for service "containerd" to be "up", extension service config (4m18s ago)
         [Starting]: Starting service (4m18s ago)

apiVersion: v1alpha1
kind: ExtensionServiceConfig
name: mergerfs
configFiles:
  - content: |-
      [mount]
      path = /var/mnt/merged
      fstype = fuse.mergerfs
      options = defaults,allow_other,use_ino,nonempty,minfreespace=50G,category.create=mfs,moveonenospc=true,fsname=mergerFS
      [paths]
      source = /var/mnt/disk*
    mountPath: /usr/local/etc/mergerfs/config.ini

[frezbo]

I'm also little confused as to why it would try to create /var/mnt, instead of bind mounting it..

ID       ext-mergerfs
STATE    Failed
HEALTH   ?
EVENTS   [Failed]: Failed to create runner: mkdir /var/mnt: read-only file system (3m45s ago)
         [Preparing]: Creating service runner (3m45s ago)
         [Preparing]: Running pre state (3m46s ago)
         [Waiting]: Waiting for extension service config (4m11s ago)
         [Waiting]: Waiting for service "containerd" to be "up", extension service config (4m13s ago)
         [Waiting]: Waiting for service "containerd" to be registered, extension service config (4m16s ago)
         [Waiting]: Waiting for service "containerd" to be "up", extension service config (4m18s ago)
         [Starting]: Starting service (4m18s ago)

apiVersion: v1alpha1
kind: ExtensionServiceConfig
name: mergerfs
configFiles:
  - content: |-
      [mount]
      path = /var/mnt/merged
      fstype = fuse.mergerfs
      options = defaults,allow_other,use_ino,nonempty,minfreespace=50G,category.create=mfs,moveonenospc=true,fsname=mergerFS
      [paths]
      source = /var/mnt/disk*
    mountPath: /usr/local/etc/mergerfs/config.ini

you'd need a dependency on service: cri to wait for /var to be mounted

[samip5]

I'm also little confused as to why it would try to create /var/mnt, instead of bind mounting it..

ID ext-mergerfs

STATE Failed

HEALTH ?

EVENTS [Failed]: Failed to create runner: mkdir /var/mnt: read-only file system (3m45s ago)

     [Preparing]: Creating service runner (3m45s ago)

     [Preparing]: Running pre state (3m46s ago)

     [Waiting]: Waiting for extension service config (4m11s ago)

     [Waiting]: Waiting for service "containerd" to be "up", extension service config (4m13s ago)

     [Waiting]: Waiting for service "containerd" to be registered, extension service config (4m16s ago)

     [Waiting]: Waiting for service "containerd" to be "up", extension service config (4m18s ago)

     [Starting]: Starting service (4m18s ago)

apiVersion: v1alpha1

kind: ExtensionServiceConfig

name: mergerfs

configFiles:

• content: |-

  [mount]

  path = /var/mnt/merged

  fstype = fuse.mergerfs

  options = defaults,allow_other,use_ino,nonempty,minfreespace=50G,category.create=mfs,moveonenospc=true,fsname=mergerFS

  [paths]

  source = /var/mnt/disk*

mountPath: /usr/local/etc/mergerfs/config.ini

you'd need a dependency on service: cri to wait for /var to be mounted

That results in other problems as cri will not come up if all user disks are not able to be mounted because they're eg non xfs. The whole point of mergerfs is to use disks that are separate and potentially different file systems.

Related: siderolabs/talos#9023

[samip5]

Closed due to better wait for siderolabs/talos#8367 and this requires a lot of the filesystem tools to exist on rootfs which they currently don't.

Like ext* related utilities, and btrfs to name a few.

Also related to: siderolabs/talos#9026