scrutiny docker container - /dev/sd*

[ericderace]

Hi,

I'd like to use the scrutiny docker container (https://github.com/AnalogJ/scrutiny) which is an app that has a nice GUI for HDD smart tests. It requires access to /dev/sd*.

As per the jlmkr documentations, /dev binds are not recommended. How would one approach this? Can we bind mount /dev/sd* devices as read-only, and would that cause any issues with the TrueNAS system?

Here is the docker configuration for scrutiny:
docker run -it --rm -p 8080:8080 -p 8086:8086 \ -vpwd/scrutiny:/opt/scrutiny/config \ -v pwd/influxdb2:/opt/scrutiny/influxdb \ -v /run/udev:/run/udev:ro \ --cap-add SYS_RAWIO \ --device=/dev/sda \ --device=/dev/sdb \ --name scrutiny \ ghcr.io/analogj/scrutiny:master-omnibus

[Jip-Hop]

I don't know the answer to this unfortunately. If you're worried then I recommend to experiment on a non-production system first to see if it works.

[ericderace]

Alright,

I added this to systemd_nspawn_user_args=
--bind-ro=/dev/sda --bind-ro=/dev/sdb --bind-ro=/dev/sdc --bind-ro=/dev/sdd --bind-ro=/dev/sde --bind-ro=/dev/sdf --bind-ro=/dev/sdg --bind-ro=/dev/sdh --bind-ro=/dev/sdi --bind-ro=/dev/sdj --bind-ro=/dev/sdk --bind-ro=/dev/sdl --bind-ro=/dev/sdm --bind-ro=/dev/sdn --bind-ro=/dev/sdo --bind-ro=/dev/sdp --bind-ro=/dev/sdq --bind-ro=/dev/sdr --bind-ro=/dev/sds --bind-ro=/dev/sdt --bind-ro=/dev/sdu --bind-ro=/dev/sdv --bind-ro=/dev/sdw --bind-ro=/dev/sdx --bind-ro=/dev/sdy --bind-ro=/dev/sdz

and re-started the jail. It works with the scrutiny docker container running within the jail, and I don't see any errors yet on truenas (dmesg).

Seems to be working fine, thanks.

[BenRoe]

startup=0
gpu_passthrough_intel=0
gpu_passthrough_nvidia=0

# Use macvlan networking to provide an isolated network namespace,
# so docker can manage firewall rules
# Alternatively use --network-bridge=br1 instead of --network-macvlan
# Ensure to change eno1/br1 to the interface name you want to use
# You may want to add additional options here, e.g. bind mounts
systemd_nspawn_user_args=--network-bridge=br0
        --resolv-conf=bind-host
        --system-call-filter='add_key keyctl bpf'
        --bind-ro=/dev/sda

# Only used while creating the jail
distro=debian
release=bookworm

# You generally will not need to change the options below
systemd_run_default_args=--property=KillMode=mixed
        --property=Type=notify
        --property=RestartForceExitStatus=133
        --property=SuccessExitStatus=133
        --property=Delegate=yes
        --property=TasksMax=infinity
        --collect
        --setenv=SYSTEMD_NSPAWN_LOCK=0

systemd_nspawn_default_args=--keep-unit
        --quiet
        --boot
        --bind-ro=/sys/module
        --inaccessible=/sys/module/apparmor

--bind=/dev/sda:/dev/sda also Read Device Identity failed: Permission denied

[Jip-Hop]

Please try again with docker_compatible=1. I know it sounds strange but this option does more than the name implies. I'm planning to replace this option in the future.

[BenRoe]

Thanks @Jip-Hop docker_compatible=1 solved the problem.

docker_compatible=1

systemd_nspawn_user_args=--network-bridge=br0
        --resolv-conf=bind-host
        --system-call-filter='add_key keyctl bpf'
        --bind-ro=/dev/sda

[BenRoe]

@Jip-Hop in v1.1.5 the docker_compatible option is deprecated. Under the hood it did something that i was able to --bind-ro=/dev/sda.
Without docker_compatible it did not work.
How to deal with that in v1.1.5?

[Jip-Hop]

Is it a possibility for you to test with v1.1.5 with just --bind-ro=/dev/sda and not using docker_compatible? I made some changes which I think may have helped. Otherwise you may need to add the required capabilities with --capability and/or add --system-call-filter= with the appropriate syscalls (or disable seccomp with the new seccomp config option). I don't know from the top of my head which combination you need. The docker_compatible made it work, but it was basically an "all limits off" mode which is not great as a general recommendation.

[mooglestiltzkin]

for scrutiny in regards to jailmaker the only issue i encountered was it was missing the capability required in order to deploy it.

there has been discussion about how to add the missing capability
https://forums.truenas.com/t/qnap-ts-877-truenas-journal/1646/382

when i tried it myself i couldn't start up docker so i rolled back without issue.

but other users had done it and managed to have some success.

i posted my scrutiny docker compose. it was what worked on qnap qts. that was the setting i used.

[Jip-Hop]

#62 (reply in thread)

[SeiyaGame]

I used the jailmaker docker template and modified it like this :

systemd_nspawn_user_args=--network-bridge=br0
        --resolv-conf=bind-host
        --system-call-filter='add_key keyctl bpf'
+       --capability=all
+       --bind-ro=/run/udev
+       --bind-ro=/dev/sda
+       --bind-ro=/dev/sdb

Scrutiny docker compose I used :

services:
  scrutiny:
    container_name: scrutiny
    image: ghcr.io/analogj/scrutiny:master-omnibus
    cap_add:
      - SYS_RAWIO
    ports:
      - "8080:8080" # webapp
      - "8086:8086" # influxDB admin
    volumes:
      - /run/udev:/run/udev:ro
      - ./config:/opt/scrutiny/config
      - ./influxdb:/opt/scrutiny/influxdb
    devices:
      - "/dev/sda"
      - "/dev/sdb"

With these settings, Scrutiny works perfectly, I hope this helps

[mooglestiltzkin]

+       --capability=all

yes it works if you do that, but i think we were told not to add capabilities all, because potentially bad things could happen? correct me if i am wrong.

[SeiyaGame]

If you don't want to add all the capacities, you can only add the ones you want, as described in the documentation

It goes something like this :

--capability='CAP_SYS_MODULE,CAP_SYS_RAWIO,CAP_SYS_ADMIN'

For Scrutiny, just add CAP_SYS_RAWIO,CAP_SYS_ADMIN (HDD/NVME):

--capability='CAP_SYS_RAWIO,CAP_SYS_ADMIN'

Edit : Forgot to add CAP_SYS_ADMIN for nvme

[vinicentus]

Yes, CAP_SYS_RAWIO is the only additional capability needed for scrutiny, as per https://github.com/AnalogJ/scrutiny/blob/master/README.md#docker (EDIT: unless using NVMe drives).
I didn't want to mount /run/udev on the truenas host to /run/udev in the jail, so i instead mounted it in a different directory.