feat: `dnf` module #377

fiftydinar · 2024-12-22T13:53:07Z

Changes

Recipe format

Recipe format changed a bit, to satisfy the new feature of dnf flags.

Existing features code changes

Adding repos is now using dnf -y config-manager addrepo --from-repofile=$repo for repo URLs & repo files, while COPR repos are using native dnf -y copr enable user/project.
COPR repos are written in separate copr: array & in simpler user/project format in recipe, instead of long URL in repos: array.
rpm-ostree override remove --install=$pkg is replaced with 2 operations of dnf removal & then install.
Single operation of removal + installation of dnf doesn't currently exist, but it's not a breaking behavior to not have it.
rpm-ostree override replace is replaced with dnf -y distro-sync --refresh --repo $repo $packages, which is compatible with all repos.
Adding the repository along with replacement is disabled, so it must be done before replacement in repos: array if repo doesn't exist.
Fixed the bug with symlinking /opt/ if it's already symlinked in both dnf & rpm-ostree module (thanks to @lorduskordus for the fix)

New features

dnf group removal & installation. Those are declared as group-remove & group-install array in recipe.
They run before the packages installation.
Option for additional flags when doing installation or replacement of packages.
- install-weak-dependencies: option for declaring if weak dependencies (such as Recommended) should be installed.
  It modifies the install & replace commands itself using --setopt=install_weak_deps=True/False flag, it doesn't modify the dnf config file. It defaults to true (reflecting dnf defaults).
- skip-unavailable-packages: option for skipping unavailable packages when not available in repositories, or when not available on the system in case of replacing. Passes --skip-unavailable flag. Defaults to false.
- skip-broken-packages: option for skipping the installation/replacement of broken packages. Passes --skip-broken flag. Defaults to false.
- allow-erasing-packages: option for erasing/removing the problematic packages during dependency problems. Passes --allowerasing flag. Defaults to false.
Option for additional flags when doing removal of packages.
- remove-unused-dependencies is true by default. If false is specified, it passes the --no-autoremove flag, which won't remove any unused dependencies during removal operation.

Issues to be closed

Fixes: #361
Fixes: #370
Fixes: #365
Fixes: #325
Fixes: #282

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

shellcheck

⚠️ [shellcheck] _{reported by reviewdog 🐶}
Did you forget to close this double quoted string? SC1078

modules/modules/dnf/dnf.sh

Line 91 in cd91b05

CLASSIC_PKGS+=("${PKG}")

📝 [shellcheck] _{reported by reviewdog 🐶}
This is actually an end quote, but due to next char it looks suspect. SC1079

modules/modules/dnf/dnf.sh

Line 98 in cd91b05

echo "Installing: ${CLASSIC_PKGS[*]}"

🚫 [shellcheck] _{reported by reviewdog 🐶}
'(' is invalid here. Did you forget to escape it? SC1036

modules/modules/dnf/dnf.sh

Line 100 in cd91b05

echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"

🚫 [shellcheck] _{reported by reviewdog 🐶}
Expected 'fi' matching previously mentioned 'if'. SC1047

modules/modules/dnf/dnf.sh

Line 100 in cd91b05

echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"

🚫 [shellcheck] _{reported by reviewdog 🐶}
Expected 'fi'. Fix any mentioned problems and try again. SC1072

modules/modules/dnf/dnf.sh

Line 100 in cd91b05

echo "Installing package(s) directly from URL: ${HTTPS_PKGS[*]}"

modules/dnf/module.yml

github-actions · 2024-12-22T21:04:11Z

modules/rpm-ostree/rpm-ostree.sh

📝 [shellcheck] _{reported by reviewdog 🐶}
Possible misspelling: REPOS may not be assigned. Did you mean REPO? SC2153

modules/modules/rpm-ostree/rpm-ostree.sh

Line 8 in 1137c8b

if [[ ${#REPOS[@]} -gt 0 ]]; then

github-actions · 2024-12-22T21:04:12Z

modules/rpm-ostree/rpm-ostree.sh

[shellcheck] _{reported by reviewdog 🐶}
$/${} is unnecessary on arithmetic variables. SC2004

modules/modules/rpm-ostree/rpm-ostree.sh

Line 72 in 1137c8b

INSTALL_PKGS[$i]="${PKG//%OS_VERSION%/${OS_VERSION}}"

github-actions · 2024-12-22T21:04:13Z

modules/rpm-ostree/rpm-ostree.sh

⚠️ [shellcheck] _{reported by reviewdog 🐶}
Quote this to prevent word splitting. SC2046

modules/modules/rpm-ostree/rpm-ostree.sh

Line 115 in 1137c8b

rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")

github-actions · 2024-12-22T21:04:13Z

modules/rpm-ostree/rpm-ostree.sh

⚠️ [shellcheck] _{reported by reviewdog 🐶}
Quote this to prevent word splitting. SC2046

modules/modules/rpm-ostree/rpm-ostree.sh

Line 117 in 1137c8b

rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")

github-actions · 2024-12-22T21:04:14Z

modules/rpm-ostree/rpm-ostree.sh

⚠️ [shellcheck] _{reported by reviewdog 🐶}
Quote this to prevent word splitting. SC2046

modules/modules/rpm-ostree/rpm-ostree.sh

Line 120 in 1137c8b

rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")

github-actions · 2024-12-22T21:04:15Z

modules/rpm-ostree/rpm-ostree.sh

⚠️ [shellcheck] _{reported by reviewdog 🐶}
Quote this to prevent word splitting. SC2046

modules/modules/rpm-ostree/rpm-ostree.sh

Line 123 in 1137c8b

rpm-ostree override remove "${REMOVE_PKGS[@]}" $(printf -- "--install=%s " "${CLASSIC_PKGS[@]}")

github-actions · 2024-12-22T21:04:16Z

modules/rpm-ostree/rpm-ostree.sh

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

modules/modules/rpm-ostree/rpm-ostree.sh

Line 185 in 1137c8b

rpm-ostree override replace --experimental --from "repo=copr:copr.fedorainfracloud.org:${MAINTAINER}:${REPO_NAME}" ${REPLACE_STR}

…d `--refresh` for install operation

modules/dnf/dnf.sh

Co-authored-by: xyny <[email protected]>

modules/dnf/dnf.tsp

Co-authored-by: Gerald Pinder <[email protected]>

modules/dnf/module.yml

Co-authored-by: Gerald Pinder <[email protected]>

modules/dnf/dnf.sh

github-actions · 2024-12-23T21:39:16Z

modules/dnf/dnf.sh

+  for i in "${!REPOS[@]}"; do
+      repo="${REPOS[$i]}"
+      repo="${repo//%OS_VERSION%/${OS_VERSION}}"
+      REPOS[$i]="${repo//[$'\t\r\n ']}"


[shellcheck] _{reported by reviewdog 🐶}
$/${} is unnecessary on arithmetic variables. SC2004

modules/dnf/dnf.sh

Co-authored-by: Gerald Pinder <[email protected]>

gmpinder

I'm getting an error when running a basic install:

l/jp-desktop-nvidia:latest => #25 0.447 ============================== Start 'dnf' Module ==============================
l/jp-desktop-nvidia:latest => #25 0.531 Installing RPMs
l/jp-desktop-nvidia:latest => #25 0.531 Installing: @kde-desktop
l/jp-desktop-nvidia:latest => #25 0.578 Updating and loading repositories:
l/jp-desktop-nvidia:latest => #25 0.800  Fedora 41 openh264 (From Cisco) - x86_ 100% |   6.5 KiB/s | 989.0   B |  00m00s
l/jp-desktop-nvidia:latest => #25 1.161  Fedora 41 - x86_64                     100% |  85.6 KiB/s |  30.8 KiB |  00m00s
l/jp-desktop-nvidia:latest => #25 1.256  Fedora 41 - x86_64 - Updates Archive   100% |  36.2 KiB/s |   3.4 KiB |  00m00s
l/jp-desktop-nvidia:latest => #25 1.416  Fedora 41 - x86_64 - Updates           100% | 184.9 KiB/s |  29.4 KiB |  00m00s
l/jp-desktop-nvidia:latest => #25 1.483 Repositories loaded.
l/jp-desktop-nvidia:latest => #25 1.952 Failed to resolve the transaction:
l/jp-desktop-nvidia:latest => #25 1.952 No match for argument:
l/jp-desktop-nvidia:latest => #25 1.952 No match for argument:
l/jp-desktop-nvidia:latest => #25 1.952 No match for argument:
l/jp-desktop-nvidia:latest => #25 1.952 Package "toolbox-0.1.1-1.fc41.x86_64" is already installed.
l/jp-desktop-nvidia:latest => #25 1.952 Package "udisks2-2.10.1-6.fc41.x86_64" is already installed.
l/jp-desktop-nvidia:latest => #25 1.952 You can try to add to command line:
l/jp-desktop-nvidia:latest => #25 1.952   --skip-unavailable to skip unavailable packages
l/jp-desktop-nvidia:latest => #25 1.963 ============================= Failed 'dnf' Module =============================

This indicates to me that it's seeing the empty args as packages. This might be a case where we actually don't want to encapsulate the args in ".

modules/dnf/dnf.sh

gmpinder · 2024-12-24T03:32:56Z

I'm getting an error when running a basic install:

l/jp-desktop-nvidia:latest => #25 0.447 ============================== Start 'dnf' Module ==============================
l/jp-desktop-nvidia:latest => #25 0.531 Installing RPMs
l/jp-desktop-nvidia:latest => #25 0.531 Installing: @kde-desktop
l/jp-desktop-nvidia:latest => #25 0.578 Updating and loading repositories:
l/jp-desktop-nvidia:latest => #25 0.800  Fedora 41 openh264 (From Cisco) - x86_ 100% |   6.5 KiB/s | 989.0   B |  00m00s
l/jp-desktop-nvidia:latest => #25 1.161  Fedora 41 - x86_64                     100% |  85.6 KiB/s |  30.8 KiB |  00m00s
l/jp-desktop-nvidia:latest => #25 1.256  Fedora 41 - x86_64 - Updates Archive   100% |  36.2 KiB/s |   3.4 KiB |  00m00s
l/jp-desktop-nvidia:latest => #25 1.416  Fedora 41 - x86_64 - Updates           100% | 184.9 KiB/s |  29.4 KiB |  00m00s
l/jp-desktop-nvidia:latest => #25 1.483 Repositories loaded.
l/jp-desktop-nvidia:latest => #25 1.952 Failed to resolve the transaction:
l/jp-desktop-nvidia:latest => #25 1.952 No match for argument:
l/jp-desktop-nvidia:latest => #25 1.952 No match for argument:
l/jp-desktop-nvidia:latest => #25 1.952 No match for argument:
l/jp-desktop-nvidia:latest => #25 1.952 Package "toolbox-0.1.1-1.fc41.x86_64" is already installed.
l/jp-desktop-nvidia:latest => #25 1.952 Package "udisks2-2.10.1-6.fc41.x86_64" is already installed.
l/jp-desktop-nvidia:latest => #25 1.952 You can try to add to command line:
l/jp-desktop-nvidia:latest => #25 1.952   --skip-unavailable to skip unavailable packages
l/jp-desktop-nvidia:latest => #25 1.963 ============================= Failed 'dnf' Module =============================

This indicates to me that it's seeing the empty args as packages. This might be a case where we actually don't want to encapsulate the args in ".

Ok just tested my suggested changes locally. They do fix the problem I ran into.

… part 1 Co-authored-by: Gerald Pinder <[email protected]>

… part 2 Co-authored-by: Gerald Pinder <[email protected]>

… part 3 Co-authored-by: Gerald Pinder <[email protected]>

… part 4 Co-authored-by: Gerald Pinder <[email protected]>

… part 5 Co-authored-by: Gerald Pinder <[email protected]>

… part 6 Co-authored-by: Gerald Pinder <[email protected]>

github-actions · 2024-12-25T14:26:50Z

modules/dnf/dnf.sh

+    echo "Removing: ${GROUP_REMOVE[*]}"
+    echo "Installing: ${GROUP_INSTALL[*]}"
+    dnf -y group remove "${GROUP_REMOVE[@]}"
+    dnf -y ${WEAK_DEPS_FLAG} group install --refresh ${SKIP_UNAVAILABLE_FLAG} ${SKIP_BROKEN_FLAG} ${ALLOW_ERASING_FLAG} "${GROUP_INSTALL[@]}"
