Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve management of CAPI images #978

Merged
merged 9 commits into from
Aug 12, 2024

Conversation

martinmo
Copy link
Contributor

@martinmo martinmo commented Aug 7, 2024

Hi, as part of SovereignCloudStack/standards#643, I want to make it as easy as possible to get the latest CAPI images in an SCS compliant way and found osism manage image clusterapi to be the best way to do it, with the following adjustments.

(Potentially) breaking changes:

  • Changed the absolute invocation of openstack-image-manager, to make it work if installed in other places. So now it depends on PATH. I think this shouldn't hurt. (Actually, it will even improve things in case someone installs stuff into a virtualenv, for example.)
  • Changed image name Kubernetes CAPI to ubuntu-capi-image as recommended by SCS.

Otherwise it just fixes bugs (hard-coded --cloud) and adds new features (--dry-run, --tag) in a backwards compatible way.

Resulting help output:

$ osism manage image clusterapi --help                                      
[... some unrelated TripleDES deprecation warnings ...]
usage: osism manage image clusterapi [-h] [--base-url BASE_URL] [--cloud CLOUD] [--dry-run] [--tag TAG] [--filter FILTER]

options:
  -h, --help            show this help message and exit
  --base-url BASE_URL
                        Base URL
  --cloud CLOUD
                        Cloud name in clouds.yaml (will be overruled by OS_AUTH_URL envvar)
  --dry-run             Do not perform any changes (--dry-run passed to openstack-image-manager)
  --tag TAG     Name of the tag used to identify managed images (use openstack-image-manager's default if unset)
  --filter FILTER
                        Filter the version to be managed (e.g. 1.28)

Example invocation:

$ osism manage image clusterapi --dry-run --cloud l1a --tag 'managed_by_C&H'
[... some unrelated TripleDES deprecation warnings ...]
2024-08-07 12:34:06 | INFO     | date: 2024-07-18
2024-08-07 12:34:06 | INFO     | image: ubuntu-2204-kube-v1.28/ubuntu-2204-kube-v1.28.12.qcow2
2024-08-07 12:34:06 | INFO     | version: 1.28.12
2024-08-07 12:34:06 | INFO     | url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.28/ubuntu-2204-kube-v1.28.12.qcow2
2024-08-07 12:34:06 | INFO     | checksum_url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.28/ubuntu-2204-kube-v1.28.12.qcow2.CHECKSUM
2024-08-07 12:34:06 | INFO     | checksum: 45c75763bdb53c17d3482abd6d0a55b2429efd7df7927fd2a1d68c4cf859b146
2024-08-07 12:34:06 | INFO     | date: 2024-07-18
2024-08-07 12:34:06 | INFO     | image: ubuntu-2204-kube-v1.29/ubuntu-2204-kube-v1.29.7.qcow2
2024-08-07 12:34:06 | INFO     | version: 1.29.7
2024-08-07 12:34:06 | INFO     | url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.29/ubuntu-2204-kube-v1.29.7.qcow2
2024-08-07 12:34:06 | INFO     | checksum_url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.29/ubuntu-2204-kube-v1.29.7.qcow2.CHECKSUM
2024-08-07 12:34:07 | INFO     | checksum: 35d3f1e124c0a3f428487d2266923bbed066f87da78ad45245075cb00cfbd33f
2024-08-07 12:34:07 | INFO     | date: 2024-07-18
2024-08-07 12:34:07 | INFO     | image: ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2
2024-08-07 12:34:07 | INFO     | version: 1.30.3
2024-08-07 12:34:07 | INFO     | url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2
2024-08-07 12:34:07 | INFO     | checksum_url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2.CHECKSUM
2024-08-07 12:34:07 | INFO     | checksum: c248a2def6110e65dd93c7fb8fd3d9d51904efce7aa54f0581200339ef849386
2024-08-07 12:34:09 | INFO     | Processing image 'ubuntu-capi-image 1.30.3'
2024-08-07 12:34:09 | INFO     | Tested URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2: 200
2024-08-07 12:34:09 | INFO     | Skipping required import of image 'ubuntu-capi-image 1.30.3', running in dry-run mode
2024-08-07 12:34:09 | INFO     | Processing image 'ubuntu-capi-image 1.29.7'
2024-08-07 12:34:09 | INFO     | Tested URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.29/ubuntu-2204-kube-v1.29.7.qcow2: 200
2024-08-07 12:34:09 | INFO     | Skipping required import of image 'ubuntu-capi-image 1.29.7', running in dry-run mode
2024-08-07 12:34:09 | INFO     | Processing image 'ubuntu-capi-image 1.28.12'
2024-08-07 12:34:10 | INFO     | Tested URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.28/ubuntu-2204-kube-v1.28.12.qcow2: 200
2024-08-07 12:34:10 | INFO     | Skipping required import of image 'ubuntu-capi-image 1.28.12', running in dry-run mode

Signed-off-by: Martin Morgenstern <[email protected]>
* this is the recommendation in SCS
* precondition for hw_scsi_model property

Signed-off-by: Martin Morgenstern <[email protected]>
Signed-off-by: Martin Morgenstern <[email protected]>
@@ -41,6 +52,7 @@ def take_action(self, parsed_args):
base_url = parsed_args.base_url
cloud = parsed_args.cloud
filter = parsed_args.filter

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable name filter shadows the built-in Python function filter(). This can lead to unexpected behavior and bugs, especially if the built-in function is needed later in the code.

Recommended Solution: Rename the variable to something more descriptive, such as image_filter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably makes sense and can be tackled In a separate PR.

args.extend(["--tag", tag])
if parsed_args.dry_run:
args.append("--dry-run")
subprocess.call(args)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of subprocess.call can be risky if any of the arguments are derived from user input, as it may lead to command injection vulnerabilities. Although the current arguments seem to be controlled, it's safer to use subprocess.run with a list of arguments to avoid shell interpretation issues.

Recommended Solution: Replace subprocess.call with subprocess.run and ensure all arguments are passed as a list.

Signed-off-by: Martin Morgenstern <[email protected]>
@berendt
Copy link
Member

berendt commented Aug 7, 2024

@martinmo The change of the image name is required because of the SCS standard?

@martinmo
Copy link
Contributor Author

martinmo commented Aug 7, 2024

@berendt Yes, in https://github.com/SovereignCloudStack/standards/blob/2d7f7d31ad5612c9db0699a626e2eace4e41c383/Tests/iaas/scs-0104-v1-images.yaml#L7-L10 we use this name scheme:

- name: "ubuntu-capi-image"
  name_scheme: "ubuntu-capi-image v[0-9]\\.[0-9]+(\\.[0-9]+)?"
  source: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube
  status: recommended

Of course, it says recommended and tests won't fail if there are only Kubernetes CAPI images, but I don't see why someone would not use the recommended name. Also, the KaaS reference implementation is supposed to use this name, albeit there is a small inconcistency at the moment which prevents that (SovereignCloudStack/cluster-stacks#156).

If you disagree, I could also add another flag --custom-name to set a custom name and let it default to the existing value Kubernetes CAPI. And in the implementation notes for the standard, I'll refer users to use --custom-name ubuntu-capi-image.

Signed-off-by: Martin Morgenstern <[email protected]>
@martinmo
Copy link
Contributor Author

martinmo commented Aug 7, 2024

Just noticed that my changes to make the image names SCS compliant were not complete: versions need to be prefixed with v as in v1.29.3. I adjusted the template accordingly.

Example dry run:

$ rm -r /tmp/clusterapi 
$ osism manage image clusterapi --dry-run --cloud l1a --filter 1.30
[... some unrelated TripleDES deprecation warnings ...]
2024-08-07 19:39:01 | INFO     | date: 2024-07-18
2024-08-07 19:39:01 | INFO     | image: ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2
2024-08-07 19:39:01 | INFO     | version: 1.30.3
2024-08-07 19:39:01 | INFO     | url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2
2024-08-07 19:39:01 | INFO     | checksum_url: https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2.CHECKSUM
2024-08-07 19:39:01 | INFO     | checksum: c248a2def6110e65dd93c7fb8fd3d9d51904efce7aa54f0581200339ef849386
2024-08-07 19:39:03 | INFO     | Processing image 'ubuntu-capi-image v1.30.3'
2024-08-07 19:39:03 | INFO     | Tested URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-k8s-capi-images/ubuntu-2204-kube-v1.30/ubuntu-2204-kube-v1.30.3.qcow2: 200
2024-08-07 19:39:03 | INFO     | Skipping required import of image 'ubuntu-capi-image v1.30.3', running in dry-run mode

@berendt berendt merged commit 158d183 into osism:main Aug 12, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants