Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle latest iso's not being immutable #1334

Open
AverageMarcus opened this issue Oct 27, 2023 · 6 comments
Open

Handle latest iso's not being immutable #1334

AverageMarcus opened this issue Oct 27, 2023 · 6 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@AverageMarcus
Copy link
Member

Is your feature request related to a problem? Please describe.

For our providers that build based on top of an iso image we've seen a problem several times now with Ubuntu (maybe others?) where the latest release is only available as a non-specific URL that is updated as new releases are made.

For example, Ubuntu 23.04 is currently (at time of writing) only available at the following location:

https://releases.ubuntu.com/23.04/ubuntu-23.04-live-server-amd64.iso

When a new release of Ubuntu 23.04 is made, this image is replaced but the URL remains the same. There isn't a location to get the specific patch version of the release that we've been able to find.

This causes the following problems:

  • checksums become out of date so we need to keep updating them even though the URL isn't changing
  • breaking changes could possibly be introduced into a new release of Ubuntu that could break for users of image-builder without there being any changes made to image-builder or the provided variables. This means image-builder can't guarentee reproducible builds.

Describe the solution you'd like

Ideally there would be a location where we can get the patch versions of Ubuntu images but I'm not aware of such a place existing.

Describe alternatives you've considered

We could fetch the checksum on the fly from the https://releases.ubuntu.com/23.04/SHA256SUMS file but this doesn't solve the reproducibility issue.

Additional context

For old releases, they become available at a different URL (e.g. https://old-releases.ubuntu.com/releases/jammy/ubuntu-22.04.2-live-server-amd64.iso) but the same doesn't seem to be possible with the latest patch release.

/kind feature
@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 27, 2023
@AverageMarcus AverageMarcus mentioned this issue Oct 27, 2023
Merged
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 31, 2024
@AverageMarcus
Copy link
Member Author

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 31, 2024
@AverageMarcus
Copy link
Member Author

AverageMarcus commented Apr 5, 2024 •
edited
Loading

A little more investigation...

The current latest Ubuntu release is available at:

https://releases.ubuntu.com/22.04.4/ubuntu-22.04.4-live-server-amd64.iso

With curl:

curl https://releases.ubuntu.com/22.04.4/ubuntu-22.04.4-live-server-amd64.iso
HTTP/1.1 200 OK
Date: Fri, 05 Apr 2024 07:23:52 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Fri, 16 Feb 2024 23:52:36 GMT
ETag: "7d6eb800-61188703a4abd"
Accept-Ranges: bytes
Content-Length: 2104408064
Content-Type: application/x-iso9660-image

+-----------------------------------------+
| NOTE: binary data not shown in terminal |
+-----------------------------------------+

The previous Ubuntu release can be fetched using:

https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso

But this actually returns a HTTP 302 redirect to https://old-releases.ubuntu.com/releases/22.04.2/ubuntu-22.04.2-live-server-amd64.iso

With curl:

curl https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://old-releases.ubuntu.com/releases/22.04.2/ubuntu-22.04.2-live-server-amd64.iso">here</a>.</p>
<hr>
<address>Apache/2.4.29 (Ubuntu) Server at releases.ubuntu.com Port 443</address>
</body></html>
HTTP/1.1 301 Moved Permanently
Date: Fri, 05 Apr 2024 07:46:28 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://old-releases.ubuntu.com/releases/22.04.2/ubuntu-22.04.2-live-server-amd64.iso
Content-Length: 379
Content-Type: text/html; charset=iso-8859-1

I'm going to check if Packer correctly follows these redirects. If so, this should be resolved as long as we use the full version in the URL.

If that doesn't work, I have discovered that at least some Packer builders support a iso_urls property in place of iso_url which can be used to support multiple URLs. This will try the next in the list if one fails. We could use this to supply both the releases.ubuntu.com and the old-releases.ubuntu.com URLs so it should fallback when needed.

@AverageMarcus
Copy link
Member Author

I just confirmed that this works by using the following vars with make build-qemu-ubuntu-2204:

{
  "boot_command_prefix": "c<wait>linux /casper/vmlinuz --- autoinstall ds='nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/22.04/'<enter><wait><wait><wait>initrd /casper/initrd<enter><wait><wait><wait>boot<enter>",
  "build_name": "ubuntu-2204",
  "distribution_version": "2204",
  "distro_name": "ubuntu",
  "guest_os_type": "ubuntu-64",
  "iso_checksum": "5e38b55d57d94ff029719342357325ed3bda38fa80054f9330dc789cd2d43931",
  "iso_checksum_type": "sha256",
  "iso_url": "https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso",
  "os_display_name": "Ubuntu 22.04",
  "shutdown_command": "shutdown -P now",
  "unmount_iso": "true"
}

So as long as we use the https://releases.ubuntu.com/XX.XX.X/ style URLs (needs the patch value) we should be ok.

@AverageMarcus
Copy link
Member Author

It seems this is only true of the recent LTS releases and doesn't apply to 23.04 yet as it's still in beta.

@AverageMarcus AverageMarcus mentioned this issue Apr 5, 2024
Merged
@AverageMarcus
Copy link
Member Author

😩

So, the patch version URLs are only available from the first patch version. The initial release of Ubuntu 24.04 (e.g. 24.04.0) doesn't have an immutable URL and is only available from the one that points to the latest for that version. 😞

@AverageMarcus AverageMarcus mentioned this issue Jun 17, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AverageMarcus @k8s-ci-robot @k8s-triage-robot