copy image contents to local filesystem

[rittneje]

With buildkit I can output my image to a local directory via --output type=local, which copies over the root file system. I am looking for a similar feature to copy the root file system from an existing image in a remote registry to the local file system. Is there any tool in the containers project that can support this use case?

Obviously I can feed a one line Dockerfile to buildkit to accomplish this, just wanted to see if there were any alternatives.

[rhatdan]

podman image mount will mount up the image, for you to use?

Or do you want one totally separate from the local image?

[rittneje]

From what I can tell, podman image mount requires the container to run as privileged, which I cannot do. To be clear, I am trying to do this copy operation from within a Kubernetes pod, and the hope is for something unprivileged/rootless.

Can you clarify what you mean by "totally separate from the local image"?

[mtrmac]

Skopeo does not have any facility to “extract an image”, combining layers into a single filesystem reflecting all the overlay operations, and it’s been considered out of scope so far.

WRT the “buildkit” reference, if this is within the context of a build, isn’t some kind of Buildah mount command the natural way to approach this? I have no idea what kinds of privileges that needs exactly, but it can’t be more privileges than the privileges to do the build in the first place, can it?

[rhatdan]

podman image mount does not require any running of a container, it does require podman unshare, to enter the user namespace. And everything can be done in rootless mode.

$ podman pull IMAGE
$ podman unshare 
# mnt=$(podman image mount IMAGE)
# cd $mnt ...

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Thoughts, if you can not mount, then most likely this will not work. You also can not copy correctly if you don't have more then 1 UID and have CAP_CHOWN (or CAP_CHOWN within the User Namespace) you are running. This is needed in order to set different UIDs then the root UID of the user namespace.

[rhatdan]

# podman pull $IMAGE; podman create --name $IMAGE_ctr $IMAGE; podman export $IMAGE_ctr | tar xf -; podman rm $IMAGE_ctr

Or

# podman pull $IMAGE; mnt=$(podman image mount $IMAGE); cp -R --preserve $MNT .

[rittneje]

How do I do that? I tried installing fuse-overlayfs but still it gives the same error message.

[rhatdan]

Are you running in rootless mode or rootfull? What OS are you running.
Does your system have a /etc/containers/storage.conf?

[rittneje]

I started with a standard ubuntu:20.10 docker container and installed podman via apt-get.
There is no /etc/containers/storage.conf but there is a /etc/containers/containers.conf (everything is commented out).

[rittneje]

I fixed my docker run instruction to run as my uid (1000:1000) instead of root, and that yields a different error.

$ podman pull hello-world:latest
Error: default OCI runtime "runc" not found: invalid argument

Tried using crun instead but that gives an error as well.

$ podman --runtime crun pull hello-world:latest
cannot clone: Operation not permitted
Error: cannot re-exec process

[rhatdan]

You are attempting to run podman within a container? That is a different can of worms.

Please read https://www.redhat.com/sysadmin/podman-inside-container

Also you might want to try playing with quay.io/podman/stable image.

[yveszoundi]

@rhatdan , I have a simlar problem for a Live CD (x86_64 and arm64) that needs to bundle podman with a pre-configured image.

I would like to be able to "copy/export" the full structure of ~/.local/share/containers for a single container image.

• preferrably no hardcoded path references in any files (json, etc.) in the /exported_local_share_containers folder
• preferrably all the required files/folders pre-populated, as my Live CD disables internet access by default

Essentially I'd like to copy an image locally-built into a chroot environment, with the following "pseudo-commands"

$ podman export_to_chroot myimage /livecd/chroot/tmp_local_share_containers_folder
$ chroot /livecd/chroot/bin/adduser-newuser_and-do-some-stuff
$ chroot /livecd/chroot/bin/copy-to-newuser-folder /livecd/chroot/tmp_local_share_containers_folder
$ rm -rf /livecd/chroot/tmp_local_share_containers_folder
$ build_iso_image_with_prebuilt_podman_container_image /livecd/chroot

Running podman commands inside the chroot environment gives me the same errors as @rittneje (cannot re-exec process). My workaround involves building locally an image with a temporary newuser matching my chroot newuser, and then I copy /home/newuser/.local/share/containers into the live CD chroot environment.

[mtrmac]

Nothing about ~/.local/share/containers is documented, or maintained, to be a stable filesystem format suitable for moving across computers or software versions. Obviously the “daemonless” design of Podman imposes some constraints on how the file formats must be, but it still leaves many ways for changes to happen in a way which is incompatible with other direct accesses to that data.

(Also, “a single container image” is not all that well-defined when a major aspect of that store is cross-image layer sharing. What if you export an image, change a file in one of the underlying layers, and then re-import that into a store, and later an image reusing that layer is pulled?)

In the example above, I would expect the first four commands to be replaced by an ordinary image build + push (even if to a private short-term registry). Then, the ISO can, in principle, be built by installing software, running an ordinary podman pull inside the ISO image’s filesystem, and freezing the resulting filesystem.

[yveszoundi]

Thanks for the quick reply @mtrmac

Understood for the ~/.local/share/containers filesystem, I'm trying to simplify the Live CD build process on my end.

What I described for the "single container image" export is very specific to my use-case indeed. I just need a "one-time" export, and just for "copying/moving" the container image to a Live CD ISO filesystem (chroot).

In my previous example, I did skip the ordinary podman build commands, as I didn't feel that it was "important".

• I'm building the image locally and not pushing it to any remote repository.
• Any podman pull inside the ISO image filesystem (chroot environment), triggers errors similar to what @rittneje faced (cannot re-exec process).

In any case I do have a "workaround", but I'd rather prefer something like this (which is not possible):

for selected_local_image in ('new_image1', 'new_image2')
   create_moveable_local_share_containers_with_selected_localimages
   # stop_creating_local_temp_user_for_podman_builds_that_matches_username_in_livecd