Separate image and container runtime locations for HPC environment

[stirlingalgermissen]

I'd like to be able to separate where Podman saves container images and where Podman runs containers. Context: this is an HPC environment with a number of Lustre and NFSv3 mounts and a single NFSv4 mount that is on a slow machine with slow disks. Podman is running in rootless mode.

My goal is to store the container images on the NFSv4 mount (no other options besides a ramdrive) and run the containers on a much faster NFSv3/Lustre mount.

Is this possible? I've tried modifying runroot/graphroot/rootless_storage_path with no luck.

[rhatdan]

NFS Servers will not allow a rootless user to chown files within a User Namespace. From the NFS servers point of view it looks like UID=1000 is attempting to chown a file to something other then 1000 and does permission denied.
If you ignore_chown_errors in storage.conf the images in container storage, this should be possible, but all of the processes and files within the container will be owned by your UID.

man storage.conf
...
       ignore_chown_errors = "false"
         ignore_chown_errors can be set to allow a non privileged user  running
       with  a   single UID within a user namespace to run containers. The user
       can pull and use any image even those with multiple uids.  Note multiple
       UIDs will be squashed down to the default uid in the  container.   These
       images  will have no separation between the users in the container. (de‐
       fault: false)

[stirlingalgermissen]

Thanks for confirming it should be possible. I'm seeing this error - is there any way I can override?

GIDMap:[]idtools.IDMap(nil), Excludes:[]string(nil), ExpandArchives:false, ChownDirs:(*idtools.IDPair)(nil), ChmodDirs:(*fs.FileMode)(nil), ChownFiles:(*idtools.IDPair)(nil), ChmodFiles:(*fs.FileMode)(nil), StripSetuidBit:false, StripSetgidBit:false, StripStickyBit:false, StripXattrs:false, KeepDirectoryNames:false, Rename:map[string]string(nil), NoDerefSymlinks:false, IgnoreUnreadable:false, NoCrossDevice:false}, PutOptions:copier.PutOptions{UIDMap:[]idtools.IDMap(nil), GIDMap:[]idtools.IDMap(nil), DefaultDirOwner:(*idtools.IDPair)(nil), DefaultDirMode:(*fs.FileMode)(nil), ChownDirs:(*idtools.IDPair)(nil), ChmodDirs:(*fs.FileMode)(nil), ChownFiles:(*idtools.IDPair)(nil), ChmodFiles:(*fs.FileMode)(nil), StripSetuidBit:false, StripSetgidBit:false, StripStickyBit:false, StripXattrs:false, IgnoreXattrErrors:false, IgnoreDevices:false, NoOverwriteDirNonDir:false, NoOverwriteNonDirDir:false, Rename:map[string]string(nil)}, MkdirOptions:copier.MkdirOptions{UIDMap:[]idtools.IDMap(nil), GIDMap:[]idtools.IDMap(nil), ChownNew:(*idtools.IDPair)(nil), ChmodNew:(*fs.FileMode)(nil)}, RemoveOptions:copier.RemoveOptions{All:false}}: copier: put: error setting extended attributes on "/sigstore": setting value of extended attribute "user.containers.override_stat" on "/sigstore": operation not supported

Looks like buildah copier related code