Problem with container restore - ProcessLabel and Mountlabel

[seanrmurphy]

Hi,

I'm trying to understand what is possible with the checkpointing/restore mechanism and whether it can be used in our context.

I was able to perform a checkpoint but when I try to restore the container, I get the rather cryptic:

# podman container restore -i ./container-test-checkpoint.tar.gz 
Error: error creating container storage: ProcessLabel and Mountlabel must either not be specified or both specified
# 

Obviously, it relates to storage and the error seems to be generated by podman (rather than criu) but I've
no idea how to troubleshoot it - any pointers appreciated.

Here are some versions....

[root@ip-10-0-2-204 fedora]# podman --version
podman version 3.1.2
[root@ip-10-0-2-204 fedora]# criu --version
Version: 3.15
[root@ip-10-0-2-204 fedora]# crun --version
crun version 0.19.1
commit: 1535fedf0b83fb898d449f9680000f729ba719f5
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
[root@ip-10-0-2-204 fedora]# cat /etc/os-release
NAME=Fedora
VERSION="33 (Cloud Edition)"
ID=fedora
VERSION_ID=33
<snip>

Thanks in advance!

[mheon]

@rhatdan Any thoughts here?

[mheon]

Are these older checkpoints? I'm wondering if this is a version issue - checkpoint from Podman N-1 doesn't restore on Podman N. Maybe the container was created with an older Podman?

If this is the same Podman version on both, this is a significant issue.

[seanrmurphy]

I'm just working on a single machine, just to see if I can get this working; as such, there is only a single podman/crun/criu stack in play using the versions noted above.

Might be worth pointing out that I'm using podman run as root on a container image which was built for docker, pulled down from AWS ECR - not sure if that might be a factor which could result in the ProcessLabel being empty; also SELinux is in permissive mode and the container is run as privileged...

[seanrmurphy]

So I can confirm that no ProcessLabel is generated when I run the container in privileged mode - see below. @mheon - does this mean that it is, in principle, impossible to checkpoint and restore a privileged container?

Test script...

[root@ip-10-0-2-102 fedora]# cat ./run_nginx.sh 
#! /usr/bin/env bash

# run it in non privileged mode...

echo
echo "Stopping and removing old nginx-test containers (if they exist)..."
podman stop nginx-test
podman rm nginx-test

echo
echo "Running nginx unprivileged..."
podman run --name nginx-test \
	-d \
	docker.io/library/nginx 

sleep 5

echo
echo "Inspecting container..."
podman inspect nginx-test | grep -b2 ProcessLabel

# run it in privileged mode...

echo
echo "Removing unprivileged nginx-test container..."
podman stop nginx-test
podman rm nginx-test

echo
echo "Running nginx privileged..."
podman run --name nginx-test \
	--privileged -d \
	docker.io/library/nginx 

sleep 5

echo
echo "Inspecting container..."
podman inspect nginx-test | grep -b2 ProcessLabel

[root@ip-10-0-2-102 fedora]# 

Output of test script...

[root@ip-10-0-2-102 fedora]# ./run_nginx.sh 

Stopping and removing old nginx-test containers (if they exist)...
nginx-test
8c9b94b789b93c749b815ff84c7a5681b3ad107a0afccb9c83aaed2e5a35ee97

Running nginx unprivileged...
e08e6526d43633abd25c3fc4d62328a03169703bd958806e767a9101700fc642

Inspecting container...
2067-        "Driver": "overlay",
2096-        "MountLabel": "system_u:object_r:container_file_t:s0:c701,c727",
2169:        "ProcessLabel": "system_u:system_r:container_t:s0:c701,c727",
2239-        "AppArmorProfile": "",
2270-        "EffectiveCaps": [

Removing unprivileged nginx-test container...
nginx-test
e08e6526d43633abd25c3fc4d62328a03169703bd958806e767a9101700fc642

Running nginx privileged...
4611a211f53b488eb8ba456ea033cba8b0a1b84f5d76adcb1881fe32b984cf91

Inspecting container...
2067-        "Driver": "overlay",
2096-        "MountLabel": "system_u:object_r:container_file_t:s0:c349,c782",
2169:        "ProcessLabel": "",
2197-        "AppArmorProfile": "",
2228-        "EffectiveCaps": [
[root@ip-10-0-2-102 fedora]#

[rhatdan]

No this looks like a bug. So a confined process works properly but a privileged one fails. The resstore code should handle this case and just allow the container to run without a label. (Note the container will run as spc_t, when it starts.)

[rhatdan]

Please open an issue for this, and I will look into it.