Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

builder-jammy-tiny fails to build spring-boot native jar: permission denied #24405

Open
shineeng opened this issue Oct 29, 2024 · 5 comments
Open
Labels
kind/bug Categorizes issue or PR as related to a bug. remote Problem is in podman-remote

Comments

@shineeng
Copy link

Issue Description

the build made with paketo buildpack fails upon final cleaning phase with permission error

Steps to reproduce the issue

Steps to reproduce the issue

  1. Create a spring-boot application
  2. compile it with -Pnative flag
  3. the cleanup phase can't be completed with a permission error

Describe the results you received

[INFO] [creator] 22.2s (9.3% of total time) in 1321 GCs | Peak RSS: 6.36GB | CPU load: 9.61
[INFO] [creator] --------------------------------------------------------------------------------
[INFO] [creator] Produced artifacts:
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/it.eng.silcloud.anagrafiche.AnagraficheApplication (executable)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libawt.so (jdk_library)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libawt_headless.so (jdk_library)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libawt_xawt.so (jdk_library)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libfontmanager.so (jdk_library)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libfreetype.so (jdk_library)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libjava.so (jdk_library_shim)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libjavajpeg.so (jdk_library)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/libjvm.so (jdk_library_shim)
[INFO] [creator] /layers/paketo-buildpacks_native-image/native-image/liblcms.so (jdk_library)
[INFO] [creator] ================================================================================
[INFO] [creator] Finished generating 'it.pem.bimbam.anagrafiche.AnagraficheApplication' in 3m 57s.
[INFO] [creator] Removing bytecode
[INFO] [creator] unable to invoke layer creator
[INFO] [creator] unable to remove /workspace/BOOT-INF
[INFO] [creator] unlinkat /workspace/BOOT-INF: permission denied
[INFO] [creator] ERROR: failed to build: exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:58 min

Describe the results you expected

a compiled image published on my local repo

podman info output

host:
arch: amd64
buildahVersion: 1.37.3
cgroupControllers: []
cgroupManager: cgroupfs
cgroupVersion: v1
conmon:
package: conmon-2.1.12-2.fc40.x86_64
path: /usr/bin/conmon
version: 'conmon version 2.1.12, commit: '
cpuUtilization:
idlePercent: 92.37
systemPercent: 0.37
userPercent: 7.25
cpus: 14
databaseBackend: sqlite
distribution:
distribution: fedora
variant: container
version: "40"
eventLogger: journald
freeLocks: 2043
hostname: ALE
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 524288
size: 65536
uidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 524288
size: 65536
kernel: 5.15.153.1-microsoft-standard-WSL2
linkmode: dynamic
logDriver: journald
memFree: 15564402688
memTotal: 16480219136
networkBackend: netavark
networkBackendInfo:
backend: netavark
dns:
package: aardvark-dns-1.12.2-2.fc40.x86_64
path: /usr/libexec/podman/aardvark-dns
version: aardvark-dns 1.12.2
package: netavark-1.12.2-1.fc40.x86_64
path: /usr/libexec/podman/netavark
version: netavark 1.12.2
ociRuntime:
name: crun
package: crun-1.17-1.fc40.x86_64
path: /usr/bin/crun
version: |-
crun version 1.17
commit: 000fa0d4eeed8938301f3bcf8206405315bc1017
rundir: /run/user/1000/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
os: linux
pasta:
executable: /usr/bin/pasta
package: passt-0^20240906.g6b38f07-1.fc40.x86_64
version: |
pasta 0^20240906.g6b38f07-1.fc40.x86_64
Copyright Red Hat
GNU General Public License, version 2 or later
https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
remoteSocket:
exists: true
path: /run/user/1000/podman/podman.sock
rootlessNetworkCmd: pasta
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: true
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 4294967296
swapTotal: 4294967296
uptime: 2h 10m 30.00s (Approximately 0.08 days)
variant: ""
plugins:
authorization: null
log:

  • k8s-file
  • none
  • passthrough
  • journald
    network:
  • bridge
  • macvlan
  • ipvlan
    volume:
  • local
    registries:
    search:
  • docker.io
    store:
    configFile: /home/user/.config/containers/storage.conf
    containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
    graphDriverName: overlay
    graphOptions: {}
    graphRoot: /home/user/.local/share/containers/storage
    graphRootAllocated: 1081101176832
    graphRootUsed: 1792749568
    graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
    imageCopyTmpDir: /var/tmp
    imageStore:
    number: 3
    runRoot: /run/user/1000/containers
    transientStore: false
    volumePath: /home/user/.local/share/containers/storage/volumes
    version:
    APIVersion: 5.2.3
    Built: 1727136000
    BuiltTime: Tue Sep 24 02:00:00 2024
    GitCommit: ""
    GoVersion: go1.22.7
    Os: linux
    OsArch: linux/amd64
    Version: 5.2.3

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

Yes

Additional environment details

Compiling spring-boot 3.3.3, Java 21

Additional information

Crossposted on paketo tracker and on stackoverflow

@shineeng shineeng added the kind/bug Categorizes issue or PR as related to a bug. label Oct 29, 2024
@github-actions github-actions bot added the remote Problem is in podman-remote label Oct 29, 2024
@vcutarelli1974
Copy link

cazzo anche io ho 'sto problema

@shineeng shineeng changed the title builder-jammy-tiny fails to build spring-boot: permission denied builder-jammy-tiny fails to build spring-boot native jar: permission denied Oct 29, 2024
@Luap99
Copy link
Member

Luap99 commented Oct 29, 2024

You will need to provide a proper podman reproducer (ideally minimal), I have no idea what spring boot does. We need the actual podman commands or API calls for to work with.

@amoscatelli
Copy link

amoscatelli commented Nov 14, 2024

I can reproduce this too on one of my projects

[INFO]     [creator]
[INFO]     [creator]     Packed 1 file.
[INFO]     [creator]       Removing bytecode
[INFO]     [creator]     unable to invoke layer creator
[INFO]     [creator]     unable to remove /workspace/BOOT-INF
[INFO]     [creator]     unlinkat /workspace/BOOT-INF: permission denied
[INFO]     [creator]     ERROR: failed to build: exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  11:18 min
[INFO] Finished at: 2024-11-14T16:40:35+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:3.2.4:build-image (default-cli) on project sem-anas-gics-be-edge-shm-dfm: Execution default-cli of goal org.springframework.boot:spring-boot-maven-plugin:3.2.4:build-image failed: Builder lifecycle 'creator' failed with status code 51 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

@cmoulliard
Copy link

The error that you got is exactly what we discovered last week during the tests we made using java buildpack client - snowdrop/java-buildpack-client#92 with podman 5.2.5.

The problem has been discussed actively between @BarDweller and @mheon and occurs when a volume and/or mountpoint is created on a container before it is started. Correct me if this is not 100% exact

@mheon
Copy link
Member

mheon commented Nov 25, 2024

#24655 ought to fix

mheon added a commit to mheon/libpod that referenced this issue Nov 26, 2024
This solves several problems with copying into volumes on a
container that is not running.

The first, and most obvious, is that we were previously entirely
unable to copy into a volume that required mounting - like
image volumes, volume plugins, and volumes that specified mount
options.

The second is that this fixed several permissions and content
issues with a fresh volume and a container that has not been run
before. A copy-up will not have occurred, so permissions on the
volume root will not have been set and content will not have been
copied into the volume.

If the container is running, this is very low cost - we maintain
a mount counter for named volumes, so it's just an increment in
the DB if the volume actually needs mounting, and a no-op if it
doesn't.

Unfortunately, we also have to fix permissions, and that is
rather more complicated. This involves an ugly set of manual
edits to the volume state to ensure that the permissions fixes
actually worked, as the code was never meant to be used in this
way. It's really ugly, but necessary to reach full Docker
compatibility.

Fixes containers#24405

Signed-off-by: Matthew Heon <[email protected]>
mheon added a commit to mheon/libpod that referenced this issue Nov 26, 2024
This solves several problems with copying into volumes on a
container that is not running.

The first, and most obvious, is that we were previously entirely
unable to copy into a volume that required mounting - like
image volumes, volume plugins, and volumes that specified mount
options.

The second is that this fixed several permissions and content
issues with a fresh volume and a container that has not been run
before. A copy-up will not have occurred, so permissions on the
volume root will not have been set and content will not have been
copied into the volume.

If the container is running, this is very low cost - we maintain
a mount counter for named volumes, so it's just an increment in
the DB if the volume actually needs mounting, and a no-op if it
doesn't.

Unfortunately, we also have to fix permissions, and that is
rather more complicated. This involves an ugly set of manual
edits to the volume state to ensure that the permissions fixes
actually worked, as the code was never meant to be used in this
way. It's really ugly, but necessary to reach full Docker
compatibility.

Fixes containers#24405

Signed-off-by: Matthew Heon <[email protected]>
mheon added a commit to mheon/libpod that referenced this issue Nov 26, 2024
This solves several problems with copying into volumes on a
container that is not running.

The first, and most obvious, is that we were previously entirely
unable to copy into a volume that required mounting - like
image volumes, volume plugins, and volumes that specified mount
options.

The second is that this fixed several permissions and content
issues with a fresh volume and a container that has not been run
before. A copy-up will not have occurred, so permissions on the
volume root will not have been set and content will not have been
copied into the volume.

If the container is running, this is very low cost - we maintain
a mount counter for named volumes, so it's just an increment in
the DB if the volume actually needs mounting, and a no-op if it
doesn't.

Unfortunately, we also have to fix permissions, and that is
rather more complicated. This involves an ugly set of manual
edits to the volume state to ensure that the permissions fixes
actually worked, as the code was never meant to be used in this
way. It's really ugly, but necessary to reach full Docker
compatibility.

Fixes containers#24405

Signed-off-by: Matthew Heon <[email protected]>
mheon added a commit to mheon/libpod that referenced this issue Nov 26, 2024
This solves several problems with copying into volumes on a
container that is not running.

The first, and most obvious, is that we were previously entirely
unable to copy into a volume that required mounting - like
image volumes, volume plugins, and volumes that specified mount
options.

The second is that this fixed several permissions and content
issues with a fresh volume and a container that has not been run
before. A copy-up will not have occurred, so permissions on the
volume root will not have been set and content will not have been
copied into the volume.

If the container is running, this is very low cost - we maintain
a mount counter for named volumes, so it's just an increment in
the DB if the volume actually needs mounting, and a no-op if it
doesn't.

Unfortunately, we also have to fix permissions, and that is
rather more complicated. This involves an ugly set of manual
edits to the volume state to ensure that the permissions fixes
actually worked, as the code was never meant to be used in this
way. It's really ugly, but necessary to reach full Docker
compatibility.

Fixes containers#24405

Signed-off-by: Matthew Heon <[email protected]>
mheon added a commit to mheon/libpod that referenced this issue Nov 27, 2024
This solves several problems with copying into volumes on a
container that is not running.

The first, and most obvious, is that we were previously entirely
unable to copy into a volume that required mounting - like
image volumes, volume plugins, and volumes that specified mount
options.

The second is that this fixed several permissions and content
issues with a fresh volume and a container that has not been run
before. A copy-up will not have occurred, so permissions on the
volume root will not have been set and content will not have been
copied into the volume.

If the container is running, this is very low cost - we maintain
a mount counter for named volumes, so it's just an increment in
the DB if the volume actually needs mounting, and a no-op if it
doesn't.

Unfortunately, we also have to fix permissions, and that is
rather more complicated. This involves an ugly set of manual
edits to the volume state to ensure that the permissions fixes
actually worked, as the code was never meant to be used in this
way. It's really ugly, but necessary to reach full Docker
compatibility.

Fixes containers#24405

Signed-off-by: Matthew Heon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. remote Problem is in podman-remote
Projects
None yet
Development

No branches or pull requests

6 participants