Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggshield secret scan docker fails with recent version of Docker #469

Open
2 tasks
agateau-gg opened this issue Feb 2, 2023 · 0 comments
Open
2 tasks

ggshield secret scan docker fails with recent version of Docker #469

agateau-gg opened this issue Feb 2, 2023 · 0 comments
Labels
status:confirmed This issue has been reviewed and confirmed type:bug Something isn't working

Comments

@agateau-gg
Copy link
Collaborator

Environment

  • ggshield version: 1.14.3
  • Operating system (Linux, macOS, Windows): macOS
  • Operating system version: -
  • Python version: -

Describe the bug

ggshield secret scan docker fails with an error message like this:

$ ggshield secret scan docker --verbose gitguardian/ggshield
Saving docker image... OK

ERROR: invalid header.

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/cmd/secret/scan/docker.py", line 54, in docker_name_cmd
    scan = docker_scan_archive(
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 257, in docker_scan_archive
    files = get_files_from_docker_archive(archive)
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 62, in get_files_from_docker_archive
    return Files(list(chain((config_file_to_scan,), layer_files_to_scan)))
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 136, in _get_layers_files
    yield from _get_layer_files(archive, layer_info)
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 157, in _get_layer_files
    layer_archive = tarfile.TarFile(
  File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tarfile.py", line 1542, in __init__
    self.firstmember = self.next()
  File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tarfile.py", line 2362, in next
    raise ReadError(str(e)) from None
tarfile.ReadError: invalid header

Steps to reproduce:

  1. Make sure you have a recent enough version of Docker. This version is known to cause the crash:
$ docker version
Client:
 Cloud integration: v1.0.29
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:41 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      trueServer: Docker Desktop 4.16.2 (95914)
 Engine:
  Version:          22.06.0-beta.0-917-gf888bd4531.m
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.18.4
  Git commit:       f888bd4531
  Built:            Wed Jan 18 09:46:59 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
  1. Run command ggshield secret scan docker <anyimage>

Actual result:

It crashes.

Expected result:

Scans work as expected.

Explanation

It seems Docker changed the file format used to save images. It used to use its own format, documented here: https://github.com/moby/moby/blob/master/image/spec/v1.md. This is the format we currently supports. Nowadays it uses the OCI file format, documented here: https://github.com/opencontainers/image-spec/blob/main/spec.md.

TODO

  • Add support for the new format
  • The old format must continue to be supported: the Docker version shipped in many Linux distributions (in particular the current Ubuntu LTS: 22.04) still use this format
@agateau-gg agateau-gg added type:bug Something isn't working status:confirmed This issue has been reviewed and confirmed labels Feb 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:confirmed This issue has been reviewed and confirmed type:bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant