Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Podman either as alias to Docker or natively #3170

Open
DerGut opened this issue Jan 6, 2022 · 8 comments · May be fixed by #5154
Open

Support Podman either as alias to Docker or natively #3170

DerGut opened this issue Jan 6, 2022 · 8 comments · May be fixed by #5154
Labels
size/M We should be able to deliver roughly 1 medium issue in a sprint. type/feature Issues that are new feature requests. type/request Issues that are created by customers.

Comments

@DerGut
Copy link

DerGut commented Jan 6, 2022

It would be great if Podman could be used in addition to Docker for building images.

I had to install Docker to be able to use Copilot. But I don't think this has to be necessary as most of the Docker commands can be replaced by running podman in place of docker (as far as I know).

I realise, that it might be easier to accept a docker alias from the shell environment than it is to implement a new client like in dockerengine.go.
Under e2e there already seems to be an implementation that makes use of the bash shell context by running bash -c docker ...:

func (d *Docker) exec(command string, opts ...cmd.Option) error {
return BashExec(fmt.Sprintf("docker %s", command), opts...)
}

Would this be a possible future addition or is there any technical limitation preventing Podman from being used here?

@efekarakus
Copy link
Contributor

Hi @DerGut !

Would this be a possible future addition or is there any technical limitation preventing Podman from being used here?

It's definitely possible :) there are no blockers for Copilot to use Podman. I've added the feature request label to the issue, thank you!

@efekarakus efekarakus added size/M We should be able to deliver roughly 1 medium issue in a sprint. type/feature Issues that are new feature requests. type/request Issues that are created by customers. labels Jan 7, 2022
@matthewhembree
Copy link

I wonder if Podman is the best target or if something like Colima or Rancher Desktop would be a better direction.

My team also uses AWS SAM and there are efforts to enable Lima based container engines (Colima / Rancher Desktop). It seems unusual to use two AWS toolchains and need to have two separate container engines for each toolchain.

My organization looked at Podman and the lack of bind mount functionality on Mac and Windows is a blocker for us. That definitely blocks SAM local invocation as the SAM container bind mounts the source code. We don't predict that we'll be using volumes/EFS in Copilot/ECS; but for organizations that would, they would be blocked.

@sombriks
Copy link

yes offer podman as option would be great, tried the command alias but didn't work.

@nathanpeck
Copy link
Member

Hey all, I have done a deep investigation into Podman + AWS Copilot. I was able to get Copilot to use Podman by adding a symlink: ln -s /usr/local/bin/podman /usr/local/bin/docker

Unfortunately there is a difference between how Podman does image digests and how Docker Desktop does image digests, which blocks the use of Podman at this time. I've created an issue on the Podman repo: containers/podman#14779

We will need to investigate deeper workarounds to how we handle image digests, or hope for a patch to Podman which causes Podman to generate consistent image digests.

@mjrlee
Copy link

mjrlee commented Jan 11, 2023

A little more digging here, the nasty hack below works when using podman.

I'm happy to help work on an a useful PR for this, though not sure how best to fit it in to the existing project.

+++ b/internal/pkg/docker/dockerengine/dockerengine.go
@@ -151,6 +151,9 @@ func (c CmdClient) Push(uri string, tags ...string) (digest string, err error) {
                images = append(images, imageName(uri, tag))
        }
        var args []string
+       digestFile, _ := os.CreateTemp("", "copilot")
+       defer os.Remove(digestFile.Name())
+       args = append(args, "--digestfile", digestFile.Name())
        if ci, _ := c.lookupEnv("CI"); ci == "true" {
                args = append(args, "--quiet")
        }
@@ -160,16 +163,9 @@ func (c CmdClient) Push(uri string, tags ...string) (digest string, err error) {
                        return "", fmt.Errorf("docker push %s: %w", img, err)
                }
        }
-       buf := new(strings.Builder)
-       if err := c.runner.Run("docker", []string{"inspect", "--format", "'{{json (index .RepoDigests 0)}}'", uri}, exec.Stdout(buf)); err != nil {
-               return "", fmt.Errorf("inspect image digest for %s: %w", uri, err)
-       }
-       repoDigest := strings.Trim(strings.TrimSpace(buf.String()), `"'`) // remove new lines and quotes from output
-       parts := strings.SplitAfter(repoDigest, "@")
-       if len(parts) != 2 {
-               return "", fmt.Errorf("parse the digest from the repo digest '%s'", repoDigest)
-       }
-       return parts[1], nil
+       repoDigest, _ := os.ReadFile("./digest")
+       part := strings.TrimPrefix(string(repoDigest), "sha256:")
+       return part, nil
 }

@Lou1415926
Copy link
Contributor

@MartinLeedotOrg Yay thank you for volunteering ❤️!

It'd be helpful to discuss the design, perhaps as well as implementation plan before working on the PR - this would save both you and us a lot of back and forth. Here is an example of a discussion that we had with a contributor before.

With this integration, we'd want to keep the experience of building with docker the same as before.

If this sounds good to you, I can kick off a discussion: what would the experience for a user that builds with Podman be like with your proposed snippet? The code snippet seems to be dealing with the image digest. Would users need to first alias Podman? Does this affect users that are building with docker?

@efekarakus also raised several questions here previously. The second question is answered (i.e. just alias-ing is not enough), but the first and the third are still interesting for us to explore. I would be curious to know your take on 1 and 3 as well!

Thank you again ❤️

@mjrlee
Copy link

mjrlee commented Jan 13, 2023

Do you know of any other tools that integrate with podman? Is there a reason why you preferred environment variable as a way of toggling?
I'm looking to see if there is a precedent for using env variables to build with podman vs. an option in the manifest such as:

image:
  build:
     engine: 'podman' # default is 'docker'

I haven't seen a consistent approach here... I like the idea of specifying the build engine in the manifest though. Another approach could simply be running docker version and parsing the output.

What are the downsides today of just doing alias docker=podman like in the homepage of podman? I'm trying to understand if it's worthwhile adding and maintaining new code in the repo if the mitigation already works for users.

That's a very common approach. It would make the approach above of parsing the output from docker version simpler too.

How about users that use the copilot pipelines? podman won't be installed in the build project, do we deem it acceptable if the images are build with docker in the pipeline but with podman locally?

I'd say yes. There's many ways to make OCI images, we shouldn't care which the user opts for and should just make an image happen using the tools available in their environment.

@efekarakus
Copy link
Contributor

Another approach could simply be running docker version and parsing the output.

Sorry for the naive question @MartinLeedotOrg, how can we tell if podman is used as the client from running docker version?
As far as I can tell, there is no information from the output that helps us find out other clients besides Docker.

@mjrlee mjrlee linked a pull request Aug 3, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/M We should be able to deliver roughly 1 medium issue in a sprint. type/feature Issues that are new feature requests. type/request Issues that are created by customers.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants