Bug: docker with aws' sam build on M1 for amd64 cannot build

[stationname]

Disclaimer

This issue is merely for reference purposes. It's not necessarily intended to be used as triaging for fixing.

Reason: This might be a difficult edge case and I don't have a from-scratch-like clean environment to make sure the steps will lead to a failure.

Description:

Docker build fails when triggered by AWS' sam build on Mac M1 with target amd64 (cross-plattform).

Error: Build Failed Error: NotFound: content digest sha256:[...]: not found

Notes:

• The build succeeds if it's directly invoked by docker build.
• The build succeeds if the target is arm64
• It's not clear whether this is a sam problem or a docker problem.

Workaround stated at the bottom.

Steps to reproduce:

• Configuration
  • AWS SAM version: version 1.125.0
  • Host: Mac M1
  • Docker 4.34.3 or any other version until 4.29
  • Target platform: amd64
  • buildkit: activated

Files

*template.yaml`

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: asdf

Resources:
  HjksDataCrawler:
    Type: AWS::Serverless::Function
    Properties:
      Description: function
      FunctionName: AFunction
      PackageType: Image
      Architectures:
        - x86_64
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./

Dockerfile

FROM --platform=linux/amd64 public.ecr.aws/lambda/python:3.11
COPY app.py /var/task
CMD [ "app.lambda_handler" ]

(↑ I hope I didn't strip too much)

app.py

def lambda_handler():
  print("hello world")

samconfig.toml

version = 0.1
[dev]
[dev.build]
[dev.build.parameters]
region="ap-east-1"
skip_pull_image=true
use_container=true
template = "template.yml"
base_dir = "."

# s3_bucketやimage_repositoryは仮置き
[dev.deploy]
[dev.deploy.parameters]
stack_name = "teststack"
image_repository = # TODO: NEEDS TO BE FILLED IN
region = "ap-east-1"
confirm_changeset = true
capabilities = "CAPABILITY_IAM"
s3_bucket = # TODO: NEEDS TO BE FILLED IN
s3_prefix = "docker_bug_test_m1_x86"

Command to run

export DOCKER_HOST=unix://"$HOME/.docker/run/docker.sock"
sam build --config-env dev 

Note:

• For the DOCKER_HOST setting you can also use this more cryptic command, which leads to the same result

Observed result:

Error output: Build Failed Error: NotFound: content digest sha256:[...]: not found

I can't reproduce the debug output, because I already workaround-ed the problem. 🙏

Expected result:

Build should succeed.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: macOS Sonoma 14.5 (14.5 (23F79))
2. sam --version: version 1.125.0
3. AWS region: ap-east-1
4. Docker 4.34.3 or any other version until 4.29

sam --info

{
  "version": "1.125.0",
  "system": {
    "python": "3.12.6",
    "os": "macOS-14.5-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "20.10.14",
    "aws_cdk": "2.61.1 (build d319d9c)",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Workaround

Downgrade to Docker 4.7, downloadable here: https://desktop.docker.com/mac/main/arm64/77141/Docker.dmg
( @docker maintainers: Please don't remove the download link in order to keep the workaround working 🙏 )

[hnnasit]

Hi @stationname, thanks for reporting the issue and providing how you managed to get it to work. Marking it as needs more investigation to see if this is a sam or docker issue.