Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added s390x and ppc64le support #10766

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Added s390x and ppc64le support #10766

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a6b6951
Added s390x and ppc64le support after rebasing
R3hankhan123 Sep 20, 2024
5d47c2e
Fixed argo installation in Dockerfile
R3hankhan123 Sep 23, 2024
b573e98
Made the requested changes
R3hankhan123 Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions backend/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# Copyright 2021-2022 The Kubeflow Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -35,10 +36,15 @@ RUN python3 -m pip install -r requirements.txt --no-cache-dir

# Downloading Argo CLI so that the samples are validated
ENV ARGO_VERSION v3.4.17
RUN curl -sLO https://github.com/argoproj/argo-workflows/releases/download/${ARGO_VERSION}/argo-linux-amd64.gz && \
gunzip argo-linux-amd64.gz && \
chmod +x argo-linux-amd64 && \
mv ./argo-linux-amd64 /usr/local/bin/argo
#Support s390x and ppc64le
RUN ARCH=$(uname -m); \
if [ "$ARCH" = "x86_64" ]; then \
ARCH="amd64"; \
fi \
&& curl -sLO https://github.com/argoproj/argo-workflows/releases/download/${ARGO_VERSION}/argo-linux-${ARCH}.gz && \
gunzip argo-linux-${ARCH}.gz && \
chmod +x argo-linux-${ARCH} && \
mv ./argo-linux-${ARCH} /usr/local/bin/argo

WORKDIR /
COPY ./samples /samples
Expand Down
8 changes: 7 additions & 1 deletion backend/Dockerfile.driver
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# Copyright 2023 The Kubeflow Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -17,7 +18,12 @@ FROM golang:1.21.7-alpine3.19 as builder
WORKDIR /go/src/github.com/kubeflow/pipelines
COPY . .

RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags netgo -ldflags '-extldflags "-static"' -o /bin/driver ./backend/src/v2/cmd/driver/*.go
RUN ARCH=$(uname -m) && \
if [ "$ARCH" = "x86_64" ]; then \
ARCH="amd64"; \
fi && \
GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=$ARCH go build -tags netgo -ldflags '-extldflags "-static"' -o /bin/driver ./backend/src/v2/cmd/driver/*.go


# Check licenses and comply with license terms.
RUN ./hack/install-go-licenses.sh
Expand Down
7 changes: 6 additions & 1 deletion backend/Dockerfile.launcher
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

R3hankhan123 marked this conversation as resolved.
Show resolved Hide resolved
# Copyright 2023 The Kubeflow Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -17,7 +18,11 @@ FROM golang:1.21.7-alpine3.19 as builder
WORKDIR /go/src/github.com/kubeflow/pipelines
COPY . .

RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags netgo -ldflags '-extldflags "-static"' -o /bin/launcher-v2 ./backend/src/v2/cmd/launcher-v2/*.go
RUN ARCH=$(uname -m) && \
if [ "$ARCH" = "x86_64" ]; then \
ARCH="amd64"; \
fi && \
GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=$ARCH go build -tags netgo -ldflags '-extldflags "-static"' -o /bin/launcher-v2 ./backend/src/v2/cmd/launcher-v2/*.go

# Check licenses and comply with license terms.
RUN ./hack/install-go-licenses.sh
Expand Down
15 changes: 8 additions & 7 deletions backend/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ CSV_PATH=backend/third_party_licenses

# Container Build Params
CONTAINER_ENGINE ?= docker
PLATFORMS ?= linux/amd64,linux/s390x,linux/ppc64le
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
PLATFORMS ?= linux/amd64,linux/s390x,linux/ppc64le
PLATFORMS ?= linux/amd64

Can we remove s390x and ppc64le from this PR? So this PR is focused on just enabling the ability to build other architectures with minimal effort. We can revisit the "build new architectures by default" topic in future. @HumairAK @gregsheremeta thoughts?

Copy link

@dilipgb dilipgb Nov 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anishasthana we can have another rule to build multiarch build and add build platform there for cross platform builds and we can use current rule for native build (This is something other communities like contour uses for multiarch builds). code may looks something like this.

# Platforms to build the multi-arch image for.
IMAGE_PLATFORMS ?= linux/amd64,linux/s390x,linux/ppc64le

multiarch-build: 
	@mkdir -p $(shell pwd)/image
	docker buildx build $(IMAGE_RESULT_FLAG) \
		--platform $(IMAGE_PLATFORMS) \
		--build-arg "BUILD_GOPRIVATE=$(BUILD_GOPRIVATE)" \
		--build-arg "BUILD_GOPROXY=$(BUILD_GOPROXY)" \
		--build-arg "BUILD_GOSUMDB=$(BUILD_GOSUMDB)" \
		--build-arg "BUILD_BASE_IMAGE=$(BUILD_BASE_IMAGE)" \
		--build-arg "BUILD_VERSION=$(BUILD_VERSION)" \
		--build-arg "BUILD_BRANCH=$(BUILD_BRANCH)" \
		--build-arg "BUILD_SHA=$(BUILD_SHA)" \
		--build-arg "BUILD_CGO_ENABLED=$(BUILD_CGO_ENABLED)" \
		--build-arg "BUILD_EXTRA_GO_LDFLAGS=$(BUILD_EXTRA_GO_LDFLAGS)" \
		--build-arg "BUILD_GOEXPERIMENT=$(BUILD_GOEXPERIMENT)" \
		$(DOCKER_BUILD_LABELS) \
		$(IMAGE_TAGS) \
		$(shell pwd)


build: 
	docker build \
		--build-arg "BUILD_GOPRIVATE=$(BUILD_GOPRIVATE)" \
		--build-arg "BUILD_GOPROXY=$(BUILD_GOPROXY)" \
		--build-arg "BUILD_GOSUMDB=$(BUILD_GOSUMDB)" \
		--build-arg "BUILD_BASE_IMAGE=$(BUILD_BASE_IMAGE)" \
		--build-arg "BUILD_VERSION=$(BUILD_VERSION)" \
		--build-arg "BUILD_BRANCH=$(BUILD_BRANCH)" \
		--build-arg "BUILD_SHA=$(BUILD_SHA)" \
		--build-arg "BUILD_CGO_ENABLED=$(BUILD_CGO_ENABLED)" \
		--build-arg "BUILD_EXTRA_GO_LDFLAGS=$(BUILD_EXTRA_GO_LDFLAGS)" \
		--build-arg "BUILD_GOEXPERIMENT=$(BUILD_GOEXPERIMENT)" \
		$(DOCKER_BUILD_LABELS) \
		$(shell pwd) \
		--tag $(IMAGE):$(VERSION)

So primary architecture which is x86/native platform here don't have build the multiarch image for every time.

IMG_TAG_APISERVER ?= apiserver
IMG_TAG_PERSISTENCEAGENT ?= persistence-agent
IMG_TAG_CACHESERVER ?= cache-server
Expand Down Expand Up @@ -66,25 +67,25 @@ image_all: image_apiserver image_persistence_agent image_cache image_swf image_v

.PHONY: image_apiserver
image_apiserver:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_APISERVER} -f backend/Dockerfile .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS) -t ${IMG_TAG_APISERVER} -f backend/Dockerfile --push .
.PHONY: image_persistence_agent
image_persistence_agent:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_PERSISTENCEAGENT} -f backend/Dockerfile.persistenceagent .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS) -t ${IMG_TAG_PERSISTENCEAGENT} -f backend/Dockerfile.persistenceagent --push .
.PHONY: image_cache
image_cache:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_CACHESERVER} -f backend/Dockerfile.cacheserver .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS) -t ${IMG_TAG_CACHESERVER} -f backend/Dockerfile.cacheserverbuildx build --platform $(PLATFORMS)
.PHONY: image_swf
image_swf:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_SCHEDULEDWORKFLOW} -f backend/Dockerfile.scheduledworkflow .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS) -t ${IMG_TAG_SCHEDULEDWORKFLOW} -f backend/Dockerfile.scheduledworkflow --push .
.PHONY: image_viewer
image_viewer:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_VIEWERCONTROLLER} -f backend/Dockerfile.viewercontroller .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS) -t ${IMG_TAG_VIEWERCONTROLLER} -f backend/Dockerfile.viewercontroller --push .
.PHONY: image_visualization
image_visualization:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_VISUALIZATION} -f backend/Dockerfile.visualization .
.PHONY: image_driver
image_driver:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_DRIVER} -f backend/Dockerfile.driver .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS) -t ${IMG_TAG_DRIVER} -f backend/Dockerfile.driver --push .
.PHONY: image_launcher
image_launcher:
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} build -t ${IMG_TAG_LAUNCHER} -f backend/Dockerfile.launcher .
cd $(MOD_ROOT) && ${CONTAINER_ENGINE} buildx build --platform $(PLATFORMS)-t ${IMG_TAG_LAUNCHER} -f backend/Dockerfile.launcher --push .
R3hankhan123 marked this conversation as resolved.
Show resolved Hide resolved
26 changes: 23 additions & 3 deletions backend/api/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

R3hankhan123 marked this conversation as resolved.
Show resolved Hide resolved
# Copyright 2021 The Kubeflow Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -23,7 +24,8 @@ ENV GOBIN=/go/bin

# Install protoc.
RUN apt-get update -y && apt-get install -y jq sed unzip
RUN curl -L -o protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-x86_64.zip
RUN ARCH=$(uname -m); \
&& curl -L -o protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-${ARCH}.zip
RUN unzip -o protoc.zip -d /usr/ bin/protoc
RUN unzip -o protoc.zip -d /usr/ 'include/*'
RUN rm -f protoc.zip
Expand All @@ -41,8 +43,26 @@ RUN go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway@la
RUN go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger@latest

# Download go-swagger binary.
RUN curl -LO "https://github.com/go-swagger/go-swagger/releases/download/${GO_SWAGGER_VERSION}/swagger_linux_amd64"
RUN chmod +x swagger_linux_amd64 && mv swagger_linux_amd64 /usr/bin/swagger
#building go-swagger binary for s390x and ppc64le since pre built binaries are not available for these platforms
RUN /bin/sh -c ARCH=$(uname -m) && \
if [ "$ARCH" = "x86_64" ]; then \
ARCH="amd64" && \
curl -LO "https://github.com/go-swagger/go-swagger/releases/download/${GO_SWAGGER_VERSION}/swagger_linux_${ARCH}" && \
chmod +x swagger_linux_${ARCH} && \
mv swagger_linux_${ARCH} /usr/bin/swagger; \
else \
dir=$(mktemp -d) && \
git clone https://github.com/go-swagger/go-swagger "$dir" && \
cd "$dir" && \
git checkout v${GO_SWAGGER_VERSION} && \
LDFLAGS="-s -w -X github.com/$GITHUB_REPOSITORY/cmd/swagger/commands.Commit=${GITHUB_SHA}"; \
LDFLAGS="$LDFLAGS -X github.com/$GITHUB_REPOSITORY/cmd/swagger/commands.Version=${GITHUB_REF_NAME-dev}"; \
out="swagger_linux_$ARCH"; \
GOOS=linux GOARCH=$ARCH CGO_ENABLED=0 go build -ldflags "$LDFLAGS" -o "./dist/bin/$out" ./cmd/swagger; \
mv "./dist/bin/$out" /usr/bin/swagger && \
cd / && \
rm -rf "$dir"; \
fi

# Need protobuf source code for -I in protoc command.
RUN mkdir golang && cd golang && git clone --depth 1 --branch $GOLANG_PROTOBUF_VERSION https://github.com/golang/protobuf.git
Expand Down
10 changes: 8 additions & 2 deletions backend/src/cache/deployer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
FROM gcr.io/google.com/cloudsdktool/google-cloud-cli:alpine
FROM alpine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this.
@rimolive @chensun thoughts?

Copy link

@dilipgb dilipgb Sep 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hbelmiro gcr.io/google.com/cloudsdktool/google-cloud-cli:alpine image is not supporting multiple architecture. It only amd/arm architectures.

Since gcloud component installed is only kubectl (https://github.com/kubeflow/pipelines/blob/master/backend/src/cache/deployer/Dockerfile#L8) we are taking plain vanila alpine and installing all the needed packages including kubectl from binary distributions as you see in below code snippet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dilipgb I see. But I'm not sure how able we are to change images without a broader involvement of the community. Like asking in the community call, for example.

cc @HumairAK

RUN apk add --update \
curl \
jq \
openssl

RUN gcloud components install kubectl
RUN ARCH=$(uname -m); \
if [ "$ARCH" = "x86_64" ]; then \
ARCH="amd64"; \
fi \
&& curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/${ARCH}/kubectl \
&& chmod +x ./kubectl \
&& mv ./kubectl /usr/local/bin/kubectl

ADD backend/src/cache/deployer/* /kfp/cache/deployer/
RUN chmod -R 777 /kfp/cache/deployer
Expand Down
13 changes: 12 additions & 1 deletion backend/src/cache/deployer/deploy-cache-service.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

R3hankhan123 marked this conversation as resolved.
Show resolved Hide resolved
#!/bin/bash
#
# Copyright 2020 The Kubeflow Authors
Expand Down Expand Up @@ -35,11 +36,21 @@ export PATH="$HOME/bin:$PATH"
{
server_version_major_minor=$(kubectl version --output json | jq --raw-output '(.serverVersion.major + "." + .serverVersion.minor)' | tr -d '"+')
stable_build_version=$(curl -s "https://storage.googleapis.com/kubernetes-release/release/stable-${server_version_major_minor}.txt")
kubectl_url="https://storage.googleapis.com/kubernetes-release/release/${stable_build_version}/bin/linux/amd64/kubectl"
if [ "$ARCH" = "amd64" ]; then
kubectl_url="https://storage.googleapis.com/kubernetes-release/release/${stable_build_version}/bin/linux/amd64/kubectl"
elif [ "$ARCH" = "s390x" ]; then #support for s390x
kubectl_url="https://storage.googleapis.com/kubernetes-release/release/${stable_build_version}/bin/linux/s390x/kubectl"
elif [ "$ARCH" = "ppc64le" ]; then # support for ppc64le
kubectl_url="https://storage.googleapis.com/kubernetes-release/release/${stable_build_version}/bin/linux/ppc64le/kubectl"
else
echo "Unsupported architecture: $ARCH"
exit 1
fi
curl -L -o "$HOME/bin/kubectl" "$kubectl_url"
chmod +x "$HOME/bin/kubectl"
} || true


R3hankhan123 marked this conversation as resolved.
Show resolved Hide resolved
# This should fail if there are connectivity problems
# Gotcha: Listing all objects requires list permission,
# but when listing a single oblect kubectl will fail if it's not found
Expand Down
3 changes: 2 additions & 1 deletion third_party/minio/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ RUN sh /third_party/download_source.sh </third_party/minio/repo-MPL.txt


# Minio image
FROM minio/minio:RELEASE.2019-08-14T20-37-41Z
# Minio image compatible with s390x and ppc64le
R3hankhan123 marked this conversation as resolved.
Show resolved Hide resolved
FROM minio/minio:RELEASE.2020-12-18T03-27-42Z
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this required for this PR?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HumairAK the image used here is published few years back. We enabled the s390x and ppc64le builds later after this image being build. Hence we are requesting to update this. You can see from tag, the image used is released in 2019 and we are having image supported from 2020.


# Copy concatenated license file
COPY third_party/minio/license.txt /third_party/license.txt
Expand Down
Loading