Add binary testing for arm64 platform via Github Actions #4591
Comments
|
Better test coverage for Linux/arm64 (and macos/arm64) would be desirable! I've tried some stuff using the binfmt ext stuff earlier, but hit a blocker that led us to only provide Linux/arm64 docker images with our static binary ... bytecodealliance/wasmtime-go#119 How does worksonarm work? Are there publicly accessible workers? |
|
Btw -- I was brief because on mobile for the other message -- any improvements to CI that increase our confidence in the arm64 builds, or just bring them on par with the amd64 builds, are very welcome. 👍 |
|
Hi @srenatus, Works on Arm is an initiative that provides free access to Arm based clusters (on Equinix). This is for the purpose of giving open source projects access to Arm HW to support build-test loops. To apply for access, you just have to open an issue with the project (linked below). Given this is a CNCF project, I would imagine it would get accepted. https://github.com/WorksOnArm/equinix-metal-arm64-cluster Keep in mind, this is for access to Arm based HW. The actual maintenance of the build systems deployed on the Arm cluster would still need to be done by the project. If you decide to apply, let us know so we can push it along (I'm an Arm employee by the way, which is separate from Equinix). |
|
Hi @srenatus I’m testing open policy agent on arm64 platform on kubernetes using this documentation. But I’m failing on this step because But when I'm using this image, opa pods are crashing with below logs. Can you please provide us with some pointers on this problem? |
|
Hmm those logs don't contain anything helpful, do they? As you've noticed, we don't publish the combination of rootless + static; static being the only thing we build for linux/arm64. I think the quickest way forward is to build your own rootless container using the static binary, something along the lines of FROM gcr.io/distroless/cc
USER 1000:1000
# TARGETOS and TARGETARCH are automatic platform args injected by BuildKit
# https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
ARG TARGETOS
ARG TARGETARCH
COPY --from=openpolicyagent/opa:latest-static /opa /opa
ENTRYPOINT ["/opa"]
CMD ["run"]where we're basically snatching the static arm binary from the published |
|
To build the binary yourself, something along the lines of what happens in CI is probably the way to go: https://github.com/open-policy-agent/opa/blob/main/.github/workflows/post-merge.yaml#L95-L100 |
|
Just to add to that: all images (including -static) will be rootless by default later this year. |
I have tried this dockerfile that you have shared. It is successfully working for me on arm64 platform.
Can you please let me know that for future release of docker image which binary are you going to use static binary or non-static binary? Please let me know I’m ready to help. |
|
I think we'll continue to have both the static and the non-static images around for the near future. I don't think the Wasm stuff works in the non-static context still, but perhaps @srenatus knows if this is about to change. |
Yay. So I guess it comes down to this: Is the workaround too cumbersome to be acceptable until all images switch to rootless?
We'll probably keep static and "default" (non-static, dynamic build). The latter is able to run wasm modules, the former is not. The only planned change is to collapse the "rootless" variant with the "default" variants, i.e. go from
to
and having them all be rootless. The reason we're not having "default" (non-static) rootless images for linux/arm64 is that we couldn't properly test them (i.e. basically this issue). Using the binfmt_ext emulation led to bytecodealliance/wasmtime-go#119. So, if we had sorted out a way to properly test the linux/arm64 binaries (or run the golang tests for that matter) on linux/arm64 including the pieces that need wasm libs, we'd be able to build the default (and default-rootless) variants for linux/arm64. At this point, however, we haven't planned to take on the work required to set up a self-hosted GHA runner on the equinix-hosted hardware, even if they accepted our proposal. I don't think this is going to be a onetime thing, these runners will need to be maintained, and cared for... 🤔 |
|
@srenatus, just to be clear,works on Arm would allow for use to run and maintain the runners for more than just a one time thing. |
|
@jsrz sorry if my statement wasn't worded properly -- I think I understand that the works-on-arm stuff would give us shell access to some Arm hardware, and it would be our task to set that up as a github actions runner. By "not a one time thing" I was referring to the work involved in maintaining that setup. That won't be a do-and-forget kind of thing, there are going to be updates to do, etc. Am I mistaken? 😃 |
|
@srenatus got, I understand. You are not mistaken, it would certainly need maintenance. |
|
This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. |
|
I don't think this is likely to be something we'll add anytime soon, so let's revisit this in the future if we feel it's needed. Closing for now. |
|
Hi Team, I have successfully tested the opa binary for arm64 platform using opa-wasm-builder docker image. I have created opa-wasm-builder docker image using this README.md file for arm64 platform. Using this image, I have successfully built the opa and opa-static binaries. Also, I successfully tested both the binaries for the arm64 platform. I have used below mentioned command to test the build and test the binary: -
May I know which test you are running which leads you to this error? |
Sure! I think it's been the wasm e2e tests: Line 144 in c54ddd3 |
|
For completeness' sake, most our tests are Golang unit tests: the binary smoke tests only assert that the binary does something; as opposed to failing to find some lib or such a thing. We'd need to pass |
|
Hi @srenatus I have run the test that you have suggested and both of the tests successfully passed on the arm64 platform using |
|
Thanks a lot @odidev. This is certainly reassuring, but we can't expect you to manually run these things before each release 😅 Is there any way to run the tests on arm64 without maintaining manual GHA runners? Is there anything we could "plug in" and forget...? 🤔 |
Hi Team,
I am working with open-policy-agent/opa on the Linux/ARM64 platform.
We know that open-policy-agent/opa already supports Linux/ARM64 platform, and releasing the ARM64 binaries as well using the cross-compilation.
But I can see in Github Actions that the AMD64 binaries are also tested before releasing.
May I know, are you also planning to include the testing of Linux/ARM64 binaries before releasing, via Github Actions, in the future?
If interested there are machines hosted through the works on arm (https://github.com/WorksOnArm), for arm64 platform.
Please share your opinion on the same. If required, I will be happy to contribute.
The text was updated successfully, but these errors were encountered: