Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add package-based soroban-rpc Dockerfile #710

Merged
merged 5 commits into from
Jun 22, 2023
Merged

Conversation

jacekn
Copy link
Contributor

@jacekn jacekn commented Jun 20, 2023

What

This PR adds new soroban-rpc Dockerfile that uses official SDF deb packages as source of stellar-soroban-rpc and stellar-core

Why

This Dockerfile will allow us to reuse release packages. It will also allow us to run more lighweight soroban-rpc deployments than those using the stellar/quickstart image

### What

This PR adds new soroban-rpc Dockerfile that uses official
SDF deb packages as source of stellar-soroban-rpc and stellar-core

### Why

This Dockerfile will allow us to reuse release packages.
It will also allow us to run more lighweight soroban-rpc deployments
than those using the stellar/quickstart image
@jacekn jacekn requested a review from sreuland June 20, 2023 13:28
cmd/soroban-rpc/docker/Makefile Outdated Show resolved Hide resolved
@leighmcculloch
Copy link
Member

leighmcculloch commented Jun 20, 2023

It will also allow us to run more lighweight soroban-rpc deployments than those using the stellar/quickstart image

It's possible for us to create a docker container that isn't quickstart, without introducing .deb installers into the mix. There have been some challenges with building our docker contains this way (how horizon's was built):

Do we need the debs, or do we just need a docker image?

@tsachiherman
Copy link
Contributor

I agree with @leighmcculloch 's point here. The fact that we're tying our docker image generation here to a prebuilt debian release ( from jenkins ) is a serious limitation. When building from source, we need always build from... source.

@sreuland
Copy link
Contributor

I agree with @leighmcculloch 's point here. The fact that we're tying our docker image generation here to a prebuilt debian release ( from jenkins ) is a serious limitation. When building from source, we need always build from... source.

yes, 'build from source' has become an approach used in other places to enable multi-arch situations, quickstart does it and system-test followed suit, but it has introduced an unintended side effect, it requires compute resources and time to compile core from source. one other idea getting some discussion was pre-built multi-arch docker images for core, which has caveats also, like which repos can push them and which are most common permutation of build flags to build images on, but, if that existed, then docker's like this rpc one can safely copy the core bin from that image as a build stage.

* specify amd64 platform explicitly as we currently only support amd64 core
* make STELLAR_CORE_VERSION required to avoid hardcoding versions in the files.
  We will provide version from a dropdown in the build job
* if soroban-rpc version is not provided read it from GitHub and install latest
  package matching the version
@jacekn
Copy link
Contributor Author

jacekn commented Jun 21, 2023

Do we need the debs, or do we just need a docker image?

We wanted to build images that use supported, tested version of core for production use. Currently we only test and validate amd64 builds of core. amd64 debs are also the only builds that can be cryptographically verified to be coming from SDF. This is a well established release process that we've been using for many years so moving to custom compiled core binaries for soroban-rpc images is a fairly big deal IMO.
I appreciate that the build process is more limited but that's, unfortunately, the price of having higher security guarantees and thorough testing that those builds go through. Maybe there will be ways to integrate things more for easier testing in GHA?

Having said that it looks like we really need to bump priority of arm64 support. It likely won't matter much for production use cases, where most systems run amd64, but being able to run the images on arm laptops would benefit developers. This can be a significant project of course, depends on how much testing infrastructure we need to duplicate on arm64 architecture.

@jacekn jacekn marked this pull request as ready for review June 22, 2023 09:06
@jacekn jacekn merged commit 94af5fa into stellar:main Jun 22, 2023
@jacekn jacekn deleted the docker branch June 22, 2023 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants