Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add visionOS deployment example #1073

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

blakehatch
Copy link
Member

@blakehatch blakehatch commented Jul 2, 2024

Description

Adds a deployment example for the visionOS example from rules_apple.

Please delete options that aren't relevant.

  • New feature (non-breaking change which adds functionality)

This change is Reviewable

Copy link
Member

@aaronmondal aaronmondal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great example!

I left some comments explaining why we need certain changes. Since this seems to work on Mac, do you think we should/can add this to CI?

Last time i tried running the K8s setup on mac (two weeks ago or so) it didn't even build. Does it magically fix itself? I think the issues were orignally caused by some rust deps not building on aarch64-linux, but after the many updates we did recently maybe that's why it works now?

If the K8s example works on MacOS now we should definitely add this example to CI to prevent regressions - this stuff is super fragile.

Reviewed 7 of 7 files at r1, all commit messages.
Reviewable status: 0 of 1 LGTMs obtained, and 4 discussions need to be resolved (waiting on @blakehatch)


deployment-examples/visionOS/01_operations.sh line 19 at r1 (raw file):

curl -v \
    -H 'content-Type: application/json' \
    -d '{"flakeOutput": "./src_root#nativelink-worker-siso-visionOS"}' \

This means that you need a flake output called nativelink-worker-siso-visionOS in the flake. See:

siso-chromium = buildImage {

This curl command triggers a tekton pipeline that fetches the image into the cluster and stores it in a cluster-internal OCI registry.


deployment-examples/visionOS/02_application.sh line 16 at r1 (raw file):

resources:
  - worker-visionOS.yaml
EOF

fyi: This creates a kustomization.yaml file in the deployment-examples/visionOS directory.


deployment-examples/visionOS/02_application.sh line 23 at r1 (raw file):

    nativelink-worker-init=localhost:5001/nativelink-worker-init:"$(\
        nix eval .#nativelink-worker-init.imageTag --raw)" \
    nativelink-worker-visionOS=localhost:5001/nativelink-worker-siso-visionOS:"$(\

fyi: This part looks a bit complicated due to the formatting, but it's just a kustomizae edit set image someimage=someotherimagewithatag.

We use nix eval .#nativelink-worker-init.imageTag --raw to get the tag for the image. The name that it has in the cluster (created by the 01 script) is localhost:5001/nativelink-worker-visionOS.

So all this kustomize command does is write the new image names and tags into the kustomization.yaml file that is generated by the cat command above.

Then we apply that kustomization. If you go through the deployment-examples/kuberentes/base directory you'll find that the "container" fields in the manifests are called nativelink. The kustomization.yaml we generate rewrites those nativelink strings to the fully qualified new image name and tag.


deployment-examples/visionOS/README.md line 5 at r1 (raw file):

This deployment sets up a 4-container deployment with separate CAS, scheduler
and worker. Don't use this example deployment in production. It's insecure.

nit: You might want to add a note like This tutorial should be run from within the <url of this file> directory. so that we can reuse this file in the autogenerated docs.

At the moment you'll need to add it in these two places:


deployment-examples/visionOS/README.md line 15 at r1 (raw file):

> 21.0`.
> - You need to install the [Docker](https://docs.docker.com/engine/install/ubuntu/) Engine in Ubuntu.
> - To get your Nix environment set up see the [official Nix installation documentation](https://nix.dev/install-nix).

nit: I'd suggest the experimental nix installer as it already sets up flake support an is known to be more "stable" than the official one: https://github.com/NixOS/experimental-nix-installer. This is the one we also use in our own CI.


deployment-examples/visionOS/README.md line 18 at r1 (raw file):

All commands should be run from nix to ensure all dependencies exist in the environment.
## NativeLink Community

nit: Add blank lines around headings. I wouldn't trust our doc generators "too" much 😅


deployment-examples/visionOS/worker-visionOS.yaml line 28 at r1 (raw file):

        - name: nativelink-worker-visionOS
          # This image will be edited by kustomize.
          image: nativelink-worker-visionOS

fyi: This is also changed by kustomize when applying the generated file. Same with the nativelink-worker-init above.

Copy link
Member

@aaronmondal aaronmondal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Looks like my fyi comments got autominimized - might need to expand those in the Reviewable ui)

Reviewable status: 0 of 1 LGTMs obtained, and 4 discussions need to be resolved (waiting on @blakehatch)

Copy link
Collaborator

@MarcusSorealheis MarcusSorealheis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After you address Aaron’s comments
:lgtm:

Reviewable status: 0 of 1 LGTMs obtained, and 4 discussions need to be resolved (waiting on @blakehatch)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants