feat: add openziti network extension #454
+180
−0
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # OpenZiti | ||
|
|
||
| Runs OpenZiti's edge tunneler in run-host mode allowing to access talos nodes resources on the overlay network | ||
|
|
||
| https://openziti.io | ||
|
|
||
| https://openziti.io/docs/reference/tunnelers/docker/#use-case-hosting-openziti-services | ||
|
|
||
| ## Installation | ||
|
|
||
| See [Installing Extensions](https://github.com/siderolabs/extensions#installing-extensions). | ||
|
|
||
| ## Usage | ||
|
|
||
| Create an identity for the machine. | ||
| Manually enroll it using(for example using ziti edge enroll ...) and output the enrolled identity into a file or stdout | ||
| Copy the contents of the enrolled identity | ||
| Create `ExtensionServiceConfig` as the example below with replacing `JSON_IDENTITY` with the JSON of the identity you created and in mountPath replace `IDENTITY_FILE_NAME` with the filename of the identity(preserve the json in the filename) | ||
|
|
||
| ```yaml | ||
| --- | ||
| apiVersion: v1alpha1 | ||
| kind: ExtensionServiceConfig | ||
| name: openziti | ||
| configFiles: | ||
| - content: 'JSON_IDENTITY' | ||
| mountPath: /var/lib/ziti/etc/identities/IDENTITY_FILE_NAME.json | ||
| ``` | ||
| Then apply the patch to your node's MachineConfigs | ||
| ```bash | ||
| talosctl patch mc -p @openziti.talos.yaml | ||
| ``` | ||
|
|
||
| You will then be able to verify that it is in place with the following command | ||
| ```bash | ||
| talosctl get extensionserviceconfigs | ||
|
|
||
| NODE NAMESPACE TYPE ID VERSION | ||
| 192.168.10.10 runtime ExtensionServiceConfig openziti 1 | ||
| ``` | ||
|
|
||
| Example of creation of ziti service to serve talos api on the overlay network(note that 192.168.10.10 is the address of the node): | ||
| ``` | ||
| ziti edge create config talosctl-controlplane.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses": ["talosctl-controlplane.ziti.internal"], "portRanges":[{"low": 50000, "high":50000}]}' | ||
| ziti edge create config talosctl-controlplane.host.v1 host.v1 '{"protocol": "tcp","address":"'"192.168.10.10"'", "port": 50000}' | ||
| ziti edge create service talosctrl-controlplane.svc --configs talosctl-controlplane.intercept.v1,talosctl-controlplane.host.v1 | ||
| ziti edge create service-policy talosctl-controlplane.policy.dial Dial --service-roles "@talosctrl-controlplane.svc" --identity-roles "@macos" | ||
| ziti edge create service-policy talosctl-controlplane.policy.bind Bind --service-roles "@talosctrl-controlplane.svc" --identity-roles "@talos-cluster-test-identity" | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| version: v1alpha1 | ||
| metadata: | ||
| name: openziti | ||
| version: "$VERSION" | ||
| author: Ruslan Chernenko | ||
| description: | | ||
| OpenZiti is the world’s most used and widely integrated open source secure networking platform. | ||
| This extension deploys a ziti-edge-tunneler in run-host mode which allows to access node's net resources | ||
| through the overlay network. | ||
| compatibility: | ||
| talos: | ||
| version: ">= v1.7.0" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| name: openziti | ||
| depends: | ||
| - service: cri | ||
| - network: | ||
| - addresses | ||
| - connectivity | ||
| - etcfiles | ||
| - configuration: true | ||
| container: | ||
| entrypoint: /usr/local/bin/ziti-edge-tunnel | ||
| args: | ||
| - run-host | ||
| - --verbose=4 | ||
| - --identity-dir=/var/lib/ziti/etc/identities | ||
| security: | ||
| writeableRootfs: false | ||
| writeableSysfs: true | ||
| mounts: | ||
| # libssl lib | ||
| - source: /usr/lib | ||
| destination: /usr/lib | ||
| type: bind | ||
| options: | ||
| - bind | ||
| - ro | ||
| - source: /sbin | ||
| destination: /sbin | ||
| type: bind | ||
| options: | ||
| - bind | ||
| - ro | ||
| - source: /dev/net/tun | ||
| destination: /dev/net/tun | ||
| type: bind | ||
| options: | ||
| - bind | ||
| - rw | ||
| - source: /var/lib/ziti/etc/identities/ | ||
| destination: /var/lib/ziti/etc/identities/ | ||
| type: bind | ||
| options: | ||
| - bind | ||
| - rw | ||
| - source: /var/lib/ziti/ | ||
| destination: /var/lib/ziti/ | ||
| type: bind | ||
| options: | ||
| - bind | ||
| - rw | ||
| restart: always |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| name: openziti | ||
| variant: alpine | ||
| shell: /toolchain/bin/bash | ||
| install: | ||
| - unzip | ||
| - zip | ||
| - ninja-build | ||
| - ninja | ||
| - zlib-static | ||
| dependencies: | ||
| - image: "ghcr.io/siderolabs/tools:v1.7.0-4-gc844dc3" | ||
| steps: | ||
| - sources: | ||
| - url: https://github.com/openziti/ziti-tunnel-sdk-c/archive/refs/tags/{{ .OPENZITI_TUNNELER_VERSION }}.tar.gz | ||
| destination: edge-tunnel-source.tar.gz | ||
| sha256: d455672bf3b6ff28fd0ff864f868f7d6e3de99e6c666a120227fa9dab2d91f78 | ||
| sha512: 5aeb05347381124e9d75693f12e0d234499a8c6482d322019576a24c622d986d43f159454a43c844623f171a456062a5e1afffcee7f7cbb03740216dbd3bb74c | ||
| env: | ||
| VCPKG_ROOT: /vcpkg | ||
| VCPKG_FORCE_SYSTEM_BINARIES: 1 | ||
| CC: /toolchain/bin/gcc | ||
| CXX: /toolchain/bin/g++ | ||
| prepare: | ||
| - | | ||
| # vcpkg depends on git as a delivery tool | ||
| # if to download .tar.gz from releases it would fail saying it cannot retrieve versions/baseline.json version using git show | ||
| # tried my best to find any CMAKE variable within vcpkg which disables this behaviour | ||
| # we fetch here commit to which tag 2024.07.12 is assigned | ||
| git clone https://github.com/microsoft/vcpkg.git /vcpkg | ||
| git --git-dir=/vcpkg/.git --work-tree=/vcpkg checkout 1de2026f28ead93ff1773e6e680387643e914ea1 | ||
| - | | ||
| sed -i 's#$VERSION#{{ .VERSION }}#' /pkg/manifest.yaml | ||
| - | | ||
| mkdir -p /vcpkg-git /vcpkg /ziti-tunnel-sdk-c/build | ||
|
There was a problem hiding this comment. why are we building an sdk? doesn't make sense, shouldn't it be using this https://github.com/openziti/ziti/? There was a problem hiding this comment.
No we shouldn't. OpenZiti tunneler is exactly for hosting or tunneling. Main ziti repo is for the other part of the OpenZiti project which provides a controller, router and etc. There was a problem hiding this comment. Tunneler is just a daemon that runs on the host machines and allows to put the network communication on or from the overlay network it creates. For Talos its impossible to change routing from the extension, so the tunneler runs in a 'run-host' mode which allows offloading of a overlay traffic on the underlay network. |
||
| tar -xzvf edge-tunnel-source.tar.gz --strip-components=1 -C /ziti-tunnel-sdk-c | ||
| build: | ||
| - | | ||
| export PATH=${PATH}:/toolchain/bin | ||
| /vcpkg/bootstrap-vcpkg.sh | ||
| - | | ||
| export PATH=${PATH}:/toolchain/bin | ||
| PRESET="ci-linux-x64" | ||
| if [[ "$(uname -m)" == "arm64" || "$(uname -m)" == "aarch64" ]]; then | ||
| PRESET="ci-linux-arm64" | ||
| ln -s /toolchain/bin/g++ /toolchain/bin/aarch64-linux-gnu-g++ | ||
| ln -s /toolchain/bin/gcc /toolchain/bin/aarch64-linux-gnu-gcc | ||
| fi | ||
| cmake -DBUILD_SHARED_LIBS=OFF -DCMAKE_EXE_LINKER_FLAGS="-static" -DDISABLE_SEMVER_VERIFICATION=ON -DDISABLE_LIBSYSTEMD_FEATURE=ON -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja --preset $PRESET -S /ziti-tunnel-sdk-c -B /ziti-tunnel-sdk-c/build | ||
| - | | ||
| cmake --build /ziti-tunnel-sdk-c/build --config Release | ||
| install: | ||
| - | | ||
| mkdir -p /rootfs/usr/local/lib/containers/openziti/usr/local/bin/ | ||
| mv /ziti-tunnel-sdk-c/build/programs/ziti-edge-tunnel/Release/ziti-edge-tunnel /rootfs/usr/local/lib/containers/openziti/usr/local/bin/ | ||
| finalize: | ||
| - from: /rootfs | ||
| to: /rootfs | ||
| - from: /pkg/manifest.yaml | ||
| to: / | ||
| - from: /pkg/openziti.yaml | ||
| to: /rootfs/usr/local/etc/containers/ | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| VERSION: "{{ .OPENZITI_TUNNELER_VERSION }}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,3 @@ | ||
| # renovate: datasource=github-releases extractVersion=^v(?<version>.*)$ depName=tailscale/tailscale | ||
| TAILSCALE_VERSION: 1.70.0 | ||
| OPENZITI_TUNNELER_VERSION: v1.1.2 |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this looks a bit strange... we usually build from
baselayer which contains toolchain (tools) and don't use alpine for the build. is there anything missing?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no zlib-static, zip, unzip when using scratch and its impossible to use install directive as
apkmanager doesnt exist in `scratch' variant.Vcpkg relies on having zip, unzip as it downloads dependencies in that format.
Zlib-static is required to link an executable against it as
there is no zlib on the host /usr/lib folderthe build process doesn't build zlib from sources and it has to be manually retrieved from the alpine packages repo in this case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
zlib should be available though, zip and unzip, there will be no
apkof course.vcpkgdownloads dependencies? does it pin them in a secure way?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I could just change to using 'base' stage instead of explicitly binding to a tools image here.
Build seems to work fine with that. But idk what to do with those zip, unzip, zlib-static, ninja deps
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes it does by verifying their sha512sum as I skimmed the code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, then this extension needs to add it's build time dependencies as like qemu-guest-agent for example
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess would be hard to do as it heavily depends on vcpkg as the dependency manager.
Maybe I could try working on creating static builds and then pulling them within the build process. Would this be okay?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean pulling them from the releases page, as you suggested above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, let's try static builds from the releases page. how big are those?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, around 5MB, I am currently working on buildin those using gh workflows, just having a linker issue with CMAKE trying to link libpthread.so instead of libpthread.a.
openziti/ziti-tunnel-sdk-c#938