Any suggestion on a local OCI proxy in orchard cluster

[eecsmap]

This is not directly related to Orchard itself. But I assume you guys have similar consideration already and probably have some best practice. So allow me to ask here to get some suggestion.

I have set up four mac studio as a Orchard cluster in my local high bandwidth network, But all my images are hosted in a remote JFrog based OCI registry. Whenever each worker needs a new image, they all pull from the remote, which is slow and unnecessary.
I want to setup a local pass through proxy which simply cache the remote OCI content, and point all my orchard workers to it.

Since the proxy is local, and all workers are 10Gb/s ethernet enabled. This should save a lot network data request to the remote and way faster when new image published.

If you guys have something work as so, please share some ideas. Thanks!

[fkorotkov]

Right now we struggle with the same issue. When an update takes a little while to download. In our case we use a hosted registry which can handle parallel downloads from a lot of hosts without performance degradation but it's still not 10Gb/s.

We plan to address it at some point in the future but at the moment it's not the highest priority. I think your idea with a local proxy will work for 4 workers but won't scale beyond 10 when network to the single proxy will become a bottleneck.

[eecsmap]

now: N multiple workers -> gateway -> remote registry
If all workers need to pull M GB data, it generates N X M GB data on the link from gateway to remote registry.
with local proxy: N multiple workers -> local proxy -> gateway -> remote registry
It generate M GB data on the link local proxy -> gateway -> remote registry, which scales to 1/N in data and time.
And you are right, if we only have one central local proxy, the stress goes to it now since every workers pulls data from it. But this is handled by the local switch with no problem.
I've looked around, and seems there are some candidates in the field. I will give them a try and come back if anything interesting.
Thanks @fkorotkov

[fkorotkov]

On that note, many years ago we used Anka with their local registry and it didn't scale beyond 10. I've created cirruslabs/tart#814 to investigate incremental pulls. PTAL if you have the same scenario and maybe we'll be able to improve this scenario without a local proxy.

[nocsi]

You can run zot registry locally and keep it synced up against another upstream instance.. while it proxies to other registries. There’s a few examples of configuring it to be a caching oci registry in their docs. You’ll probably want to tuck the registry behind haproxy or something similar, don’t let clients directly reach out to the oci registries