Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-cloud comm optimization #5

Open
camelop opened this issue May 12, 2022 · 0 comments
Open

Multi-cloud comm optimization #5

camelop opened this issue May 12, 2022 · 0 comments
Labels
cl-design New design, research ideas, thoughts, or paper plans. enhancement New feature or request

Comments

@camelop
Copy link

camelop commented May 12, 2022

Migrated from: https://github.com/camelop/dds-dev/issues/17

Idea

When considering decentralized data science in a multi-cloud setting, we might be able to do some system level optimization for existing abstractions. For example, consider two parties running a protocol, party A has a core running on cloud AC with storage on cloud AS1, AS2 and protocol operators running on cloud AP1, AP2; similarly party B has a core on BC, storage backends on BS1, BS2 and protocol operators BP1, BP2.

  1. In normal settings, we have AC=AS1=AS2=AP1=AP2, BC=BS1=BS2=BP1=BP2. In that case, there isn't much need for a multi-cloud optimization. Because if AC=BC, everyone is using the same cloud. If AC!=BC, there's no chance for data/execution migration towards optimization.
  2. However, if party A is a large company that owns abundant cloud resource, it might also be the case that AS1!=AS2 and AS2=BP1 (e.g. party A has both storage on AWS and GCP), with a upgraded core that supports operating across clouds, an internal migration before execution might be helpful to improve performance, because BP1 can directly take data from AS2.
  • It might also be the case that a party C want to provide a service to help A, B to accelerate their protocol execution. Serving as something similar to a proxy, for setting 1, there might be CS1=AS1 and CS2=BS1 which provides an opportunity for tunneling.

Note

@suquark might have some better ideas on this.

@camelop camelop added enhancement New feature or request cl-design New design, research ideas, thoughts, or paper plans. labels May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cl-design New design, research ideas, thoughts, or paper plans. enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant