Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new basic_cas that resembles production setup #1122

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

blakehatch
Copy link
Member

@blakehatch blakehatch commented Jul 9, 2024

Description

This variant of basic_cas is based on a production setup combined with the convience of using basic_cas locally as an all-in-one CAS, AC, and Worker,

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

This change is Reviewable

@blakehatch blakehatch requested a review from allada July 9, 2024 07:07
Copy link
Member

@allada allada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: 0 of 1 LGTMs obtained, and 1 discussions need to be resolved


nativelink-config/examples/basic_cas_recommended.json line 39 at r1 (raw file):

      }
    },
    "CAS_FAST_SLOW_STORE": {

nit: I don't think this will work. If used as a worker store, it needs to have the fast store be a filesystem and the slow store can be whatever.

What you actually want is to split this into a few different stores and use different ones in different places.

The CAS should use fast_slow { fast -> size_partitioning { lower -> memory, upper -> noop }, slow -> filesystem }

The worker should use: fast_slow { fast -> filesystem, slow -> noop } (but put comment that noop should be replaced with a remote storage store (S3, GRPC, Redis, exc...).

The filesystem stores both are referencing should be the same store.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants