[Refactor] Move scheduler state behind mutex #1062
Conversation
There was a problem hiding this comment.
Reviewed 12 of 13 files at r1, all commit messages.
Reviewable status: 0 of 1 LGTMs obtained, and pending CI: pre-commit-checks
allada
force-pushed
the
move-scheduler-state-behind-mutex
branch
from
b3b9128
to
5dd9dec
Compare
Refactoring to include the `awaited_action` watch channel `notify_channel: watch::Sender<Arc<ActionState>>` to be provided via `MatchingEngineActionStateResult`. This aids the matching engine's need for access to `OperationId`.
There was a problem hiding this comment.
Reviewed 1 of 13 files at r1.
Reviewable status: 1 of 1 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Installation / macos-13, Installation / macos-14, Installation / ubuntu-22.04, Local / ubuntu-22.04, Publish image, Publish nativelink-worker-init, Publish nativelink-worker-lre-cc, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, ubuntu-20.04 / stable, ubuntu-22.04, ubuntu-22.04 / stable, windows-2022 / stable
In prep to support a distributed/redis scheduler, prepare the state interface to no longer take mutable references. This is a partial PR and should be landed immediately with followup PRs that will remove many of the locking in the SimpleScheduler. towards: TraceMachina#359
allada
force-pushed
the
move-scheduler-state-behind-mutex
branch
from
5dd9dec
to
be1e41a
Compare
There was a problem hiding this comment.
Reviewed 14 of 14 files at r2.
Reviewable status:complete! 1 of 1 LGTMs obtained
There was a problem hiding this comment.
Reviewable status: 1 of 1 LGTMs obtained, and 1 discussions need to be resolved
nativelink-scheduler/src/simple_scheduler.rs line 109 at r3 (raw file):
&self, unique_qualifier: &ActionInfoHashKey, ) -> Result<Option<watch::Receiver<Arc<ActionState>>>, Error> {
Would it be ok to just do an error event here detailing to show how and why the check failed and then just return None instead of propagating and handling it all the way up the call stack?
An alternative could be to return just a Result instead of a Result<Option> and have the Error code be ActionNotFound or something.
Either way works but I think wrapping this twice is overkill.
In prep to support a distributed/redis scheduler, prepare the state interface to no longer take mutable references.
This is a partial PR and should be landed immediately with follow up PRs that will remove many of the locking in the SimpleScheduler.
towards: #359
This change is