-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a slurm workflow manager #789
Open
linsword13
wants to merge
16
commits into
GoogleCloudPlatform:develop
Choose a base branch
from
linsword13:workflow
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+772
−20
Open
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
1feb5e2
Scaffolding for a new workflow_manager object type
linsword13 f81ed19
Add a slurm workflow object
linsword13 1c026e5
Allow for injecting arbitrary sbatch headers
linsword13 bef1fb0
Add workflow_manager_variable directive
linsword13 7b37d6b
Add test
linsword13 4047b45
Add a slurm command runner
linsword13 7ca416d
Improve on test
linsword13 d2f4651
Improve on the query script
linsword13 c6c8c96
Add initial support for setting status from workflow_manager
linsword13 e86f832
Add wm directive info
linsword13 ab8443e
Add a hostlist workflow manager variable
linsword13 2e3ad95
Move conditional expand logic to base class
linsword13 94e5b98
Rename query and cancel script names
linsword13 ee99c5b
Suppress squeue stderr
linsword13 aff3677
Include job name in status print
linsword13 c06088d
Move status checking logic to python (for analyze only)
linsword13 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Add initial support for setting status from workflow_manager
commit c6c8c9698e087f84badd66b9537636de80fff5b8
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,3 +19,5 @@ | |
CommandRunner, | ||
RunnerError, | ||
) | ||
|
||
from ramble.util.logger import logger |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure how picky I am about this, but I would almost prefer the status map be in python rather than in bash.
And maybe some of the logic mapped into the base class too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call. I moved the get_status call inside the runner to be in python instead of relying on this bash script, so now
ramble workspace analyze
would rely on python. Do you prefer even the--executor "{batch_query}"
to be handled in python instead of the bash script right now?