Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document HPC workflows #179

Open
jpdunc23 opened this issue Jan 15, 2024 · 2 comments
Open

Document HPC workflows #179

jpdunc23 opened this issue Jan 15, 2024 · 2 comments
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@jpdunc23
Copy link
Contributor

Create an example using multiple nodes on a Slurm cluster to run a simulation.

@danielinteractive
Copy link

Thanks @jpdunc23 , that would be super helpful. I am currently trying to use future.batchtools with the LSF backend to run some simChef simulations on our HPC, and I am not really sure how to get it right, and if it is possible to asynchronously run the simulations, i.e. start them, and collect results later, without letting the controlling R session run over the whole time period.

@jpdunc23
Copy link
Contributor Author

@danielinteractive Glad you're trying this out! I'll try to take another look at this sometime soon. In the meantime, if you get it working we would greatly appreciate a PR extending the parallelization vignette or a new vignette altogether.

On the other hand, if you just want to get your simulations working as quickly as possible then I suggest dividing the simulation up into logical chunks (e.g., subsets of Method) if possible and run separate single-node LSF jobs for each subset. Please let us know how it goes one way or another.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

2 participants