Peak Memory Usage #5116
Replies: 1 comment 8 replies
-
|
The execution report will give you the peak mem usage ( |
Beta Was this translation helpful? Give feedback.
-
|
Please correct me if i am missing something |
Beta Was this translation helpful? Give feedback.
-
|
As long as there is enough memory for the largest task, the pipeline will work. But if you want to know how much memory you need to run the pipeline with the highest level of parallelism, that is a much more complicated question that depends on the pipeline structure. I would render the pipeline DAG and inspect it manually to get a sense for where the "widest" part of your pipeline is in terms of memory usage. |
Beta Was this translation helpful? Give feedback.
-
|
Thats a good suggestion, but the DAG for the rnaseq pipeline that I am running is massive. I will try. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
Yes, it might not be feasible if the workflow is too large. There are some config options to control the level of detail for the DAG which might help a little. But the bigger problem I see is, for a sufficiently large workflow, even if you find the maximum possible amount of memory that could be used at once, it could vary greatly based on the order in which tasks are executed, and it might not ever even reach that theoretical maximum. So you might end up over-provisioning by taking this analytical approach. You might be better off inspecting the waterfall for a few small runs and finding some heuristic. People usually get around this problem with auto-scaling in the cloud. But I assume you are preparing some fixed allocation of compute nodes. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, the order is very important for this problem and the maximum cant be 100% correct. |
Beta Was this translation helpful? Give feedback.
-
Is there a way to find out what was the peak of memory (ram) usage of a pipeline?
Beta Was this translation helpful? Give feedback.
All reactions