Heap size larger than resident size?

[andronat]

Hello everyone! I'm amazed with memray! It's an epic tool that I've been using extensively for the last month to benchmark some pytorch models!

I recently stumbled upon the following graph:

I'm not sure how can I interpret the above result. My initial idea was that probably part of the heap has been swapped by the OS, but my RAM is significantly larger than the sizes requested by python. Any pointers on how to understand the results?

[pablogsal]

Thanks for opening this discussion!

Without looking at the code is difficult to tell for sure but this is kind of behaviour normally indicates that your application is requesting a lot of memory that’s not using. When you call malloc() you increase your heap size because you retested the memory but until you actually touch touch that memory by writing something to it, the OS won’t actually give do anything important so your resident size won’t increase.

You can do an easy experiment to see this: with numpy, allocate a big array with numpy.empty() and another one with numpy.zeros(). In the first case your resident size won’t increase but in the second it will (because writing zeros to the array needs the actual memory).

Another possibility is heavy fragmentation of the allocator or preallocation of data structures but that’s more difficult to confirm specially without the original code.