CoSimulation Application is too slow

[mnabideltares]

Dear commity,

We built thermal elements in GeoMechanicsApplication. The thermal element interact with water-pressure element by CoSim. So, actually we use cosim to run 2 simulations of GeoMechanics, one is for water pressure, and the other is for thermal.

We have exactly same grid geometry. All the node coordinates and element geometries are identical. So, we expect the simulation time to be the sum of simulation times for pressure and thermal cases, with a small extra fraction as communication time.

However, the simulation take far more long than expected. It may take several times longer, or sometimes even more. It may reach to 10's time langer. We think (if we are correct) it may come from a searching algorithm in CoSim or Mapper as it needs to find the nodes of those two seperate runs (pressure and thermal) and interpolate the nodal variables (in our case, we use nearest neighbour).

As our meshes for pressure and thermal are identical, is it possible to avoid the interpolation phase and use a direct mapping of nodes?

I thank you in advance

Best regards
Mohamed Nabi
Deltares

[loumalouomega]

Did you try to run it witha profiler to see what is taking most of the time?, Intel VTune is free, and it is the tool we ussually consider in our performance analyses.

[mnabideltares]

@loumalouomega Thank you for your suggestion. I still didn't run it with VTune. I am going to do it.

[philbucher]

that sounds like sth is wrong with the setup. I tested it with really big problems, and CoSim was never the issue
In particular the mapper should be super fast, I have been optimizing it for years

Increasing the echo-level should print some timings, those should give some first answers

Nevertheless I might be wrong, maybe I screwed up sth.

Can you post some characteristics of the problem, or maybe a full example setup?

[loumalouomega]

I know that this accelerator was reimplementd in C++ by @rubenzorrilla, but also I think is not compatible with CoSim. I also know that there is a version that allows to be used in large system because otherwise it takes to much memmory. Also developed adn implemented by @rubenzorrilla .

[mnabideltares]

@loumalouomega
Thank you for the fast reply.
So, do you suggest to comment it out always? Or is there any other accelerator type which may help optimizing the mapping in CoSim?

[loumalouomega]

I don't know details about accelerators. I know MVQN because it was the subject of the master thesis of @rubenzorrilla when I started my PhD and I remember how he implemented the algorith originally and the problems they found, like speed and memmory. I don't know more details, particularly for CoSim app

[rubenzorrilla]

Just pointing that the Python version in the CoSimulationApplication is a reimplementation of my C++ one. Nevertheless, I don't think there is any error related to such reimplementation as it has been actively used. If the number of interface DOFs is large, I'd suggest to avoid its usage. The point is that it requires storing and operating with a dense square matrix. Though this makes possible to approximate the previous iteration Jacobian without any user-definable parameter, it results in a quadratic computational cost.

[mnabideltares]

@rubenzorrilla , Thank you for the clarification.

We also believe there is no error inside, but it may be problem-dependent. Based on the physical properties of a problem, methods may show different behavior. We are using it for GeoMechanicsApplication to couple thermal and pressure processes. It is an imbalanced problem, namely, it is strongly non-linear for the temperature part but weakly non-linear for pressure. Even in our applications, this shows different performance based on the test case.