How to get the correct event position when object_fit='contain'?

[GuillaumeFavelier]

In mne-python, we rely on ipywidgets and ipyvtklink to design apps for MEG and EEG processing and visualization.

AFAIK, ipyvtklink uses ipyevents to get the mouse event locations. Then it does a little bit of offset computation to adjust the given coordinates from the canvas size (which depends on the notebook) to the render window size (which is fixed by the Renderer at init), correct?

Well, in our last application, we embed the ViewInteractiveWidget in a HBox for a new design:

output.mp4

What happens when all the group boxes are folded, the values given by ipyevents for boundingRectHeight and boundingRectTop used for offset calculation are correct because the image and its bounding box have the same size.

But we use viewer.layout.object_fit='contain' to preserve the aspect ratio and when the group boxes are unfolded, the bounding box of the image is stretched with the accordion and now, I think the sizes sent by ipyevents do not match anymore.

This is a problem for us because we need the correct position of the event for picking:

output.mp4

Is there a way to update the offset calculation so that GetEventPosition() returns the correct value when object_fit='contain'?