Incorporating IMU data

[robinlindh]

Hi,

Thanks for a great project and also one of the few times I get an easy demo colab notebook which just works <3

I am running the reconstruction successfully on my own dataset which is a sequence of images, and already getting fairly good results. However, I'd like to explore adding additional constraints based on IMU data, to get more robustness with challenging data like visual symmetry or segments with few visual features.

I already have approximate relative pose between the images based on IMU which are fairly accurate, so I was thinking to somehow incorporate that knowledge as an extra constraint in the bundle adjustment. For example either as a hard constraint on the relative pose < twice the prior known offset. Or as an extra penalty in the cost function if distance. This is common in other visual-inertial approaches to bundle adjustment in SLAM for example.

Any pointers to where in the code I should look, or other advice?
Thanks a lot, and if this works well I might also submit it as a PR.