How to increase output depth map resolution?

[m0o0scar]

Hi. Great work and thanks for sharing the colab notebook! I noticed that the on the colab notebook, output depth map appears to be of relatively low resolution compared to the input image, which could result in some fuzziness. As such, I am curious to know whether there is a way to increase the output depth map resolution.

If it is not too much trouble, could you kindly point me to the relevant section of the notebook where I can make the necessary changes? I would be incredibly grateful for your help in this regard :)

[talasalim]

I also used the notebook, do you know how we can add code to extract the metric depth at each pixel?

[razorx89]

I am also curious how we could improve on the depth map resolution. I think the current pre-trained models won't be able to generate higher resolutions. However, are there any plans to use backbones with larger image sizes? Or are there any tricks to generate e.g. a sliding window inference with recalibration based on other predictions? For example, I cropped a high resolution image into multiple non-overlapping images and stitched the resulting depth maps together, but the boundary artifacts are clearly visible and there are also some depth inconsistencies between the individual crops.

A hierarchical approach could be the way to go, using ZoeDepth for depth estimation at the full image but coarse resolution and afterwards using this as an additional input to give guidance to a decoder which predicts on a fine resolution crop.

[m0o0scar]

Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging – Yağız Aksoy this research looks like a good place to start @razorx89

[BillFSmith]

The https://github.com/compphoto/BoostYourOwnDepth work is great and can be used with ZoeDepth, but I found it gave banding, which shows up as pixelation in the depth axis when converting the depth map to a 3d model.

[BillFSmith]

I've written a proof of principle colab script to increase the apparent output resolution from ZoeDepth (which already gives amazing results). An example is shown below

Original:

High resolution using tiling and combining:

[shariqfarooq123]

This looks great! Thanks for sharing your work!

[m0o0scar]

This is amazing!!

[razorx89]

Great work, thank you! 👍

[kwea123]

the idea is good, but it becomes non-metric after your operations.

[codikzng]

Great work! Is this what you came up with, or are there any similar research findings?