Stereo matching

Author: Tobit Flatscher (January 2020)

Overview

Left image	Right image	Depth image

This small tool is a manual implementation of simple stereo-matching in Python 3. Two rectified images taken from different views are combined to a depth image by means of two matching algorithms:

• a simple winner-takes-it-all (WTA) or
• a more sophisticated semi-global matching (SGM)

with several matching costs:

• Sum of Absolute Differences (SAD),
• Sum of Squared Differences (SSD) or
• Normalized Cross-Correlation (NCC).

The results are compared to a ground-truth using the accX accuracy measure excluding occluded pixels with a mask.

For the precise details of the involved formulas (matching cost, matching algorithms and accuracy measure) refer to doc/Theory.pdf.

The repository is structured as follows:

.
├── data/                  # Directory for the input images (left and right eye)
├── doc/                   # Further documentation, in particular the computational approach
├── docker/                # contains a Docker container as well as a Docker-Compose configuration file
├── output/                # contains the resulting depth-image output
├── src/
│   ├── main.ipynb         # The Jupyter notebook that allows a convenient access to the underlying Python functions
│   └── stereo_matching.py # The Python3 implementation of the core functions with Scipy, Scimage, Numba, Numpy and Matplotlib
├── test/                  # contains parametrized unit tests for the implementations
├── .devcontainer/         # contains configuration files for containers in Visual Studio Code
└── .vscode/               # contains configuration files for Visual Studio Code

1. Download it

Either download and copy this folder manually or directly clone this repository by typing

$ git clone https://github.com/2b-t/stereo-matching.git

2. Launch it

Now you have two options for launching the code. Either you can install all libraries on your system and launch the code there or you can use the Docker container located in docker/.

2.1 On your system

For launching the code directly on your system make sure Scipy, Numba, Numpy and potentially also Jupyter are installed on your system by typing. If they are not installed on your system yet, install them - ideally with Anaconda or use the supplied Docker as described below.

2.1.1 Jupyter notebook

For debugging purposes it can be pretty helpful to launch the Jupyter notebook by typing

$ jupyter notebook

Browse and open the Jupyter notebook src/main.ipynb and run it by pressing the play-button.

2.1.2 Command line interface

Alternatively you can also edit the Python-file src/main.py in your editor of choice (e.g. Visual Studio Code) and launch it from there or from the console. When launching it with $ python3 main.py -h it will tell you the available options that you can set.

2.1.3 Library

Finally you can also use this package as a library. For this purpose have a look at src/main.py, src/main.ipynb as well as at the unit tests located in test/ for a reference.

2.2 Run from Docker

This is discussed in detail in the document doc/Docker.md.