libBICOS

BInary COrrespondence Search for multi-shot stereo imaging, with optional CUDA acceleration.

Citing:

This is the implementation of the corresponding paper which appeared in Optical 3D Metrology 2024:

@article{liebender2024libbicos,
  title={libBICOS -- An Open Source GPU-Accelerated Library implementing BInary COrrespondence Search for 3D Reconstruction},
  author={Liebender, Christoph and Bleier, Michael and N{\"u}chter, Andreas},
  journal={The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  volume={48},
  pages={57--64},
  year={2024},
  publisher={Copernicus Publications G{\"o}ttingen, Germany}
}

Build:

Dependencies:

• gcc or equivalent C compiler with C++17 support (build)
• meson >= 1.1.0 (build)
• opencv 4.x with cuda support
• cuda 12.x including toolkit
• cog (build, cuda, for generating sources)
• benchmark (optional, for executing benchmarks)

# recommended: clone the most recent release
$ git clone --depth 1 --branch v2.2.0 https://github.com/JMUWRobotics/libBICOS
$ cd libBICOS

$ meson setup builddir --buildtype release

# optional but recommended if you have access to a CUDA-capable GPU
$ meson configure -Dimplementation=cuda builddir

$ meson install -C builddir

The versioning scheme of the library is Semantic Versioning.

Usage

Linking

After installing, you can include /usr/local/include/BICOS/* and link against /usr/local/lib/libBICOS.so.

Python module

With an available python installation, meson will build a (somewhat experimental) python module pybicos. It is a wrapper around the C++ API for more convenient experimentation:

import pybicos
import cv2 as cv

lstack = [cv.imread(f"data/left/{i}.png", cv.IMREAD_UNCHANGED) for i in range(20)]
rstack = [cv.imread(f"data/right/{i}.png", cv.IMREAD_UNCHANGED) for i in range(20)]

cfg = pybicos.Config()
cfg.nxcorr_threshold = 0.9

disparity, correlation_map = pybicos.match(lstack, rstack, cfg)

Commandline-interface

Alternatively, this project builds bicos-cli. To give you a feel for the parameters of BICOS, you can download an example dataset with data/prepare.sh that contains rectified imagery from a stereo camera, where images per side only differ in the projected light pattern. Calling:

$ bicos-cli data/{left,right} -q data/Q.yaml --threshold 0.96 --stacksize 33 --limited --variance 2.0 --step 0.1 -o /tmp/result.png

will get you:

Disparity

Pointcloud

While only requiring ~44ms (RTX4090) and ~1.6GB VRAM for matching on two stacks of 33 images each.

The most significant parameters can be summarized:

• --threshold: increasing this reduces match-outliers. High values rely on a reliable pattern.
• --variance: high values reduce coarse outliers, e.g. pixels where no pattern was projected. May reduce correct matches on low pattern contrast.
• --step: optional value for subpixel-interpolation around matches.
• --lr-maxdiff: use a maximum left-right disparity difference as a postfilter instead of no-duplicates.

Other settings are available; Try bicos-cli -h for details.

Benchmarking:

$ meson test --benchmark -C builddir --interactive

Light projection:

For starters, you may find https://github.com/Sir-Photch/VRRTest useful for projecting a light pattern using an aftermarket projector.

License

This library is licensed under the GNU Lesser General Public License v3.0 (or later). Please see COPYING and COPYING.LESSER for details.