feature-extraction-opencl/README.md

Feature Extraction & Sparse Image Alignment with OpenCL

A GPU-accelerated Visual Inertial Odometry (VIO) system implementing sparse image alignment and feature detection algorithms using OpenCL.

This project is based on the SVO (Semi-direct Visual Odometry) framework from the Robotics and Perception Group at the University of Zurich, adapted for GPU acceleration using OpenCL.

Features

• GPU-Accelerated Image Processing: Utilizes OpenCL for parallel computation of image features and alignment
• FAST Corner Detection: GPU-optimized FAST (Features from Accelerated Segment Test) algorithm for corner detection
• FREAK Descriptors: GPU implementation of FREAK (Fast Retina Keypoint) feature descriptors
• Sparse Image Alignment: Photometric error minimization using Gauss-Newton optimization
• Pose Optimization: SE(2) transformation estimation for camera pose refinement
• Multi-level Pyramid Processing: Coarse-to-fine image pyramid for robust alignment

Requirements

System Dependencies

• CMake >= 3.7
• C++14 compiler (g++, clang, or MSVC)
• OpenCL headers and libraries
• Intel GPU drivers (for Intel GPU support) or other OpenCL-compatible GPU drivers

Libraries

• OpenCV (>= 3.0): Image processing and manipulation
• Eigen (>= 3.2): Linear algebra computations
• Sophus (>= 1.0): Lie group operations for SE(2) transformations

Installation

Prerequisites (Ubuntu/Debian)

# Install basic build tools
sudo apt-get update
sudo apt-get install -y cmake build-essential git

# Install OpenCL headers and runtime
sudo apt-get install -y opencl-headers ocl-icd-opencl-dev

# Install OpenCV
sudo apt-get install -y libopencv-dev

# Install Eigen
sudo apt-get install -y libeigen3-dev

# Install Sophus
sudo apt-get install -y libsophus-dev
# Or build from source: https://github.com/strasdat/Sophus

For Intel GPUs (Optional)

# Add Intel graphics repository
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654
sudo apt-get install -y software-properties-common
sudo apt-add-repository 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu bionic main'
sudo apt-get update

# Install Intel GPU drivers and OpenCL support
sudo apt-get install -y intel-level-zero-gpu level-zero intel-opencl intel-igc-opencl-devel level-zero-devel

Build

mkdir build
cd build
cmake ..
make

The executable Vio_OpenCL will be created in the build directory.

Usage

Basic Execution

./Vio_OpenCL

The program will:

1. Initialize OpenCL context and compile kernels
2. Load test data (images, poses, features)
3. Execute sparse image alignment iterations
4. Output residual and pose estimates

Configuration

Key parameters can be modified in the source code:

• Pyramid Levels: max_level, min_level - control coarse-to-fine processing
• Iterations: n_iter - number of optimization iterations per level
• Optimization Method: GaussNewton, LevenbergMarquardt - solver selection
• Patch Size: patch_halfsize_ - correlation patch size (currently 12×12)
• Feature Count: FAST_COUNT - maximum number of detected corners

Input Data Format

The program expects:

• Images: Grayscale images with known intrinsic parameters
• Features: Corner positions and 3D world coordinates
• Poses: Initial camera pose estimates (SE2 transformation)

Test data location: test/data/

Project Structure

.
├── CMakeLists.txt                 # CMake build configuration
├── main2.cpp                      # Entry point
├── include/
│   ├── cl_class.h                # OpenCL wrapper classes
│   ├── sparse_img_align_gpu.h    # Image alignment algorithm header
│   ├── math_utils.h              # Mathematical utilities
│   └── robust_cost.h             # Robust cost functions
├── src/
│   ├── cl_class.cpp              # OpenCL context and kernel management
│   ├── sparse_img_align_gpu.cpp  # Sparse alignment implementation
│   ├── math_utils.cpp            # Math utility implementations
│   ├── robust_cost.cpp           # Robust cost function implementations
│   ├── FastGrayStep.cc           # FAST detector preprocessing
│   └── fastcl.cc                 # FAST detector wrapper
├── kernel/                        # OpenCL kernels
│   ├── compute-residual.cl       # Photometric residual computation
│   ├── fast-gray.cl              # FAST corner detection
│   ├── fast-best.cl              # FAST non-maximum suppression
│   ├── freak.cl                  # FREAK descriptor computation
│   ├── integral_image.cl         # Integral image computation
│   ├── is_corner_10_*.cl         # FAST corner test variants
│   └── prefast-gray.cl           # Preprocessing for FAST
└── test/
    ├── data/                      # Test images and feature data
    └── res/                       # Result output directory

Algorithm Overview

Sparse Image Alignment

The algorithm optimizes camera pose by minimizing the photometric error:

minimize: Σ ρ(I(x+∇x) - I'(x))²

Where:

• I and I' are reference and current frame intensities
• ∇x is the image warp from pose change
• ρ() is a robust weighting function

Optimization uses:

• Gauss-Newton or Levenberg-Marquardt solvers
• Coarse-to-fine multi-level pyramid for robustness
• Huber/Tukey robust cost functions for outlier rejection

Feature Detection

• FAST algorithm for rapid corner detection with automatic scale adaptation
• FREAK descriptors for compact feature representation
• GPU parallel processing across multiple image regions

Performance Notes

• Kernel compilation happens at runtime; first execution will be slower
• Memory transfers between CPU and GPU are optimized using pinned memory
• Floating-point precision: Single precision (32-bit) for GPU kernels

Limitations & Known Issues

• Currently supports single-level (SE2) 2D transformation
• Limited to single-threaded execution on CPU side
• Requires compatible OpenCL-capable GPU for optimal performance
• Memory access patterns require careful alignment (see error handling in code)

References

• SVO: Forster, C., Pizzoli, M., & Scaramuzza, D. (2014). "SVO: Semi-direct visual odometry for monocular and stereo cameras." ICRA.
• FAST: Rosten, E., & Drummond, T. (2006). "Machine learning for high-speed corner detection." ECCV.
• FREAK: Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). "FREAK: Fast retina keypoint." CVPR.
• OpenCL: https://www.khronos.org/opencl/

License

This project is licensed under the GNU General Public License v3.0 (GPLv3), consistent with the original SVO framework.

See LICENSE file for details.

Contributing

Bug reports and improvements are welcome. Please ensure:

• Code follows C++14 standard
• OpenCL kernels are optimized for parallel execution
• GPU memory constraints are considered
• Changes include corresponding kernel updates if needed

Authors

Original SVO Framework: Christian Forster, Robotics and Perception Group, University of Zurich

GPU Acceleration: Feature-extraction team (gfuse-show-cases project)

Support

For issues or questions:

• Check test data compatibility
• Verify OpenCL drivers and hardware support
• Review kernel compilation logs for runtime errors
• Ensure sufficient GPU memory (minimum 1GB recommended)