From 1fa13081ca6bf8a027ae109979e0451bdef48d5f Mon Sep 17 00:00:00 2001 From: msjid Date: Fri, 28 Aug 2026 18:53:58 +0200 Subject: [PATCH] Add comprehensive README documentation Includes project description, features, requirements, installation instructions, usage guide, project structure overview, algorithm explanation, and references to original papers and frameworks. Co-Authored-By: Claude Haiku 4.5 Claude-Session: https://claude.ai/code/session_01EC6AeDvAUwUCU6EhkLpCQF --- README.md | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..3aa41cd --- /dev/null +++ b/README.md @@ -0,0 +1,212 @@ +# Feature Extraction & Sparse Image Alignment with OpenCL + +A GPU-accelerated Visual Inertial Odometry (VIO) system implementing sparse image alignment and feature detection algorithms using OpenCL. + +This project is based on the [SVO (Semi-direct Visual Odometry)](http://rpg.ifi.uzh.ch/svo/) framework from the Robotics and Perception Group at the University of Zurich, adapted for GPU acceleration using OpenCL. + +## Features + +- **GPU-Accelerated Image Processing**: Utilizes OpenCL for parallel computation of image features and alignment +- **FAST Corner Detection**: GPU-optimized FAST (Features from Accelerated Segment Test) algorithm for corner detection +- **FREAK Descriptors**: GPU implementation of FREAK (Fast Retina Keypoint) feature descriptors +- **Sparse Image Alignment**: Photometric error minimization using Gauss-Newton optimization +- **Pose Optimization**: SE(2) transformation estimation for camera pose refinement +- **Multi-level Pyramid Processing**: Coarse-to-fine image pyramid for robust alignment + +## Requirements + +### System Dependencies +- CMake >= 3.7 +- C++14 compiler (g++, clang, or MSVC) +- OpenCL headers and libraries +- Intel GPU drivers (for Intel GPU support) or other OpenCL-compatible GPU drivers + +### Libraries +- **OpenCV** (>= 3.0): Image processing and manipulation +- **Eigen** (>= 3.2): Linear algebra computations +- **Sophus** (>= 1.0): Lie group operations for SE(2) transformations + +## Installation + +### Prerequisites (Ubuntu/Debian) + +```bash +# Install basic build tools +sudo apt-get update +sudo apt-get install -y cmake build-essential git + +# Install OpenCL headers and runtime +sudo apt-get install -y opencl-headers ocl-icd-opencl-dev + +# Install OpenCV +sudo apt-get install -y libopencv-dev + +# Install Eigen +sudo apt-get install -y libeigen3-dev + +# Install Sophus +sudo apt-get install -y libsophus-dev +# Or build from source: https://github.com/strasdat/Sophus +``` + +### For Intel GPUs (Optional) + +```bash +# Add Intel graphics repository +sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654 +sudo apt-get install -y software-properties-common +sudo apt-add-repository 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu bionic main' +sudo apt-get update + +# Install Intel GPU drivers and OpenCL support +sudo apt-get install -y intel-level-zero-gpu level-zero intel-opencl intel-igc-opencl-devel level-zero-devel +``` + +### Build + +```bash +mkdir build +cd build +cmake .. +make +``` + +The executable `Vio_OpenCL` will be created in the build directory. + +## Usage + +### Basic Execution + +```bash +./Vio_OpenCL +``` + +The program will: +1. Initialize OpenCL context and compile kernels +2. Load test data (images, poses, features) +3. Execute sparse image alignment iterations +4. Output residual and pose estimates + +### Configuration + +Key parameters can be modified in the source code: + +- **Pyramid Levels**: `max_level`, `min_level` - control coarse-to-fine processing +- **Iterations**: `n_iter` - number of optimization iterations per level +- **Optimization Method**: `GaussNewton`, `LevenbergMarquardt` - solver selection +- **Patch Size**: `patch_halfsize_` - correlation patch size (currently 12×12) +- **Feature Count**: `FAST_COUNT` - maximum number of detected corners + +### Input Data Format + +The program expects: +- **Images**: Grayscale images with known intrinsic parameters +- **Features**: Corner positions and 3D world coordinates +- **Poses**: Initial camera pose estimates (SE2 transformation) + +Test data location: `test/data/` + +## Project Structure + +``` +. +├── CMakeLists.txt # CMake build configuration +├── main2.cpp # Entry point +├── include/ +│ ├── cl_class.h # OpenCL wrapper classes +│ ├── sparse_img_align_gpu.h # Image alignment algorithm header +│ ├── math_utils.h # Mathematical utilities +│ └── robust_cost.h # Robust cost functions +├── src/ +│ ├── cl_class.cpp # OpenCL context and kernel management +│ ├── sparse_img_align_gpu.cpp # Sparse alignment implementation +│ ├── math_utils.cpp # Math utility implementations +│ ├── robust_cost.cpp # Robust cost function implementations +│ ├── FastGrayStep.cc # FAST detector preprocessing +│ └── fastcl.cc # FAST detector wrapper +├── kernel/ # OpenCL kernels +│ ├── compute-residual.cl # Photometric residual computation +│ ├── fast-gray.cl # FAST corner detection +│ ├── fast-best.cl # FAST non-maximum suppression +│ ├── freak.cl # FREAK descriptor computation +│ ├── integral_image.cl # Integral image computation +│ ├── is_corner_10_*.cl # FAST corner test variants +│ └── prefast-gray.cl # Preprocessing for FAST +└── test/ + ├── data/ # Test images and feature data + └── res/ # Result output directory +``` + +## Algorithm Overview + +### Sparse Image Alignment + +The algorithm optimizes camera pose by minimizing the photometric error: + +``` +minimize: Σ ρ(I(x+∇x) - I'(x))² +``` + +Where: +- `I` and `I'` are reference and current frame intensities +- `∇x` is the image warp from pose change +- `ρ()` is a robust weighting function + +Optimization uses: +- **Gauss-Newton** or **Levenberg-Marquardt** solvers +- **Coarse-to-fine** multi-level pyramid for robustness +- **Huber/Tukey** robust cost functions for outlier rejection + +### Feature Detection + +- **FAST** algorithm for rapid corner detection with automatic scale adaptation +- **FREAK** descriptors for compact feature representation +- GPU parallel processing across multiple image regions + +## Performance Notes + +- Kernel compilation happens at runtime; first execution will be slower +- Memory transfers between CPU and GPU are optimized using pinned memory +- Floating-point precision: Single precision (32-bit) for GPU kernels + +## Limitations & Known Issues + +- Currently supports single-level (SE2) 2D transformation +- Limited to single-threaded execution on CPU side +- Requires compatible OpenCL-capable GPU for optimal performance +- Memory access patterns require careful alignment (see error handling in code) + +## References + +- **SVO**: Forster, C., Pizzoli, M., & Scaramuzza, D. (2014). "SVO: Semi-direct visual odometry for monocular and stereo cameras." ICRA. +- **FAST**: Rosten, E., & Drummond, T. (2006). "Machine learning for high-speed corner detection." ECCV. +- **FREAK**: Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). "FREAK: Fast retina keypoint." CVPR. +- **OpenCL**: https://www.khronos.org/opencl/ + +## License + +This project is licensed under the GNU General Public License v3.0 (GPLv3), consistent with the original SVO framework. + +See `LICENSE` file for details. + +## Contributing + +Bug reports and improvements are welcome. Please ensure: +- Code follows C++14 standard +- OpenCL kernels are optimized for parallel execution +- GPU memory constraints are considered +- Changes include corresponding kernel updates if needed + +## Authors + +Original SVO Framework: Christian Forster, Robotics and Perception Group, University of Zurich + +GPU Acceleration: Feature-extraction team (gfuse-show-cases project) + +## Support + +For issues or questions: +- Check test data compatibility +- Verify OpenCL drivers and hardware support +- Review kernel compilation logs for runtime errors +- Ensure sufficient GPU memory (minimum 1GB recommended)