Add comprehensive README documentation
Includes project description, features, requirements, installation instructions, usage guide, project structure overview, algorithm explanation, and references to original papers and frameworks. Co-Authored-By: Claude Haiku 4.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EC6AeDvAUwUCU6EhkLpCQF
This commit is contained in:
msjid
co-authored by
Claude Haiku 4.5
Claude Haiku 4.5
parent
722a4a5ab2
commit
1fa13081ca
@@ -0,0 +1,212 @@
|
||||
# Feature Extraction & Sparse Image Alignment with OpenCL
|
||||
|
||||
A GPU-accelerated Visual Inertial Odometry (VIO) system implementing sparse image alignment and feature detection algorithms using OpenCL.
|
||||
|
||||
This project is based on the [SVO (Semi-direct Visual Odometry)](http://rpg.ifi.uzh.ch/svo/) framework from the Robotics and Perception Group at the University of Zurich, adapted for GPU acceleration using OpenCL.
|
||||
|
||||
## Features
|
||||
|
||||
- **GPU-Accelerated Image Processing**: Utilizes OpenCL for parallel computation of image features and alignment
|
||||
- **FAST Corner Detection**: GPU-optimized FAST (Features from Accelerated Segment Test) algorithm for corner detection
|
||||
- **FREAK Descriptors**: GPU implementation of FREAK (Fast Retina Keypoint) feature descriptors
|
||||
- **Sparse Image Alignment**: Photometric error minimization using Gauss-Newton optimization
|
||||
- **Pose Optimization**: SE(2) transformation estimation for camera pose refinement
|
||||
- **Multi-level Pyramid Processing**: Coarse-to-fine image pyramid for robust alignment
|
||||
|
||||
## Requirements
|
||||
|
||||
### System Dependencies
|
||||
- CMake >= 3.7
|
||||
- C++14 compiler (g++, clang, or MSVC)
|
||||
- OpenCL headers and libraries
|
||||
- Intel GPU drivers (for Intel GPU support) or other OpenCL-compatible GPU drivers
|
||||
|
||||
### Libraries
|
||||
- **OpenCV** (>= 3.0): Image processing and manipulation
|
||||
- **Eigen** (>= 3.2): Linear algebra computations
|
||||
- **Sophus** (>= 1.0): Lie group operations for SE(2) transformations
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites (Ubuntu/Debian)
|
||||
|
||||
```bash
|
||||
# Install basic build tools
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y cmake build-essential git
|
||||
|
||||
# Install OpenCL headers and runtime
|
||||
sudo apt-get install -y opencl-headers ocl-icd-opencl-dev
|
||||
|
||||
# Install OpenCV
|
||||
sudo apt-get install -y libopencv-dev
|
||||
|
||||
# Install Eigen
|
||||
sudo apt-get install -y libeigen3-dev
|
||||
|
||||
# Install Sophus
|
||||
sudo apt-get install -y libsophus-dev
|
||||
# Or build from source: https://github.com/strasdat/Sophus
|
||||
```
|
||||
|
||||
### For Intel GPUs (Optional)
|
||||
|
||||
```bash
|
||||
# Add Intel graphics repository
|
||||
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654
|
||||
sudo apt-get install -y software-properties-common
|
||||
sudo apt-add-repository 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu bionic main'
|
||||
sudo apt-get update
|
||||
|
||||
# Install Intel GPU drivers and OpenCL support
|
||||
sudo apt-get install -y intel-level-zero-gpu level-zero intel-opencl intel-igc-opencl-devel level-zero-devel
|
||||
```
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
make
|
||||
```
|
||||
|
||||
The executable `Vio_OpenCL` will be created in the build directory.
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Execution
|
||||
|
||||
```bash
|
||||
./Vio_OpenCL
|
||||
```
|
||||
|
||||
The program will:
|
||||
1. Initialize OpenCL context and compile kernels
|
||||
2. Load test data (images, poses, features)
|
||||
3. Execute sparse image alignment iterations
|
||||
4. Output residual and pose estimates
|
||||
|
||||
### Configuration
|
||||
|
||||
Key parameters can be modified in the source code:
|
||||
|
||||
- **Pyramid Levels**: `max_level`, `min_level` - control coarse-to-fine processing
|
||||
- **Iterations**: `n_iter` - number of optimization iterations per level
|
||||
- **Optimization Method**: `GaussNewton`, `LevenbergMarquardt` - solver selection
|
||||
- **Patch Size**: `patch_halfsize_` - correlation patch size (currently 12×12)
|
||||
- **Feature Count**: `FAST_COUNT` - maximum number of detected corners
|
||||
|
||||
### Input Data Format
|
||||
|
||||
The program expects:
|
||||
- **Images**: Grayscale images with known intrinsic parameters
|
||||
- **Features**: Corner positions and 3D world coordinates
|
||||
- **Poses**: Initial camera pose estimates (SE2 transformation)
|
||||
|
||||
Test data location: `test/data/`
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
.
|
||||
├── CMakeLists.txt # CMake build configuration
|
||||
├── main2.cpp # Entry point
|
||||
├── include/
|
||||
│ ├── cl_class.h # OpenCL wrapper classes
|
||||
│ ├── sparse_img_align_gpu.h # Image alignment algorithm header
|
||||
│ ├── math_utils.h # Mathematical utilities
|
||||
│ └── robust_cost.h # Robust cost functions
|
||||
├── src/
|
||||
│ ├── cl_class.cpp # OpenCL context and kernel management
|
||||
│ ├── sparse_img_align_gpu.cpp # Sparse alignment implementation
|
||||
│ ├── math_utils.cpp # Math utility implementations
|
||||
│ ├── robust_cost.cpp # Robust cost function implementations
|
||||
│ ├── FastGrayStep.cc # FAST detector preprocessing
|
||||
│ └── fastcl.cc # FAST detector wrapper
|
||||
├── kernel/ # OpenCL kernels
|
||||
│ ├── compute-residual.cl # Photometric residual computation
|
||||
│ ├── fast-gray.cl # FAST corner detection
|
||||
│ ├── fast-best.cl # FAST non-maximum suppression
|
||||
│ ├── freak.cl # FREAK descriptor computation
|
||||
│ ├── integral_image.cl # Integral image computation
|
||||
│ ├── is_corner_10_*.cl # FAST corner test variants
|
||||
│ └── prefast-gray.cl # Preprocessing for FAST
|
||||
└── test/
|
||||
├── data/ # Test images and feature data
|
||||
└── res/ # Result output directory
|
||||
```
|
||||
|
||||
## Algorithm Overview
|
||||
|
||||
### Sparse Image Alignment
|
||||
|
||||
The algorithm optimizes camera pose by minimizing the photometric error:
|
||||
|
||||
```
|
||||
minimize: Σ ρ(I(x+∇x) - I'(x))²
|
||||
```
|
||||
|
||||
Where:
|
||||
- `I` and `I'` are reference and current frame intensities
|
||||
- `∇x` is the image warp from pose change
|
||||
- `ρ()` is a robust weighting function
|
||||
|
||||
Optimization uses:
|
||||
- **Gauss-Newton** or **Levenberg-Marquardt** solvers
|
||||
- **Coarse-to-fine** multi-level pyramid for robustness
|
||||
- **Huber/Tukey** robust cost functions for outlier rejection
|
||||
|
||||
### Feature Detection
|
||||
|
||||
- **FAST** algorithm for rapid corner detection with automatic scale adaptation
|
||||
- **FREAK** descriptors for compact feature representation
|
||||
- GPU parallel processing across multiple image regions
|
||||
|
||||
## Performance Notes
|
||||
|
||||
- Kernel compilation happens at runtime; first execution will be slower
|
||||
- Memory transfers between CPU and GPU are optimized using pinned memory
|
||||
- Floating-point precision: Single precision (32-bit) for GPU kernels
|
||||
|
||||
## Limitations & Known Issues
|
||||
|
||||
- Currently supports single-level (SE2) 2D transformation
|
||||
- Limited to single-threaded execution on CPU side
|
||||
- Requires compatible OpenCL-capable GPU for optimal performance
|
||||
- Memory access patterns require careful alignment (see error handling in code)
|
||||
|
||||
## References
|
||||
|
||||
- **SVO**: Forster, C., Pizzoli, M., & Scaramuzza, D. (2014). "SVO: Semi-direct visual odometry for monocular and stereo cameras." ICRA.
|
||||
- **FAST**: Rosten, E., & Drummond, T. (2006). "Machine learning for high-speed corner detection." ECCV.
|
||||
- **FREAK**: Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). "FREAK: Fast retina keypoint." CVPR.
|
||||
- **OpenCL**: https://www.khronos.org/opencl/
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the GNU General Public License v3.0 (GPLv3), consistent with the original SVO framework.
|
||||
|
||||
See `LICENSE` file for details.
|
||||
|
||||
## Contributing
|
||||
|
||||
Bug reports and improvements are welcome. Please ensure:
|
||||
- Code follows C++14 standard
|
||||
- OpenCL kernels are optimized for parallel execution
|
||||
- GPU memory constraints are considered
|
||||
- Changes include corresponding kernel updates if needed
|
||||
|
||||
## Authors
|
||||
|
||||
Original SVO Framework: Christian Forster, Robotics and Perception Group, University of Zurich
|
||||
|
||||
GPU Acceleration: Feature-extraction team (gfuse-show-cases project)
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- Check test data compatibility
|
||||
- Verify OpenCL drivers and hardware support
|
||||
- Review kernel compilation logs for runtime errors
|
||||
- Ensure sufficient GPU memory (minimum 1GB recommended)
|
||||
Reference in New Issue
Block a user