Add comprehensive README documentation
Includes project description, features, requirements, installation instructions, usage guide, project structure overview, algorithm explanation, and references to original papers and frameworks. Co-Authored-By: Claude Haiku 4.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EC6AeDvAUwUCU6EhkLpCQF
This commit is contained in:
msjid
co-authored by
Claude Haiku 4.5
Claude Haiku 4.5
parent
722a4a5ab2
commit
1fa13081ca
@@ -0,0 +1,212 @@
|
|||||||
|
# Feature Extraction & Sparse Image Alignment with OpenCL
|
||||||
|
|
||||||
|
A GPU-accelerated Visual Inertial Odometry (VIO) system implementing sparse image alignment and feature detection algorithms using OpenCL.
|
||||||
|
|
||||||
|
This project is based on the [SVO (Semi-direct Visual Odometry)](http://rpg.ifi.uzh.ch/svo/) framework from the Robotics and Perception Group at the University of Zurich, adapted for GPU acceleration using OpenCL.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **GPU-Accelerated Image Processing**: Utilizes OpenCL for parallel computation of image features and alignment
|
||||||
|
- **FAST Corner Detection**: GPU-optimized FAST (Features from Accelerated Segment Test) algorithm for corner detection
|
||||||
|
- **FREAK Descriptors**: GPU implementation of FREAK (Fast Retina Keypoint) feature descriptors
|
||||||
|
- **Sparse Image Alignment**: Photometric error minimization using Gauss-Newton optimization
|
||||||
|
- **Pose Optimization**: SE(2) transformation estimation for camera pose refinement
|
||||||
|
- **Multi-level Pyramid Processing**: Coarse-to-fine image pyramid for robust alignment
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
### System Dependencies
|
||||||
|
- CMake >= 3.7
|
||||||
|
- C++14 compiler (g++, clang, or MSVC)
|
||||||
|
- OpenCL headers and libraries
|
||||||
|
- Intel GPU drivers (for Intel GPU support) or other OpenCL-compatible GPU drivers
|
||||||
|
|
||||||
|
### Libraries
|
||||||
|
- **OpenCV** (>= 3.0): Image processing and manipulation
|
||||||
|
- **Eigen** (>= 3.2): Linear algebra computations
|
||||||
|
- **Sophus** (>= 1.0): Lie group operations for SE(2) transformations
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
### Prerequisites (Ubuntu/Debian)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install basic build tools
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y cmake build-essential git
|
||||||
|
|
||||||
|
# Install OpenCL headers and runtime
|
||||||
|
sudo apt-get install -y opencl-headers ocl-icd-opencl-dev
|
||||||
|
|
||||||
|
# Install OpenCV
|
||||||
|
sudo apt-get install -y libopencv-dev
|
||||||
|
|
||||||
|
# Install Eigen
|
||||||
|
sudo apt-get install -y libeigen3-dev
|
||||||
|
|
||||||
|
# Install Sophus
|
||||||
|
sudo apt-get install -y libsophus-dev
|
||||||
|
# Or build from source: https://github.com/strasdat/Sophus
|
||||||
|
```
|
||||||
|
|
||||||
|
### For Intel GPUs (Optional)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add Intel graphics repository
|
||||||
|
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654
|
||||||
|
sudo apt-get install -y software-properties-common
|
||||||
|
sudo apt-add-repository 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu bionic main'
|
||||||
|
sudo apt-get update
|
||||||
|
|
||||||
|
# Install Intel GPU drivers and OpenCL support
|
||||||
|
sudo apt-get install -y intel-level-zero-gpu level-zero intel-opencl intel-igc-opencl-devel level-zero-devel
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir build
|
||||||
|
cd build
|
||||||
|
cmake ..
|
||||||
|
make
|
||||||
|
```
|
||||||
|
|
||||||
|
The executable `Vio_OpenCL` will be created in the build directory.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Basic Execution
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./Vio_OpenCL
|
||||||
|
```
|
||||||
|
|
||||||
|
The program will:
|
||||||
|
1. Initialize OpenCL context and compile kernels
|
||||||
|
2. Load test data (images, poses, features)
|
||||||
|
3. Execute sparse image alignment iterations
|
||||||
|
4. Output residual and pose estimates
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
Key parameters can be modified in the source code:
|
||||||
|
|
||||||
|
- **Pyramid Levels**: `max_level`, `min_level` - control coarse-to-fine processing
|
||||||
|
- **Iterations**: `n_iter` - number of optimization iterations per level
|
||||||
|
- **Optimization Method**: `GaussNewton`, `LevenbergMarquardt` - solver selection
|
||||||
|
- **Patch Size**: `patch_halfsize_` - correlation patch size (currently 12×12)
|
||||||
|
- **Feature Count**: `FAST_COUNT` - maximum number of detected corners
|
||||||
|
|
||||||
|
### Input Data Format
|
||||||
|
|
||||||
|
The program expects:
|
||||||
|
- **Images**: Grayscale images with known intrinsic parameters
|
||||||
|
- **Features**: Corner positions and 3D world coordinates
|
||||||
|
- **Poses**: Initial camera pose estimates (SE2 transformation)
|
||||||
|
|
||||||
|
Test data location: `test/data/`
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── CMakeLists.txt # CMake build configuration
|
||||||
|
├── main2.cpp # Entry point
|
||||||
|
├── include/
|
||||||
|
│ ├── cl_class.h # OpenCL wrapper classes
|
||||||
|
│ ├── sparse_img_align_gpu.h # Image alignment algorithm header
|
||||||
|
│ ├── math_utils.h # Mathematical utilities
|
||||||
|
│ └── robust_cost.h # Robust cost functions
|
||||||
|
├── src/
|
||||||
|
│ ├── cl_class.cpp # OpenCL context and kernel management
|
||||||
|
│ ├── sparse_img_align_gpu.cpp # Sparse alignment implementation
|
||||||
|
│ ├── math_utils.cpp # Math utility implementations
|
||||||
|
│ ├── robust_cost.cpp # Robust cost function implementations
|
||||||
|
│ ├── FastGrayStep.cc # FAST detector preprocessing
|
||||||
|
│ └── fastcl.cc # FAST detector wrapper
|
||||||
|
├── kernel/ # OpenCL kernels
|
||||||
|
│ ├── compute-residual.cl # Photometric residual computation
|
||||||
|
│ ├── fast-gray.cl # FAST corner detection
|
||||||
|
│ ├── fast-best.cl # FAST non-maximum suppression
|
||||||
|
│ ├── freak.cl # FREAK descriptor computation
|
||||||
|
│ ├── integral_image.cl # Integral image computation
|
||||||
|
│ ├── is_corner_10_*.cl # FAST corner test variants
|
||||||
|
│ └── prefast-gray.cl # Preprocessing for FAST
|
||||||
|
└── test/
|
||||||
|
├── data/ # Test images and feature data
|
||||||
|
└── res/ # Result output directory
|
||||||
|
```
|
||||||
|
|
||||||
|
## Algorithm Overview
|
||||||
|
|
||||||
|
### Sparse Image Alignment
|
||||||
|
|
||||||
|
The algorithm optimizes camera pose by minimizing the photometric error:
|
||||||
|
|
||||||
|
```
|
||||||
|
minimize: Σ ρ(I(x+∇x) - I'(x))²
|
||||||
|
```
|
||||||
|
|
||||||
|
Where:
|
||||||
|
- `I` and `I'` are reference and current frame intensities
|
||||||
|
- `∇x` is the image warp from pose change
|
||||||
|
- `ρ()` is a robust weighting function
|
||||||
|
|
||||||
|
Optimization uses:
|
||||||
|
- **Gauss-Newton** or **Levenberg-Marquardt** solvers
|
||||||
|
- **Coarse-to-fine** multi-level pyramid for robustness
|
||||||
|
- **Huber/Tukey** robust cost functions for outlier rejection
|
||||||
|
|
||||||
|
### Feature Detection
|
||||||
|
|
||||||
|
- **FAST** algorithm for rapid corner detection with automatic scale adaptation
|
||||||
|
- **FREAK** descriptors for compact feature representation
|
||||||
|
- GPU parallel processing across multiple image regions
|
||||||
|
|
||||||
|
## Performance Notes
|
||||||
|
|
||||||
|
- Kernel compilation happens at runtime; first execution will be slower
|
||||||
|
- Memory transfers between CPU and GPU are optimized using pinned memory
|
||||||
|
- Floating-point precision: Single precision (32-bit) for GPU kernels
|
||||||
|
|
||||||
|
## Limitations & Known Issues
|
||||||
|
|
||||||
|
- Currently supports single-level (SE2) 2D transformation
|
||||||
|
- Limited to single-threaded execution on CPU side
|
||||||
|
- Requires compatible OpenCL-capable GPU for optimal performance
|
||||||
|
- Memory access patterns require careful alignment (see error handling in code)
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- **SVO**: Forster, C., Pizzoli, M., & Scaramuzza, D. (2014). "SVO: Semi-direct visual odometry for monocular and stereo cameras." ICRA.
|
||||||
|
- **FAST**: Rosten, E., & Drummond, T. (2006). "Machine learning for high-speed corner detection." ECCV.
|
||||||
|
- **FREAK**: Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). "FREAK: Fast retina keypoint." CVPR.
|
||||||
|
- **OpenCL**: https://www.khronos.org/opencl/
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This project is licensed under the GNU General Public License v3.0 (GPLv3), consistent with the original SVO framework.
|
||||||
|
|
||||||
|
See `LICENSE` file for details.
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
Bug reports and improvements are welcome. Please ensure:
|
||||||
|
- Code follows C++14 standard
|
||||||
|
- OpenCL kernels are optimized for parallel execution
|
||||||
|
- GPU memory constraints are considered
|
||||||
|
- Changes include corresponding kernel updates if needed
|
||||||
|
|
||||||
|
## Authors
|
||||||
|
|
||||||
|
Original SVO Framework: Christian Forster, Robotics and Perception Group, University of Zurich
|
||||||
|
|
||||||
|
GPU Acceleration: Feature-extraction team (gfuse-show-cases project)
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
For issues or questions:
|
||||||
|
- Check test data compatibility
|
||||||
|
- Verify OpenCL drivers and hardware support
|
||||||
|
- Review kernel compilation logs for runtime errors
|
||||||
|
- Ensure sufficient GPU memory (minimum 1GB recommended)
|
||||||
Reference in New Issue
Block a user