Computer Vision: Basic

Feature Points and Camera Calibration

Overview

Feature Detection Harris, SIFT, ORB Corners / DoG extrema / FAST Scale handling differs by method Feature Description Descriptor BRIEF, FREAK Rotation handling (ORB adds orientation) Feature Matching Nearest Neighbor Search RANSAC Outlier Rejection Camera Calibration Checkerboard Intrinsic / Extrinsic Params Lens Distortion Correction Find corresponding points across images and estimate the geometric properties of the camera

The Basic level covers methods for detecting distinctive points in images and establishing correspondences between images. It also treats calibration techniques for estimating the intrinsic and extrinsic parameters of a camera (the chapters are in preparation).

These techniques form the foundation of applications such as 3D reconstruction, SLAM, image stitching (panorama creation), and augmented reality (AR). Recovering geometric relationships from feature correspondences is the starting point for many advanced tasks.

Learning Objectives

  • Understand the major feature detectors (Harris, SIFT, ORB)
  • Understand the role and types of feature descriptors
  • Understand robust estimation with RANSAC
  • Be able to perform camera calibration

Table of Contents

  1. Chapter 1 Corner Detection

    Harris corner detection, Shi-Tomasi

  2. Chapter 2 SIFT Features

    Scale space, DoG, descriptors

  3. Chapter 3 ORB and Binary Features

    FAST, BRIEF, ORB

  4. Chapter 4 Feature Matching

    Brute-Force, FLANN, ratio test

  5. Chapter 5 RANSAC

    Robust estimation, outlier rejection

  6. Chapter 6 Camera Calibration

    Zhang's method, distortion coefficients

  7. Chapter 7 Lens Distortion Correction

    Radial distortion, tangential distortion

  8. Chapter 8 Exercises

    Wrap-up of the Basic level

Supplementary Reading: 10 Illustrated Pieces

In addition to the 8 main chapters, we provide a collection of 10 columns on filtering, edges, pyramids, calibration, and distortion.

1.The Principle of Gaussian Smoothing
2.Median vs Bilateral
3.Sobel vs Canny
4.Line Detection with the Hough Transform
5.The Idea of Image Pyramids
6.Scale Selection in SIFT
7.Intuition for Homography
8.Reprojection Error and Calibration Evaluation
9.Lens Aberration and Distortion Models
10.Rolling Shutter and Motion Blur

Prerequisites

  • The content of Computer Vision: Introduction
  • Calculus (gradients, Hessian matrix)
  • Linear algebra (eigenvalue decomposition, SVD)
  • Probability (basic concepts; helpful for understanding RANSAC)

Basic Concepts

Harris Corner Detection

Construct the structure tensor $M$ from image gradients over a window around each point (the sums run over the window, generally with weights):

$$M = \begin{pmatrix} \displaystyle\sum I_x^2 & \displaystyle\sum I_x I_y \\ \displaystyle\sum I_x I_y & \displaystyle\sum I_y^2 \end{pmatrix}$$

Corner response function: $R = \det(M) - k(\text{trace}(M))^2$

Scale Space and DoG

By progressively blurring an image to build a Gaussian pyramid and taking the difference between adjacent blurred images (Difference of Gaussian, DoG), one takes points that are extrema in both position and scale as candidate feature points. Through this mechanism, SIFT obtains feature points that are robust to scaling.

RANSAC

Randomly select a sample and estimate a model:

  1. Estimate a model from the minimal number of samples
  2. Count inliers (points that fit the model)
  3. Repeat, and adopt the model with the most inliers (inliers are decided by a residual threshold; finally re-estimate from those inliers)

Lens Distortion Model

Radial distortion:

$$x' = x(1 + k_1 r^2 + k_2 r^4 + k_3 r^6), \quad y' = y(1 + k_1 r^2 + k_2 r^4 + k_3 r^6)$$

$(x, y)$ are normalized image coordinates, $r^2 = x^2 + y^2$, and $k_i$ are the distortion coefficients

Reading Reading

Before diving into the equations, a story to relax and build intuition. We look at "why a computer picks corners as feature points" through the familiar analogy of sliding a window.

Frequently Asked Questions (FAQ)

Q1. How does Harris corner detection work?

It constructs the structure tensor $M$ from image gradients and evaluates the corner response function $R = \det(M) - k(\text{trace}(M))^2$. Points with large $R$ are those where the intensity within the window (sum of squared differences) changes significantly no matter which direction the window is shifted slightly — that is, corners.

Q2. What is the difference between SIFT and ORB?

SIFT detects scale-invariant keypoints using scale space and DoG, and produces real-valued descriptors from gradient histograms. ORB combines FAST corner detection with a BRIEF-based binary descriptor and adds rotation invariance. ORB is generally used for real-time applications where speed matters, while SIFT is robust to changes in scale and rotation (which is better depends on the images and implementation).

Q3. Why is RANSAC needed?

Feature matches often contain wrong correspondences (outliers), and least-squares estimation over all points can be pulled far off by them. RANSAC randomly picks a minimal sample, estimates a model, and keeps the model that explains the most inliers, giving an estimate that is robust to outliers.

Q4. What does camera calibration estimate?

It estimates the intrinsic parameters such as focal length and principal point, the extrinsic parameters describing the camera pose, and the distortion coefficients for radial and tangential lens distortion. Zhang's method, which images a known planar checkerboard in several poses with different tilts, is a representative approach.