Camera Calibration & Augmented Reality

Quick Insights:

Problem: Rendering virtual 3D content in real scenes requires accurate camera intrinsics, distortion removal and pose estimation from real-world markers.

Solution: Calibrated the camera with multiple checkerboard images, estimated rotation and translation vectors using solvePnP, and projected the vertices of a virtual prism with projectPoints.

Result: A stable augmented reality effect where the virtual object tracks the checkerboard marker’s position, orientation and perspective in real time.

Introduction

This project implements a classical computer vision AR system without deep learning. Using a printed checkerboard, the camera’s intrinsic matrix and distortion parameters were estimated. With these, the pose relative to the marker was recovered frame-by-frame and a virtual 3D prism was rendered at the correct physical location on the board.

Computed intrinsics + distortion from multiple checkerboard views

Used solvePnP to obtain rotation + translation vectors

Projected a 3D object model using projectPoints

Maintained alignment even under changes in angle and distance

Demonstration

The pipeline below shows the two key stages: pose estimation via checkerboard detection and virtual object rendering.

Detected checkerboard corners with color-coded lines — **Camera Calibration:** Detected checkerboard corners used to compute intrinsic matrix and distortion coefficients.

AR prism rendered using solvePnP based pose estimation — **AR Rendering Stage:** Prism rendered using projected 3D points from the estimated camera pose.

Second AR prism view with perspective alignment — Another view of the AR overlay showing correct perspective alignment.

Representative Code Snippet

// Estimate pose from checkerboard points
solvePnP(objectPoints, imagePoints,
         cameraMatrix, distCoeffs,
         rvec, tvec);

// Project 3D prism vertices into image space
projectPoints(prismVertices, rvec, tvec,
              cameraMatrix, distCoeffs,
              projected2D);

// Draw edges between projected points
for (auto &edge : prismEdges) {
    line(frame, projected2D[edge.first],
         projected2D[edge.second],
         Scalar(0, 255, 255), 2);
}

Why It Matters

This project recreates the core of ARKit/ARCore-style tracking using only geometry and OpenCV. It demonstrates practical understanding of camera projection, pose estimation and real-time rendering—all foundational skills for AR, robotics, SLAM and computer vision work.

Strengthened understanding of intrinsic/extrinsic calibration
Implemented real-time AR without ML or heavy frameworks
Built end-to-end geometric intuition for projection pipelines
Created a clean demonstration of classical CV-based AR