Simple triangulation with OpenCV from Harley & Zisserman [w/ code]

I sense that a lot of people are looking for a simple triangulation method with OpenCV, when they have two images and matching features.
While OpenCV contains the function cvTriangulatePoints in the triangulation.cpp file, it is not documented, and uses the arcane C API.
Luckily, Hartley and Zisserman describe in their excellent book "Multiple View Geometry" (in many cases considered to be "The Bible" of 3D reconstruction), a simple method for linear triangulation. This method is actually discussed earlier in Hartley's article "Triangulation".
I implemented it using the new OpenCV 2.3+ C++ API, which makes it super easy, and here it is before you.

Edit (4/25/2015): In a new post I am using OpenCV's cv::triangulatePoints() function. The code is available online in a gist.

Edit (6/5/2014): See some of my more recent work on structure from motion in this post on SfM and that post on the recent Qt GUI and SfM library.

Update 2017: See the new Mastering OpenCV3 book with a deeper discussion, and a more recent post on the implications of using OpenCV3 for SfM.

Continue reading "Simple triangulation with OpenCV from Harley & Zisserman [w/ code]"


Spherical harmonics face relighting using OpenCV, OpenGL [w/ code]

I've been working on implementing a face image relighting algorithm using spherical harmonics, one of the most elegant methods I've seen lately.
I start up by aligning a face model with OpenGL to automatically get the canonical face normals, which brushed up my knowledge of GLSL. Then I continue to estimating real faces "spharmonics", and relighting.

Let's start!
Continue reading "Spherical harmonics face relighting using OpenCV, OpenGL [w/ code]"


Identity Transfer in Photographs


I would like to present something I have been working on recently, a work that immensely affect what I wrote in the blog in the past two years...

To use it:
Go on this page,
Watch the short instruction video,
download the application (MacOSX-Intel-x64 Win32)
and make yourself a model!
It takes just a couple of minutes and it's very simple...

This work is an academic research project, Please please, take the time to fill out the survey! It is very short..
The results of the survey (the survey alone, no photos of your work) will possibly be published in an academic paper.

Note: No information is sent anywhere in any way outside of your machine (you may even unplug the network). All results are saved locally on your computer, and no inputs are recorded or transmitted. The application contains no malware. The source is available here.

Note II: All stock photos of models used in the application are released under Creative Commons By-NC-SA 2.0 license. Creator: If you wish to distribute your results, they should also be released under a CC-By-NC-SA 2.0 license.

Thank you!


A simple object classifier with Bag-of-Words using OpenCV 2.3 [w/ code]

Just wanted to share of some code I've been writing.
So I wanted to create a food classifier, for a cool project down in the Media Lab called FoodCam. It's basically a camera that people put free food under, and they can send an email alert to the entire building to come eat (by pushing a huge button marked "Dinner Bell"). Really a cool thing.

OK let's get down to business.
Continue reading "A simple object classifier with Bag-of-Words using OpenCV 2.3 [w/ code]"


Simple Kalman filter for tracking using OpenCV 2.2 [w/ code]

I wanted to put up a quick note on how to use Kalman Filters in OpenCV 2.2 with the C++ API, because all I could find online was using the old C API. Plus the kalman.cpp example that ships with OpenCV is kind of crappy and really doesn't explain how to use the Kalman Filter.
I'm no expert on Kalman filters though, this is just a quick hack I got going as a test for a project. It worked, so I'm posting the results.
Continue reading "Simple Kalman filter for tracking using OpenCV 2.2 [w/ code]"


Neat OpenCV smoothing trick when Kineacking (Kinect Hacking) [w/ code]

I found a nice little trick to ease the work with the very noisy depth image the Kinect is giving out. The image is filled with these "blank" values that basically note where the data is unreadable. The secret is to use inpainting to cover these areas and get a cleaner image. And as always, no need to dig deep - OpenCV has it all included.
Continue reading "Neat OpenCV smoothing trick when Kineacking (Kinect Hacking) [w/ code]"


Hand gesture recognition via model fitting in energy minimization w/OpenCV

hands with model fittedHi

Just wanted to share a thing I made - a simple 2D hand pose estimator, using a skeleton model fitting. Basically there has been a crap load of work on hand pose estimation, but I was inspired by this ancient work. The problem is setting out to find a good solution, and everything is very hard to understand and implement. In such cases I like to be inspired by a method, and just set out with my own implementation. This way, I understand whats going on, simplify it, and share it with you!

Anyway, let's get down to business.

Edit (6/5/2014): Also see some of my other work on hand gesture recognition using smart contours and particle filters

Continue reading "Hand gesture recognition via model fitting in energy minimization w/OpenCV"


20-lines AR in OpenCV [w/code]


Just wanted to share a bit of code using OpenCV's camera extrinsic parameters recovery, camera position and rotation - solvePnP (or it's C counterpart cvFindExtrinsicCameraParams2). I wanted to get a simple planar object surface recovery for augmented reality, but without using any of the AR libraries, rather dig into some OpenCV and OpenGL code.
This can serve as a primer, or tutorial on how to use OpenCV with OpenGL for AR.

Update 2/16/2015: I wrote another post on OpenCV-OpenGL AR, this time using the fine QGLViewer - a very convenient Qt OpenGL widget.

The program is just a straightforward optical flow based tracking, fed manually with four points which are the planar object's corners, and solving camera-pose every frame. Plain vanilla AR.

Well the whole cpp file is ~350 lines, but there will only be 20 or less interesting lines... Actually much less. Let's see what's up
Continue reading "20-lines AR in OpenCV [w/code]"