Neat OpenCV smoothing trick when Kineacking (Kinect Hacking) [w/ code]

I found a nice little trick to ease the work with the very noisy depth image the Kinect is giving out. The image is filled with these "blank" values that basically note where the data is unreadable. The secret is to use inpainting to cover these areas and get a cleaner image. And as always, no need to dig deep - OpenCV has it all included.
Continue reading "Neat OpenCV smoothing trick when Kineacking (Kinect Hacking) [w/ code]"


Hand gesture recognition via model fitting in energy minimization w/OpenCV

hands with model fittedHi

Just wanted to share a thing I made - a simple 2D hand pose estimator, using a skeleton model fitting. Basically there has been a crap load of work on hand pose estimation, but I was inspired by this ancient work. The problem is setting out to find a good solution, and everything is very hard to understand and implement. In such cases I like to be inspired by a method, and just set out with my own implementation. This way, I understand whats going on, simplify it, and share it with you!

Anyway, let's get down to business.

Edit (6/5/2014): Also see some of my other work on hand gesture recognition using smart contours and particle filters

Continue reading "Hand gesture recognition via model fitting in energy minimization w/OpenCV"


20-lines AR in OpenCV [w/code]


Just wanted to share a bit of code using OpenCV's camera extrinsic parameters recovery, camera position and rotation - solvePnP (or it's C counterpart cvFindExtrinsicCameraParams2). I wanted to get a simple planar object surface recovery for augmented reality, but without using any of the AR libraries, rather dig into some OpenCV and OpenGL code.
This can serve as a primer, or tutorial on how to use OpenCV with OpenGL for AR.

Update 2/16/2015: I wrote another post on OpenCV-OpenGL AR, this time using the fine QGLViewer - a very convenient Qt OpenGL widget.

The program is just a straightforward optical flow based tracking, fed manually with four points which are the planar object's corners, and solving camera-pose every frame. Plain vanilla AR.

Well the whole cpp file is ~350 lines, but there will only be 20 or less interesting lines... Actually much less. Let's see what's up
Continue reading "20-lines AR in OpenCV [w/code]"


Image Recoloring using Gaussian Mixture Model and Expectation Maximization [OpenCV, w/Code]

I'll present a quick and simple implementation of image recoloring, in fact more like color transfer between images, using OpenCV in C++ environment. The basis of the algorithm is learning the source color distribution with a GMM using EM, and then applying changes to the target color distribution. It's fairly easy to implement with OpenCV, as all the "tools" are built in.

I was inspired by Lior Shapira's work that was presented in Eurographics 09 about image appearance manipulation, and a work about recoloring for the colorblind by Huang et al presented at ICASSP 09. Both works deal with color manipulation using Gaussian Mixture Models.

Update 5/28/2015: Adrien contributed code that works with OpenCV v3! Thanks!

Let's see how it's done!
Continue reading "Image Recoloring using Gaussian Mixture Model and Expectation Maximization [OpenCV, w/Code]"


Iterative Closest Point (ICP) for 2D curves with OpenCV [w/ code]

ICP - Iterative closest point, is a very trivial algorithm for matching object templates to noisy data. It's also super easy to program, so it's good material for a tutorial. The goal is to take a known set of points (usually defining a curve or object exterior) and register it, as good as possible, to a set of other points, usually a larger and noisy set in which we would like to find the object. The basic algorithm is described very briefly in wikipedia, but there are a ton of papers on the subject.

I'll take you through the steps of programming it with OpenCV.
Continue reading "Iterative Closest Point (ICP) for 2D curves with OpenCV [w/ code]"


Bust out your own graphcut based image segmentation with OpenCV [w/ code]

This is a tutorial on using Graph-Cuts and Gaussian-Mixture-Models for image segmentation with OpenCV in C++ environment.

Update 10/30/2017: See a new implementation of this method using OpenCV-Python, PyMaxflow, SLIC superpixels, Delaunay and other tricks.

Been wokring on my masters thesis for a while now, and the path of my work came across image segmentation. Naturally I became interested in Max-Flow Graph Cuts algorithms, being the "hottest fish in the fish-market" right now if the fish market was the image segmentation scene.

So I went looking for a CPP implementation of graphcut, only to find out that OpenCV already implemented it in v2.0 as part of their GrabCut impl. But I wanted to explore a bit, so I found this implementation by Olga Vexler, which is build upon Kolmogorov's framework for max-flow algorithms. I was also inspired by Shai Bagon's usage example of this implementation for Matlab.

Let's jump in...
Continue reading "Bust out your own graphcut based image segmentation with OpenCV [w/ code]"


Quick and Easy Head Pose Estimation with OpenCV [w/ code]

Update: check out my new post about this

Just wanted to share a small thing I did with OpenCV - Head Pose Estimation (sometimes known as Gaze Direction Estimation). Many people try to achieve this and there are a ton of papers covering it, including a recent overview of almost all known methods.

I implemented a very quick & dirty solution based on OpenCV's internal methods that produced surprising results (I expected it to fail), so I decided to share. It is based on 3D-2D point correspondence and then fitting of the points to the 3D model. OpenCV provides a magical method - solvePnP - that does this, given some calibration parameters that I completely disregarded.

Here's how it's done

Continue reading "Quick and Easy Head Pose Estimation with OpenCV [w/ code]"