WTH OpenGL 4? Rendering elements arrays with VAOs and VBOs in a QGLWidget

I spent an entire day getting OpenGL 4 to display data from a VAO with VBOs so I thought I'd share the results with you guys, save you some pain.

I'm using the excellent GL wrappers from Qt, and in particular QGLShaderProgram.
This is pretty straightforward, but the thing to remember is that OpenGL is looking for the vertices/other elements (color? tex coords?) to come from some bound GL buffer or from the host. So if your app is not working and nothing appears on screen, just make sure GL has a bound buffer and the shader locations match up and consistent (see the const int I have on the class here).


Head Pose Estimation with OpenCV & OpenGL Revisited [w/ code]

So I was contacted earlier by someone asking about the Head Pose Estimation work I put up a while back. And I remembered that I needed to go back to that work and fix some things, so it was a great opportunity.
I ended up making it a bit nicer, and it's also a good chance for us to review some OpenCV-OpenGL interoperation stuff. Things like getting a projection matrix in OpenCV and translating it to an OpenGL ModelView matrix, are very handy.
Let's get down to the code.
Continue reading "Head Pose Estimation with OpenCV & OpenGL Revisited [w/ code]"


Spherical harmonics face relighting using OpenCV, OpenGL [w/ code]

I've been working on implementing a face image relighting algorithm using spherical harmonics, one of the most elegant methods I've seen lately.
I start up by aligning a face model with OpenGL to automatically get the canonical face normals, which brushed up my knowledge of GLSL. Then I continue to estimating real faces "spharmonics", and relighting.

Let's start!
Continue reading "Spherical harmonics face relighting using OpenCV, OpenGL [w/ code]"


20-lines AR in OpenCV [w/code]


Just wanted to share a bit of code using OpenCV's camera extrinsic parameters recovery, camera position and rotation - solvePnP (or it's C counterpart cvFindExtrinsicCameraParams2). I wanted to get a simple planar object surface recovery for augmented reality, but without using any of the AR libraries, rather dig into some OpenCV and OpenGL code.
This can serve as a primer, or tutorial on how to use OpenCV with OpenGL for AR.

Update 2/16/2015: I wrote another post on OpenCV-OpenGL AR, this time using the fine QGLViewer - a very convenient Qt OpenGL widget.

The program is just a straightforward optical flow based tracking, fed manually with four points which are the planar object's corners, and solving camera-pose every frame. Plain vanilla AR.

Well the whole cpp file is ~350 lines, but there will only be 20 or less interesting lines... Actually much less. Let's see what's up
Continue reading "20-lines AR in OpenCV [w/code]"


Quick and Easy Head Pose Estimation with OpenCV [w/ code]

Update: check out my new post about this http://www.morethantechnical.com/2012/10/17/head-pose-estimation-with-opencv-opengl-revisited-w-code/

Just wanted to share a small thing I did with OpenCV - Head Pose Estimation (sometimes known as Gaze Direction Estimation). Many people try to achieve this and there are a ton of papers covering it, including a recent overview of almost all known methods.

I implemented a very quick & dirty solution based on OpenCV's internal methods that produced surprising results (I expected it to fail), so I decided to share. It is based on 3D-2D point correspondence and then fitting of the points to the 3D model. OpenCV provides a magical method - solvePnP - that does this, given some calibration parameters that I completely disregarded.

Here's how it's done

Continue reading "Quick and Easy Head Pose Estimation with OpenCV [w/ code]"


Implementing PTAM: stereo, tracking and pose estimation for AR with OpenCV [w/ code]


Been working hard at a project for school the past month, implementing one of the more interesting works I've seen in the AR arena: Parallel Tracking and Mapping (PTAM) [PDF]. This is a work by George Klein [homepage] and David Murray from Oxford university, presented in ISMAR 2007.

When I first saw it on youtube [link] I immediately saw the immense potential - mobile markerless augmented reality. I thought I should get to know this work a bit more closely, so I chose to implement it as a part of advanced computer vision course, given by Dr. Lior Wolf [link] at TAU.

The work is very extensive, and clearly is a result of deep research in the field, so I set to achieve a few selected features: Stereo initialization, Tracking, and small map upkeeping. I chose not to implement relocalization and full map handling.

This post is kind of a tutorial for 3D reconstruction with OpenCV 2.0. I will show practical use of the functions in cvtriangulation.cpp, which are not documented and in fact incomplete. Furthermore I'll show how to easily combine OpenCV and OpenGL for 3D augmentations, a thing which is only briefly described in the docs or online.

Here are the step I took and things I learned in the process of implementing the work.

Update: A nice patch by yazor fixes the video mismatching - thanks! and also a nice application by Zentium called "iKat" is doing some kick-ass mobile markerless augmented reality.
Continue reading "Implementing PTAM: stereo, tracking and pose estimation for AR with OpenCV [w/ code]"