3d code graphics opencv opengl programming Recommended vision Website

Quick and Easy Head Pose Estimation with OpenCV [w/ code]

Update: check out my new post about this
Just wanted to share a small thing I did with OpenCV – Head Pose Estimation (sometimes known as Gaze Direction Estimation). Many people try to achieve this and there are a ton of papers covering it, including a recent overview of almost all known methods.
I implemented a very quick & dirty solution based on OpenCV’s internal methods that produced surprising results (I expected it to fail), so I decided to share. It is based on 3D-2D point correspondence and then fitting of the points to the 3D model. OpenCV provides a magical method – solvePnP – that does this, given some calibration parameters that I completely disregarded.
Here’s how it’s done

code graphics opencv opengl programming Recommended school video vision Website

Implementing PTAM: stereo, tracking and pose estimation for AR with OpenCV [w/ code]

Been working hard at a project for school the past month, implementing one of the more interesting works I’ve seen in the AR arena: Parallel Tracking and Mapping (PTAM) [PDF]. This is a work by George Klein [homepage] and David Murray from Oxford university, presented in ISMAR 2007.
When I first saw it on youtube [link] I immediately saw the immense potential – mobile markerless augmented reality. I thought I should get to know this work a bit more closely, so I chose to implement it as a part of advanced computer vision course, given by Dr. Lior Wolf [link] at TAU.
The work is very extensive, and clearly is a result of deep research in the field, so I set to achieve a few selected features: Stereo initialization, Tracking, and small map upkeeping. I chose not to implement relocalization and full map handling.
This post is kind of a tutorial for 3D reconstruction with OpenCV 2.0. I will show practical use of the functions in cvtriangulation.cpp, which are not documented and in fact incomplete. Furthermore I’ll show how to easily combine OpenCV and OpenGL for 3D augmentations, a thing which is only briefly described in the docs or online.
Here are the step I took and things I learned in the process of implementing the work.
Update: A nice patch by yazor fixes the video mismatching – thanks! and also a nice application by Zentium called “iKat” is doing some kick-ass mobile markerless augmented reality.

graphics Mobile phones programming video vision Website

iPhone OS 3.x Raw data of camera frames

Hi All
It looks like it’s finally here – a way to grab the raw data of the camera frames on the iPhone OS 3.x.
Update: Apple officially supports this in iOS 4.x using AVFoundation, here’s sample code from Apple developer.
A gifted hacker named John DeWeese was nice enough to comment on a post from May 09′ with his method of hacking the APIs to get the frames. Though cumbersome, it looks like it should work, but I haven’t tried it yet. I promise to try it soon and share my results.
Way to go John!
Some code would be awesome…

code Java linux programming school video Website

SmartHome – Embedded computing course project

In the past few weeks I have been working hard at a few projects for end-of-term at Uni. One of the projects is what I called “SmartHome”, for Embedded computing [link] course, is a home monitoring [link] application. In the course the students were given an LPC2148 arm7-MCU (NXP) based education board, implemented by Embedded Artists [link]. My partner Gil and I decided to work with ZigBee extension modules [link] to enable remote communication.
Here are the steps we took to bring this project to life.

Recommended Website

Links of the week,7124,s6-240-319–13001-0,00.html
Shoes tying hacks
Very nice! A digital guitar…
An interesting concept – see-through walls w/ augmentd reality
The “Youth market”‘s little brother – the “Toddler market” – is booming
Some goodies from iPhone OS 4 – where is video-pixel-bytes access already?!
I like! A helper webapp to memorize text—
This is awesome.
YouTube without flash: I tried it on Chrome, the video was choppy, volume control didn’t work proerly and the progressing download & play made the position marker bounce around. But in the end, anything that replaces Flash, and Adobe’s reign over internet interactive animation,  is good..
C ya’ll next week!

Recommended Website

Never fear – the links are here [Links of the week]

Stuff I picked up on the web the last week:
Compare the leading smartphones on the market
Mobile web usage stats: iPhone 65%, Android 12%, RIM 9%
A transparent screen – Cool? yes. Practical? Not so much.
Augmented reality is officially more popular than virtual reality.
You don’t need a mouse anymore (if you have a 154 frames-per-second camera, and very steady hands)
Samsung’s projector mobile phone in action in CES
Apple is putting proximity sensors on new device to allow for 3D desktop manipulation.
AT&T goes app-store on dumbphones, releases SDK for BREW
C y’all next week!

Recommended Website

Just links [Links of the week]

Stuff I picked up on the web the last week:
Never tired of infographics: World map of social networks
Awe-some and then some. Enhance.
Give Ribbit credit for the bold stand in front of G-Voice.
A G netbook! That’s what we’ve been missing! Not.
3D interface by Apple, based on position of user.
Orange Israel promoting iPhones in a cute way: iPhone inside iPhone with AR.
JaJah sold to Telefonica (O2) – for 145 million euros!
See ya’ll next week!

Recommended Website

Leenky wiks! [Links of the week]

Hello people of high measure,
Forth are listed the hyperlinks thy humble servant hath collected in past days:
Nice ad by G for Chrome!
Very interesting: huge database of geo-tagged information with API for developers.
I saw this contraption in the lab and the guy demoed it for me. It’s strange looking, but an interesting concept.
A G phone!
This is only a fraction of the online buzz about…
Air-Pen. Nice implementation. But, is this is how we’re going to interface with computers in the future? Don’t think so.
Välkommen till Sverige – LTE! (“Welcome to Sweden – LTE!” in Swedish)
TeliaSonera launching LTE
Free Text-To-Speech from G! Hurrah!
TAT are as usual a good indicator of future UI (Social AR…). This time: 3D UI, haptic interface.
Some stats on the Android market, looks good.
G seem very committed to making the web better. First tools for webmasters to make their sites faster, and making sure the resolution fits
Anthropomorphism: Look at the video, (though it’s an AD) it will put a smile on your face!
and then go shoot some anthropomorphous objects.
Mobile OS stats (Feb-Oct): Apple and RIM skyrocket! WinMo, Symb, Palm, Android – stagnate.
Enjoy thy weekend!

graphics programming Website work

Extending Justin Talbot's GrabCut Impl [w/ code]

Justin Talbot has done a tremendous job implementing the GrabCut algorithm in C [link to paper, link to code]. I was missing though, the option to load ANY kind of file, not just PPMs and PGMs.
So I tweaked the code a bit to receive a filename and determine how to load it: use the internal P[P|G]M loaders, or offload the work to the OpenCV image loaders that take in many more type. If the OpenCV method is used, the IplImage is converted to the internal GrabCut code representation.

Image<Color>* load( std::string file_name )
 if( file_name.find( ".pgm" ) != std::string::npos )
 return loadFromPGM( file_name );
 else if( file_name.find( ".ppm" ) != std::string::npos )
 return loadFromPPM( file_name );
 return loadOpenCV(file_name);
void fromImageMaskToIplImage(const Image<Real>* image, IplImage* ipli) {
 for(int x=0;x<image->width();x++) {
 for(int y=0;y<image->height();y++) {
 //Color c = (*image)(x,y);
 Real r = (*image)(x,y);
 CvScalar s = cvScalarAll(0);
 if(r == 0.0) {
 s.val[0] = 255.0;
 cvSet2D(ipli,ipli->height - y - 1,x,s);
Image<Color>* loadIplImage(IplImage* im) {
 Image<Color>* image = new Image<Color>(im->width, im->height);
 for(int x=0;x<im->width;x++) {
 for(int y=0;y<im->height;y++) {
 CvScalar v = cvGet2D(im,im->height-y-1,x);
 Real R, G, B;
 R = (Real)((unsigned char)v.val[2])/255.0f;
 G = (Real)((unsigned char)v.val[1])/255.0f;
 B = (Real)((unsigned char)v.val[0])/255.0f;
 (*image)(x,y) = Color(R,G,B);
 return image;
Image<Color>* loadOpenCV(std::string file_name) {
 IplImage* im = cvLoadImage(file_name.c_str(),1);
 Image<Color>* i = loadIplImage(im);
 return i;

Well, there’s nothing fancy here, but it does give you a fully working GrabCut implementation on top of OpenCV… so there’s the contribution.

GrabCutNS::Image<GrabCutNS::Color>* imageGC = GrabCutNS::loadIplImage(orig);
 GrabCutNS::Image<GrabCutNS::Color>* maskGC = GrabCutNS::loadIplImage(mask);
 GrabCutNS::GrabCut *grabCut = new GrabCutNS::GrabCut( imageGC );
 IplImage* __GCtmp = cvCreateImage(cvSize(orig->width,orig->height),8,1);

I also added the GrabCutNS namespace, to differentiate the Image class from the rest of the code (that probably has an Image already).
Code is as usual available online in the SVN repo.

Recommended Website

The links are back! [Links of the week]

Hi good people,
Well after a two-week break, I’m back with some more interesting weekly links..
So here’s what I’ve picked up on the web the past 2 weeks:
Awesome 3D reconstruction work with only a webcam! I dig that stuff.
Me like! An old-new mobile social concept comes to life: Ad-Hoc Social Networks meet Foursquare.
Current mobile-phones OSs and devices usage numbers:
iPhone is slightly rising still: 48% Sep09 to 55% Oct09, Android is growing – 17% Sep09 to 20% Oct09, and Palm is losing height – 10% Sep09 to 5% Oct09.
LinkedIn API – well it’s about time!
The new Virtual Keyboard typing method: SWYPE, takes on the good ol’ QWERTY.
Check out Layar 3.0, awesome 3Ds in AR:
The HTML 5.0 craze is upon us, and it is going to change everything we know and think about web applications:
Instant wallpaper photo maker: Slow shutter and just throw the camera in the air!
This is what HTC has in store for us next year
Surface UI! not new, but this advancement is very interesting..
Nice video lecture, take a look at 5:00 – speaking about innovative ideas and solutions while negotiating:
Instead of thinking “what will I gain right now”, think about make a long-term partnership with the other side.
My intuition says this is the beginning of a war on YouTube, in the Copyrights arena. Sporadic things have been done already, now let’s have a showdown!

Google is taking over the world – again! Corner
Google coupons!
And then, just as they started to pick up, came Google and busted them all down.
Big G steps in the restaurants ranking business. Yelp, Zagat – behind you!!
Big G goes realtime web… The REAL realtime web.
Big G keeps on rolling… now conquering the visual search domain. what else?
Speech-to-text is piece of cake for Google…
Have a cracking weekend and See ya’ll next week!