Cylindrical image warping for panorama stitching

Hey-o
Just sharing a code snippet to warp images to cylindrical coordinates, in case you're stitching panoramas in Python OpenCV...

This is an improved version from what I had in class some time ago... http://hi.cs.stonybrook.edu/cse-527
It runs VERY fast. No loops involved, all matrix operations. In C++ this code would look gnarly.. Thanks Numpy!

Enjoy!
Roy

Share

Take a SWIG out of the Gesture Recognition Toolkit (GRT)

Reporting on a project I worked on for the last few weeks - porting the excellent Gesture Recognition Toolkit (GRT) to Python.
Right now it's still a pull request: https://github.com/nickgillian/grt/pull/151.

Not exactly porting, rather I've simply added Python bindings to GRT that allow you to access the GRT C++ APIs from Python.
Did it using the wonderful SWIG project. Such a wondrous tool, SWIG is. Magical.

Here are the deets
Continue reading "Take a SWIG out of the Gesture Recognition Toolkit (GRT)"

Share

Aligning faces with py opencv-dlib combo

Face alignment with Dlib and OpenCV

This is my first trial at using Jupyter notebook to write a post, hope it makes sense.

I've recently taught a class on generative models: http://hi.cs.stonybrook.edu/teaching/cdt450

In class we've manipulated face images with neural networks.

One important thing I found that helped is to align the images so the facial features overlap.
It helps the nets learn the variance in faces better, rather than waste their "representation power" on the shift between faces.

The following is some code to align face images using the excellent Dlib (python bindings) http://dlib.net. First I'm just using a standard face detector, and then using the facial fatures extractor I'm using that information for a complete alignment of the face.

After the alignment - I'm just having fun with the aligned dataset 🙂
Continue reading "Aligning faces with py opencv-dlib combo"

Share

Build your AWS Lambda Machine Learning Function with Docker

I've recently made a tutorial on using Docker for machine learning purposes, and I thought also to publish it in here: http://hi.cs.stonybrook.edu/teaching/docker4ml

It includes videos, slides and code, with hands-on demonstrations in class.

A GitHub repo holds the code: https://github.com/royshil/Docker4MLTutorial

I made several scripts to make it easy to upload python code that performs an ML inference ("prediction") operation on AWS Lambda.

Enjoy!
Roy.

Share

An automatic Tensorflow-CUDA-Docker-Jupyter machine on Google Cloud Platform


For a class I'm teaching (on deep learning and art) I had to create a machine that auto starts a jupyter notebook with tensorflow and GPU support. Just create an instance and presto - Jupyter notebook with TF and GPU!
How awesome is that?

Well... building it wasn't that simple.
So for your enjoyment - here's my recipe:
Continue reading "An automatic Tensorflow-CUDA-Docker-Jupyter machine on Google Cloud Platform"

Share

Projector-Camera Calibration - the "easy" way

First let me open by saying projector-camera calibration is NOT EASY. But it's technically not complicated too.

It is however, an amalgamation of optimizations that accrue and accumulate error with each step, so that the end product is not far from a random guess.
So 3D reconstructions I was able to get from my calibrated pro-cam were just a distorted mess of points.

Nevertheless, here come the deets.
Continue reading "Projector-Camera Calibration - the "easy" way"

Share

Revisiting graph-cut segmentation with SLIC and color histograms [w/Python]

As part of the computer vision class I'm teaching at SBU I asked students to implement a segmentation method based on SLIC superpixels. Here is my boilerplate implementation.

This follows the work I've done a very long time ago (2010) on the same subject.

For graph-cut I've used PyMaxflow: https://github.com/pmneila/PyMaxflow, which is very easily installed by just pip install PyMaxflow

The method is simple:

  • Calculate SLIC superpixels (the SKImage implementation)
  • Use markings to determine the foreground and background color histograms (from the superpixels under the markings)
  • Setup a graph with a straightforward energy model: Smoothness term = K-L-Div between superpix histogram and neighbor superpix histogram, and Match term = inf if marked as BG or FG, or K-L-Div between SuperPix histogram and FG and BG.
  • To find neighbors I've used Delaunay tessellation (from scipy.spatial), for simplicity. But a full neighbor finding could be implemented by looking at all the neighbors on the superpix's boundary.
  • Color histograms are 2D over H-S (from the HSV)

Result

Share

[Python] OpenCV capturing from a v4l2 device

I tried to set the capture format on a webcam from OpenCV's cv2.VideoCapture and ran into a problem: it's using the wrong IOCTL command.
So I used python-v4l2capture to get images from the device, which allows more control.
Here is the gist:

Enjoy!
Roy

Share

OpenCV Python YAML persistance

I wasn't able to find online a complete example on how to persist OpenCV matrices in Python (so really NumPy arrays) to YAML like what cv::FileStorage will give you.

So here's a short snippet:

import numpy as np
import yaml

# A yaml constructor is for loading from a yaml node.
# This is taken from: http://stackoverflow.com/a/15942429
def opencv_matrix_constructor(loader, node):
    mapping = loader.construct_mapping(node, deep=True)
    mat = np.array(mapping["data"])
    mat.resize(mapping["rows"], mapping["cols"])
    return mat
yaml.add_constructor(u"tag:yaml.org,2002:opencv-matrix", opencv_matrix_constructor)

# A yaml representer is for dumping structs into a yaml node.
# So for an opencv_matrix type (to be compatible with c++'s FileStorage) we save the rows, cols, type and flattened-data
def opencv_matrix_representer(dumper, mat):
    mapping = {'rows': mat.shape[0], 'cols': mat.shape[1], 'dt': 'd', 'data': mat.reshape(-1).tolist()}
    return dumper.represent_mapping(u"tag:yaml.org,2002:opencv-matrix", mapping)
yaml.add_representer(np.ndarray, opencv_matrix_representer)


#example
with open('output.yaml', 'w') as f:
    f.write("%YAML:1.0")
    yaml.dump({"a matrix": np.zeros((10,10)), "another_one": np.zeros((2,4))}, f)

#   a matrix: !!opencv-matrix
#     cols: 10
#     data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0]
#     dt: d
#     rows: 10
#   another_one: !!opencv-matrix
#     cols: 4
#     data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
#     dt: d
#     rows: 2

with open('output.yaml', 'r') as f:
    print yaml.load(f)
  
#  {'a matrix': array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]]), 'another_one': array([[ 0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.]])}

There you go

Share