Categories
code programming python Solutions Web

A script to scrape PDFs from a page using Python+Mechanize

A friend asked me for a way to download all the PDFs from a page, and I made this simple script with Python and Mechanize. It’s very straightforward…
It does hack the user agent, which is not nice. So use at your discretion.

Categories
3d code opencv programming vision

Structure-from-Motion Toy Lib Upgrades to OpenCV 3

sfm toy lib
Hello again!
After a long hiatus I’m back with an update. Recently I’ve been upgrading the Structure-from-Motion Toy Library (https://github.com/royshil/SfM-Toy-Library/) to OpenCV 3.x from OpenCV 2.4.x.

Categories
3d code graphics gui opengl qt

WTH OpenGL 4? Rendering elements arrays with VAOs and VBOs in a QGLWidget

I spent an entire day getting OpenGL 4 to display data from a VAO with VBOs so I thought I’d share the results with you guys, save you some pain.
I’m using the excellent GL wrappers from Qt, and in particular QGLShaderProgram.
This is pretty straightforward, but the thing to remember is that OpenGL is looking for the vertices/other elements (color? tex coords?) to come from some bound GL buffer or from the host. So if your app is not working and nothing appears on screen, just make sure GL has a bound buffer and the shader locations match up and consistent (see the const int I have on the class here).

Categories
code opencv python video

[Python] OpenCV capturing from a v4l2 device

I tried to set the capture format on a webcam from OpenCV’s cv2.VideoCapture and ran into a problem: it’s using the wrong IOCTL command.
So I used python-v4l2capture to get images from the device, which allows more control.
Here is the gist:

Enjoy!
Roy

Categories
code opencv python

OpenCV Python YAML persistance

I wasn’t able to find online a complete example on how to persist OpenCV matrices in Python (so really NumPy arrays) to YAML like what cv::FileStorage will give you.
So here’s a short snippet:

import numpy as np
import yaml
# A yaml constructor is for loading from a yaml node.
# This is taken from: http://stackoverflow.com/a/15942429
def opencv_matrix_constructor(loader, node):
    mapping = loader.construct_mapping(node, deep=True)
    mat = np.array(mapping["data"])
    mat.resize(mapping["rows"], mapping["cols"])
    return mat
yaml.add_constructor(u"tag:yaml.org,2002:opencv-matrix", opencv_matrix_constructor)
# A yaml representer is for dumping structs into a yaml node.
# So for an opencv_matrix type (to be compatible with c++'s FileStorage) we save the rows, cols, type and flattened-data
def opencv_matrix_representer(dumper, mat):
    mapping = {'rows': mat.shape[0], 'cols': mat.shape[1], 'dt': 'd', 'data': mat.reshape(-1).tolist()}
    return dumper.represent_mapping(u"tag:yaml.org,2002:opencv-matrix", mapping)
yaml.add_representer(np.ndarray, opencv_matrix_representer)
#example
with open('output.yaml', 'w') as f:
    f.write("%YAML:1.0")
    yaml.dump({"a matrix": np.zeros((10,10)), "another_one": np.zeros((2,4))}, f)
#   a matrix: !!opencv-matrix
#     cols: 10
#     data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
#       0.0, 0.0, 0.0, 0.0, 0.0]
#     dt: d
#     rows: 10
#   another_one: !!opencv-matrix
#     cols: 4
#     data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
#     dt: d
#     rows: 2
with open('output.yaml', 'r') as f:
    print yaml.load(f)
#  {'a matrix': array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]]), 'another_one': array([[ 0.,  0.,  0.,  0.],
#         [ 0.,  0.,  0.,  0.]])}

There you go

Categories
apache code Java programming Web

Simple Loading Spinner Tapestry 5 Mixin w/ spin.js

Sharing a small snippet on creating a loading spinner in a Tapestry 5.3+ Mixin, using spin.js.
It creates a convenient way to add spinners to your long-loading-times ajax zone updates, with all the code hidden away from the template .tml and page class object.
Sorry I can’t show a working example, that would entail running a Tapestry application server.
But it’s very straightforward, just grab the spin.min.js and the rest falls into place (it also depends on jQuery).

Categories
code Web

Adding radial labels to Dimple.JS pie chart

Too bad Dimple.JS doesn’t have this feature built-in. They only have a boring old legend, where pie charts scream for labels that circle the actual pie (or donut).
Anyway, here’s how to get it done using some d3.js love.

Categories
code Solutions Web

Bootstrap3 fluid container with custom width sidebar

I was looking for a way to get a fluid container live side-by-side with a custom width sidebar.
A custom width sidebar can’t be achieved with a Bootstrap column, and is a total mess to get right with floats if you then need a fluid container to get a grid system for the main section.
So, here’s one solution:

JSFiddle: https://jsfiddle.net/6sfog80k/

Categories
code graphics programming video

YUYV to JPEG conversion with libjpeg

Sharing a small libjpeg snippet.
Some SO questions about it have only partial snippets:


Enjoy!
Roy

Categories
code graphics opencv

Quickly: How to render a PDF to an image in C++?

Using Poppler, of course!
Poppler is a very useful tool for handling PDF, so I’ve discovered lately. Having tried both muPDF and ImageMagick’s Magick++ and failed, Poppler stepped up to the challenge and paid off.
So here’s a small example of how work the API (with OpenCV, naturally):

#include <iostream>
#include <fstream>
#include <sstream>
#include <opencv2/opencv.hpp>
#include <poppler-document.h>
#include <poppler-page.h>
#include <poppler-page-renderer.h>
#include <poppler-image.h>
using namespace cv;
using namespace std;
using namespace poppler;
Mat readPDFtoCV(const string& filename,int DPI) {
    document* mypdf = document::load_from_file(filename);
    if(mypdf == NULL) {
        cerr << "couldn't read pdf\n";
        return Mat();
    }
    cout << "pdf has " << mypdf->pages() << " pages\n";
    page* mypage = mypdf->create_page(0);
    page_renderer renderer;
    renderer.set_render_hint(page_renderer::text_antialiasing);
    image myimage = renderer.render_page(mypage,DPI,DPI);
    cout << "created image of  " << myimage.width() << "x"<< myimage.height() << "\n";
    Mat cvimg;
    if(myimage.format() == image::format_rgb24) {
        Mat(myimage.height(),myimage.width(),CV_8UC3,myimage.data()).copyTo(cvimg);
    } else if(myimage.format() == image::format_argb32) {
        Mat(myimage.height(),myimage.width(),CV_8UC4,myimage.data()).copyTo(cvimg);
    } else {
        cerr << "PDF format no good\n";
        return Mat();
    }
    return cvimg;
}

All you have to do is give it the DPI (say you want to render in 100 DPI) and a filename.
Keep in mind it only renders the first page, but getting the other pages is just as easy.
That’s it, enjoy!
Roy.