A friend asked me for a way to download all the PDFs from a page, and I made this simple script with Python and Mechanize. It’s very straightforward…
It does hack the user agent, which is not nice. So use at your discretion.
Category: code
Hello again!
After a long hiatus I’m back with an update. Recently I’ve been upgrading the Structure-from-Motion Toy Library (https://github.com/royshil/SfM-Toy-Library/) to OpenCV 3.x from OpenCV 2.4.x.
I spent an entire day getting OpenGL 4 to display data from a VAO with VBOs so I thought I’d share the results with you guys, save you some pain.
I’m using the excellent GL wrappers from Qt, and in particular QGLShaderProgram.
This is pretty straightforward, but the thing to remember is that OpenGL is looking for the vertices/other elements (color? tex coords?) to come from some bound GL buffer or from the host. So if your app is not working and nothing appears on screen, just make sure GL has a bound buffer and the shader locations match up and consistent (see the const int I have on the class here).
I tried to set the capture format on a webcam from OpenCV’s cv2.VideoCapture and ran into a problem: it’s using the wrong IOCTL command.
So I used python-v4l2capture to get images from the device, which allows more control.
Here is the gist:
Enjoy!
Roy
I wasn’t able to find online a complete example on how to persist OpenCV matrices in Python (so really NumPy arrays) to YAML like what cv::FileStorage will give you.
So here’s a short snippet:
import numpy as np
import yaml
# A yaml constructor is for loading from a yaml node.
# This is taken from: http://stackoverflow.com/a/15942429
def opencv_matrix_constructor(loader, node):
mapping = loader.construct_mapping(node, deep=True)
mat = np.array(mapping["data"])
mat.resize(mapping["rows"], mapping["cols"])
return mat
yaml.add_constructor(u"tag:yaml.org,2002:opencv-matrix", opencv_matrix_constructor)
# A yaml representer is for dumping structs into a yaml node.
# So for an opencv_matrix type (to be compatible with c++'s FileStorage) we save the rows, cols, type and flattened-data
def opencv_matrix_representer(dumper, mat):
mapping = {'rows': mat.shape[0], 'cols': mat.shape[1], 'dt': 'd', 'data': mat.reshape(-1).tolist()}
return dumper.represent_mapping(u"tag:yaml.org,2002:opencv-matrix", mapping)
yaml.add_representer(np.ndarray, opencv_matrix_representer)
#example
with open('output.yaml', 'w') as f:
f.write("%YAML:1.0")
yaml.dump({"a matrix": np.zeros((10,10)), "another_one": np.zeros((2,4))}, f)
# a matrix: !!opencv-matrix
# cols: 10
# data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
# 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
# 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
# 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
# 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
# 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
# 0.0, 0.0, 0.0, 0.0, 0.0]
# dt: d
# rows: 10
# another_one: !!opencv-matrix
# cols: 4
# data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
# dt: d
# rows: 2
with open('output.yaml', 'r') as f:
print yaml.load(f)
# {'a matrix': array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]]), 'another_one': array([[ 0., 0., 0., 0.],
# [ 0., 0., 0., 0.]])}
There you go
Sharing a small snippet on creating a loading spinner in a Tapestry 5.3+ Mixin, using spin.js.
It creates a convenient way to add spinners to your long-loading-times ajax zone updates, with all the code hidden away from the template .tml and page class object.
Sorry I can’t show a working example, that would entail running a Tapestry application server.
But it’s very straightforward, just grab the spin.min.js and the rest falls into place (it also depends on jQuery).
I was looking for a way to get a fluid container live side-by-side with a custom width sidebar.
A custom width sidebar can’t be achieved with a Bootstrap column, and is a total mess to get right with floats if you then need a fluid container to get a grid system for the main section.
So, here’s one solution:
JSFiddle: https://jsfiddle.net/6sfog80k/
Sharing a small libjpeg snippet.
Some SO questions about it have only partial snippets:
- http://stackoverflow.com/questions/16390783/how-to-save-yuyv-raw-data-to-jpeg-using-libjpeg
- http://stackoverflow.com/questions/17029136/weird-image-while-trying-to-compress-yuv-image-to-jpeg-using-libjpeg
- http://stackoverflow.com/questions/19282402/how-to-compress-a-yuyv-image-into-a-jpeg
Enjoy!
Roy
Using Poppler, of course!
Poppler is a very useful tool for handling PDF, so I’ve discovered lately. Having tried both muPDF and ImageMagick’s Magick++ and failed, Poppler stepped up to the challenge and paid off.
So here’s a small example of how work the API (with OpenCV, naturally):
#include <iostream>
#include <fstream>
#include <sstream>
#include <opencv2/opencv.hpp>
#include <poppler-document.h>
#include <poppler-page.h>
#include <poppler-page-renderer.h>
#include <poppler-image.h>
using namespace cv;
using namespace std;
using namespace poppler;
Mat readPDFtoCV(const string& filename,int DPI) {
document* mypdf = document::load_from_file(filename);
if(mypdf == NULL) {
cerr << "couldn't read pdf\n";
return Mat();
}
cout << "pdf has " << mypdf->pages() << " pages\n";
page* mypage = mypdf->create_page(0);
page_renderer renderer;
renderer.set_render_hint(page_renderer::text_antialiasing);
image myimage = renderer.render_page(mypage,DPI,DPI);
cout << "created image of " << myimage.width() << "x"<< myimage.height() << "\n";
Mat cvimg;
if(myimage.format() == image::format_rgb24) {
Mat(myimage.height(),myimage.width(),CV_8UC3,myimage.data()).copyTo(cvimg);
} else if(myimage.format() == image::format_argb32) {
Mat(myimage.height(),myimage.width(),CV_8UC4,myimage.data()).copyTo(cvimg);
} else {
cerr << "PDF format no good\n";
return Mat();
}
return cvimg;
}
All you have to do is give it the DPI (say you want to render in 100 DPI) and a filename.
Keep in mind it only renders the first page, but getting the other pages is just as easy.
That’s it, enjoy!
Roy.