«

»

Jan 28

Recoloring via Histogram Matching with OpenCV [w/ code]

Hi

I wanted to do the simplest recoloring/color-transfer I could find - and the internet is just a bust. Nothing free, good and usable available online... So I implemented the simplest color transfer algorithm in the wolrd - Histogram Matching.

Here's the implementation with OpenCV

// Compute histogram and CDF for an image with mask
void do1ChnHist(const Mat& _i, const Mat& mask, double* h, double* cdf) {
Mat _t = _i.reshape(1,1);
Mat _tm;
mask.copyTo(_tm);
_tm = _tm.reshape(1,1);
for(int p=0;p<_t.cols;p++) {
if(_tm.at(0,p) > 0) {
uchar c = _t.at(0,p);
h += 1.0;
}
}

//normalize hist
Mat _tmp(1,256,CV_64FC1,h);
double minVal,maxVal;
minMaxLoc(_tmp,&minVal,&maxVal);
_tmp = _tmp / maxVal;

cdf[0] = h[0];
for(int j=1;j<256;j++) {
cdf[j] = cdf[j-1]+h[j];
}

//normalize CDF
_tmp.data = (uchar*)cdf;
minMaxLoc(_tmp,&minVal,&maxVal);
_tmp = _tmp / maxVal;
}

// match histograms of 'src' to that of 'dst', according to both masks
void histMatchRGB(Mat& src, const Mat& src_mask, const Mat& dst, const Mat& dst_mask) {
#ifdef BTM_DEBUG
	namedWindow("original source",CV_WINDOW_AUTOSIZE);
	imshow("original source",src);
	namedWindow("original query",CV_WINDOW_AUTOSIZE);
	imshow("original query",dst);
#endif

	vector<Mat> chns;
	split(src,chns);
	vector<Mat> chns1;
	split(dst,chns1);
	Mat src_hist = Mat::zeros(1,256,CV_64FC1);
	Mat dst_hist = Mat::zeros(1,256,CV_64FC1);
	Mat src_cdf = Mat::zeros(1,256,CV_64FC1);
	Mat dst_cdf = Mat::zeros(1,256,CV_64FC1);
	Mat Mv(1,256,CV_8UC1);
	uchar* M = Mv.ptr<uchar>();

	for(int i=0;i<3;i++) {
		src_hist.setTo(cvScalar(0));
		dst_hist.setTo(cvScalar(0));
		src_cdf.setTo(cvScalar(0));
		src_cdf.setTo(cvScalar(0));

		do1ChnHist(chns&#91;i&#93;,src_mask,src_hist,src_cdf);
		do1ChnHist(chns1&#91;i&#93;,dst_mask,dst_hist,dst_cdf);

		uchar last = 0;
		double* _src_cdf = src_cdf.ptr<double>();
		double* _dst_cdf = dst_cdf.ptr<double>();

		for(int j=0;j<src_cdf.cols;j++) {
			double F1j = _src_cdf&#91;j&#93;;

			for(uchar k = last; k<dst_cdf.cols; k++) {
				double F2k = _dst_cdf&#91;k&#93;;
				if(abs(F2k - F1j) < HISTMATCH_EPSILON || F2k > F1j) {
					M[j] = k;
					last = k;
					break;
				}
			}
		}

		Mat lut(1,256,CV_8UC1,M);
		LUT(chns[i],lut,chns[i]);
	}

	Mat res;
	merge(chns,res);

#ifdef BTM_DEBUG
	namedWindow("matched",CV_WINDOW_AUTOSIZE);
	imshow("matched",res);

	waitKey(BTM_WAIT_TIME);
#endif

	res.copyTo(src);
}

Edit 31/1/10: nicer code in histMatch, better memory consumption.

The code is fairly simple.. I started using the OpenCV 2.0 C++ API and I must say it's great, not having to worry about silly memory managment, everything is instanced on the stack and get's released automatically.

Just a quick overview:

  • Compute histogram and comulative distibution function (CDF) for each image.
  • Create the lookup table (LUT) from one CDF to the other
  • Apply the LUT to the image

External usage is simple

Mat src=cvLoadImage("image008.jpg");
Mat dst=cvLoadImage("image003.jpg");
Mat src_mask = cvLoadImage("image008_mask.bmp",0);
Mat dst_mask = cvLoadImage("image003_mask.bmp",0);

histMatchRGB(dst,dst_mask,src,src_mask);

Enjoy!
Roy.

Share
  • Zac

    What value are you using for HISTMATCH_EPSILON?

  • http://www.morethantechnical.com Roy

    #define HISTMATCH_EPSILON 0.000001

  • Zac

    Thanks very much Roy, I am currently looking at modifying your code to work with 16 bit images.

    Zac

  • http://www.b4silio.com Basilio

    Thanks a lot for your effort! there's a couple of hiccups that needed to be fixed (at least for me) for the code to run but it does, and very smoothly! Very nice and elegant! (and it finally got me to switch to proper cv2.0...)

    Much appreciated!

  • http://www.morethantechnical.com Roy

    Hi Basilio,
    Thanks
    Just saw your online portfolio - some awesome stuff you got there!
    R.

  • Andrea

    Hi, this code is great, i'm using it but i have some trouble with some artifacts, when there is a great chromatic difference betwen the images some strange halo, with wrong colors, appears :(

    How i can solve it?

  • Umar

    i need to match the histogram of one image to with another so that the histogram of first image looks like the histogram of second image.
    We can do this in matlab in this way
    imgA=imread(some img);
    [hist x]=imhist(imgB);
    imgA_new= histeq(imgA,hist);

    Here we just need two images and there histograms. But I am unable to understand that what is the purpose of following images in the code above
    Mat src_mask = cvLoadImage("image008_mask.bmp",0);
    Mat dst_mask = cvLoadImage("image003_mask.bmp",0);
    Can u plz elaborate it a lil bit more. I am new in computer vision and opencv. And kindly also explain dat wat do u mean by image mask in the code above

  • http://www.morethantechnical.com Roy

    @Umar, The masks tell the algorithm which pixels in the source and target images should change colors. In the mask, every pixel that has a 255 value is included, and 0 value is excluded.
    If you are looking for global color-transfer, just make the masks be white (255) all over.

  • Ingo

    Hey there,

    I'm trying your code with the new OpenCV 2.2 C++ API and I have discovered some difficulties. First, I need to specify the data type of some template calls as in _tm.at(0, p). I changed it to _tm.at(0, p) and now it seems ok.

    The second issue is with the calls of the do1ChnHist function. You are calling this function with variables of type cv::Mat as third and fourth parameter, but it is specified to take double* instead. Could you please explain to me how that is supposed to work?

    I'm really only beginning with using C++ and OpenCV, so maybe it's quite obvious how to fix this, but I wasn't able to figure it out by now.

    Thanks a lot in advance for your answer and of course for sharing your work with us!

  • http://www.shervinemami.co.cc/ Shervin Emami

    Hi Roy,

    Thanks for this nice code example. But like others had noted, it has some problems with most compilers, so here is a working version of the code (tested on GCC):

    void do1ChnHist(const cv::Mat& _i, const cv::Mat* mask, double* h, double* cdf);

    // Match Red, Green and Blue histograms of 'src' to that of 'dst', according to both masks.
    // Based on: http://www.morethantechnical.com/2010/01/28/recoloring-via-histogram-matching-with-opencv-w-code/
    // Modified by Shervin Emami so it can also pass NULL for the masks if you want to process the whole image.
    void histMatchRGB(cv::Mat& src, const cv::Mat* src_mask, const cv::Mat& dst, const cv::Mat* dst_mask)
    {
    vector chns;
    cv::split(src,chns);
    vector chns1;
    cv::split(dst,chns1);
    cv::Mat src_hist = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat dst_hist = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat src_cdf = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat dst_cdf = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat Mv(1,256,CV_8UC1);
    uchar* M = Mv.ptr();

    for(int i=0;i<3;i++) {
    src_hist.setTo(cvScalar(0));
    dst_hist.setTo(cvScalar(0));
    src_cdf.setTo(cvScalar(0));
    src_cdf.setTo(cvScalar(0));

    double* _src_cdf = src_cdf.ptr();
    double* _dst_cdf = dst_cdf.ptr();
    double* _src_hist = src_hist.ptr();
    double* _dst_hist = dst_hist.ptr();

    do1ChnHist(chns[i], src_mask, _src_hist, _src_cdf);
    do1ChnHist(chns1[i], dst_mask, _dst_hist, _dst_cdf);

    uchar last = 0;
    double const HISTMATCH_EPSILON = 0.000001;

    for(int j=0;j<src_cdf.cols;j++) {
    double F1j = _src_cdf[j];

    for(uchar k = last; k<dst_cdf.cols; k++) {
    double F2k = _dst_cdf[k];
    // Note: Two tests were combined into one for efficiency, by Shervin Emami, Apr 24th 2011.
    //if (abs(F2k - F1j) F1j) {
    if (F2k > F1j - HISTMATCH_EPSILON) {
    M[j] = k;
    last = k;
    break;
    }
    }
    }

    cv::Mat lut(1,256,CV_8UC1,M);
    cv::LUT(chns[i],lut,chns[i]);
    }

    cv::Mat res;
    cv::merge(chns,res);

    res.copyTo(src);
    }

    // Compute histogram and CDF for an image with mask
    void do1ChnHist(const cv::Mat& _i, const cv::Mat* mask, double* h, double* cdf)
    {
    cv::Mat _t = _i.reshape(1,1);

    // Get the histogram with or without the mask
    if (mask) {
    cv::Mat _tm;
    mask->copyTo(_tm);
    _tm = _tm.reshape(1,1);
    for(int p=0; p<_t.cols; p++) {
    uchar m = _tm.at(0,p);
    if(m > 0) { // Mask value
    uchar c = _t.at(0,p); // Image value
    h += 1.0; // Histogram
    }
    }
    }
    else {
    for(int p=0; p<_t.cols; p++) {
    uchar c = _t.at(0,p); // Image value
    h += 1.0; // Histogram
    }
    }

    //normalize hist to a max value of 1.0
    cv::Mat _tmp(1,256,CV_64FC1,h);
    double minVal,maxVal;
    cv::minMaxLoc(_tmp, &minVal, &maxVal);
    _tmp = _tmp / maxVal;

    // Calculate the Cumulative Distribution Function
    cdf[0] = h[0];
    for(int j=1; j<256; j++) {
    cdf[j] = cdf[j-1]+h[j];
    }

    //normalize CDF to a max value of 1.0
    _tmp.data = (uchar*)cdf; // Array of doubles, but gets a byte pointer.
    cv::minMaxLoc(_tmp, &minVal, &maxVal);
    _tmp = _tmp / maxVal;
    }

  • rz

    my minor changes

    void do1ChnHist(const cv::Mat& _i, const cv::Mat* mask, double* h, double* cdf);

    // Match Red, Green and Blue histograms of ‘src’ to that of ‘dst’, according to both masks.
    // Based on: http://www.morethantechnical.com/2010/01/28/recoloring-via-histogram-matching-with-opencv-w-code/
    // Modified by Shervin Emami so it can also pass NULL for the masks if you want to process the whole image.
    void histMatchRGB(cv::Mat& src, const cv::Mat* src_mask, const cv::Mat& dst, const cv::Mat* dst_mask)
    {
    double const HISTMATCH_EPSILON = 0.000001;

    vector chns;
    cv::split(src,chns);
    vector chns1;
    cv::split(dst,chns1);
    cv::Mat src_hist = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat dst_hist = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat src_cdf = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat dst_cdf = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat Mv(1,256,CV_8UC1);
    uchar* M = Mv.ptr();

    for(int i=0;i<3;i++)
    {
    src_hist.setTo(0);
    dst_hist.setTo(0);
    src_cdf.setTo(0);
    src_cdf.setTo(0);

    double* _src_cdf = src_cdf.ptr();
    double* _dst_cdf = dst_cdf.ptr();
    double* _src_hist = src_hist.ptr();
    double* _dst_hist = dst_hist.ptr();

    do1ChnHist(chns[i], src_mask, _src_hist, _src_cdf);
    do1ChnHist(chns1[i], dst_mask, _dst_hist, _dst_cdf);

    uchar last = 0;

    for(int j=0;j<src_cdf.cols;j++)
    {
    double F1j = _src_cdf[j];

    for(uchar k = last; k (F1j - HISTMATCH_EPSILON))
    {
    M[j] = k;
    last = k;
    break;
    }
    }
    }

    cv::Mat lut(1,256,CV_8UC1,M);
    cv::LUT(chns[i],lut,chns[i]);
    }

    cv::Mat res;
    cv::merge(chns,res);

    res.copyTo(src);
    }

    // Compute histogram and CDF for an image with mask
    void do1ChnHist(const cv::Mat& _i, const cv::Mat* mask, double* h, double* cdf)
    {
    cv::Mat _t = _i.reshape(1,1);

    // Get the histogram with or without the mask
    if (mask)
    {
    cv::Mat _tm;
    mask->copyTo(_tm);
    _tm = _tm.reshape(1,1);
    for(int p=0; p<_t.cols; p++)
    {
    CvPoint po;
    po.x = 0;
    po.y = p;
    uchar m = _tm.at(po);

    if(m > 0)
    { // Mask value
    uchar c = _t.at(0,p); // Image value
    h += 1.0; // Histogram
    }
    }
    }
    else
    {
    //uchar* Uptr = _t.ptr(0);
    for(int p = 0; p < _t.cols; p++)
    {
    //Uptr[p] = saturate_cast((Uptr[p]-128)/2 + 128);
    uchar c = _t.at(0, p);
    //Vxy = saturate_cast((Vxy-128)/2 + 128);

    //uchar c = _t(0,p); // Image value
    h+= 1.0; // Histogram
    }
    }

    //normalize hist to a max value of 1.0
    cv::Mat _tmp(1,256,CV_64FC1,h);
    double minVal,maxVal;
    cv::minMaxLoc(_tmp, &minVal, &maxVal);
    _tmp = _tmp / maxVal;

    // Calculate the Cumulative Distribution Function
    cdf[0] = h[0];
    for(int j=1; j<256; j++)
    {
    cdf[j] = cdf[j-1]+h[j];
    }

    //normalize CDF to a max value of 1.0
    _tmp.data = (uchar*)cdf; // Array of doubles, but gets a byte pointer.
    cv::minMaxLoc(_tmp, &minVal, &maxVal);
    _tmp = _tmp / maxVal;
    }

  • rz

    void do1ChnHist(const cv::Mat& _i, const cv::Mat* mask, double* h, double* cdf);

    // Match Red, Green and Blue histograms of ‘src’ to that of ‘dst’, according to both masks.
    // Based on: http://www.morethantechnical.com/2010/01/28/recoloring-via-histogram-matching-with-opencv-w-code/
    // Modified by Shervin Emami so it can also pass NULL for the masks if you want to process the whole image.
    void histMatchRGB(cv::Mat& src, const cv::Mat* src_mask, const cv::Mat& dst, const cv::Mat* dst_mask)
    {
    double const HISTMATCH_EPSILON = 0.000001;

    vector chns;
    cv::split(src,chns);
    vector chns1;
    cv::split(dst,chns1);
    cv::Mat src_hist = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat dst_hist = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat src_cdf = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat dst_cdf = cv::Mat::zeros(1,256,CV_64FC1);
    cv::Mat Mv(1,256,CV_8UC1);
    uchar* M = Mv.ptr();

    for(int i=0;i<3;i++)
    {
    src_hist.setTo(0);
    dst_hist.setTo(0);
    src_cdf.setTo(0);
    src_cdf.setTo(0);

    double* _src_cdf = src_cdf.ptr();
    double* _dst_cdf = dst_cdf.ptr();
    double* _src_hist = src_hist.ptr();
    double* _dst_hist = dst_hist.ptr();

    do1ChnHist(chns[i], src_mask, _src_hist, _src_cdf);
    do1ChnHist(chns1[i], dst_mask, _dst_hist, _dst_cdf);

    uchar last = 0;

    for(int j=0;j<src_cdf.cols;j++)
    {
    double F1j = _src_cdf[j];

    for(uchar k = last; k (F1j - HISTMATCH_EPSILON))
    {
    M[j] = k;
    last = k;
    break;
    }
    }
    }

    cv::Mat lut(1,256,CV_8UC1,M);
    cv::LUT(chns[i],lut,chns[i]);
    }

    cv::Mat res;
    cv::merge(chns,res);

    res.copyTo(src);
    }

    // Compute histogram and CDF for an image with mask
    void do1ChnHist(const cv::Mat& _i, const cv::Mat* mask, double* h, double* cdf)
    {
    cv::Mat _t = _i.reshape(1,1);

    // Get the histogram with or without the mask
    if (mask)
    {
    cv::Mat _tm;
    mask->copyTo(_tm);
    _tm = _tm.reshape(1,1);
    for(int p=0; p<_t.cols; p++)
    {
    CvPoint po;
    po.x = 0;
    po.y = p;
    uchar m = _tm.at<>(po);

    if(m > 0)
    { // Mask value
    uchar c = _t.at(0,p); // Image value
    h += 1.0; // Histogram
    }
    }
    }
    else
    {
    //uchar* Uptr = _t.ptr(0);
    for(int p = 0; p < _t.cols; p++)
    {
    //Uptr[p] = saturate_cast((Uptr[p]-128)/2 + 128);
    uchar c = _t.at(0, p);
    //Vxy = saturate_cast((Vxy-128)/2 + 128);

    //uchar c = _t(0,p); // Image value
    h+= 1.0; // Histogram
    }
    }

    //normalize hist to a max value of 1.0
    cv::Mat _tmp(1,256,CV_64FC1,h);
    double minVal,maxVal;
    cv::minMaxLoc(_tmp, &minVal, &maxVal);
    _tmp = _tmp / maxVal;

    // Calculate the Cumulative Distribution Function
    cdf[0] = h[0];
    for(int j=1; j<256; j++)
    {
    cdf[j] = cdf[j-1]+h[j];
    }

    //normalize CDF to a max value of 1.0
    _tmp.data = (uchar*)cdf; // Array of doubles, but gets a byte pointer.
    cv::minMaxLoc(_tmp, &minVal, &maxVal);
    _tmp = _tmp / maxVal;
    }

  • http://www.morethantechnical.com Roy

    @rz
    Thanks, but, what is changed in the code?

  • stephan schulz

    thanks for the code.

    i noticed that some symbols got replaced by the internet; like
    &amp
    &lt

  • Ravi Teja

    Can any one please give me links of examples images..including masks

  • Nimish

    Dear Roy and other developers

    I have tried this sample code but it gives me few error.Can anyone please provide me the working sample code for this as I am in urgent need for this.
    Thanks in advance.