Image segmentation using OpenCV’s Expectation Maximization


Sorry for the shilling, but here’s my upcoming project:

Please register your email address if you’re interested in it.


Here’s sample implementation of OpenCV’s Expectation Maximization (EM) that I pilfered out of StackOverflow:

I’m no EM expert (well, I’m expert in nothing), but it seems that EM is an algorithm that basically tries to estimate hidden / unobserved parameters that maximize the likelihood parameters in order to describe the statistical models of a given data. So, I assume (I might be wrong, I’m probably wrong), if the dataset is governed by a number of independent gaussian distributions, then EM tries to find unobserved parameters that describe those gaussian models. Since I’ve already thoroughly confused you, I’d recommend you to google for better EM explanations.

Anyway, here’s the implementation code.

* Create a sample vector out of RGB image
Mat asSamplesVectors( Mat& img ) {
    Mat float_img;
    img.convertTo( float_img, CV_32F );

    Mat samples( img.rows * img.cols, 3, CV_32FC1 );

    /* Flatten  */
    int index = 0;
    for( int y = 0; y < img.rows; y++ ) {
        Vec3f * row = float_img.ptr(y);
        for( int x = 0; x < img.cols; x++ )
  <Vec3f>(index++, 0) = row[x];
        return samples;

Perform segmentation (clustering) using EM algorithm
vector EMSegmentation( Mat& image, int no_of_clusters = 2 ){
    Mat samples = asSamplesVectors( image );

    cout << "Starting EM training" << endl;
    EM em( no_of_clusters );
    em.train( samples );
    cout << "Finished training EM" << endl;

    vector<Mat> segmented;
    for( int i = 0; i < no_of_clusters; i++ )
        segmented.push_back( Mat::zeros( image.rows, image.cols, CV_8UC3 ) );

    int index = 0;
    for( int y = 0; y < image.rows; y++ ) {
        for( int x = 0; x < image.cols; x++ ) {
            int result = em.predict( samples.row(index++) )[1];
            segmented[result].at<Point3i>(y, x, 0) =<Point3i>(y, x, 0);

    return segmented;

Here’s the sample test data and the result of applying EM algorithm:

Screen Shot 2014-03-24 at 11.37.10

Screen Shot 2014-03-24 at 11.37.32

And here’s a sample result of applying k-means to it. Which looks quite similar to EM’s result, but if you were to observed carefully part of the head is not segmented correctly, due to the similarity between its color and the color of the background.

Screen Shot 2014-03-24 at 11.59.50

Images used in this example is obtained from 3D photography dataset which is available here:


3 thoughts on “Image segmentation using OpenCV’s Expectation Maximization

    • Just run the EMSegmentation function on an OpenCV Mat image. Btw, wordpress removed some of the code due to formatting, e.g. is removed, just FYI.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s