Monday, January 02, 2012

More K-Means Clustering Experiments on Images

I spent a little more time experimenting with k-means clustering on images and realized that I could use these clusters to recolor the image in interesting ways.

I wrote the function save_recolor to replace pixels from the given clusters (replacements) with new ones of equal intensity, as specified by the rgb_factors vector. For example, the following code will convert pixels of the first two clusters to greyscale.

> save_recolor("baby.jpeg", "baby_new.jpg", replacements=c(1,2),
               rgb_factors=c(1/3, 1/3, 1/3))

It's greyscale because the rgb_factors distributes the pixel intensity evenly among the channels. A factor of c(20/100, 60/100, 20/100) would make pixels from the cluster 60% more green.

Let's get to some examples. Here's an unprocessed image, alongside its color clusters. I picked k=10. You can set k by specifying the palette_size parameter to save_recolor.

Here's what happens when I remove the red (the first cluster).

> save_recolor("baby.jpeg", "baby_new.jpg", replacements=1)

In the next image, I keep the red, and remove everything else.

> save_recolor("baby.jpeg", "baby_new.jpg", replacements=2:10)

Below, I replace the red cluster pixels, with green ones of corresponding intensity.

> save_recolor("baby.jpeg", "baby_new.jpg", replacements=1,
               rgb_factors=c(10/100, 80/100, 10/100))

And this is a fun one: Get rid of everything, keep just the grass.

> save_recolor("baby.jpeg", "baby_new.jpg", replacements=c(1,3:10))

I tried this on various images, using different cluster sizes, replacements, and RGB factors, with lots of interesting results. Anyhow, you should experiment with this yourselves and let me know what you find.

I should point out that nothing here is novel or new -- it's all well known in image processing circles. It's still pretty impressive what you can do when you apply simple machine learning algorithms to other areas.

Okay, as in all my posts, the code is available in my GitHub repository:

https://github.com/0xfe/experiments/blob/master/r/recolor.rscript

Happy new year!

3 comments:

  1. What images should i use? Am getting this error on my RGui - null device.

    By the way am noob in R!

    NB: I manged to run your first sample on building color palettes ...

    ReplyDelete
  2. The color work is very nice stuff---but of no immediate use for me. What I am interested in is your music display API. In particular I'd like to request (alright; beg :) ) for a Irish Flute tablature. Don't get me wrong, the Guitar Tabs are awesome, but I've lately been bitten by those silly little sticks with 6 holes. My vision going forward is something along the lines of---first drag and drop note entry, note name display as a result. Then with help conversion to score presentation with tablature. In other words; given an existing score to be able to quickly mouse it in and be be able to produce either the guitar tablature or the flute tablature with minimal hassle. BTW if you know of something like this that already exists, I'd like to know of it. I'm currently using ABCexplorer with some success, but the data entry still requires 'reading' the original followed by conversion to the correct note---'A2' etc. in order to get what I want. Very error prone. I can see down the road perhaps output as ABC for conversion into other formats. Right now I'm looking for baby steps. If interested, my email is hsmyers.gmail.com My background is user interface design since roughly 1975 :) so I might be of some help for you as well. Thanks,

    --hsm

    ReplyDelete
  3. Cool stuff. I'm going to spend some time soon doing this with OpenCV in Scala. I'll post a gist when I get a good demo. I think I might try to add some locality factor to make it more like a segmentation algo then just a per-pixel operation. Or pipe each cluster into a graphcut algo to segment.

    ReplyDelete