Why such a complex title?

It all started with a strange idea I had today.  I was thinking of common ways to randomize image data (don’t ask why), and it struck me that the most common randomization method – varying RGB data of single pixels – is not the most interesting way to go about it.  Why not use lines, triangles, or other polygons to randomize an image?  How would that look?

To test my theory, I wrote a quick program that selects two random pixels in an image, averages their colors, then draws a line of that averaged color between the two points. When repeated over and over again, such an algorithm leads to some interesting effects…

I started with this God of War 3 image...

...and got this (100,000 iterations of lines with max length 42).

Here's the same image, but with lines of max length 85.

Kinda cool.  I’m not sure what to call this effect… although it looks “furry” to me.  Should we invent a new word – furrification?

Once I had lines working, my next curiosity involved polygons.  Here’s the same picture, but with triangle randomization:

Same parameters as the first line-based randomization above (100,000 iterations, 42 max length).

Same parameters as the second line-based image above (100,000 iterations, max length 85)

This is also a cool effect, especially when you watch it in action.  (The program refreshes the screen every 100 iterations.)

While I didn’t go to the trouble of implementing additional polygons, the code is primed and ready for it.  In fact, it would be trivial to draw polygons of any segment count.

Once I had my newly randomized images, I decided to pop into GIMP and do a bit of post-processing.  It was then that I realized this could be used to create pretty sweet stained glass images:

Sweet!

It’s trivial to create an image like this – simply open up your base image, then add a triangle randomized copy over the top as a new layer.  Set the layer mode to “difference” and bam: stained glass!

Same effect, but with a larger triangle size.

Other blending modes provide interesting effects – for example, multiply:

Same two images as the first stained glass example - just the blending mode has changed.

Anyway, I thought this was an interesting exploration in using a randomized copy of an image as an overlay.

Bayonetta is quite possibly the best action game I've ever played - yes, even better than God of War III

I’ve had this code ready for months, but I couldn’t bring myself to post it until I added something more exciting than just “emboss” and “engrave.”

Today is that day!

First, let me mention what emboss/engrave actually do. These are two of the simplest image processing filters, and they can be efficiently implemented in pretty much any programming language. They both operate on the same principle – for each pixel, subtract the RGB values of one or more neighboring pixels in a particular direction. This leads to an image where low-contrast areas are all black, while high-contrast areas (edges) are varying colors of brighter intensity. Most emboss/engrave filters add 127 to the RGB values so that uniformly contrasted areas are gray. As a bonus feature, I’ve added a “color” option to this code, so you can emboss/engrave an image to any hue. In the Bayonetta example above, the left side of the picture is embossed to something around #81a3fe.

Relief is a variation on emboss/engrave, where the base color is not artificially generated but is based off the current pixel. This acts almost like a sharpen filter, but because the filter doesn’t operate in all directions, it lends the image a more artificial look. (Hence the “relief” moniker – on certain images, it looks like an image has been chipped out of stone and then painted.) Try it on a photo – they tend to work best!

One possible view of the Mandelbrot set (generated with this project)

Today’s very cool project demonstrates a proof of concept implementation for rendering the famed Mandelbrot set (or “Mandelbrot fractal”) using VB6.  It’s a bit of a feat, since VB6 isn’t exactly optimized for recursion-heavy calculations…but you know me.  :)  I love making VB do things it was never meant to do!

I’m very happy with how the project turned out.  It may not be the fastest or prettiest or most accurate implementation (for example, at very high zoom-in you’ll start to see noticeable quality degradation), but I’ve deliberately kept the project as short and sweet as possible to maximize adaptability and learning potential.

To enhance the experience, I’ve added several user-controlled options to help you get a feel for how the rendering works.  First is the scroll-bar at the top of the screen, which allows you to preferentially set speed or accuracy as the program’s objective.  (For those familiar with how the Mandelbrot set works, the scroll bar controls the “escape time” of the recursive function.)

Another neat option is a click-and-drag feature that allows you to draw new rendering boundaries.  This makes it simple to zoom as deep as you want into the fractal.

Finally, rather than a drab grayscale rendering, I’ve chosen a more purplish gradient.  The code can be easily modified to favor another color if you so choose.

The code is reasonably optimized, so a complete view should render within several seconds on most modern hardware.  If speed is a concern, note that the white areas of the image take the longest to process – so minimizing the amount of white you select will maximize render speed.  Also, due to the inherent limits of the “Double” type in VB6, if you zoom in really, really far the program may lock up.  (Simply click the close button to force-terminate the project if this happens.)

The download link can be found below these sample images.

Anaphase, courtesy of Lothar Schermelleh

Today’s project is something new to this site – some bioinformatics code!  I feel a tad ridiculous that it’s taken me so many years to post code related to my field, but hey – better late than never.  (That seems to be a recurring theme around here…)

First, a primer.  If you don’t know what the title of this article refers to, here are some Wikipedia links to get you started:

Hidden Markov models (including some examples in Python)

Viterbi algorithm / Viterbi paths (including even more Python examples)

CpG islands

As a bonus, I’m including sections from my original write-up on this program (it began as a university project) in hopes that they’ll shed some light on how the code works and what its potential usefulness is.

The algorithm implementations are quite fast, and the visual feedback is an excellent addition (IMO). Full source code and a sample executable are provided, as well as a FASTA file with data from chromosome #1 in the human genome (bases 6000-12000, I believe).

(FYI, my original lab assignment link is here: http://dna.cs.byu.edu/bio465/Labs/hmm.shtml.  It contains quite a bit of helpful information and links if you’re looking to write your own HMM implementation.)

INTRODUCTION

CpG islands are regions of DNA characterized by a large number of adjacent cytosine and guanine nucletoides linked by phosphodiester bonds. Additionally, CpG islands appear in some 70% of promoters of human genes (40% of mammalian genes). Unlike CpG sites in the coding region of a gene, in most instances the CpG sites in CpG islands are unmethylated if genes are expressed. This observation led to the speculation that methylation of CpG sites in the promoter of a gene may inhibit the expression of a gene. (Wikipedia, retrieved Feb 2007)

In this project, I was particularly concerned with generating an accurate Hidden Markov model to define the relationship between normal states (B) and island states (I) within a region of the human chromosome. Such an implementation is useful for gene finding, as CpG islands tend to appear near the promoters of important mammalian genes. The automation of this process is critical because such islands are impossible to locate by simply looking at a strand of DNA (they may cover several hundred bases and their C/G content may be only slightly higher than expected values).

PROCEDURE

I opted to code this project in the Microsoft Visual Basic 6.0 programming language. I believe this implementation is highly appropriate, as I can quickly generate both graphs for visualization and a simple, easy-to-modify GUI (since finding CpG islands can require a lot of trial-and-error using different parameters, and this is not conducive to a command-line program). VB6 is excellent for quickly assembling GUIs, and for algorithms of this nature its speed is comparable to that of a raw C++ implementation.  (BTW, if you are a Linux user please note that this project works flawlessly under Wine 1.1.27, and likely earlier versions as well.)

I will refer to the following image as I walk through my implementation of a HMM.

Step 1: Loading a FASTA file
This routine isn’t of much interest, except to mention a programming quirk particular to VB6. A, C, G, and T entries from the FASTA file are reassigned into a byte array as the numbers 0, 1, 2, and 3. In VB6, number comparison and processing is generally faster than string comparison and processing. (To any VB experts reading this – please note that I am aware of mechanisms for overcoming this, but the fact of the matter is that I prefer working with byte arrays. :)

Step 2: HMM Parameters
As per the lab specifications, the user can change 14 HMM-related parameters. Additionally, I have chosen to add an option to let the program estimate the eight state probabilities. This is a trivial operation, but it seems to markedly increase the accuracy of the algorithm. The estimation of A, C, G and T’s being emitted in a B state is computed by the percentage of occurrences of each of these entries in the original FASTA data. I predict that given the low percentage of I states in a length of DNA, this method of estimating emission probabilities is likely quite accurate. Once those categories have been determined, the I state probabilities are computed by doubling the C and G probabilities and halving the A and T probabilities.

Step 3: The HMM and Viterbi algorithms
This step begins by moving all HMM parameters into look-up tables. The logs of these values are pre-calculated, allowing us to add them (instead of multiplying) for future HMM calculations.

Next, all HMM probabilities are calculated and stored.

This is followed by a 2nd loop backward through the data while calculating all data necessary for the Viterbi algorithm. (Note: this is not necessarily the most efficient way to do this, but the speed difference is negligible for sequences less than 100,000 bases long.)

After all Viterbi data has been pre-calculated, the traceback part of the Viterbi algorithm becomes extremely fast. (Another note: execution speed of this chunk of code is slightly reduced because I translate the states into a string and display that string in the textbox in the upper-right of the program window. This is mainly for convenience and/or to satisfy my curiosity.)

Step 4: The Sliding Window
This step is one where I felt a proper GUI was essential. I have chosen to display two graphs; the top graph displays raw counts of C and G occurrences as compared to the expected value (where 0.5 is used as the expected ratio). The bottom graph displays the ratio of I and B states, and it is this data that is used to recommend the location of any CpG islands.

A noteworthy shortcoming of this implementation is that it is designed to only find the most probable CpG island (instead of multiple islands). In a more formal implementation, returning any/all potential CpG island locations would be more useful.

(And for the record, this step is my favorite part of the program. I find it to be well-implemented and surprisingly graphically pleasing for a bioinformatics tool. :)

RESULTS
By my estimation, the program’s most accurate CpG prediction for this FASTA file is bases 2018-2045 (which corresponds to bases 6018-6045 in the original data, I believe). If you run the program yourself, you will find that most of its estimates fall around this range. Adjusting the sliding window doesn’t have tremendous effect on the estimates, although larger windows will result in larger CpG island predictions (not surprising). The percentage of I states varies dramatically; certain parameters result in levels as high as 30-40% while others are less than 5%. For my best estimate (above), the percentage of I states was 22%.

Finally, code execution is swift – typically a matter of seconds – and if the text display in the top-right is disabled the program runs more-or-less instantaneously on modern hardware.

"Antiquified" version of a BBT promo photo

I’m guessing you’ve seen this style of image before – a sort of pseudo-antique filter than can make any photo look like it was taken with a very old camera (or even a daguerreotype – how’s that for a cool word?).  There are many ways to programmatically generate images like this, and in this article I’ve put together one that does more than just make the image look “brown.”  This filter involves several steps (fading, multiplicative brightness, and gamma correction, among others) and results in a conversion that not only adds a sepia coloring, but also gives an image a histogram more in keeping with older photos.

As with all graphics code on this site, the algorithm is fast enough to be applied in real-time.

Below, you can see how each filter looks on a promotional image from Final Fantasy Versus XIII.

Original:

Final Fantasy Versus XIII

Atmosphere:

Final Fantasy Versus XIII - Atmosphere Effect

Burn:

Final Fantasy Versus XIII - Burn Effect

Fog:

Final Fantasy Versus XIII - Fog Effect

Freeze:

Final Fantasy Versus XIII - Freeze Effect

Lava:

Final Fantasy Versus XIII - Lava Effect

Ocean:

Final Fantasy Versus XIII - Ocean Effect

Rainbow:

Final Fantasy Versus XIII - Rainbow Effect

Metal:

Final Fantasy Versus XIII - Metal Effect

Underwater:

Final Fantasy Versus XIII - Underwater Effect

In this project, I’ve provided a 5×5 custom filter engine with support for scaling/weighting and biasing/offsetting.  This is identical to the custom filter engine provided by Photoshop, and mine is even slightly faster (at least for the screen-sized images I’ve been testing).

Besides being fast, the engine is also smart.  It is clever enough to estimate edge pixels correctly – even for scaled/weighted filters – and it will automatically validate all input values to make sure they’re appropriate.

Finally, I’ve included 7 sample filters for you to play with, including blur, sharpen, emboss, engrave, grease, unfocus, and vibrate. Below, you can see how each filter looks on a promotional image from Castle, an excellent new NYPD dramedy on ABC.

Original:

Blur:

Sharpen:

Emboss:

Engrave:

Grease:

Unfocus:

Vibrate:

For those who have never used edge detection algorithms before, here’s an idea of the output:

Original image:

Prewitt method (horizontal):

Prewitt method (vertical):

Sobel method (horizontal):

Sobel method (vertical):

Laplacian method:

Hilite method:

Tanner’s method 1:

Tanner’s method 2:

As promised, here is the second half of my histogram code project.  In this project, I’ll show you how to stretch a histogram, equalize individual channels, and – most useful of all – equalize an image’s overall luminance.  Cool, eh?

Stretching a Histogram

The most straightforward of these functions is stretching a histogram.  The idea is this: many images stretch from pretty-dark colors to pretty-light colors, but they don’t take advantage of the full luminance spectrum by stretching all the way from pure black to pure light.  (For example, a “washed-out” image typically fails to utilize the entire luminance spectrum.)  To help fix an image like this, it is necessary to adjust the image’s colors to better span all possible brightness values.

Stretching does this by finding the darkest and lightest values in an image, then performing an automatic conversion between that color range (max – min) and the ideal color range (pure white – pure black).

As an example, here is an image that doesn’t span the full brightness spectrum:

Notice how the top half of the histogram is empty?  The brightest pixels in this image are only a mid-level gray.

Here is the same image after running a stretch histogram algorithm:

Notice how the histogram now stretches all the way to pure white?  Admittedly, this picture is not the best example of how stretching can be useful – but it’s a good demonstration of what stretching does.

Stretching is a fast and simple function, but its useful is limited.  Take the following image, for example:

The colors in this tiger picture stretch all the way from pure black to pure white, but they seem to be more heavily concentrated toward the dark end of the luminance spectrum.  To really fix this picture, we need to redistribute its colors across the spectrum in a more equal way.

Enter equalization.

Equalizing a Histogram

Equalizing is a more complex and time-consuming function than stretching, but it is able to fix problems outside the reach of stretching alone.

Here is the tiger image post-equalization:

See how much more balanced the histogram has become?  As another example, here is the cloud picture from above after both stretching and equalizing:

Equalizing has brought out a bunch of details that weren’t apparent in the original image.

Rather than going through the gory details of how histogram equalizing works, I’m going to refer you to the excellent Wikipedia article on the subject:

http://en.wikipedia.org/wiki/Histogram_equalization

Also, I’d like to quote one very applicable comment from the above article (emphasis added by me):

The above describes histogram equalization on a greyscale image. However, it can also be used on color images by applying the same method separately to the Red, Green and Blue components of the RGB color values of the image. Still, it should be noted that applying the same method on the Red, Green, and Blue components of an RGB image may yield dramatic changes in the image’s color balance since the relative distributions of the color channels change as a result of applying the algorithm. However, if the image is first converted to another color space, Lab color space, or HSL/HSV color space in particular, then the algorithm can be applied to the luminance or value channel without resulting in changes to the hue and saturation of the image.

Because of this, I have supplied two equalizing algorithms in this project – one that equalizes any combination of color channels, and another that equalizes only luminance.  The luminance method provides the code for converting between RGB and HSL color spaces…and that code is very complex.  It’s probably worth your while to take that code on faith for now, and maybe at a future date I’ll discuss color space transformations in more detail.

Get the Code!

But enough chat!  Here’s the link for the code: