How to compile zLib 1.2.8 (WINAPI / WAPI / STDCALL version)

This is not a full-blown article – just a quick HOWTO for people who require the latest version of zLib and want to use it from a language (like classic VB) that requires a standard WinAPI interface.

By default, zLib uses C calling conventions (CDECL). Gilles Vollant has helpfully provided an STDCALL version of zLib in the past, but his site only provides version 1.2.5, which dates back to January 2012. zLib is currently on version 1.2.8.

Gilles has contributed his WAPI fixes to the core zLib distribution so that anyone can compile it themselves. To do this, you will need to download:

Once both of those are downloaded (and updated, as VS 2012 will require you to install several service packs), follow these steps to compile zLib yourself:

  1. Extract the entire zLib file and navigate to the /contrib/masmx86 folder. Open the “bld_ml32.bat” file in a text editor.
  2. Add the “/safeseh” switch to both lines in that file (e.g. “ml /safeseh /coff /Zi /c /Flmatch686.lst match686.asm”). Then save and exit.
  3. Navigate to the /contrib/vstudio/vc11/ folder. Open the zlibvc.sln file in your newly installed Visual Studio 2012 Express.
  4. In the Solution Explorer (top-right by default), right-click “zlibstat” then select “Properties” at the bottom.
  5. Go to Configuration Properties -> C/C++ -> Preprocessor, and in the Preprocessor Definitions line remove “ZLIB_WINAPI;” (don’t forget to remove the trailing semicolon).
  6. Now, we need to fix a recently introduced problem that relies on Win8 functionality. In the Solution Explorer, navigate to zlibvc -> iowin32.c. Double-click to open the file.
  7. Find the line of text that reads “#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_APP)”. Change this line to “#if WINAPI_FAMILY_ONE_PARTITION(WINAPI_FAMILY_DESKTOP_APP, WINAPI_PARTITION_APP)”. (Thanks to this link for this fix.)
  8. zLib uses a Version number declaration that can cause the build process to fail. To fix this, go back to the Solution Explorer, then navigate to zlibvc -> zlibvc.def. Double-click to open.
  9. Change the line that reads “VERSION 1.2.8” to read “VERSION 1.28”.
  10. Finally, go to the Build -> Configuration Manager menu and change the Active Solution Configuration to “Release”.
  11. Exit that window and press F7 (or click the Build -> Build Solution menu). The project should successfully build.
  12. You can find your newly compiled zlibwapi.dll file in the /contrib/vstudio/vc11/x86/ZlibDllRelease/ folder.

A simple algorithm for correcting lens distortion

One of the new features in the development branch of my open-source photo editor is a simple tool for correcting lens distortion. I thought I’d share the algorithm I use, in case others find it useful. (There are very few useful examples of lens correction on the Internet – most articles simply refer to existing software packages, rather than explaining how the software works.)

Lens distortion is a complex beast, and a lot of approaches have been developed to deal with it. Some professional software packages address the problem by providing a comprehensive list of cameras and lenses – then the user just picks their equipment from the list, and the software applies a correction algorithm using a table of hard-coded values. This approach requires way more resources than a small developer like myself could handle, so I chose a simpler solution: a universal algorithm that allows the user to apply their own correction, with two tunable parameters for controlling the strength of the correction.

This is what PhotoDemon's new lens correction tool looks like.
PhotoDemon’s new lens correction tool in action.

The key part of the algorithm is less than ten lines of code, so there’s not much work involved. The effect is also fast enough to preview in real-time.

Before sharing the algorithm, let me demonstrate its output. Here is a sample photo that suffers from typical spherical distortion:

This lovely demonstration photo comes from Wikipedia, courtesy of Ashley Pomeroy
This lovely demonstration photo comes from Wikipedia, courtesy of Ashley Pomeroy

Pay special attention to the lines on the floor and the glass panels on the right.

Here’s the same image, as corrected by the algorithm in this article:

Note the straight lines on both the floor and the glass panels on the right.  Not bad, eh?
Note the straight lines on both the floor and the glass panels on the right. Not bad, eh?

My use of simple bilinear resampling blurs the output slightly; a more sophisticated resampling technique would produce clearer results.

A key feature of the algorithm is that it works at any aspect ratio – rectangular images, like the one above, are handled just fine, as are perfectly square images.

Anyway, here is the required code, as pseudocode:


input:
    strength as floating point >= 0.  0 = no change, high numbers equal stronger correction.
    zoom as floating point >= 1.  (1 = no change in zoom)

algorithm:

    set halfWidth = imageWidth / 2
    set halfHeight = imageHeight / 2
    
    if strength = 0 then strength = 0.00001
    set correctionRadius = squareroot(imageWidth ^ 2 + imageHeight ^ 2) / strength

    for each pixel (x,y) in destinationImage
        set newX = x - halfWidth
        set newY = y - halfHeight

        set distance = squareroot(newX ^ 2 + newY ^ 2)
        set r = distance / correctionRadius
        
        if r = 0 then
            set theta = 1
        else
            set theta = arctangent(r) / r

        set sourceX = halfWidth + theta * newX * zoom
        set sourceY = halfHeight + theta * newY * zoom

        set color of pixel (x, y) to color of source image pixel at (sourceX, sourceY)

That’s all there is to it. Note that you’ll need to do some bounds checking, as sourceX and sourceY may lie outside the bounds of the original image. Note also that sourceX and sourceY will be floating-point values – so for best results, you’ll want to interpolate the color used instead of just clamping sourceX and sourceY to integer values.

I should mention that the algorithm works just fine without the zoom parameter. I added the zoom parameter after some experimentation; specifically, I find zoom useful in two ways:

  • On images with only minor lens distortion, zooming out reduces stretching artifacts at the edges of the corrected image
  • On images with severe distortion, such as true fish-eye photos, zooming-out retains more of the source material

As there is not a universally “correct” solution to these two scenarios, I recommend providing zoom as a tunable parameter. To give a specific example of the second circumstance, consider this fish-eye photo from Wikipedia, courtesy of Josef F. Stuefer:

Severe distortion like this is difficult to correct completely.
Severe distortion like this is difficult to fully correct.

If we attempt to correct the image without applying any zoom, the image must be stretched so far that much of the edges are lost completely:

This is hardly the same photo.  Note also the visible stretching at the edges.
This is hardly the same photo. The pier at the bottom has been completely erased!

By utilizing a zoom parameter, it is possible to include more of the image in the finished result:

Much more of the photo can be preserved by adding a simple zoom parameter to the algorithm.
Use of a zoom parameter allows us to preserve much more of the photo. When correcting severe distortion like this, you might want to apply a sharpening algorithm to the final image. (This sample image has no sharpening applied.)

Again, I only use a simple resampling technique; a more sophisticated one would produce clearer results at the edges.

If you’d like to see my actual source code, check out this GitHub link. The fun begins at line 194. I also include an optional radius parameter, which allows the user to correct only a subset of the image (rather than the entire thing), but other than that the code is identical to what you see above.

Enjoy!

P.S. For a great discussion of fish-eye distortion from a pro photographer’s perspective, check out http://photo.net/learn/fisheye/

Image Dithering: Eleven Algorithms and Source Code

Dithering: An Overview

Today’s graphics programming topic – dithering – is one I receive a lot of emails about, which some may find surprising. You might think that dithering is something programmers shouldn’t have to deal with in 2012. Doesn’t dithering belong in the annals of technology history, a relic of times when “16 million color displays” were something programmers and users could only dream of? In an age when cheap mobile phones operate in full 32bpp glory, why am I writing an article about dithering?

Actually, dithering is still a surprisingly applicable technique, not just for practical reasons (such as preparing a full-color image for output on a non-color printer), but for artistic reasons as well. Dithering also has applications in web design, where it is a useful technique for reducing images with high color counts to lower color counts, reducing file size (and bandwidth) without harming quality. It also has uses when reducing 48 or 64bpp RAW-format digital photos to 24bpp RGB for editing.

And these are just image dithering uses – dithering still has extremely crucial roles to play in audio, but I’m afraid I won’t be discussing audio dithering here. Just image dithering.

In this article, I’m going to focus on three things:

  • a basic discussion of how image dithering works
  • eleven specific two-dimensional dithering formulas, including famous ones like “Floyd-Steinberg”
  • how to write a general-purpose dithering engine

Update 11 June 2016: some of the sample images in this article have been updated to better reflect the various dithering algorithms. Thank you to commenters who noted problems with the previous images!

Dithering: Some Examples

Consider the following full-color image, a wallpaper of the famous “companion cube” from Portal:

This will be our demonstration image for this article.  I chose it because it has a nice mixture of soft gradients and hard edges.
This will be our demonstration image for this article. I chose it because it has a nice mixture of soft gradients and hard edges.

On a modern LCD or LED screen – be it your computer monitor, smartphone, or TV – this full-color image can be displayed without any problems. But consider an older PC, one that only supports a limited palette. If we attempt to display the image on such a PC, it might look something like this:

This is the same image as above, but restricted to a websafe palette.
This is the same image as above, but restricted to a websafe palette.

Pretty nasty, isn’t it? Consider an even more dramatic example, where we want to print the cube image on a black-and-white printer. Then we’re left with something like this:

At this point, the image is barely recognizable.
At this point, the image is barely recognizable.

Problems arise any time an image is displayed on a device that supports less colors than the image contains. Subtle gradients in the original image may be replaced with blobs of uniform color, and depending on the restrictions of the device, the original image may become unrecognizable.

Dithering is an attempt to solve this problem. Dithering works by approximating unavailable colors with available colors, by mixing and matching available colors in a way that mimicks unavailable ones. As an example, here is the cube image once again reduced to the colors of a theoretical old PC – only this time, dithering has been applied:

A big improvement over the non-dithered version!
A big improvement over the non-dithered version!

If you look closely, you can see that this image uses the same colors as its non-dithered counterpart – but those few colors are arranged in a way that makes it seem like many more colors are present.

As another example, here is a black-and-white version of the image with similar dithering applied:

The specific algorithm used on this image is "2-row Sierra" dithering.
The specific algorithm used on this image is “2-row Sierra” dithering.

Despite only black and white being used, we can still make out the shape of the cube, right down to the hearts on either side. Dithering is an extremely powerful technique, and it can be used in ANY situation where data has to be represented at a lower resolution than it was originally created for. This article will focus specifically on images, but the same techniques can be applied to any 2-dimensional data (or 1-dimensional data, which is even simpler!).

The Basic Concept Behind Dithering

Boiled down to its simplest form, dithering is fundamentally about error diffusion.

Error diffusion works as follows: let’s pretend to reduce a grayscale photograph to black and white, so we can print it on a printer that only supports pure black (ink) or pure white (no ink). The first pixel in the image is dark gray, with a value of 96 on a scale from 0 to 255, with zero being pure black and 255 being pure white.

Here is an example of the RGB values in the example.
Here is a visualization of the RGB values in our example.

When converting such a pixel to black or white, we use a simple formula – is the color value closer to 0 (black) or 255 (white)? 96 is closer to 0 than to 255, so we make the pixel black.

At this point, a standard approach would simply move to the next pixel and perform the same comparison. But a problem arises if we have a bunch of “96 gray” pixels – they all get turned to black, and we’re left with a huge chunk of empty black pixels, which doesn’t represent the original gray color very well at all.

Error diffusion takes a smarter approach to the problem. As you might have inferred, error diffusion works by “diffusing” – or spreading – the error of each calculation to neighboring pixels. If it finds a pixel of 96 gray, it too determines that 96 is closer to 0 than to 255 – and so it makes the pixel black. But then the algorithm makes note of the “error” in its conversion – specifically, that the gray pixel we have forced to black was actually 96 steps away from black.

When it moves to the next pixel, the error diffusion algorithm adds the error of the previous pixel to the current pixel. If the next pixel is also 96 gray, instead of simply forcing that to black as well, the algorithm adds the error of 96 from the previous pixel. This results in a value of 192, which is actually closer to 255 – and thus closer to white! So it makes this particular pixel white, and it again makes note of the error – in this case, the error is -63, because 192 is 63 less than 255, which is the value this pixel was forced to.

As the algorithm proceeds, the “diffused error” results in an alternating pattern of black and white pixels, which does a pretty good job of mimicking the “96 gray” of the section – much better just forcing the color to black over and over again. Typically, when we finish processing a line of the image, we discard the error value we’ve been tracking and start over again at an error of “0” with the next line of the image.

Here is an example of the cube image from above with this exact algorithm applied – specifically, each pixel is converted to black or white, the error of the conversion is noted, and it is passed to the next pixel on the right:

This is the simplest possible application of error diffusion dithering.
This is the simplest possible application of error diffusion dithering.

Unfortunately, error diffusion dithering has problems of its own. For better or worse, dithering always leads to a spotted or stippled appearance. This is an inevitable side-effect of working with a small number of available colors – those colors are going to be repeated over and over again, because there are only so many of them.

In the simple error diffusion example above, another problem is evident – if you have a block of very similar colors, and you only push the error to the right, all the “dots” end up in the same place! This leads to funny lines of dots, which is nearly as distracting as the original, non-dithered version.

The problem is that we’re only using a one-dimensional error diffusion. By only pushing the error in one direction (right), we don’t distribute it very well. Since an image has two dimensions – horizontal and vertical – why not push the error in multiple directions? This will spread it out more evenly, which in turn will avoid the funny “lines of speckles” seen in the error diffusion example above.

Two-Dimensional Error Diffusion Dithering

There are many ways to diffuse an error in two dimensions. For example, we can spread the error to one or more pixels on the right, one or more pixels on the left, one or more pixels up, and one or more pixels down.

For simplicity of computation, all standard dithering formulas push the error forward, never backward. If you loop through an image one pixel at a time, starting at the top-left and moving right, you never want to push errors backward (e.g. left and/or up). The reason for this is obvious – if you push the error backward, you have to revisit pixels you’ve already processed, which leads to more errors being pushed backward, and you end up with an infinite cycle of error diffusion.

So for standard loop behavior (starting at the top-left of the image and moving right), we only want to push pixels right and down.

Apologies for the crappy image - but I hope it helps illustrate the gist of proper error diffusion.
Apologies for the crappy image – but I hope it helps illustrate the gist of proper error diffusion.

As for how specifically to propagate the error, a great number of individuals smarter than I have tackled this problem head-on. Let me share their formulas with you.

(Note: these dithering formulas are available multiple places online, but the best, most comprehensive reference I have found is this one.)

Floyd-Steinberg Dithering

The first – and arguably most famous – 2D error diffusion formula was published by Robert Floyd and Louis Steinberg in 1976. It diffuses errors in the following pattern:


       X   7
   3   5   1

     (1/16)

In the notation above, “X” refers to the current pixel. The fraction at the bottom represents the divisor for the error. Said another way, the Floyd-Steinberg formula could be written as:


           X    7/16
   3/16  5/16   1/16

But that notation is long and messy, so I’ll stick with the original.

To use our original example of converting a pixel of value “96” to 0 (black) or 255 (white), if we force the pixel to black, the resulting error is 96. We then propagate that error to the surrounding pixels by dividing 96 by 16 ( = 6), then multiplying it by the appropriate values, e.g.:


           X     +42
   +18    +30    +6

By spreading the error to multiple pixels, each with a different value, we minimize any distracting bands of speckles like the original error diffusion example. Here is the cube image with Floyd-Steinberg dithering applied:

Floyd-Steinberg dithering
Floyd-Steinberg dithering

Not bad, eh?

Floyd-Steinberg dithering is easily the most well-known error diffusion algorithm. It provides reasonably good quality, while only requiring a single forward array (a one-dimensional array the width of the image, which stores the error values pushed to the next row). Additionally, because its divisor is 16, bit-shifting can be used in place of division – making it quite fast, even on old hardware.

As for the 1/3/5/7 values used to distribute the error – those were chosen specifically because they create an even checkerboard pattern for perfectly gray images. Clever!

One warning regarding “Floyd-Steinberg” dithering – some software may use other, simpler dithering formulas and call them “Floyd-Steinberg”, hoping people won’t know the difference. This excellent dithering article describes one such “False Floyd-Steinberg” algorithm:


   X   3
   3   2

   (1/8)

This simplification of the original Floyd-Steinberg algorithm not only produces markedly worse output – but it does so without any conceivable advantage in terms of speed (or memory, as a forward-array to store error values for the next line is still required).

But if you’re curious, here’s the cube image after a “False Floyd-Steinberg” application:

Much more speckling than the legit Floyd-Steinberg algorithm - so don't use this formula!
Much more speckling than the legit Floyd-Steinberg algorithm – so don’t use this formula!

Jarvis, Judice, and Ninke Dithering

In the same year that Floyd and Steinberg published their famous dithering algorithm, a lesser-known – but much more powerful – algorithm was also published. The Jarvis, Judice, and Ninke filter is significantly more complex than Floyd-Steinberg:


             X   7   5 
     3   5   7   5   3
     1   3   5   3   1

           (1/48)

With this algorithm, the error is distributed to three times as many pixels as in Floyd-Steinberg, leading to much smoother – and more subtle – output. Unfortunately, the divisor of 48 is not a power of two, so bit-shifting can no longer be used – but only values of 1/48, 3/48, 5/48, and 7/48 are used, so these values can each be calculated but once, then propagated multiple times for a small speed gain.

Another downside of the JJN filter is that it pushes the error down not just one row, but two rows. This means we have to keep two forward arrays – one for the next row, and another for the row after that. This was a problem at the time the algorithm was first published, but on modern PCs or smartphones this extra requirement makes no difference. Frankly, you may be better off using a single error array the size of the image, rather than erasing the two single-row arrays over and over again.

Jarvis, Judice, Ninke dithering
Jarvis, Judice, Ninke dithering

Stucki Dithering

Five years after Jarvis, Judice, and Ninke published their dithering formula, Peter Stucki published an adjusted version of it, with slight changes made to improve processing time:


             X   8   4 
     2   4   8   4   2
     1   2   4   2   1

           (1/42)

The divisor of 42 is still not a power of two, but all the error propagation values are – so once the error is divided by 42, bit-shifting can be used to derive the specific values to propagate.

For most images, there will be minimal difference between the output of Stucki and JJN algorithms, so Stucki is often used because of its slight speed increase.

Stucki dithering
Stucki dithering

Atkinson Dithering

During the mid-1980’s, dithering became increasingly popular as computer hardware advanced to support more powerful video drivers and displays. One of the best dithering algorithms from this era was developed by Bill Atkinson, a Apple employee who worked on everything from MacPaint (which he wrote from scratch for the original Macintosh) to HyperCard and QuickDraw.

Atkinson’s formula is a bit different from others in this list, because it only propagates a fraction of the error instead of the full amount. This technique is sometimes offered by modern graphics applications as a “reduced color bleed” option. By only propagating part of the error, speckling is reduced, but contiguous dark or bright sections of an image may become washed out.


         X   1   1 
     1   1   1
         1

       (1/8)

Atkinson dithering
Atkinson dithering

Burkes Dithering

Seven years after Stucki published his improvement to Jarvis, Judice, Ninke dithering, Daniel Burkes suggested a further improvement:


             X   8   4 
     2   4   8   4   2

           (1/32)

Burkes’s suggestion was to drop the bottom row of Stucki’s matrix. Not only did this remove the need for two forward arrays, but it also resulted in a divisor that was once again a multiple of 2. This change meant that all math involved in the error calculation could be accomplished by simple bit-shifting, with only a minor hit to quality.

Burkes dithering
Burkes dithering

Sierra Dithering

The final three dithering algorithms come from Frankie Sierra, who published the following matrices in 1989 and 1990:


             X   5   3
     2   4   5   4   2
         2   3   2
           (1/32)


             X   4   3
     1   2   3   2   1
           (1/16)


         X   2
     1   1
       (1/4)

These three filters are commonly referred to as “Sierra”, “Two-Row Sierra”, and “Sierra Lite”. Their output on the sample cube image is as follows:

Sierra (sometimes called Sierra-3)
Sierra (sometimes called Sierra-3)
Two-row Sierra
Two-row Sierra
Sierra Lite
Sierra Lite

Other dithering considerations

If you compare the images above to the dithering results of another program, you may find slight differences. This is to be expected. There are a surprising number of variables that can affect the precise output of a dithering algorithm, including:

  • Integer or floating point tracking of errors. Integer-only methods lose some resolution due to quantization errors.
  • Color bleed reduction. Some software reduces the error by a set value – maybe 50% or 75% – to reduce the amount of “bleed” to neighboring pixels.
  • The threshold cut-off for black or white. 127 or 128 are common, but on some images it may be helpful to use other values.
  • For color images, how luminance is calculated can make a big difference. I use the HSL luminance formula ( [max(R,G,B) + min(R,G,B)] / 2). Others use ([r+g+b] / 3) or one of the ITU formulas. YUV or CIELAB will offer even better results.
  • Gamma correction or other pre-processing modifications. It is often beneficial to normalize an image before converting it to black and white, and whichever technique you use for this will obviously affect the output.
  • Loop direction. I’ve discussed a standard “left-to-right, top-to-bottom” approach, but some clever dithering algorithms will follow a serpentine path, where left-to-right directionality is reversed each line. This can reduce spots of uniform speckling and give a more varied appearance, but it’s more complicated to implement.

For the demonstration images in this article, I have not performed any pre-processing to the original image. All color matching is done in the RGB space with a cut-off of 127 (values <= 127 are set to 0). Loop direction is standard left-to-right, top-to-bottom.

Which specific techniques you may want to use will vary according to your programming language, processing constraints, and desired output.

I count 9 algorithms, but you promised 11! Where are the other two?

So far I’ve focused purely on error-diffusion dithering, because it offers better results than static, non-diffusion dithering.

But for sake of completeness, here are demonstrations of two standard “ordered dither” techniques. Ordered dithering leads to far more speckling (and worse results) than error-diffusion dithering, but they require no forward arrays and are very fast to apply. For more information on ordered dithering, check out the relevant Wikipedia article.

Ordered dither using a 4x4 Bayer matrix
Ordered dither using a 4×4 Bayer matrix
Ordered dither using an 8x8 Bayer matrix
Ordered dither using an 8×8 Bayer matrix

With these, the article has now covered a total of 11 different dithering algorithms.

Writing your own general-purpose dithering algorithm

Earlier this year, I wrote a fully functional, general-purpose dithering engine for PhotoDemon (an open-source photo editor). Rather than post the entirety of the code here, let me refer you to the relevant page on GitHub. The black and white conversion engine starts at line 350. If you have any questions about the code – which covers all the algorithms described on this page – please let me know and I’ll post additional explanations.

That engine works by allowing you to specify any dithering matrix in advance, just like the ones on this page. Then you hand that matrix over to the dithering engine and it takes care of the rest.

The engine is designed around monochrome conversion, but it could easily be modified to work on color palettes as well. The biggest difference with a color palette is that you must track separate errors for red, green, and blue, rather than a single luminance error. Otherwise, all the math is identical.

 

This site - and its many free downloads - are 100% funded by donations. Please consider a small contribution to fund server costs and to help me support my family. Even $1.00 helps. Thank you!

How to Convert Temperature (K) to RGB: Algorithm and Sample Code

Converting temperature (Kelvin) to RGB: an overview

If you don’t know what “color temperature” is, start here.

While working on a “Color Temperature” tool for PhotoDemon, I spent an evening trying to track down a simple, straightforward algorithm for converting between temperature (in Kelvin) and RGB values. This seemed like an easy algorithm to find, since many photo editors provide tools for correcting an image’s color temperature in post-production, and every modern camera – including smartphones – provides a way to adjust white balance based on the lighting conditions of a shot.

Example of a camera white balance screen. Image courtesy of http://digitalcamerareviews2011online.blogspot.com

Little did I know, but it’s pretty much impossible to find a reliable temperature to RGB conversion formula. Granted, there are some algorithms out there, but most work by converting temperature to the XYZ color space, to which you could add your own RGB transformation after the fact. Such algorithms seem to be based off AR Robertson’s method, one implementation of which is here, while another is here.

Unfortunately, that approach isn’t really a mathematical formula – it’s just glorified look-up table interpolation. That might be a reasonable solution under certain circumstances, but when you factor in the additional XYZ -> RGB transformation required, it’s just too slow and overwrought for simple real-time color temperature adjustment.

So I wrote my own algorithm, and it works pretty damn well. Here’s how I did it.

Caveats for using this algorithm

Caveat 1: my algorithm provides a high-quality approximation, but it’s not accurate enough for serious scientific use. It’s designed primarily for photo manipulation – so don’t try and use it for astronomy or medical imaging.

Caveat 2: due to its relative simplicity, this algorithm is fast enough to work in real-time on reasonably sized images (I tested it on 12 megapixel shots), but for best results you should apply mathematical optimizations specific to your programming language. I’m presenting it here without math optimizations so as to not over-complicate it.

Caveat 3: this algorithm is only designed to be used between 1000 K and 40000 K, which is a nice spectrum for photography. (Actually, it’s way larger than most photographic situations will ever call for.) While it will work for temperatures outside these ranges, the estimation quality will decline.

Special thanks to Mitchell Charity

First off, I owe a big debt of gratitude to the source data I used to generate these algorithms – Mitchell Charity’s raw blackbody datafile at http://www.vendian.org/mncharity/dir3/blackbody/UnstableURLs/bbr_color.html. Charity provides two datasets, and my algorithm uses the CIE 1964 10-degree color matching function. A discussion of the CIE 1931 2-degree CMF with Judd Vos corrections versus the 1964 10-degree set is way beyond the scope of this article, but you can start here for a more comprehensive analysis if you’re so inclined.

The Algorithm: sample output

Here’s the output of the algorithm from 1000 K to 40000 K:

Output of my algorithm from 1000 K to 40000 K. The white point occurs at 6500-6600 K, which is perfect for photo manipulation purposes on a modern LCD monitor.

Here’s a more detailed shot of the algorithm in the interesting photographic range, which is 1500 K to 15000 K:

Same algorithm, but from 1500 K to 15000 K

As you can see, banding is minimal – which is a big improvement over the aforementioned look-up table methods. The algorithm also does a great job of preserving the slightly yellow cast leading up to the white point, which is important for imitating daylight in post-production photo manipulation.

How I arrived at this algorithm

My first step in reverse-engineering a reliable formula was to plot Charity’s original blackbody values. You can download my whole worksheet here in LibreOffice / OpenOffice .ods format (430kb).

Here’s how the data looks when plotted:

Mitchell Charity’s original Temperature (K) to RGB (sRGB) data, plotted in LibreOffice Calc. Again, these are based off the CIE 1964 10-degree CMFs. The white point, as desired, occurs between 6500 K and 6600 K (the peak on the left-hand side of the chart). (Source: http://www.vendian.org/mncharity/dir3/blackbody/UnstableURLs/bbr_color.html)

From this, it’s easy to note that there are a few floors and ceilings that make our algorithm easier. Specifically:

  • Red values below 6600 K are always 255
  • Blue values below 2000 K are always 0
  • Blue values above 6500 K are always 255

It’s also important to note that for purposes of fitting a curve to the data, green is best treated as two separate curves – one for temperatures below 6600 K, and a separate one for temperatures above that point.

From here, I separated the data (without the “always 0” and “always 255” segments) into individual color components. In a perfect world, a curve could then be fitted to each set of points, but unfortunately it wasn’t that simple. Because there’s a large disparity between the X and Y values in the plot – the x-values are all over 1000, and they are plotted in 100 point segments, while the y values all fall between 255 and 0 – it was necessary to transpose the x data in order to get a better fit. For optimization purposes, I stuck to first dividing the x value (the temperature) by 100 across each color, followed by an additional subtraction if it led to a significantly better fit. Here are the resultant charts for each curve, along with the best-fit curve and corresponding R-squared value:

Apologies for the horrifically poor font kerning and hinting in those charts. I love LibreOffice for many things, but its inability to do font aliasing on charts is downright shameful. I also don’t like having to extract charts from screenshots because they don’t have an export option, but that’s a rant best saved for some other day.

As you can see, the curves all fit reasonably well, with R-square values above .987. I could have spent more time really tweaking the curves, but for purposes of photo manipulation these are plenty close enough. No layperson is going to be able to tell that the curves don’t exactly fit raw idealized blackbody observations, right?

The algorithm

Using that data, here’s the algorithm, in all its glory.

First, pseudocode:



    Start with a temperature, in Kelvin, somewhere between 1000 and 40000.  (Other values may work,
     but I can't make any promises about the quality of the algorithm's estimates above 40000 K.)
    Note also that the temperature and color variables need to be declared as floating-point.

    Set Temperature = Temperature \ 100
    
    Calculate Red:

    If Temperature <= 66 Then
        Red = 255
    Else
        Red = Temperature - 60
        Red = 329.698727446 * (Red ^ -0.1332047592)
        If Red < 0 Then Red = 0
        If Red > 255 Then Red = 255
    End If
    
    Calculate Green:

    If Temperature <= 66 Then
        Green = Temperature
        Green = 99.4708025861 * Ln(Green) - 161.1195681661
        If Green < 0 Then Green = 0
        If Green > 255 Then Green = 255
    Else
        Green = Temperature - 60
        Green = 288.1221695283 * (Green ^ -0.0755148492)
        If Green < 0 Then Green = 0
        If Green > 255 Then Green = 255
    End If
    
    Calculate Blue:

    If Temperature >= 66 Then
        Blue = 255
    Else

        If Temperature <= 19 Then
            Blue = 0
        Else
            Blue = Temperature - 10
            Blue = 138.5177312231 * Ln(Blue) - 305.0447927307
            If Blue < 0 Then Blue = 0
            If Blue > 255 Then Blue = 255
        End If

    End If

In the pseudocode above, note that Ln() means natural logarithm. Note also that you can omit the “If color < 0” checks if you will only ever supply temperatures in the recommended range. (You still need to leave the “If color > 255” checks, though.)

As for actual code, here’s the exact Visual Basic function I’m using in PhotoDemon. It’s not yet optimized (for example, the logarithms would be much faster via look-up table) but at least the code is short and readable:


'Given a temperature (in Kelvin), estimate an RGB equivalent
Private Sub getRGBfromTemperature(ByRef r As Long, ByRef g As Long, ByRef b As Long, ByVal tmpKelvin As Long)

    Static tmpCalc As Double

    'Temperature must fall between 1000 and 40000 degrees
    If tmpKelvin < 1000 Then tmpKelvin = 1000
    If tmpKelvin > 40000 Then tmpKelvin = 40000
    
    'All calculations require tmpKelvin \ 100, so only do the conversion once
    tmpKelvin = tmpKelvin \ 100
    
    'Calculate each color in turn
    
    'First: red
    If tmpKelvin <= 66 Then
        r = 255
    Else
        'Note: the R-squared value for this approximation is .988
        tmpCalc = tmpKelvin - 60
        tmpCalc = 329.698727446 * (tmpCalc ^ -0.1332047592)
        r = tmpCalc
        If r < 0 Then r = 0
        If r > 255 Then r = 255
    End If
    
    'Second: green
    If tmpKelvin <= 66 Then
        'Note: the R-squared value for this approximation is .996
        tmpCalc = tmpKelvin
        tmpCalc = 99.4708025861 * Log(tmpCalc) - 161.1195681661
        g = tmpCalc
        If g < 0 Then g = 0
        If g > 255 Then g = 255
    Else
        'Note: the R-squared value for this approximation is .987
        tmpCalc = tmpKelvin - 60
        tmpCalc = 288.1221695283 * (tmpCalc ^ -0.0755148492)
        g = tmpCalc
        If g < 0 Then g = 0
        If g > 255 Then g = 255
    End If
    
    'Third: blue
    If tmpKelvin >= 66 Then
        b = 255
    ElseIf tmpKelvin <= 19 Then
        b = 0
    Else
        'Note: the R-squared value for this approximation is .998
        tmpCalc = tmpKelvin - 10
        tmpCalc = 138.5177312231 * Log(tmpCalc) - 305.0447927307
        
        b = tmpCalc
        If b < 0 Then b = 0
        If b > 255 Then b = 255
    End If
    
End Sub

This function was used to generate the sample output near the start of this article, so I can guarantee that it works.

Sample images

Here’s a great example of what color temperature adjustments can do. The image below – a promotional poster for the HBO series True Blood – nicely demonstrates the potential of color temperature adjustments. On the left is the original shot; on the right, a color temperature adjustment using the code above. In one click, a nighttime scene can been recast in daylight.

Color temperature adjustments in action. (Click for full size)

The actual color temperature tool in my PhotoDemon project looks like this:

PhotoDemon’s Color Temperature tool.

Download it here to see it in action.

addendum October 2014

Renaud Bédard has put together a great online demonstration of this algorithm. Check it out here, and thanks to Renaud for sharing!

addendum April 2015

Thank you to everyone who has suggested improvements to the original algorithm. I know there are a lot of comments on this article, but they’re worth reading if you’re planning on implementing your own version.

I’d like to call out two specific improvements. First, Neil B has helpfully provided a better version of the original curve-fitting functions, which results in slightly modified temperature coefficients. His excellent article describes the changes in detail.

Next, Francis Loch has added some comments and sample images below, which are very helpful if you want to apply these corrections to a photograph. His modifications produce a much more detailed image, as his sample images demonstrate.

How to use a scanner (or TWAIN-compatible digital camera) in VB6

Today’s project demonstrates how to implement full scanner support from within a VB6 project. As a bonus, it also provides support for TWAIN-compatible digital cameras.

Before we begin

Because VB6 does not include a native scanner library, some sort of third-party DLL is required for scanner access.  My preferred choice is the free, public-domain EZTW32 library, which I have happily used for many years.  (The first version of EZTW32 was released in 1994!)  This project uses the most recent version of EZTW32 at the time of this writing: v1.19, updated 2009.02.22.  You can check for a newer version of the library here.

There are two versions of the EZTW32 library: a free, public-domain library – called “classic” – and a more sophisticated, paid version – called “pro.”  This project utilizes only the classic version.  The paid version includes many advanced features, and if you are interested in anything beyond simply capturing images from a scanner, it may be worth a look.  A full description of the “pro” version’s feature set is available here.

While the EZTW32 library provides many ways of interacting with the scanner, this project will focus on the following:

Private Declare Function TWAIN_IsAvailable Lib "EZTW32.dll" () As Long
Private Declare Function TWAIN_SelectImageSource Lib "EZTW32.dll" (ByVal hwndApp As Long) As Long
Private Declare Function TWAIN_AcquireToFilename Lib "EZTW32.dll" (ByVal hwndApp As Long, _
 ByVal sFile As String) As Long

The sample project includes a copy of v1.19 of the EZTW32 library.  If you would like to download a newer version of the library, simply copy the new version of EZTW32.dll into the same directory as the executable file (or .VBP file).  It should work without a problem.

Bonus tip: how to load DLLs from any location

Because a 3rd-party library is required to access a scanner in VB6, it is useful to know how to load a DLL from any location.  By default, VB6 will attempt to load a DLL from the computer’s system folder.  This is not ideal when developing portable applications (e.g. applications that can be run without requiring an installer), because it requires the user to manually copy files into their system folder… and that’s bad for a variety of reasons (security, potential for mistakes, etc).  It is possible to load a DLL from other locations using the regsvr32 command, but I don’t like regsvr32 because it adds additional entries to the user’s system registry, and it must be re-run every time the project is moved to a new folder.

So the best solution, in my opinion, is to tell VB to expect the DLL to appear in the same folder as the project itself.  (Alternatively, a /plugins/ sub-directory could be used.)  We do this by adding the following code to the General Declarations section of the project:

Private Declare Function LoadLibrary Lib "kernel32" Alias "LoadLibraryA" (ByVal lpLibFileName _
 As String) As Long
Private Declare Function FreeLibrary Lib "kernel32" (ByVal hLibModule As Long) As Long
Dim hLib as Long

Then, in the Form_Load sub, add this code to determine the project directory and tell VB to load any EZTW32 functions from the DLL in that directory:

Dim ProgramPath as String
ProgramPath = App.Path
If Right(ProgramPath, 1) <> "\" Then ProgramPath = ProgramPath & "\"

hLib = LoadLibrary(ProgramPath & "EZTW32.dll")

Finally, add this to the Form_Unload sub to release the DLL when the program terminates:

FreeLibrary hLib

The sample project (available below) demonstrates this technique, so feel free to download that instead of copying-and-pasting the code from this page.

What this project demonstrates

Because this project is focused on the basics of using a scanner, it demonstrates only the following:

  • How to check if the system offers scanner support (e.g. is a scanner driver loaded?)
  • On systems with multiple scanners, allow the user to select which scanner they want to use
  • Load the scanner’s built-in software and allow the user to preview and scan an image
  • Send the scanned image to a temporary file, then load that temporary file into a VB picture box

All of the above also applies to TWAIN-compatible digital cameras, which the software treats just like a scanner.

Finally, as mentioned earlier, EZTW32 has a paid “pro” version that offers additional features. You can learn more about the “pro” version here.

Caveats

As you may have inferred from its title, EZTW32 relies on the TWAIN protocol for accessing a scanner. TWAIN is one of several ways to interact with a scanner; other common options include WIA (Windows Image Acquisition) on Windows, and SANE (Scanner Access Now Easy) on Linux.

TWAIN has some advantages and disadvantages compared to WIA and SANE. One of TWAIN’s unique characteristics is that it requires the scanner to provide its own user interface. The advantage of this approach is that on a given system, TWAIN-compatible programs all launch the same scanner user interface – the interface provided by the scanner itself. This is good for casual users, because regardless of whether they use this program or Photoshop or GIMP, the scanner interface will always be the exact same. The downside is that if you want to implement custom scanner features or options, TWAIN makes it difficult.

Canon scanner user interface
Because TWAIN relies upon the scanner to provide its own user interface, the user will see a different scan window depending on their scanner brand. My Canon scanner software looks like this.

Another advantage of TWAIN is its longevity. The TWAIN standard has been around since 1992, so pretty much every scanner made in the last 20 years will offer TWAIN drivers. By comparison, WIA didn’t exist until the year 2000, and it wasn’t until Windows Vista that most scanners offered WIA support.

One disadvantage of TWAIN is that Windows Vista and Windows 7 give WIA preferential treatment. If you set up a new scanner using the default Windows Hardware Wizard, it may only load WIA drivers – meaning you’ll need to hunt down the install CD that came with your scanner, or download the latest driver bundle from the scanner manufacturer’s website. I discovered this the hard way when testing this program with my Canon all-in-one printer/scanner/fax. If my program can’t find your scanner, download the free GIMP image software from this link and use the File -> Create… -> Scanner/Camera… option. If my program can’t find your scanner, and GIMP can’t find your scanner, you probably don’t have TWAIN drivers installed. If GIMP works but my program does not, send me a message and I’ll investigate further.

Download the sample project

My sample project is pretty minimalist:

Scanner project user interface
The sample project keeps things simple.

I tried to keep the code as small and simple as possible. Again, the latest version of the EZTW32 dll (v1.19) is included in the download. Future versions of the file should be backwards-compatible; simply replace the existing dll with a newer version.

 

DISCLAIMER: These download files are regularly scanned to ensure they remain free from malicious content. Unfortunately, some virus scanners will flag these .zip files as suspicious simply because they contain source code and/or executable files. I have submitted my projects to a number of companies in an attempt to rectify these false-positives. Some have been cooperative. Others have not. If your virus scanner alerts you regarding these files, please allow the file to be submitted for further analysis (if your program allows for that). This should help ensure that any false-positive warnings gradually disappear for all users.

This site - and its many free downloads - are 100% funded by donations. Please consider a small contribution to fund server costs and to help me support my family. Even $1.00 helps. Thank you!

VB Graphics Programming: Part 4 (Optimizations Checklist)

This last section of my graphics programming tutorials takes a different approach from the previous three sections.  Instead of discussing specific graphics routines, I’m going to give you my “Top 10 List of Graphics Code Optimizations.”  This checklist of optimization techniques will provide a simple, straightforward mechanism for speeding up your graphics application.  We’ll start with the easiest ways to speed up your code and end with the most dramatic (but effective) ways.  My hope is that you can take slow graphics-related code, adjust it according to this checklist, and end up with a much faster version with minimal impact to code readability or complexity.

Tanner’s Top 10 List of Graphics Optimizations

10. Compile to native code, enabling every advanced optimization. While it may seem obvious, this is the easiest – but oft forgotten – way to speed up your graphics code.  Enabling one optimization in particular: ‘Remove Array Bound Checks,’ will significantly speed up DIB section code, as it is heavily dependent on arrays.  When you check that box, VB stops checking array locations for validity (i.e. if you try to access location (x,y) in an array, VB won’t check to see if x or y falls within the declared range of the array).  This optimization comes at a trade-off, of course; any code involving arrays could now be as much as 10-15x faster.  Unfortunately, if you try to access an invalid array location, you won’t get an error – your code will cause a critical fault and the program will crash.  (If you write decent code, however, errors like this will never occur. Check your bounds in advance!)

9. Check your ScaleMode: use pixels, never twips. VB.Net has completely done away with twips, but ol’ VB6 decided to make this silly measurement the default ScaleMode setting for forms and picture boxes.

If your code a) doesn’t work, or b) does work but is strangely slow, make sure you’ve set the ScaleMode of all relevant picture boxes and/or forms to pixels.  As a general rule of thumb, set every object’s ScaleMode to pixels even if you don’t plan to use it for graphics.  This helps ensure that you’ll never fall victim to twips-related errors.

8. Don’t make your code refreshing. For most graphics-related code, AutoRedraw is your friend.  If you have AutoRedraw set to false and you use API-based graphics methods on a picture box, VB6 will attempt to refresh the image every time a pixel changes.  For any image larger than 1×1 pixels, this is a problem.  Refreshing an image takes a lot of time, so refresh an image only after you’re completely done messing with it.  (There are several exceptions to this rule, but they only apply to experienced users who are doing unconventional tricks with picture boxes; generally speaking, unless using a picture box as the receiver of a buffer, keep AutoRedraw on.)

Another thing worth mentioning here is progress bars – if you’re using a progress bar to track image processing code, don’t refresh it for every pixel or even every line.  Refreshing a progress bar is almost as slow as refreshing an image, so if you must use a progress bar then refresh it every 15-20 lines to minimize its impact on your code.

7. Optimize your variables. Because graphics programming can involve many variables, here are some tips for avoiding bad variable usage:

  • Variants are very slow.  Never – EVER – use Variant types in your code.  They’re slow, they take up a lot of memory, and you can always find a way to write code that doesn’t require them.  (Another thing worth mentioning is to never declare your variables like this: Dim x, y as Long. In this example, VB6 will declare x as a Variant and y as a Long. The proper programming technique is: Dim x as Long, y as Long.)
  • Don’t use decimal variables.  Generally speaking, Singles and Doubles are slower, Longs and Integers are faster.  In every Visual Basic case I’ve researched (there are exceptions in other languages, particularly ASM, and under different hardware configurations), floating-point/decimal math is slower than integer/whole number math.  Therefore, avoid decimals if possible.  For example, Red = Red * 1.5 tends to be slower than Red = (Red * 15) \ 10.
  • Longs are fastest.  Because VB6 was designed for 32-bit systems, it heavily favors Long-type variables.  This means that in most cases, Long variables will be executed faster than Integer or Byte variables – and significantly faster than Single, Double, or Variant ones (typically ~3-5x faster than Single ones, and moreso for the other data types).  Use Longs unless you absolutely need to conserve memory by using Integers or Bytes.  (This is especially important for looping variables; because they get accessed so many times, always use Long-type variables for your loops.  VBFibre – the famous VB optimization site shows the significant difference between using a Single and a Long as For...Next variables.)
  • Avoid type conversions.  Regardless of what type of variable you use, keep it consistent.  When you intermix variable types, any programming language slows down – a line like Integer = Long requires VB to temporarily change Long into an Integer, and that takes time.  (Note: this doesn’t apply to DIB section arrays – they must always be of type Byte.)

6. In-line optimization: fix your math. In-line optimization refers to optimizing your code one line at a time. This often involves using faster math functions and better programming techniques. There are many tips I could list under this category, but here are some of the most critical math optimizations you can do for VB6 code:

  • Watch for dividing code.  Two points here: 1) always use \ instead of /. You may not know this, but these functions are different in VB6. Backslash represents integer divide – it ignores any decimals that arise from the division, while frontslash allows for decimal math.  If you’re only interested in the integer results of division, use backslash. It will be faster.  2) Don’t divide if you don’t have to.  Dividing is the slowest of the four basic math functions so avoid it if you can.
  • Avoid calling functions and subs within For/Next or Do/While loops.  Though convenient, functions and subs in VB6 are generally slower than manually inserting that code into the loop.  The slow-down generally isn’t noticeable unless you’re calling functions from within a loop, because then the speed loss is accentuated thousands of times over.
  • If you must use functions and subs, use ByRef instead of ByValByRef variables simply send a function the address of a variable.  ByVal variables require VB to make a copy of the variable before it sends it – and creating a new variable takes time.
  • Lengthy math equations take a lengthy time to process.  Short lines of math allow the compiler to better optimize their order and purpose.  One giant line of code might impress your friends, but VB6 can’t optimize it nearly as well as a bunch of smaller lines.

For a much more in-depth discussion of VB optimization techniques, visit VBFibre at http://www.persistentrealities.com/vbfibre/index.php.  Every VB6 programmer should spend some time there – it’s the most accurate and well-explained information ANYWHERE about VB6 optimizations.  Phenomenal site, donate if you can.

5. Use look-up tables. Few things can speed up code as dramatically as a look-up table.  For those who don’t know, consider the following code example (used to invert a pixel’s color):

For x = 0 to 99
For y = 0 to 99

   'Color = GetPixel()… goes here

   'RGB extraction goes here

   R = 255 - R
   G = 255 - G
   B = 255 - B

   'SetPixel()… goes here

Next y
Next x

For every pixel, VB has to do three ‘Subtract’ functions. This is bad coding – for this simple 100×100 picture example that’s over 30,000 ‘Subtracts.’  Try the following code instead:

Dim LookUpTable(0 to 255) as Byte

 'Fill the look-up table with every possible color value and it's corresponding inverted value
For x = 0 to 255
   LookUpTable(x) = 255 - x
Next x

For x = 0 to 99
For y = 0 to 99

    'Color = GetPixel… goes here

    'RGB extraction goes here

   R = LookUpTable(R)
   G = LookUpTable(G)
   B = LookUpTable(B)

    'SetPixel… goes here

Next y
Next x

With this code, VB only does 256 ‘Subtracts’ and then uses the table of values to change R, G, and B for each pixel.  This makes a big difference in execution time – so use look-up tables every chance you get.

(There is one disclaimer associated with look-up tables in VB6: unless you follow step 10 of this tutorial (particularly the “Remove Array Bound Checks” option), look-up tables can accidentally slow down code that works on small images.  Note that the steps of this tutorial are in order for a reason!)

An excellent example of look-up tables can be found in the Real-time Brightness program.

4. Forget about PSet and Point – use the API. If you haven’t read the previous three sections of these tutorials, now is the time to do it.  PSet and Point are terrible choices for per-pixel image processing.  Use GetPixel and SetPixel/V for a huge speed increase.

3. Forget about GetPixel and SetPixel/V – use DIB sections. While GetPixel and SetPixel/V are nice, they’re still slow.  DIB sections (or BitmapBits) will give you significantly better results.

2. DIB Sections do better in streams. If you read these tutorials in order, you’ll remember that tutorial three discussed how to declare an ImageData() array.  Specifically, a method like Redim ImageData(0 to 2, 0 to Width, 0 to Height) is used.  While this creates an easy-to-use array, its possible to speed it up by declaring it differently.  Redim ImageData (0 to (Width * Height * 3)) gives us an array the exact same size as the first statement, but we use only one dimension instead of three – making our array more than 3x faster for VB to access (and for large arrays, the gain can surpass 10x).  The only problem with this is that we can no longer access direct pixels or colors, but for many graphics functions (like the ‘Invert’ example on (5), or brightness, or contrast, etc.) we do the same thing to every color within a pixel so it doesn’t matter.  For example:

For x = 0 to (Width * Height * 3)
   ImageData(x) = 255 - ImageData(x)
Next x

would invert an image just the same as the example in optimization (5).  The reason I call this method a “stream” is that we treat the image as a continuous stream of values, not as separate pixels or colors. For a more in-depth example of this, see the Real-Time Brightness program.

(One additional disclaimer is in order: make sure to read Section VI: the infamous 4-byte alignment issue in the previous tutorial. Image streams must be declared in a way that adheres to this rule, which limits them to working on images whose width is a multiple of 4.)

1. If you do all this and your program is still too slow, try something extreme. I hate to say it, but if you’ve done everything above and your program is still too slow, you may be out of luck with traditional methods. Here are some advanced techniques you might consider:

  • Consider switching to SafeArrays. This archive from the old Students of Game Design site is an excellent tutorial detailing the structure and use of “SafeArrays”, the internal VB format for arrays.  SafeArrays won’t give you a speed increase in editing your array information, but they will get and set pixel information faster than GetDIBits and Set/StretchDIBits.  If your code requires a large amount of getting and setting pixels, SafeArrays may give you the performance boost you need without a lot of extra coding (as they are structured almost identically to the arrays returned by GetDIBits).
  • Look into assembly language extensions. Planet-Source-Code has several excellent programs by the aforementioned Robert Rayment demonstrating how to use ASM (assembly language) within VB to create graphics routines slightly faster than traditional VB methods. Assembly language is often the fastest programming language available to Windows programmers (next to machine language, but you don’t want to write that :) ), and with some clever work you can use it within your VB6 program.  Be forewarned, however, that this is most definitely not an easy thing to do – but it may give you a little extra speed.
  • Search the net for 3rd party SDKs or ActiveX controls dealing with graphics (DirectX, OpenGL, ActiveX controls, etc.).  In almost every case, DirectX, OpenGL, or similar SDKs (software development kits) won’t help your per-pixel graphics code to be any faster.  They may, however, provide you with hardware support for your effect.  For example, DirectX provides hardware acceleration for gamma correction, which will be much faster than trying to perform gamma correction in code.  Also, you could buy a VB-compatible OCX or DLL written in another language that does your effect for you.  This can, however, be an expensive option.
  • Learn another programming language. The sad but true fact is that some other programming languages provide faster graphics programming techniques via pointers and other tricks.  Even java contains many powerful image editing routines as part of the language, so if you’re really desperate than you can try learning (or rewriting your project) in another programming language.  Also, if you rewrite your routine in C/C++ you could compile it into a DLL and use that from VB, should you want to.
  • Go to the library and check out some graphics programming books. Some of my favorite programming techniques were learned from non-VB graphics programming books.  Even if you don’t understand the code associated with a book, you can still learn a lot about different filters, optimization routines, etc.  If your library doesn’t have anything good, search Amazon.com – you’ll be amazed at how many graphics programming books are out there.
  • Use your imagination. Sometimes the best optimizations are written by programmers who aren’t planning on writing a good optimization.  Your ingenuity is your best programming tool – try to think of new, clever ways to speed up your code. If you think up something great, be sure to let me know about it!

As a demonstration of many of these techniques, feel free to check out the Real-time Brightness example below:

And that, my friends, this is the end of these tutorials.  It’s taken me many, many hours to write these up, so I hope you’ve gained something from reading them.  If you have any comments, questions, suggestions, ideas for future tutorials, or want to send me a generous donation, visit my contact page.

Happy programming!

 

This site - and its many free downloads - are 100% funded by donations. Please consider a small contribution to fund server costs and to help me support my family. Even $1.00 helps. Thank you!

VB Graphics Programming: Part 3 (Advanced API)

Advanced API Pixel Routines

Next comes two advanced ways of getting and setting pixels in Visual Basic: the API routines of GetBitmapBits/SetBitmapBits and GetDIBits/StretchDIBits.  If you haven’t already, I strongly recommend reading the previous two tutorials, “Pure VB Pixel Routines” and “Basic API Pixel Routines,” as they provide the foundation for the advanced graphics principles discussed in this section.

Assuming that you now understand how to use both Visual Basic and the API to get per-pixel data, it’s time to move to the next – and most difficult – section of these tutorials.  Next we are going to discuss the API routines of GetBitmapBits/SetBitmapBits and GetDIBits/StretchDIBits.  Both sets of routines are very fast and very powerful, but they come with a strong disclaimer – pay close attention to any red warnings on this page.  Because these API routines directly interface the heap (dynamically allocated memory), you can easily crash both the VB IDE and/or Windows with a page fault or worse if you use them incorrectly.  Believe me – it’s not a pretty sight to watch your entire machine freeze because you accidentally allocated your array to the wrong size.  (But it gives you a good taste of programming in non-BASIC languages, heh heh.)

But on the happy side of things, these are about as fast as graphics get in VB.  There are ways to use CopyMemory, in-line assembly language, and other freakish routines to get slightly faster effects, but they are not designed specifically for graphics programming so I’m going to avoid them here.

On to the programming!

I – Declaring the Necessary API Functions

At this point in your VB career you are probably used to interacting with images one pixel at a time (using something like the afore-taught .Point/.PSet or GetPixel/SetPixel/V).  These functions are simple to use, but that ease comes at the cost of speed. Many things cause these functions to be slow (as discussed in the first page of this tutorial), so you must be wondering – is there a way to remove some of those speed barriers?

Enter GetBitmapBits and SetBitmapBits.  The big advantage of these two API calls is this: rather than extracting or setting each pixel in an image individually, we pass each of these functions an array and let them fill the whole thing at once with the picture’s pixel data (or set the picture’s pixel data all at once with the information in the array).  This is obviously much more efficient.  The trade-off, of course, is that these techniques involve a little more programming and significantly more risk.  If the array dimensions are off by a mere 1 byte all kinds of things can happen – the picture won’t appear at all, your program will shut itself down, or VB will freeze.  But these only happen if you’re careless and don’t heed my warnings, so pay close attention and you’ll be fine.

We start by declaring a whole bunch of things:

Private Type Bitmap
   bmType As Long
   bmWidth As Long
   bmHeight As Long
   bmWidthBytes As Long
   bmPlanes As Integer
   bmBitsPixel As Integer
   bmBits As Long
End Type

Private Declare Function GetObject Lib "gdi32" Alias "GetObjectA" (ByVal hObject As Long, _
  ByVal nCount As Long, ByRef lpObject As Any) As Long

Private Declare Function GetBitmapBits Lib "gdi32" (ByVal hBitmap As Long, ByVal dwCount As Long, _
  ByRef lpBits As Any) As Long

Private Declare Function SetBitmapBits Lib "gdi32" (ByVal hBitmap As Long, ByVal dwCount As Long, _
  ByRef lpBits As Any) As Long

This might seem a little extreme, so let me go through each of these one at a time.

The Bitmap type is required for the GetObject call – if you’ll look at the GetObject declaration, you’ll notice that the last parameter is of type Any. This is where we will be passing our Bitmap object. As for the individual elements of the Bitmap type, typical graphics programming only cares about four out of the seven variables. They are:

  • bmWidth – the width of the bitmap, in pixels
  • bmHeight – the height of the bitmap, in pixels
  • bmWidthBytes – the width of a bitmap in bytes. If a bitmap is 24 bits-per-pixel (bpp), that means that each pixel occupies 3 bytes. So in this color mode, bmWidthBytes would be (bmWidth * 3). If a bitmap were 16 bpp, each pixel would occupy 2 bytes. In that color mode, bmWidthBytes would be (bmWidth * 2).
  • bmBitsPixel – the number of bits per pixel in the image. In 24bpp mode, the number is 24. In 16bpp mode, the number is 16. Pretty straightforward. Divide this number by 8 to get the number of bytes per pixel.

The other three variables aren’t needed for getting and setting pixels; we simply include them to ensure that our Bitmap type matches the Windows Bitmap type.

Next we have the GetObject call.  The purpose of this API call is to…well, get an object.  You’ll see how this works in a moment.

  • hObject is an object (containing a picture) that we want to get information about – most likely PictureBox.Image or Form.Image.  The properties of hObject will be transferred into lpObject (see below).
  • nCount is the size of the type that is going to receive the information (in our case, the size of the Bitmap type).
  • lpObject is the variable that is going to hold all of the information that we get from hObject (a variable of the Bitmap type we’ve just declared, in fact).  Notice that it is passed ByRef – this allows the API call to edit that variable directly.

GetBitmapBits and SetBitmapBits have identical parameters, which in turn are almost identical to the GetObject parameters.

  • hBitmap represents an object containing a picture (like hObject above, most likely PictureBox.Image)
  • dwCount is the total size of the array holding the image’s pixel data
  • lpBits is the starting address of the place in memory where we want to place the image data (almost always the first spot of an array). Notice, again, that it is declared as ByRef – this allows the API call to edit the array directly (which is a good thing, because that’s how we get the image data!).

Okay – that’s a whole lot of information in a small space, so take a quick break to make sure you understand those declarations.  If some of this is a little hazy, that’s okay, because we’re about to see how they work.

II – Getting Pixels Using GetBitmapBits

Ready?  If so, here’s how we use GetBitmapBits:

Dim bm As Bitmap

GetObject PictureBox.Image, Len(bm), bm

Dim ImageData() as Byte

ReDim ImageData(0 To (bm.bmBitsPixel \ 8) - 1, 0 To bm.bmWidth - 1, 0 To bm.bmHeight - 1)

GetBitmapBits PictureBox.Image, bm.bmWidthBytes * bm.bmHeight, ImageData(0, 0, 0)

Surprisingly, this procedure is very straightforward.  First, we declare a Bitmap object and call the GetObject function.  When called, GetObject will analyze the specified PictureBox and assign the appropriate values to our Bitmap object, which we can then use to prepare our array to receive the image’s pixel data.

Once we have all the picture’s information available to us in the form of a Bitmap object, we declare an array. This ImageData array is of critical importance – we’re going to use it to hold all of the picture’s pixel information.  To make sure it is the right size, we use ReDim to make its dimensions just perfect:

  • The first dimension will contain the values of each pixel’s red, green, and blue values. (bm.bmBitsPixel should equal 24 because your computer is in 24 bit color mode; thus we get 24/8 = 3 bytes per pixel. Because we start the dimension at zero, we subtract one from the upper bound to give us three total spots: 0, 1, and 2, which correspond to red (2), green (1), and blue (0))
  • The second dimension will be used to address the x coordinates of the image’s pixels.
  • The third dimension will used to address the y coordinates of the image’s pixels.

SIDE NOTE ABOUT ‘GETOBJECT’: You may be wondering why we use the GetObject call at all – couldn’t we just resize the ImageData array using the picture box’s ScaleWidth and ScaleHeight properties?  In theory, you could.  However, VB5 does strange things to the ScaleWidth and ScaleHeight properties depending on what is stored there.  For example, the same image might report different ScaleWidth and ScaleHeight properties at different execution times.  In my experience, JPEGs are notoriously bad at this – when you load one, VB5 sometimes thinks that the picture’s width is one pixel less in the picture box than it is in memory.  Honestly, I have no idea as to why VB5 has this problem.  VB6 seems to work fine.  GetObject is always accurate so I use it instead, and I recommend that you do too.  There is no measurable speed difference between these two mechanisms.

SIDE NOTE ABOUT DECLARING YOUR ARRAY: Arrays supplied to GetBitmapBits (and later in this tutorial, GetDIBits) must have a width that is a multiple of 4, e.g. 4, 8, 16, 256, 360, etc. If your image has a width that is a multiple of four, no worries – but if it is not a multiple of four, you will need to adjust your code accordingly. See the bottom of this page for details (in the section titled “VI – OPTIONAL: The Infamous 4-Byte Alignment Issue”).

ANOTHER SIDE NOTE ABOUT DECLARING YOUR ARRAY: You can use any number of dimensions in your array, so long as the total size is accurate. For example, you could also do something like ReDim ImageData(0 to bm.bmWidth * bm.bmHeight * 3) and the function would still work fine. The API call could care less about how the array is declared – all it gets is the address of the first element in the array and the number of bytes that it’s allowed to work with. The way the array is dimensioned is only for your convenience.  I like the above way because it makes editing the image very easy. This issue will be discussed further in the next section of this tutorial.

WARNING!! This ReDim statement is where you can really screw your computer.  If ImageData is too small, GetBitmapBits will attempt to put the picture data in unallocated memory – causing a general protection fault, a page fault, or some other nasty illegal operation.  Make sure that ImageData is the right size!

The GetBitmapBits call itself is very straightforward: it takes the array (in this case, ImageData()) and fills it with the pixel data located in PictureBox.  Now you can edit the values any way you want.  For example, the following loop would invert all of the pixels in the image:

'First, get the image data using the above code section

Dim X as long, Y as long

For X = 0 to PictureBox.ScaleWidth - 1
For Y = 0 to PictureBox.ScaleHeight - 1

   'Invert the R value
   ImageData(2, X, Y) = 255 - ImageData(2, X, Y)
   'Invert the G value
   ImageData(1, X, Y) = 255 - ImageData(1, X, Y)
   'Invert the B value
   ImageData(0, X, Y) = 255 - ImageData(0, X, Y)
Next Y
Next X

Really, GetBitmapBits is as easy as GetPixel if you understand the API structure.

III – Setting Pixels Using SetBitmapBits

SetBitmapBits is almost identical to GetBitmapBits:

Dim bm As Bitmap

GetObject PictureBox.Image, Len(bm), bm

SetBitmapBits PictureBox.Image, bm.bmWidthBytes * bm.bmHeight, ImageData(0, 0, 0)

If PictureBox.AutoRedraw = True Then
   PictureBox.Picture = PictureBox.Image
   PictureBox.Refresh
End If

Everything is the same as GetBitmapBits, except that we aren’t resizing the array (because it is already the right size and resizing it would erase all of its information!).  The last If/Then statement is included because SetBitmapBits won’t automatically initialize the AutoRedraw event, so we have to tell it to replace the Picture property (what is shown on the screen) with the Image property (what is stored in memory).

I hope you’re finding this easier than expected!  In fact, it’s almost too easy… so of course, there is a slight problem with this method: both GetBitmapBits and SetBitmapBits only work in 24/32-bit color mode (16.7 million colors). Actually, they work in 16 and 8 bit color modes too, but the image data no longer occupies 3bpp (bpp = bits per pixel) so editing the image data is significantly more complicated.  It can be done, but you have to write a function to translate 2 bits into 3 as well as transferring the data into a separate array while you edit it.  Then, to draw it, you have to translate the 3 bits back into 2 bits and then transfer your editing array back into the original one.  It’s messy and time-intensive, so I wouldn’t recommend this method

So of course, someone is going to ask “but how can I do fast graphics in 16 or 8 bit color mode?” That is what DIB sections are for, so if you want to know about them then keep reading.

(Personally, I would recommend using DIB sections for all of your graphics programs because you’ll never get unexpected color-mode errors with them, and it’s a great way to add additional functionality to your graphics program.  I have only discussed BitmapBits because they make for an excellent introduction to DIB sections.)

IV – A Crash Course in Declaring DIB Sections

DIB section stands for ‘Device Independent Bitmap.’  The name is pretty self-explanatory: DIBs are simply a way of interacting with bitmaps in any color mode or on any computer and getting consistent results.  There are actually two varieties of DIBs – OS/2 encoded and Windows encoded, so I guess “device independent’ isn’t totally accurate… but that’s okay.

DIBs share many characteristics with BitmapBits.  The calls share certain parameters and the underlying logic is very much the same.  However, DIB sections have several major differences you need to be aware of: they’re slightly more confusing to use, they require more code, and they return the image data upside-down.  The most important difference, however, is that DIB sections work in any color mode – and, as a bonus, the StretchDIBits call is much more powerful than SetBitmapBits. Below are the required DIB section declarations:

Private Type BITMAP
   bmType As Long
   bmWidth As Long
   bmHeight As Long
   bmWidthBytes As Long
   bmPlanes As Integer
   bmBitsPixel As Integer
   bmBits As Long
End Type

Private Declare Function GetObject Lib "gdi32" Alias "GetObjectA" (ByVal hObject As Long, _
  ByVal nCount As Long, ByRef lpObject As Any) As Long

Private Type RGBQUAD
   rgbBlue As Byte
   rgbGreen As Byte
   rgbRed As Byte
   rgbAlpha As Byte
End Type

Private Type BITMAPINFOHEADER
   bmSize As Long
   bmWidth As Long
   bmHeight As Long
   bmPlanes As Integer
   bmBitCount As Integer
   bmCompression As Long
   bmSizeImage As Long
   bmXPelsPerMeter As Long
   bmYPelsPerMeter As Long
   bmClrUsed As Long
   bmClrImportant As Long
End Type

Private Type BITMAPINFO
   bmHeader As BITMAPINFOHEADER
   bmColors(0 To 255) As RGBQUAD
End Type

Private Declare Function GetDIBits Lib "gdi32" (ByVal hDC As Long, ByVal hBitmap As Long, _
  ByVal nStartScan As Long, ByVal nNumScans As Long, lpBits As Any, lpBI As BITMAPINFO, _
  ByVal wUsage As Long) As Long

Private Declare Function StretchDIBits Lib "gdi32" (ByVal hDC As Long, ByVal x As Long, _
  ByVal y As Long, ByVal dWidth As Long, ByVal dHeight As Long, ByVal SrcX As Long, _
  ByVal SrcY As Long, ByVal SrcWidth As Long, ByVal SrcHeight As Long, lpBits As Any, _
  lpBI As BITMAPINFO, ByVal wUsage As Long, ByVal RasterOp As Long) As Long

Quite the mess of declarations, isn’t it? You should notice some similarities between these declarations and the BitmapBits ones. Here’s a quick explanation of the DIBits calls:

  • The first type and declaration are just the same old GetObject stuff – you already know all about that.
  • The next type, RGBQuad, represents basic pixel data – red, green, and blue values, along with an alpha channel.

SIDE NOTE ON ALPHA CHANNELS: For those who are interested: both DIBs and regular bitmaps can contain transparency information.  If you have (what used to be, heh) an expensive monitor and video card and run them at 32-bit color mode, that extra byte contains transparency data (a number from 0-255, 0 being opaque and 255 being transparent).  So technically, 32-bit bitmaps and DIBs could be used like GIFs or PNGs and displayed transparently.  There’s actually an API call with Win2K/ME/XP called AlphaBlend that’s similar to BitBlt/StretchBlt except that it utilizes an alpha channel (this method is very similar to DirectX and a transparent key color); you can read all about the AlphaBlend call at MSDN.

  • The BITMAPINFOHEADER type contains all of a particular bitmap’s information, unlike the stripped-down version we use for GetObject.  This is what makes DIB sections “Device Independent” – by knowing all of that extra information about the bitmap we can display it accurately on any device at any color resolution.
  • Lastly, the BITMAPINFO class combines a header and an array of RGBQUADs (used for the palette in 8-bit images).  This header is capable of holding data for DIB sections of any color depth – from 1bpp to 32bpp.  Again, this is part of making DIB sections “Device Independent.”  It also comes in handy when using 8bpp color modes because you can directly edit the palette for super-fast graphics effects.

The GetDIBits call is somewhat more complicated than the GetBitmapBits one, so let’s go through it one part at a time.

  • hDC is the same as it was in GetPixel – it is the handle/”address” of the device we want to get the pixel data from. Most likely, this is PictureBox.hDC or Form.hDC.
  • hBitmap is the location of the pixel data itself. This is most commonly PictureBox.Image or Form.Image
  • nStartScan is the line we want to start reading the pixel data from.  This will always be 0, unless for some odd reason you want to read the data from the middle of the image.
  • nNumScans is the number of horizontal lines that you want to read from the image.  This will always be the height of the image (in pixels), unless you want to extract every-other-line or something strange like that.  (Scanlines also come in handy for doing fast image rotations, but I’m not going to discuss those here ;) )
  • lpBits is the same as it was in GetBitmapBits.  This is the array that the image’s data will be copied into.
  • lpBI is a BitmapInfo variable that contains all of the desired information of the bitmap we are want to get.  This includes the width, height, and – important! – the color depth.  This is how we let Windows know that even though the computer may be in 16bpp or 8bpp color modes, we want the bitmap information to be in 24bpp mode (color depth is handled via the .bmBitCount property).
  • Last is the wUsage variable, which is related to referencing source-DC-based palettes. Because this tutorial focuses only on 24-bit (i.e. non-paletted) image processing, we’re going to ignore this variable completely – just always leave it as zero.

We’re almost done!

Unfortunately, if you thought GetDIBits was long-winded, you’re not going to like StretchDIBitsStretchDIBits is a very powerful call, but this means there are a lot of parameters.  If you are familiar with BitBlt and/or StretchBlt this part will probably make a lot of sense to you.  The StretchDIBits parameters are:

  • hDC: same as GetDIBits
  • x: the x coordinate of the top-left corner you want to draw the pixels to
  • y: the y coordinate of the top-left corner you want to draw the pixels to
  • dWidth: the desired width of the destination image.  This is usually the same size as the image you got the data from, although you can stretch or shrink the destination size if you wish.
  • dHeight: the desired height of the destination image.  Refer to the dWidth notes.
  • SrcX: the x coordinate of the top-left pixel in the source array (for us, ImageData()) that you want to start taking the pixels from. Usually zero, but you can change this value to only take a portion of the source image.
  • SrcY: the y coordinate of the top-left pixel in the source array that you want to start taking pixel data from.  Refer to the SrcX notes.
  • SrcWidth: the desired width of the pixel selection from the source image. This is usually the same size as the original image, although you can stretch or shrink the source size if you wish.
  • SrcHeight: the desired height of the pixel selection from the source image. Refer to the SrcWidth notes.
  • lpBits: same as GetDIBits. This is the array that the pixel data will be taken from (again, ImageData())
  • lpBi: same as GetDIBits. This is the BitmapInfo variable that contains all of the correct parameters for our pixel data.
  • wUsage: same as GetDIBits – leave it as zero.
  • RasterOp: a raster operation, exactly the same as you would use for BitBlt or StretchBlt. There is a complete list of RasterOp constants listed in the VB help files (or MSDN collection), but the most common one is vbSrcCopy, which will simply copy the pixels from the source array to the destination picture box.

V – Using DIB Sections

Now that your brain has had some time to digest all of those declarations, let’s demonstrate the GetDIBits call.  Here’s a full-blown example of how to get an image’s data using the GetDIBits call, minus the declarations above:

'Routine to get an image's pixel information into an array dimensioned (rgb, x, y)
Public Sub GetImageData(ByRef SrcPictureBox As PictureBox, ByRef ImageData() As Byte)

'Declare variables of the necessary bitmap types
Dim bm As Bitmap
Dim bmi As BITMAPINFO

'Now we fill up the bmi (Bitmap information variable) with all the necessary data
bmi.bmHeader.bmSize = 40 'Size, in bytes, of the header (always 40)
bmi.bmHeader.bmPlanes = 1 'Number of planes (always one)
bmi.bmHeader.bmBitCount = 24 'Bits per pixel (always 24 for image processing)
bmi.bmHeader.bmCompression = 0 'Compression: none or RLE (always zero)

'Calculate the size of the bitmap type (in bytes)
Dim bmLen As Long
bmLen = Len(bm)

'Get the picture box information from SrcPictureBox and put it into our 'bm' variable
GetObject SrcPictureBox.Image, bmLen, bm

'Build a correctly sized array.
ReDim ImageData(0 To 2, 0 To bm.bmWidth - 1, 0 To bm.bmHeight - 1)

'Finish building the 'bmi' variable we want to pass to the GetDIBits call
bmi.bmHeader.bmWidth = bm.bmWidth
bmi.bmHeader.bmHeight = bm.bmHeight

'Now that we've filled the 'bmi' variable, we use GetDIBits to take the data from SrcPictureBox and put
'  it into the ImageData() array using the settings specified in 'bmi'
GetDIBits SrcPictureBox.hDC, SrcPictureBox.Image, 0, bm.bmHeight, ImageData(0, 0, 0), bmi, 0

End Sub

We’re almost done with DIB sections – all that’s left is StretchDIBits.

The procedure required to set up our variables for StretchDIBits is almost identical to the procedure we used for GetDIBits. In fact, everything up to the actual GetDIBits call is the same – everything except the ReDim ImageData() line, of course.  (If we ReDimmed the array before setting it, we would erase all of the pixel data!).  Here’s a full example:

'Routine to set an image's pixel information from an array dimensioned (rgb, x, y)
Public Sub SetImageData(ByRef DstPictureBox As PictureBox, ByRef ImageData() As Byte)

'Variables for the necessary bitmap types
Dim bm As Bitmap
Dim bmi As BITMAPINFO

'Fill the bmi (Bitmap information variable) with appropriate values
bmi.bmHeader.bmSize = 40
bmi.bmHeader.bmPlanes = 1
bmi.bmHeader.bmBitCount = 24
bmi.bmHeader.bmCompression = 0

'Calculate the size of the bitmap type (in bytes)
Dim bmLen As Long
bmLen = Len(bm)

'Get the picture box information from DstPictureBox and put it into our 'bm' variable
GetObject DstPictureBox.Image, bmLen, bm

'Now that we know the object's size, finish building the temporary header to pass to StretchDIBits
bmi.bmHeader.bmWidth = bm.bmWidth
bmi.bmHeader.bmHeight = bm.bmHeight

'Use StretchDIBits to take the data from the ImageData() array and put it into SrcPictureBox using
'  the settings specified in 'bmi'
StretchDIBits DstPictureBox.hDC, 0, 0, bm.bmWidth, bm.bmHeight, 0, 0, bm.bmWidth, bm.bmHeight, _
  ImageData(0, 0, 0), bmi, 0, vbSrcCopy

'Since this doesn't automatically initialize AutoRedraw, we have to do it manually
'Note: set AutoRedraw to 'True' when using DIB sections. Otherwise, you may get unpredictable results.
If DstPictureBox.AutoRedraw = True Then
   DstPictureBox.Picture = DstPictureBox.Image
   DstPictureBox.Refresh
End If

End Sub

And there you have it – a complete explanation of how to get image data from any picture in any color mode and how to set that same data back into a picture once you’re done editing it.

VI – DIB Sections in Action

This .zip file shows these exact routines – cut and pasted out of this tutorial into a form – being used to adjust the brightness of an image.  Quite a bit better than GetPixel and SetPixel or .Point and .PSet, aren’t they?

It may surprise you, but in the next tutorial we’re going to get this program running even faster – we’re going to use some DIB section tricks to make it run in real-time.

Before we continue, however, I am including an optional section regarding the infamous 4-bit alignment issue associated with DIB sections.  If you are interested in using DIB sections only casually, this issue may not apply to you.  If, however, you plan on using DIB sections extensively, this issue is critical.  DIB sections have trouble if you use them on an image whose width is not a multiple of 4.  We’ll discuss how to deal with this problem in the optional section below.

CONTINUE TO “TANNER’S TOP 10 LIST OF GRAPHICS OPTIMIZATIONS”

VI – OPTIONAL: The Infamous 4-Byte Alignment Issue

As you may know, all versions of Windows at the time of this writing (95, 98, ME, NT4, 2000, XP) are 32-bit operating systems.  This means that memory within Windows is split up into 32-bit, or 4-byte, chunks.  Normally this means very little to a VB6 programmer, but when using DIB sections we must take this into account.

Because DIB sections are designed to be fast, they’re optimized for speed in many ways.  One such way is that they require any arrays associated with them to be 4-bytes (or 32 bits) wide.  This allows the arrays to line up in memory exactly – without any trailing bytes – which in turn allows Windows to access the information more quickly, since it doesn’t have to re-align each horizontal line of the image to match up with 32-bit memory spaces.

As it turns out, this isn’t a problem for images whose width is already a multiple of 4: 800x600, 32x32, 1920x1080, etc.  All standard image/screen sizes are multiples of 4 for a reason.

Sometimes, however, it’s necessary to work with images whose width may not be divisible by 4 (e.g. 29, 30, and 31 instead of 32).  This creates serious problems with DIB sections – try plugging such an image into the code above and you’ll see what I mean.

So how do we solve such a problem?  There’s no good way, to be honest.  Two main options exist:

1) Resize the image to make it have a width that’s a multiple of 4.

2) Manually force the array containing the image data to have a width that’s a multiple of 4.

The first option is preferable, but if that’s not available to us then we have to do some hardcore adjusting of the array we use (ImageData() in this tutorial).  This requires CopyMemory and a For loop – both of which are time killers, which is contrary to the whole point of this tutorial.

So in the interest of space and brevity, I’m not going to go through the intricacies of this method in this tutorial.  Instead, you can download a modified brightness program that demonstrates how to do this.  Robert Rayment – a profilic VB programmer in his own right – put it together, and all credit for the modifications go to him.  Please shower him with praise.

An easier fix – and my preferred method – is to use a 2-dimensional array to receive the image data instead of a 3-dimensional one, like so:

ArrayWidth = (bm.bmWidth * 3) - 1
ArrayWidth = ArrayWidth + (bm.bmWidth Mod 4)
ArrayHeight = bm.bmHeight
ReDim ImageData(0 To ArrayWidth, 0 To ArrayHeight) As Byte

This method works well, but accessing individual pixels is slightly more cumbersome:

  • Red is in location (x * 3 + 2, y)
  • Green is in location (x * 3 + 1, y)
  • Blue is in location (x * 3, y)

Using a temp variable to store the x * 3 value actually makes this faster than the 3d array used in the tutorials.  As such, this is my favorite method. It is also the method I use in PhotoDemon.

Streams, mentioned in the next tutorial, also need to be 4-byte aligned.

CONTINUE TO “TANNER’S TOP 10 LIST OF GRAPHICS OPTIMIZATIONS”

 

This site - and its many free downloads - are 100% funded by donations. Please consider a small contribution to fund server costs and to help me support my family. Even $1.00 helps. Thank you!

VB Graphics Programming: Part 2 (Beginning API)

Basic API Pixel Routines
Next, let’s discuss the basics of per-pixel graphics programming using the simple API routines of GetPixel and SetPixel/SetPixelV.  If you haven’t already, I recommend reading the previous page, “Pure VB Pixel Routines,” as it provides the foundation for the advanced graphics principles discussed in this and the next two sections.

Assuming that you now understand how to use built-in Visual Basic functions to get per-pixel data, it is time to extend that information a little bit further to include the Windows API.  For those who don’t know, the Windows API is a collection of dynamically linked libraries (so-called ‘DLL files’) that contain commonly used programming routines.  These routines, or “interfaces” (API = Application Programming Interface) range from I/O to networking to multimedia, and they generally work much faster than the intrinsic Visual Basic routines.  The API that we are going to be looking at is called “GDI32,” which stands for Graphic Device Interface: 32-bit version.  (This dll is included with every Windows OS since Win95, so no matter what OS you are using with VB 5/6 this method will work.)

GDI32 is a collection of – surprise, surprise – routines commonly used in graphic and image manipulation.  This includes brushes for drawing, bit-block transfers (BitBlt) for painting image sections, and what we’re most interested in for this tutorial: several important ways to get and set pixel data.  We will start by looking at three functions in particular: GetPixel, SetPixel, and SetPixelV.

I – Declaring the Necessary API Functions

Because the GDI32 functions aren’t an integral part of the Visual Basic programming language, we have to declare them in the General Declarations section just as we would a variable or a type.  The syntax each of our API function declarations is as follows:

Private Declare Function GetPixel Lib "GDI32" (ByVal hDC As Long, ByVal x As Long, ByVal y As Long) As Long

Private Declare Function SetPixel Lib "GDI32" (ByVal hDC As Long, ByVal x As Long, ByVal y As Long, ByVal crColor As Long) As Long

Private Declare Function SetPixelV Lib "GDI32" (ByVal hDC As Long, ByVal x As Long, ByVal y As Long, ByVal crColor As Long) As Byte

For those unfamiliar with API declarations, these might look imposing at first – but don’t worry.  They’re all pretty darn straightforward.

  • Private simply means that we only want to use these functions within the current code block. If we were to use Public instead of Private, we could use these function calls in any module, class module, or form in the current project.
  • Declare Function FunctionName simply lets VB know that we plan on using a function titled GetPixel or SetPixel.
  • Lib "GDI32" is short for Library “C:WindowsSystemGDI32.dll”‘ – it tells Visual Basic which dynamically-linked library (DLL) contains the code for the ‘FunctionName’ we just declared.

The values inside of the parentheses are nothing more than variable declarations, just as you would see for a standard sub or function.

  • hDC stands for “Handle Device Context.” This is the computer’s way of saying ‘address of an object.’ This variable tells us what picture box we want to draw on (like PictureBox1.hDC)
  • X and Y are the location – in pixels – of the pixel we want to work with. API calls always work in pixels, so you should too.  If your picture boxes use twips, inches, or any measurement other than pixels, these calls won’t work.
  • crColor in SetPixel and SetPixelV is the color that we want to set pixel (x,y) to.  This is identical to the ‘Color’ part of the PSet call.  (I’m not entirely sure why this is typically declared as “crColor”; you could call it anything you wanted, but convention seems to stick to “crColor.” Go figure…)

Looks familiar, eh? You should be able to recognize some major similarities between these calls and Point/PSet. If you don’t, you may want to reread the previous tutorial page.

II – Using Our New Friend, GetPixel

Now that we’ve told Visual Basic everything it needs to know about our “GDI32” functions, we can use them anywhere and everywhere we want to!  Yay!

Let’s demonstrate.  If you wanted to get a color from pixel (35, 42) of picture box “Picture1”, you would use the following:

Dim Color as Long
Color = GetPixel(Picture1.hDC, 35, 42)

I hope this looks painfully easy, because that’s exactly what it is.

Now that you’ve gotten your color into a variable of type Long, you still have to extract the individual red, green, and blue components just like before. I’m not going to repeat all that stuff, so just cut and paste those functions out of the last tutorial if you need them.

III – Setting Pixels Using SetPixel and SetPixelV

Again, setting a pixel’s color is almost identical to getting its color:

Dim APIReturnValue as Long
APIReturnValue = SetPixel(PictureBox.hDC, x, y, Color)

OR

SetPixel PictureBox.hDC, x, y, Color

OR

Dim APIReturnValue as Byte
APIReturnValue = SetPixelV(PictureBox.hDC, x, y, Color)

OR

SetPixelV PictureBox.hDC, x, y, Color

Ahhh! Four different ways to do exactly the same thing! Seems a little weird, doesn’t it?  Let me explain why this is.

Every function returns a value of some type.  If you’ll scroll back up and look at the SetPixel and SetPixelV declarations, you’ll notice a slight difference between the two: SetPixel is of type Long and SetPixelV is of type Byte. This is because SetPixel returns the color that it was able to set (for example, if you tried to set RGB(0, 0, 1) in 8- or 16-bit color mode, SetPixel would likely only be able to set RGB(0, 0, 0) instead), while SetPixelV only returns whether or not the pixel was set (1 or 0).  Because SetPixelV only has to return a boolean value instead of a Long (though they’re stored identically in memory), I prefer to use it.  However, SetPixel could be useful in color modes other than 24-bit, because you could determine the difference between the color you wanted to set and the color that actually got set, as mentioned above.

This is the difference between SetPixel and SetPixelV.  The reason for the other two declarations is whether or not we care what value SetPixel/SetPixelV returns.  If we don’t care, it is easier to just use the second form of the call – the one without the extra variable declaration.  If, however, we want to know what value they return, we need to use the first form.

Now you know four different ways to set pixels using the API!  Use that to impress your friends… :)

Let’s quickly demonstrate a specific example using SetPixel/SetPixelV. Using our example from the last tutorial, let’s say that you want to set pixel (35,42) of the picture box titled “Picture1” to the value of variable ‘Color:’

SetPixelV Picture1.hDC, 35, 42, Color

Again we note that Color is of type Long – you could use the RGB() function just as you did in part 1.  In that case, you could write:

SetPixelV Picture1.hDC, 35, 42, RGB(255, 0, 0)

To set pixel (35, 42) of “Picture1” to pure red.  See how easy the API calls are to use?  Once declared, they are no harder than than PSet and Point – and, as you’re about to see, they’re significantly faster.

IV – Using GetPixel and SetPixel to Edit an Image

This .zip file contains a program identical to the last one, except that this one utilizes the API for its pixel interaction.  Compare the results of this program to the PSet/Point one – notice a pretty significant difference?  As a good programming exercise, build a small function to time how long each method takes and compare the results.  The API calls can be anywhere from 3-10x faster than PSet and Point – a pretty noticeable difference for hardly any extra work.

V – In Conclusion: The Need for Speed – GetPixel/SetPixel vs. Point/PSet and Beyond…

I hope that you’re beginning to see the advantages of well-used API calls within your Visual Basic programs. If you can master using the Windows API, your VB capabilities are endless.  GetPixel/SetPixel are just the tip of the iceberg, too – the next tutorial will show you a method 10-50x faster than this one!  Get excited!!

But before we head into the next tutorial, let’s think of ways that GetPixel and SetPixel/SetPixelV could be improved.

For one, we still have to manually extract the red, green, and blue values out of a Long-type variable – this is not only annoying, but it’s slow as well.  Things would go faster if we could somehow get Windows to separate the Long variable for us.

Also, there is this problem of having to use GetPixel/SetPixel for every single pixel.  Any way you slice it, running two functions for each of our 120,000 pixels is a pretty slow proposition.  What if there was a way to get the data for every single pixel at once – that way, we’d only have to use a single API call to get ALL of our pixel data.  Now that would be fast!

Well believe-it-or-not, our next tutorial will explain how to do just that – get Windows to give us all of an image’s pixel data at once, nicely parsed into its red, green, and blue components.  These are commonly referred to as DIB sections, the most powerful API graphics call you can use from within VB.

CONTINUE TO “ADVANCED API PIXEL ROUTINES”

 

This site - and its many free downloads - are 100% funded by donations. Please consider a small contribution to fund server costs and to help me support my family. Even $1.00 helps. Thank you!

VB Graphics Programming: Part 1 (Pure VB)

Pure VB Pixel Routines

First, let’s discuss the basics of per-pixel graphics programming using only built-in Visual Basic functions.  I recommend that even hardened VB veterans glance through this document, as it provides the foundation for the advanced graphics principles discussed in the next three tutorials.  We will discuss the only VB per-pixel graphics functions (Point and PSet), and after this is done you may know a lot more than you ever wanted to about VB graphics…  :) *hehe*

Most of you have probably heard of the “horrible twins of PSet and Point”*. These two routines are usually the first per-pixel methods introduced to VB programmers (since they come included as part of the language), but – perhaps inevitably – they are also the slowest way to do things.  Avoid ever using these routines for per-pixel image processing (though they do have some uses in non-pixel-sized work). I include them here only for completeness; I would never recommend using them in an actual image processing program because they are so extremely slow.

Why are they so slow? We’ll discuss that later, after we’ve looked at their syntax.

I – Getting the Color of an Individual Pixel

You can use the Point event in VB to get the color of a specified pixel. The format is as follows:

Dim Color as Long
Color = PictureBox.Point(x,y)

PictureBox is the name of the picture box or form you want to retrieve the pixel from, and (x,y) are the pixel’s coordinates.

For example, if you wanted to get a color from spot (35, 42) of picture box “Picture1”, you would use the following:

Color = Picture1.Point(35,42)

It doesn’t get much simpler than that, folks.

So now the question is “what do I do once I’ve gotten the color?” After all, it comes in Long type, which is some strange number from -2 billion to +2 billion…and that really doesn’t lend itself to easily adjusting the color of that pixel.

Thus the challenge is figuring out how to change one 4-byte number into three 1-byte numbers (red, green, and blue).

While this may sound easy, the theory behind doing this requires some knowledge of binary encoding…which is far too large a topic to be covered here. Luckily for you, here are three functions that will automatically do RGB color extraction for you:

Public Function ExtractR(ByVal CurrentColor As Long) As Byte
   ExtractR = CurrentColor And 255
End Function

Public Function ExtractG(ByVal CurrentColor As Long) As Byte
   ExtractG = (CurrentColor \ 256) And 255
End Function

Public Function ExtractB(ByVal CurrentColor As Long) As Byte
   ExtractB = (CurrentColor \ 65536) And 255
End Function

To utilize these functions, use the following syntax:

Dim R as Byte, G as Byte, B as Byte

Dim Color as Long

Color = PictureBox.Point(0, 0)

R = ExtractR(Color)
G = ExtractG(Color)
B = ExtractB(Color)

Pretty neat, eh?

This is the first method we’ll use for getting a pixel’s data and breaking it down into its red, green, and blue components.  Now let’s quickly mention how to set this data back into a picture box.

II – Setting the Color of an Individual Pixel

Setting a pixel’s color is almost identical to getting its color. You use the VB event “PSet,” which stands for “Pixel Set.”

PictureBox.PSet (x,y), Color

Again, PictureBox is the name of the picture box or form you want to set the pixel to and (x,y) are the pixel’s coordinates. The only difference here is that we also include the color that we want to set. So, using the example above, if you wanted to set a color to pixel (35, 42) of picture box “Picture1” you would use the following:

Picture1.PSet (35,42), Color

It is worth noting that Color is of type Long, which creates the same problem we discussed above – how to change three separate red/green/blue values into a single 4-byte number. Fortunately, VB has a built-in command called RGB() that does this conversion for us. To illustrate it’s use, let’s use the same example saying that you want to change the color of the pixel at (35,42) to pure red:

Picture1.PSet (35,42), RGB(255, 0, 0)

The first RGB parameter is red, then green, last blue, so RGB(255,0,0) will set the color to pure red.  Easy!

III – Using Point and PSet to Edit an Image

First, a little disclaimer: entire books have been written on the theories behind GP and there are entire programming disciplines whose job is nothing but optimizing graphics routines.  So, while what I’m about to show you in code is a nice method, be advised that GP is an extremely complicated field and to truly succeed in it you must be willing to do a little research.  I have chosen a well-optimized and very standard method for my sample programs because it is easy to understand while still offering good results.  But, for this first tutorial, don’t be disappointed if the results aren’t particularly incredible or lightning-fast.  That’s what the next three tutorials are for!

The code in this .zip file will demonstrate how to change the brightness of an image using a standard linear brightness algorithm. The code is simple and well-commented. Read through the comments and make sure that you understand how everything works.

IV – Why are Point and PSet So Slow?

If you’ve tried out the sample code, you’re probably not impressed…and rightfully so!  Point and PSet – though easy to use – are extremely slow. Can you imagine trying to work with images 4 or 5 times the size of the demo one? Not so cool.

So why are these two functions SO slow?  To illustrate it, let’s follow the path your computer takes for changing the color of a single pixel using Point and PSet as you just saw done in the sample program. (Author’s Note: I base these conclusions on general programming knowledge, not known facts; so while I’m pretty sure that this explanation is accurate, I could be wrong on some of the details. I invite and encourage input on making this section 100% accurate.)

  1. Upon encountering a Point command, VB’s first task is to do a whole crapload of error checking. This involves things like making sure that the pixel you want is within the image’s boundaries, making sure that the PictureBox exists, seeing if an image has been loaded or if you’re working with a blank image, etc. This step is speed killer #1 – but the advantage is that Point will never crash your machine, thankfully.
  2. Once VB has decided that there is actually a pixel at point (x,y), it now has to figure out where to get the pixel color from.  This changes depending on both the status of AutoRedraw and whether or not you’ve updated the image since loading it. VB will usually go to the ‘Image’ property, but in certain cases AutoRedraw may tell it to go to the ‘Picture’ property.  (If that doesn’t make any sense, forget about it!)  This step is speed killer #2 – but again, you never crash your machine and you always get predictable results.
  3. After VB knows where the pixel data resides, it can now go and get the pixel information. This step is all handled in memory, so this is really fast.
  4. After VB gets the color of this pixel, it must transfer that information into the variable specified by the original Point command.  This is all but instantaneous – no speed problems here.
  5. Once a variable of type Long contains the color of the pixel, we must parse that long into its red, green, and blue components. This step is speed killer #3, because we gotta do three ‘Ands’ and two ‘Divides’ for every single pixel. For an image like the sample one, that’s 400×300 or 120,000 pixels… meaning there’s a grand total of 360,000 ‘Ands’ and 240,000 ‘Divides.’ This step is a very, very bad one for speed – the ‘Ands’ are fast, but the ‘Divides’ are extremely slow (times 240,000).
  6. Once we have a red, green, and blue component, we change these values to the new values specified by the look-up table. This step is very fast because, again, it’s nothing more than simple memory transfers.
  7. Next comes the PSet step.  This process is almost identical to Point, so I’m going to abbreviate its steps.  First, it does the error checking.  Speed killer #4 here.
  8. VB will automatically assign the new color to its appropriate location within the ‘Image’ property.  This is fast – again, it’s all done in memory.
  9. Now VB has to decide whether or not to refresh the image.  If AutoRedraw is set to false, VB will attempt to redraw the entire image after each pixel has been set.  Do not do this – EVER – while working with per-pixel programming routines.  If AutoRedraw is set to true, VB will only redraw the entire image after you explicitly tell it to or after you finish the loop containing the PSet calls.  Redrawing the image is very slow because your computer has to copy the information for thousands of pixels from the Picture or Image property to wherever the screen data is located (either VRAM or RAM – this is yet one more thing your computer has to figure it; it too takes time). Although your RAM is one of the faster parts of your computer, it will be slowed down by lots of huge memory chunk transfers (like graphics).  This is speed killer #5.

Steps 1, 2, 5, 7, and 9 are what’s slowing down your PSet/Point-based graphics program. Visual Basic is very nice in that it does almost all of your error checking for you, but there is a definite speed trade-off.  In other programming languages and per-pixel routines, many of these error checking steps are removed – which is one of the reasons why other languages and functions are generally faster but more dangerous to use. In the next three tutorials we will discuss alternate methods of doing graphics that cut out some of these “speed killer” steps.

V – Conclusion

Hope that all made sense to you!  Does it feel good knowing you can now program any graphics routine using nothing but VB?  Hope it does – but don’t get too comfortable yet.

To be totally honest, I hope that you completely forget that PSet and Point even exist – at least as far as per-pixel image processing is concerned – after reading the next three tutorials.  Both are extremely slow and… well, just bad programming for image processing.  VB6 is good for a lot of things, but its pixel interfacing is a total joke.

Thankfully, there are three more tutorials that will show you better, faster ways to do graphics programming – but at least you now know how to use PSet and Point if the need ever arises.

CONTINUE TO “BASIC API PIXEL ROUTINES”

*As quoted by Matt Hart, the legendary VB programmer

 

This site - and its many free downloads - are 100% funded by donations. Please consider a small contribution to fund server costs and to help me support my family. Even $1.00 helps. Thank you!

VB Graphics Programming: An Introduction

To any VB 6.0 users still out there:

In my experience, many VB programmers seem to believe that fast graphics programming is impossible in VB.  Most complain that PSet and Point (or GetPixel and SetPixel) just aren’t fast enough for their needs, and they feel like they have to switch to an alternate language for any serious graphics-related coding.

This is most definitely not the case, and I hope by the end of these tutorials to have you feeling quite differently.

Using some creativity and efficient coding, you can use VB6 to do almost anything that C++, java, DirectX or OpenGL does – and almost always as fast (and in some cases faster!).  The code for DIB sections, the ultimate end of these tutorials, is not well-documented anywhere else on the net, so I have set out to clearly explain how extremely fast graphics-related code can be used inside Visual Basic without a lot of extra work or overly-cryptic API calls.

Also, unlike other programming tutorials, this series of articles will explain not only how to do fast graphics but how fast graphics work. You will first learn about pure VB routines for graphics processing; next comes the basic API routines of GetPixel and SetPixel/V; third comes the more advanced GetBitmapBits, SetBitmapBits, and DIB sections; the last tutorial will cover additional optimizations for the real speed demons out there. By the end of this series of tutorials you will be able to effectively write any graphics application you can dream up using nothing but VB6 and the Windows API.

But before we jump into the code, it’s probably good to clarify a little bit about graphics programming in general.

General Graphics Programming Stuff

Graphics programming (hereafter referred to as GP) is, in my opinion, the most far-reaching aspect of programming. Every Windows, Linux, and Mac program that has been written in the last twenty years has utilized graphics in some way. Because of the popularity of GUI-based operating systems, every programmer has been forced to learn at least the fundamentals of GP, because any program they write must include little picture boxes, have nice buttons and toolbars, load with a pretty splash screen, etc.

Now it’s true that some programs don’t use many graphics routines – but every program you own uses some graphics.  Next time you turn on your computer, start thinking about all of the GP involved in simple, everyday routines, like clicking on your start bar, the animated folder that appears when you copy a file, that progress bar at the bottom of a window. You’ll quickly be amazed at how many GP techniques are utilized within every program you run.

As you can imagine, a lot could be said about graphics programming.  I do not intend to talk about EVERYTHING graphics related, so here’s the part of GP that this set of tutorials will cover: per-pixel image interface routines.  Basically, we’re going to discuss how to get a picture’s pixel information, how to transfer it into a data structure we can use (arrays), and how to put those pixels back onto the screen once we’re done with them. We’ll go from the easiest and slowest ways to the fastest and most advanced ways available to you, the common VB6 programmer.

What this tutorial will not cover is how to do specific image editing routines (please refer to the numerous source code examples on the internet for that), how to write graphics-based dlls or ocxs, or how to use DirectX. There are resources for each of these if you are interested; my goal is simply to outline the standard Windows API methods for per-pixel image interfacing.

Are you excited?  You should be!  On to the tutorials!

CONTINUE TO “PURE VB PIXEL ROUTINES”