Image processing – contrasting things

When we talk about image enhancement, one of the most commonly used filters involves enhancement of contrast – i.e. enhancement of the difference between objects. This is helpful in situations where there is very little contrast due to low-lighting, or to make small features stand out. Ideally contrast enhancement occurs independent of changes to colours in an image. In many cases increasing the enhancement just stretches out the histogram of intensity levels. Low contrast usually manifests itself as an image histogram which is not evenly distributed. Consider the photograph below – taken in the inside of an old workshop with low lighting (in this case only the natural lighting provided by the windows). The histogram of the image is skewed towards the lower intensities, a consequence of being underexposed. (Note that this is the “intensity” histogram of the RGB colour image, usually taken to be the green channel).


Fig 1: An underexposed image (low lighting conditions)

Many of the techniques used to enhance contrast work by manipulating histograms. There are two major categories of techniques: automatic, and manual. The automatic techniques work almost like a black-box: image goes in, processed image comes out. There are no parameters to tweak, and so they are completely automatic – and the results are sometimes good, and sometimes not so good. The manual techniques require the user to adjust parameters, so they can produce a better result, but at the expense of having to find the right set of parameters – which likely will be different for each image being processed. Consider the image above processed using four different methods. The first method, requiring no parameters is histogram equalization (HE), a process which has the effect of spreading out the most frequent intensity values in an image. The result is a vastly improved image, with pink undertones. At the other end of the spectrum is the Retinex algorithm, with three parameters. It has the effect of reducing shadows in an image, at the expense of over-highlighting other areas.


Fig 2: Histogram equalization and Retinex

One solution to this is to derive an image which is the average of both the HE and Retinex (shown in Fig.3).This has removed the pink undertones, and some of the harsh lighting of Retinex. Compare this to histogram stretching, which effectively stretches the histogram towards the higher intensity spectrum of the histogram. There are other algorithms as well, such as CLAHE (contrast-limited adaptive histogram equalization), localized HE (where each pixel is processed by applying HE to a small neighbourhood around it), or bi-histogram equalization (find the mean intensity, then apply HE separately to the sub-histogram either side). Indeed there are many algorithms for improving contrast, but the result is truly in the eyes of the beholder. Some of the four examples shown are more aesthetically pleasing to some people. Some people may not like any of them.


Fig 3: AVG(Histogram equalization, Retinex), Histogram stretching


Why image processing is an art.

There are lots of blogs that extol some piece of code that does some type of “image processing”. Classically this is some type of image enhancement – an attempt to improve the aesthetics of an image. But the problem with image processing is that there are aspects of if that are not really a science. Image processing is an art fundamentally because the quality of the outcome is often intrinsically linked to an individuals visual preferences. Some will say the operations used in image processing are inherently scientific because they are derived using mathematical formula. But so are paint colours. Paint is made from chemical substances, and deriving a particular colour is nothing more than a mathematical formula for combining different paint colours. We’re really talking about processing here, and not analysis (operations like segmentation). So what forms of processing are artistic?

  1. Anything that is termed a “filter”. The Instagram-type filters that make an ordinary photo look like a Polaroid.
  2. Anything with the word enhancement in it. This is an extremely loose term – for it literally means “an increase in quality” – what does this mean to different people? This could involve improving the contrast in an image, removing blur through sharpening, or maybe suppressing noise artifacts.

These processes are partially artistic because there is no tried-and-true method of determining whether the processing has resulted in an improvement in the quality of the image. Take an image, improve its contrast. Does it have a greater aesthetic appeal? Are the colours more vibrant? Do vibrant colours contribute to aesthetic appeal? Are the blues really blue?

Consider the photograph below. To some, the image on the left suffers from being somewhat underexposed, i.e. dark. The image in the middle is the same image processed using a filter called Retinex. Retinex helps remove unfavourable illumination conditions – the result is not perfect, however the filter can help recover detail from an image in which it is enveloped in darkness. Whilst a good portion of the image has been “lightened”, the overcast sky has darkened through the process. There is no exact science for “automagically” making an image have greater aesthetic appeal. The art of image processing often requires tweaking settings, and adjusting the image until it appears to have improved visually. In the final image of the sequence below, the original and Retinex processed images are used to create a composite by retaining only the maximum value at each pixel location. The result is a brighter, more visually appealing image.

Contrast enhancement using the Retinex filter: before (left), after (right)

Contrast enhancement: (a) original, (b) Retinex-processed, (c) MAXimum of (a) and (b)