Decorrelation Stretch Images

article by Ben Lincoln

Decorrelation Stretch[ing] ("DCS") is an image-processing algorithm which originated in the world of satellite and aerial mapping. Its intended use is to highlight differences in an image that are too subtle for a human to see.

DCS is one of the original reasons that I decided to use DaVinci as the foundation for The Mirror's Surface Breaks. There are some striking example images available on various websites, and DaVinci includes support for performing a DCS operation on images.

However, viewing DCS images has an odd effect on me. If I haven't seen any for at least a couple of months, my reaction is that they look amazing. However, my eyes seem to tire quickly of them, and so after a few days I don't want to see any more for awhile. I suppose you could say that - at least as far as my preferences - DCS is sort of the Winterkälte of the image-processing world. In any case, the functionality is there, and it is definitely useful in certain cases.

DCS is a much more complicated process than the others described on this site. There is a bit of a gaping abyss between the "easy, high-level" and "formal, accurate" descriptions of how it works. In order of ascending complexity, here are four of my attempts to explain it, and then a fifth from Jon Harman's website:

1. Magic!
2. Math!
3. It makes colours that are similar in the original image dissimilar in the processed image.
4. If each of the sets of numbers in the original image that define a colour (e.g. red, green, and blue values) is treated as a coordinate in space, decorrelation stretching moves those points in space farther apart, so that it becomes easier to see a difference between them.
5. "[It applies a] Karhunen-Loeve transform to the colors of the image. This diagonalizes the covariance (or optionally the correlation) matrix of the colors. Next the contrast for each color is stretched to equalize the color variances. At this point the colors are uncorrelated and fill the colorspace. Finally the inverse transform is used to map the colors back to an approximation of the original."[1]

If you don't know how to "calculate the covariance matrix and eigenvectors of it"[2], don't bother trying to work your way through the actual algorithm.

The aforementioned Jon Harman has found extensive use for it in examining images of petroglyphs and pictographs, and so here is an example in which I've taken one of my own photos of petroglyphs and processed it using DCS. I've also included an image that shows the histograms of the images, and a scatterplot of a sample of pixels in which the R, G, and B values are used as X, Y, and Z coordinates to illustrate the effect that the DCS process is having. As you can see, before the operation, the colours are spaced closely together in the scatterplot, and afterwards, they are very spread out. In other words, they were "highly correlated" before, and "decorrelated" afterwards.

DCS Examples 1
 Original Image
 After DCS In RGB Colourspace
 After DCS In YUV Colourspace
 After DCS In HSY Colourspace
 Scatterplot and Histogram Comparison

When I was originally researching the technique, I read on Dr. Harman's site that it gives different results in the YUV colourspace, so I gave TMSB the ability to perform it in any colourspace that that software supports. The results seem to be the most useful in the YUV colourspace, especially when larger images are being processed. YUV gives a brighter, higher-contrast appearance, probably because it is biased towards that aspect of human visual perception. The colours are not always quite as well-spaced as when using the RGB colourspace, so the tradeoff may make one or the other preferable for certain types of image. The results in any hue-based colourspace (HSL, HSV, HSY, IHSL, etc.) are quite similar and usually not outstanding, but every once in awhile will be interesting.

Running DCS

Running DCS ("RDCS") is a modified DCS in which the image is divided into sections. Each section has a DCS operation applied separately, and then the sections are blended back together into a single output image. This can dramatically increase the contrast of an image, at the expense of making it extremely noisy.

DCS Examples 2
 NIR-G-B [3C]
 NIR-G-UVA [3C]
 NIR-B-UVA [3C]
 (NIR-G-B) - DCS: YUV
 (NIR-B-UVA) - DCS: YUV
 (NIR-G-UVA) - RDCS: YUV, 25%

This is the paper from the charred paper text recovery example in the Uses of Multispectral Photography article. You can see that there are several lines which are barely visible even in the multispectral composites, but which stand out clearly when processed with DCS. The last variation was processed with a running DCS in which each segment was 25% of the width and height of the full image.

Date Shot: 2011-01-01
Camera Body: Nikon D70 (Modified)
Lens: Nikon Micro-Nikkor 105mm f/4
Filters: Standard Set
Date Processed: 2011-01-01
Version: 1.0

Here are some examples of DCS that shows it where it is usually strongest - processing satellite imagery. Many sections of the populated areas of Nazca, Peru become highly visible in the DCS-enhanced versions of the same images.

DCS Examples 3
 SRTM-Green-Blue [3C]
 IR3-IR1-Green [3C]
 IR1-Red-Green [3C]
 IR1-Green-Blue [3C]
 (SRTM-Green-Blue) - DCS: YUV
 (IR3-IR1-Green) - DCS: YUV
 (IR1-Red-Green) - RDCS: YUV, 25%
 (IR1-Green-Blue) - DCS: YUV
 (IR1-Green-Blue) - RDCS: HSL, 25%

Before and after DCS comparisons of several variations of Nazca, Peru from space. Note: in case you are looking for them, the Nazca Lines are far too small to show up on satellite imagery of this resolution.

Footnotes
 1 "Using Decorrelation Stretch to Enhance Rock Art Images", Jon Harman, Ph.D. 2 Noel Gorelick, in the comments to the dcs() function in DaVinci.