There are lots of crime shows on television, and some of them, CSI, Crossing Lines, Law and Order, and Castle use algorithms to find clues in images, usually those acquired from CCTV cameras. According to a survey by Leger* for Axis Communications, 68% of Canadians said they watch these crime dramas. 71% of Canadians think recorded surveillance footage can be enhanced in a lab using software. However here’s the debunk, the majority of surveillance cameras sold worldwide today remain analog, which is why security video often shown on the evening news is grainy and of poor quality, making identification difficult.
TV has a magical way of taking a poor quality image and improving it in many ways. Is the image too small? Then it might be possible to “zoom-in” or enlarge it. Image too blurry? It might be possible to improve the acuity by sharpening it. There’s something happening out of view of the camera, but there is person in the camera looking at the scene – we could probably extract an image from a reflection in the cornea? Are any of these at all possible?
The short answer is no. But let us delve into why.
The basic pretext is that these images have too little or missing data, and it is possible to use “intelligent” algorithms will fill in the missing pieces. Problem is that you can’t recreate what wasn’t there in the first place. Here’s a simple example. When something in an image is too small, the natural tendency is to zoom in. When they do this on television they are effectively “enlarging” that information. Consider enlarging a 2×2 image into a 4×4 image:
But the reality is that there are 12 pixels in the new image with no information – the algorithms have to somehow predict what values these “empty” spaces have. Simple algorithms do this by “averaging” the pixels around it. Below are examples of two simple algorithms applied to the task of enlarging a 4×4 image into a 200×200 image. A bilinear algorithm uses four surrounding pixels, the bicubic, 16 to calculate their averages. The result is less than optimal. It is enlarged, and offers a kind-of-approximation, but there is no way to make it crisper – or indeed to add information that is not there.