Test, test, and test again

Software testing is somewhat of a primitive notion. There are situations where you can create a program, and test it quite thoroughly – or at least to the point where you can conclusively say it is a robust algorithm. Sorting numbers is a good example. A simple sorting algorithm such as Insertion sort can be coded up in a language, and tested on an assortment of listed of unsorted numbers. Does it work for 1000 numbers? Does it work for 100,000 numbers? What about one billion? If it works for varied test cases, then it will likely work for all data sets it is given. (It may not work efficiently, but that is another matter altogether).

But what happens if we design an algorithm to do something like solve Sudoku by taking a picture of it. Now it’s not so easy. We may find that the algorithm works well for simple, clear images – but not so well for images of Sudoku that are acquired in poor lighting conditions, or which contain weird fonts. How many images with differing characteristics are needed to properly test such as algorithm? What about a program to automatically read and interpret street signs from a moving vehicle? In comparison, the Sudoku seems easy, partially because it is a printed as a black-and-white image. Street signs are more challenging because the images being analyzed are affected by some many different factors – size, shape, colour, background, motion etc. Consider the STOP sign in the photograph below:

This is a challenging image to extract the sign information from – not impossible, but demanding nonetheless. Now consider all the variations possible, and you can see how onerous testing such an algorithm would be. Mother nature does not always provide for ideal conditions with respect to lighting or background, and signs can be askew, different colours (due to hue, or lighting conditions), or there may be multiple signs? Testing a system like this involves driving around on different days in different places seeing if the system will interpret signs in real-time.

There is no alternative to testing. The more complex an algorithm is, or the task to be performed – the more complex and large-cale testing will be. There is no way around it.

good software testing = robust software = quality software


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s