I like simple code

One of the biggest problems with computer science in academia is the fact that most instructors don’t get much time to code anything. The code I write is often quite simplistic… a library here, a function there. Few academics write apps, or large programs (> 10,000 LOC) – there just isn’t the time. I am in the midst of writing a small image processing library for Julia. but it won’t compete with JuliaImages from a code perspective. I have looked at some of the code there, and honestly I can’t make heads nor tails of it – it seems well written of course, but it is extremely complex from my perspective. That, and I don’t like OO, never have. So the library I am cobbling together (it is a craft isn’t it?), will be written in basic Julia. No OO, no functional, just what is essentially pure procedural. But it will be possible to figure out what is happening in the code, which I think is important. There is nothing worse than a piece of code written by someone else, that performs its task admirably, but looking at it sends you into fits of convulsion because it is indecipherable. And this seems to happen a lot… it’s almost as bad as trying to figure out some algorithm in a paper, where the author quite clearly has no ability to actually explain how the algorithm works.

Consider this simple function to calculate a threshold (turning a grayscale image into a binary one) using the classic Otsu method:

# Function to perform image thresholding using Otsu's algorithm
#
# Type: clustering threshold algorithm
#
# Parameters:
# (in) img : gray-scale image
# (in) nBin : number of bins in the histogram (default=256)
# (out) tvalue : threshold value
#
# Example: t = otsu(img)
#
# Ref(s):
# Otsu, N., "Threshold selection using grey level histograms", IEEE
# Transactions on Systems, Man and Cybernetics, Vol.9(1), pp.62-66 (1979)
#
# Notes: Variables coincide with the original paper

function otsu(img, nBin=256)

   dx,dy = size(img)
   imgSize = dx * dy
   imgN = copy(img)

   # Get the image histogram
   hst = getIMGhist(img)

   # Normalize the histogram 0->1 (Eq.1)
   hstP = hst / imgSize

   # Total mean level of the original image (Eq.8)
   mu_T = 0.0
   for i = 1:nBin
      mu_T = mu_T + i * hstP[i]
   end

   sigma = zeros(Float32, nBin)

   for k = 1:nBin
      # Zero-order cumulative moments up to the t-th level
      w_k = sum(hstP[1:k]) # (Eq.6)

      # First-order cumulative moments up to the t-th level
      mu_k = 0.0
      for i = 1:k
         mu_k = mu_k + i * hstP[i] # (Eq.7)
      end

      # Variance of the class separability (Eq.18)
      if (w_k != 0.0) && (w_k != 1.0)
         sigma[k] = ((mu_T * w_k - mu_k)^2.0) / (w_k * (1.0-w_k))
      else
         sigma[k] = 0.0
      end
   end

   # Choose the threshold for which the variance of class 
   # separability is at its maximum. Return the index of 
   # this value (Eq.19)
   tvalue = indmax(sigma)

   return tvalue
end

It is by no means perfect, but it works, and to the novice programmer it should be easy to read. There is enough documentation to look at the code, and see where each part of the algorithm exists in the original publication that described the algorithm. This makes it easy to modify, or even to check for potential faults in the code.

It would be really cool to have the time to build an app, but the reality is, I don’t have the development skills to do so. Few instructors do. If I were writing apps as a profession it would be a different thing altogether, but I don’t. In fact it’s hard to write anything meaningful in snippets of code. So remember that when you are taking a class on software development, chances are the instructor has never designed (or implemented) a large scale piece of software (and by large scale, I mean 25,000-100,000 LOC, which is trivial in terms of todays software industry).

 

 

Advertisements

Creating photographic filters (i) – desaturation and saturation

How does one create photographic filter effects such as those found in Instagram? Are they easy to replicate? In Photoshop they likely are – there are numerous tutorials on the net about how to make a filter effect – but it requires each image to be processed separately, so make take some of the “instant coolness” away from actually processing the image. So what are these filters? Usually they are a combination of effects – the use of some curve profiles, different blending modes, and the addition of colour hues. Basically, if you can create an effect in Photoshop, converting that to an automated process. Note that there is no single way to produce an effect. There are probably thousands of ways of giving a photograph a retro or vintage effect.

What are some of the tricks to adding effects?

Desaturation (i.e. a muted look)

Desaturation is the process of reducing saturation in an image. Saturation in the context of colour images is the “colorfulness of an area judged in proportion to its brightness” – saturation can be viewed as how far a colour differs from pure white. Effectively desaturation reduces this colourfulness. Removing colour from images may be more effective than enhancing it. Having a more colourful images lends itself towards a more “cheerful” interpretation, whereas reduced colour is more of a mood enhancer. Photographs that benefit more from desaturation tend to be those that have darker undertones. This can include earthy outdoor photos, cityscapes, and scenes that contain fog, or rain.

Saturation is one component in colour spaces such as HSV and HSB, and so it is relatively easy to modify saturation. It basically involves converted an image from RGB to HSV colour space, and then modifying the “S” component by multiplying it by some value in the range 0 to 1.0. Reducing saturation to 0% effectively reduces the image to grayscale.

Here is an example of desaturation. The image below was taken in August in Oslo, Norway. It is an image with a substantial amount of “colourfulness”, both in the vegetation, and the wood on the building.

Now if the image is converted to HSB, and the saturation component of the image is multiplied by 0.6 (60% of saturation), we get an image with much muted tones.

saturation (i.e. a vibrant look)

Obviously saturation is the opposite to desaturation. This is usually applied to images that are characterized as “cheerful”, and may contain a number of vibrant colours. Saturation is often overdone, producing colours which were unlikely to appear in real life. Consider the photograph of two snow-blower railcars in Norway. The scene might be a good candidate for saturation, providing a more vibrant colour to the railcars (even though in real-life they are faded).

Again, converting to HSB colour space first, then multiplying the saturation component by 1.3 (30% increase), results in the following saturated image:

The sky becomes bluer, the the grass greener, and the railcars have had their faded paintwork restored.

 

 

 

Image mean filtering (ii) – in Fortran

Image processing in Fortran is often overlooked, for whatever reason. The lack of libraries probably doesn’t help – there are plenty of mathematical libraries, but image processing has moved off to other languages.

The reality is that arrays in Fortran is very capable, especially from the perspective of arrays – it allows whole array operations, e.g. assignment of one array to another, and array slicing.  Like Julia, Fortran’s for loops are easy to decipher, and as they are terminated with end do, there is little chance of missing something. Reading in the text image is easy, and only uses one for loop, with the read() function capable of reading an entire row into the array (through short-hand syntax).

program meanfilter
   implicit none
   integer :: dx, dy
   integer, dimension(2144,6640) :: image, imageN
   integer, dimension(5,5) :: blk
   integer :: i, j, w, sumB

   dx = 2144
   dy = 6640
   w = 2
   open(unit=20,file='pano.txt',status='old',action='read')
   do i = 1,dx
      read(20,*) image(i,1:dy)
   end do
   close(20)

   imageN = image

   do i = w+1,dx-w
      do j = w+1,dy-w
         blk = image(i-w:i+w,j-w:j+w)
         sumB = sum(blk)
         imageN(i,j) = int(sumB/25.0)
      end do
   end do

   open(unit=20,file='panoF.txt',status='new',action='write')
   do i = 1,dx
      write(20,*) imageN(i,:)
   end do
   close(20)

end program meanfilter

LEGEND: image input, mean filtering, filtered image output

The piece of code performing the mean filtering is no different than that of Julia. Runtime for this piece of code is 2.45 seconds, which you have to admit is pretty fast for processing 14 million odd pixels. If we modify the code so that the mean filtering is constrained inside a subroutine, then incredibly the runtime reduces to 1.61 seconds. Here is the code for that version:

program meanfilter
   implicit none
   integer :: dx, dy, i, w
   integer, dimension(2144,6640) :: image, imageN

   dx = 2144
   dy = 6640
   w = 2
   open(unit=20,file='pano.txt',status='old',action='read')
   do i = 1,dx
      read(20,*) image(i,1:dy)
   end do
   close(20)

   call meanFilterI(image,imageN,dx,dy)

   open(unit=20,file='panoF2.txt',status='new',action='write')
   do i = 1,dx
      write(20,*) imageN(i,:)
   end do
   close(20)

end program meanfilter

subroutine meanFilterI(img,imgF,nr,nc)
   integer, intent(in), dimension(nr,nc) :: img
   integer, intent(out), dimension(nr,nc) :: imgF
   integer, intent(in) :: nr, nc

   integer, dimension(5,5) :: blk
   integer :: i, j, w, sumB

   do i = w+1,nr-w
      do j = w+1,nc-w
         blk = img(i-w:i+w,j-w:j+w)
         sumB = sum(blk)
         imgF(i,j) = int(sumB/25.0)
      end do
   end do

end subroutine meanFilterI

 

 

 

 

 

 

Image mean filtering (i) – in Python

A long while back I tested some code to apply a mean filter to a grayscale image written in Julia (Testing Julia for speed (iii)), and compared against three other languages: C, Fortran, and Python. This is a continuation of those posts and looks at the code in the other languages. First on the list is Python.

The code below has two parts, the main program, and the function meanFilter(). One of the problems with Python is that even though it is a simple language from the perspective of language structure, it suffers from some usability issues. First is the fact that the act of reading in the text image involves a lot of somewhat cryptic code. Natively to read a series of integers from a file is not exactly trivial. It reads the image in as lines, and then splits the lines up into numbers. After it is input, the image data is converted into a numpy array. Numpy is of course the Python package incorporating n-dimensional array objects. Of course if we used numpy to its entirety, we could just use it to read in the text image:

image = numpy.genfromtxt('pano.txt', dtype=numpy.int32)

This is obviously *way* simpler. It is challenging to use Python effectively for image processing without the use of numpy.

import numpy

def meanFilter(im):
    img = im
    w = 2

    for i in range(2,im.shape[0]-2):
        for j in range(2,im.shape[1]-2):
            block = im[i-w:i+w+1, j-w:j+w+1]
            m = numpy.mean(block,dtype=numpy.float32)
            img[i][j] = int(m)
    return img

def main():
    with open('pano.txt','r') as f:
        image = []
        for line in f:
            image.append([int(x) for x in line.split()])
    f.close()
    image = numpy.asarray(image)

    imgN = meanFilter(image)
 
    fw = open('panoP.txt','w')

    for i in range(imgN.shape[0]):
        for j in range(imgN.shape[1]):
            fw.write('%s ' % imgN[i,j])
        fw.write('\n')

    fw.close()

The function meanFilter() processes every pixel in the image (apart from the image borders). Python uses the range function to determine the list of loop iterators for the for loops. Image slicing is then used to extract the 5×5 block around each pixel, and the mean is calculated using the  numpy mean() function. The result is then converted to an integer, and assigned to the filtered image. After the image has been processed, the filtered image is output to a text file.

Overall, the Python algorithm works, although it is slow. Since I ran this code previously, the run-time has improved (faster machine likely). The image to be processed (8-bit, 2144×6640 pixels), took 178 seconds. There are obviously more efficient ways to write this code in Python (e.g. vectorization), but how cryptic does the code have to become? for the novice programmer, who wants to do some basic image processing, Python is *okay*, but I still think it lacks from a usability viewpoint.

 

The worst usability = washing machines

Some of the worst usability issues relating to appliances are those associated with washing machines. It’s almost like the people who design washing machine controls have *no clue*. Some sort of design engineers who likely have never used a washing machine in their lives? Consider the interface on this washing machine:

Now the dial on the left is clearly for spin-cycle, and the numbers around the dial are likely revolutions-per-minute. It’s hard for a user to really decipher what 700 rpm means though, except that the higher the number, the faster it goes. The dial on the right, which allows a cycle to be chosen is full of the usual cryptic symbols. Many European brands of washing machines use pictograms, but unfortunately they don’t conform to any standard. What do the symbols mean?

Below I have marked the symbols as best as I can decipher them, with those in red being ones that I’m not really sure about. Even some of the more identifiable symbols are questionable. For example the symbol for cotton, looks like a tree. The symbol for synthetics looks like a bent paperclip.

This is part of the nightmare of laundry symbols in general – which are *somwhat*, but not quite universal. What is worse of course is the fact that all these settings aren’t needed. You can wash most clothes in cold water (which isn’t really tap cold, and most machines will add some warm water to make sure the water isn’t ice cold). The added bonus is that cold water will actually make things last longer – hot water can weaken fibers, fade colors, destroy elastic and contribute to shrinkage.

So interfaces on washing machines could be way simpler, because I imagine most settings aren’t ever used. I consistently use about two wash cycles on my machine. Here is an interface from a Speed Queen commercial washing machine (used in coin-operated laundries). This interface basically requires the user to push four buttons, and makes using the machine easy. Does the user really need to choose a spin speed? Likely not.

Computer humour – “real” programmers

From “Real Programmers Don’t Use Pascal, Ed Post (1983). A humorous look at what a “real” programmer was in the early 1980s… Post considers Pascal programmers to be Quiche eaters [1].

  • Real Programmers aren’t afraid to use GOTO’s.
  • Real Programmers can write five-page-long DO loops without getting confused.
  • Real Programmers like Arithmetic IF statements – they make the code more interesting.
  • Real Programmers write self-modifying code, especially if they can save 20 nanoseconds in the middle of a tight loop.
  • Real Programmers don’t need comments – the code is obvious.

An interesting and fun read – I wonder what we would consider a “real” programmer now? C? Java?

[1] Ed Post, “Real Programmers Don’t Use Pascal”

Does flash photography affect museum artifacts?

On a trip to the Louvre in Paris (10 years ago now), I noticed that the information guide stated “flash photography is strongly discouraged throughout the galleries”. The only place I really saw this enforced was in front of the Mona Lisa. Not a problem you say, everyone will abide by this. Well, not so it appears. I would imagine a good proportion of visitors have some form of digital camera, usually of the “point-and-shoot” (PS) type where the use of flash is automatic if light levels are low. There are of course two reasons for prohibiting the use of flash photography. One is that it disturbs other patrons. The second is that the flash has a direct effect, causing accelerated fading in artifacts such paintings and textiles. So what is the scientific basis for these restrictions? Well very little has actually been written about the effect of photographic flashes on exhibits. In 1994 Evans[1] wrote a small 3-page note discussing whether exhibits can be harmed by photographic flash, but there seems to be very little scientific data to back up claims that flashes cause accelerated fading. The earliest experiment was performed in 1970 using multiple flash (25,000) exposures [2]. Evans has written another article [3], which looks at the quantitative evidence behind banning flash photography in museums.

“Photographic flashes can damage art”. This is sort of a very broad statement. Strictly speaking, I would imagine the damaging affects of  1000 sweaty hands touching the Venus de Milo would greatly outweigh 1000 photographic flashes. It is doubtful that flash photography does any real damage. Should it be used? Unless you are using a professional lighting setup, you can probably achieve better pictures by not using a flash. Frankly if you are taking photographs of paintings in an art gallery you might be better off buying a book on the artist at the gallery shop. That, and flashes in enclosed spaces are annoying. Here is an example of a photo taken in the National Gallery of Norway, without the use of a flash. Actually, the biggest problem taking photographs indoors is possibly too many lights, and reflections off glass.

[1] Evans, M.H., “Photography: Can gallery exhibits be harmed by visitors using photographic flash?,” Museum Management and Curatorship, vol. 13, pp. 104-106, 1994.

[2] Hanlan, J.F.,  “The effect of electronic photographic lamps on the materials of works of art.,” Museum News, vol. 48, pp. 33, 1970.

[3] Evans, M.H., “Amateur photographers in art galleries: Assessing the harm done by flash photography”.

 

The real world of software development is not in academia

Back in the late 1980s when I went to university, computing in the workplace was largely run by people who had grown up in the trenches. Through one means or another they had learned to program on the job, or maybe had a degree in something like math and somehow had transitioned into computing. Very few people likely had degrees in computer science, and software engineering as a topic was just making inroads. Courses in university were focused largely on the programming aspects of computer science, or theory, and courses on AI, or HCI were available based on the particular interests of the faculty, they were by no means available at every institution.

Now we have forsaken teaching good programming skills for blasting students with a plethora of information on designing software, or far too much theory. Sure, all these things have their place, but in moderation. How many theory courses does one need? How many design documents do students have to write? Theory courses are interesting for individuals continuing on in academia, but honestly, few software professionals will ever care about NP-completeness. More should care about designing good user interfaces, but it seems sometimes that doesn’t seem to matter.

We don’t teach much in the way of problem solving skills, leveraging the ability to “see the larger picture”, and provide solutions. We don’t really prepare students for the world of legacy systems out there, nor the world of “big” software (and people wonder the F-35 fighter with *only* 8 million lines of code doesn’t work properly). This is largely because we can’t. Very few professors have little if any experience in the real world of software development. I teach a course on legacy programming, but  I’ve never been involved in a large scale software re-engineering project (nor likely ever will). What I teach is language-related issues, and “theoretical” work-arounds. Providing the foundation – the rest you have to figure out for yourself. Thankfully many universities have cooperative programs for students to get real-world experiences.

What has changed since the 1980s? Lots in the real-world, less in academia. We’re still teaching the same courses on algorithms, the same courses on C. They are interspersed with courses on software engineering, hardware, and boutique subjects such as AI. As I’ve mentioned before, computer science should be a craft-based discipline. Traditional methods of teaching don’t really work, and should be replaced with , experiential-based learning with knowledge interspersed in the form of burst-mode topic modules provided by experts.

Making a colour-splash effect filter

How to build a colour-splash filter? It basically involves using a spot image representing the colours to be retained.

A 128×128 region taken from the photograph above has RGB values in the following ranges: R:172-224, G:105-195, and B:150-225. This information can now be used to find all pixels within these constraints.

Here is the result of the algorithm

Choosing a more diverse spot image will produce a better outcome.

Here is the result:

When done in-camera, I imagine a point is chosen and then extrapolated to contain a broader set of colours. For example, a region with a radius of 5 around a point (253,75,13) would extrapolate to (248-258, 70-80, 8-18).

 

Reducing technology [complexity] for a simpler life

When I reflect on the 30 years I have spent programming, I realize that nothing has immeasurably changed in computer science in that time. Computers have migrated from desktops to everyday items, and chips have continued to shrink, but the way we view technology, and how we program it are fundamentally the same. Every week some new technology is released to “make our lives better”, in some microscopic way. The reality is, we could likely do without this much of this technology. It just adds additional layers to our lives, in an age when we strive to make life “easier”. What happens is that we just add layers of distraction.

To be honest, the only part of technology I enjoy are digital cameras, and the ability to obtain copious amounts of information on the net (oh, and global online shopping). The rest I could do without. True, I like being able to send text messages, and using the odd app, and I like the ability to blog,  but I don’t need AI in my life. I would be happy raising heirloom pigs on a small farm somewhere, carving spoons, and building things. The same could be said about software. Instead of continuously making software more complex and adding things such as AI, we should be simplifying it. Does an app that takes photos really need 101 different type of filters? How about natural photographs, or heaven forbid having the human actually improve their skills? I know, it’s a stretch. Look complex software isn’t just bad for humans, it’s bad for technology itself. More complex software is more likely to fail. That’s because if we shove 8 million lines of code into a jet fighter, that’s 8 million points of potential failure – and jet fighters are the least of our problems.

For years we had cars without much electronics in them – and they worked. Now we have cars with 100 million lines of code in them, and we are beginning to understand that may not be optimal. Largely because it’s impossible to test these cars thoroughly. In 2015, Toyota recalled 625,000 cars for a software glitch that caused the hybrid system to “shut off while the car was being driven”. Do we need such complex software? There is a Volvo ad that shows how a car stops abruptly when it senses a child crossing a road, but the distracted driver doesn’t. What happens if the software fails? Is the driver culpable, or the car company? We could just make life simpler and have the humans drive better.

We have come to the point where we seem to be adding complexity for the sake of adding complexity. It’s almost like we can’t stop ourselves. Simpler, leaner, easier to fix? Maybe we need to rethink the way we write software.