Image processing in Julia (i)

Some may think that C is a good language for image processing, and it is, certainly from the perspective of speed. However it suffers somewhat from usability issues with arrays (at least from my viewpoint). Partially this is attributed to forcing the user to use dynamic arrays for large 2D arrays to store images, but also because I do not want to deal with arrays that begin with an index of 0. Python on the other hand, just suffers from being slow, and frankly I don’t want to spend the energy to completely vectorize things.

COLUMN VS. ROW MAJOR

Most people use to languages such as C deal with 2D arrays in a row-major, which means arrays are stored as rows. Julia however harks back to Fortran (and MATLAB) and uses column-major arrays. So an array, z,  of the form:

1 2 3 4
5 6 7 8
9 10 11 12

would be stored as 1, 5, 9, 2, 6, 7, 3, 7, 11, 4, 8, 12. Therefore accessing z[8] would return the value 7. This is a little tricky, but it’s just about how things are stored, and shouldn’t interfere two much with processing an array. The only caveat occurs when “flattening” the array, as it is flattened by column, instead of by row. It’s really all about memory layout.

1D VS. 2D

So the way to create arrays in Julia is quite interesting. Doing the following:

z = [1 2 3 4 5 6 7 8]

Creates a 2D array, with 1 row and 8 columns. Remember Julia is column-major. To create what is essentially a vector, one writes:

z = [1, 2, 3, 4, 5, 6, 7, 8]

Trying to access the third element in both cases will work with z[3], although z[3,1] works with the vector, and z[1,3] works with the 2D array. A 3×3 array can be created in the following manner:

z = [1 2 3;4 5 6;7 8 9]

There are a lot of built-in functions which can be used on arrays, for example vec(x) which converts the array x, whatever shape into a vector.

ARRAYS begin with 1

As a person who does some image processing, dealing with arrays that begin with an index of 1 is  perfectly fine. Images are basically 2D arrays, and when processing neighborhoods in the region it is way easier to deal with 1..n than it is 0..n-1, from an algorithmic viewpoint.

ARRAY SLICING IS *SO* CONVENIENT

C does not allow for array slicing, which is an annoyance. So processing an entire image by extracting a 5×5 neighbourhood in C requires a piece of code of the form (where z=2 represents the boundary pixels to deal with the 5×5 neighbourhood):

for (i=z; i<dx-z; i=i+1)
  for (j=z; j<dy-z; j=j+1)
    for (x=-2; x<=2; x=x+1)
      for (y=-2; y<=2; y=y+1)
        block[x+2][y+2] = image[i+x][j+y]

Sure, it’s fast, but it really is inconvenient to have to write code like this. The same piece of code in Julia is:

for i=z+1:dx-z, j=z+1:dy-z
    block = img[i-2:i+2,j-2:j+2]
end

The syntax in Julia is way easier to code.

IMAGE I/O

One of the inconveniences of any programming language used for image processing is reading in specific file formats… in fact trying to read TIFF or JPEG images is often a bugbear in any language. One alternative is to store images as text files, containing the raw 8-bit values (for grayscale anyway). Julia provides an easy method of reading in the image, of the form:

imgIn = readdlm(fname,UInt8::Type)

This function readdlm reads the image in based on the the rows and columns of the image inside the file given by fname. The argument UInt8::Type specifies the format type to be stored in the array imgIn. Note that just using Int can lead to the 8-bit image being stored as an array of 32/64-bit integers, which can cause a system crash when processing large images.

IT’S ALL ABOUT THE DATA

One of the caveats of dealing with any image is how to store the data. In an ideal world, an 8-bit images would be stored as UInt8 in Julia, giving values from 0 to 255. This works well if any calculations only involve that set of discrete values, however if a function is responsible for applying a kernel with negative coefficients, using UInt8 is no longer feasible, and neither is Int8. So it is then necessary to take a step up and store/process images using Int16, which is overkill from a storage point-of-view, but saves a lot of hassle when processing images.

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s