Why floating-point numbers aren’t really real

There are two types of numbers in programming (let’s ignore complex numbers) – floating-point and integers. Integers are whole numbers, and are therefore always precise. Therefore assigning 4 to an integer variable, no matter the language, will always result in the number 4 being precisely stored. Floating-point numbers are any numbers with a decimal point in them. 3.14159 is a floating-point number. So what is a real number? A real number is the “analogue” version of a floating-point number – they have infinite precision, and are continuous, and “nonlossy”. This means that π is represented as:

3.1415926535897932384626433832795028841971693993751058209…

There are no bounds. Now the difference between reals and floating-point numbers is that the latter have limited precision – it really depends on the abilities of the computer. Floating-point numbers are essentially approximations of real numbers. Some languages, such as Fortran, use the term real to denote floating-point numbers – but they aren’t real. Also, because they are approximations, they are susceptible to errors. Here’s an example in Python: the value of the expression 0.1+0.1+0.1-0.3 is 5.551115123125783e-17. Do this on paper, and you will get zero. That’s because computers don’t store all floating-point numbers precisely.

Floating-point numbers are represented in computers as binary (base 2) fractions. Therefore to understand why floating-point numbers are approximations, it’s easiest to do a conversion from a decimal fraction to a binary fraction. This can be done using the following steps:

  1. Multiply the number by two
  2. Take decimal as the digit
  3. Take the fraction as the starting point for the next step
  4. Repeat until you either get to 0 or a periodic number
  5. Read the number starting from the top – the first result is the first digit after the comma

So converting 0.125 to binary gives:

0.125 × 2 = 0.25
0.25 × 2 = 0.5
0.5 × 2 = 1.0

So the binary number is .001. Now 0.1 *seems* like it would convert easily. But here’s the problem… let’s convert it:

0.1 × 2 = 0.2
0.2 × 2 = 0.4
0.4 × 2 = 0.8
0.8 × 2 = 1.6
0.6 × 2 = 1.2
0.2 × 2 = 0.4
0.4 × 2 = 0.8
0.8 × 2 = 1.6
0.6 × 2 = 1.2

So the binary equivalent is .00011(0011) . The 0011 will occur ad infinitum. Which means 0.1 won’t be stored exactly. Converting the number back to a decimal fraction will derive a number pretty close to, but not exactly 0.1. Now many systems will print a decimal approximation of the true decimal value of the binary approximation stored by the machine. (whew!). So you might actually see 0.1 printed. Some languages, such as Python will allow you to see what is happening behind the scenes. This is achieved using the Decimal module.

> from decimal import Decimal
> Decimal(0.1)
> Decimal('0.1000000000000000055511151231257827021181583404541015625')

This is known as a representational error – certain decimal fractions can’t be represented exactly as binary fractions.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s