Strictly, it's not because it's floating point, it's because it's binary floating point.

When decimal 1.51 is converted to a binary fraction the result is:

  1.1000_0010_1000_1111_0101_1100_0010_1000_1111_0101_1100_0010_1000_1111_0101_1100_0010_1000_1111_0101...

As you say, most decimal fractions are like this... so before any errors can be introduced by rounding and what not, most decimal fractions have a built-in "representation" error.

Most of the time you won't see the "representation" error, because conversion back to decimal rounds it off. This can lead to bafflement when two values look the same when printed out, but fail $x == $y. Addition and subtraction are the more difficult floating point operations, so you're more likely to see the problems there. Consider:

  print 1.09 - 1, "\n" ;
  print "0.84 - 0.34 == ", 0.84 - 0.34, 
       ( 0.84 - 0.34 == 0.5 ? " ==" : " but !=" ), " 0.5\n" ;
[download]

  0.0900000000000001
  0.84 - 0.34 == 0.5 but != 0.5