The binary floating point representation produces this asymmetry. It's not only what you see as decimal fraction after the point which determines how the binary mantissa is calculated, the digits before also get into the mantissa.

UPDATE:

From 23.2 to 0.2 you already skipped 2 decimal steps in exponentiation.

These precision errors resulting from base transformations are periodic, if you just shift the exponent long enough you get the same mantissa at the lower end, such that the difference is again 0 at the "tail".

EXAMPLE:

0.2 = 1/5 can't be represented as a binary fraction without infinite repetition, the same way you can hardly represent 1/7 in a decimal fraction. (NOTE: 10 and 7 are coprime, like 2 and 5 are!)

Now try to calculate (10+1/7)-1/7 to a decimal base but with a finite mantissa!

You'll get

10 + 1/7=       1.01428571428571     * 10^1 
     1/7=         1.42857142857143   * 10^-1
[download]

Clearly, the lower ends of the mantissa to the base ten are not symmetric ...

... but to the base 7 both numbers would be highly symmetric again, no matter what you add before the point. Hence no error!

 13.1 - 0.1  (base 7)

Hope this calculation helps to visualize what happens with "obvious calculations" when switching bases!

PS: Further reading: Humans have too many fingers!