Sunday, April 11, 2010

Cast from float to double error

I was surprised to find out that this line of code fails:

Assert.AreEqual(0.1d, (double)0.1f);

(double)0.1f is actually the same as ... drumbeat ... 0.100000001490116

Why does it happen?
Because float and double do NOT represent fractions precisely.
Rounding is always going on.

How to deal with this?
There are several options:
1) Round double after converting float number into it.
For example:

float f = 0.1f;
double d = Math.Round(f, 8);
would assign 0.1d to d.

2) Don't use float/Single and use double only.

double d = 0.1;
would assign exactly 0.1d to d.

3) Use decimal or bigdecimal if you are dealing with money.

decimal m = 0.1m

Hope it helps.

No comments:

Followers

About Me

My photo
Email me: blog@postjobfree.com