Sunday, October 4, 2009

[Q&D] Fix Decimal Module Operator

This is a Q&D solution for some JavaScript precision problem when we use the module operator with floating point numbers.

The Problem

If we are dealing with the module operator we could have this case:
alert(
8.25 % .05
);
// 0.049999999999999545

We expected 0 but we have a floating point inconsistency instead. This is a well known problem, and not only with JavaScript, truly annoying when we have to show run-time computed money/trends operations.

The Solution

This comes from my reply in a JS forum, knowing that the module will be a simply floating point, let's say .1 to .001, if we remove floating point from the right side of the operation everything works just fine:

function mod(num, mod){
// Another WebReflection silly idea
var pow = Math.pow(10, (("" + mod).split(".")[1] || "").length);
return ((num * pow) % (mod * pow)) / pow;
};

alert(
mod(8.25, .05)
); // 0


alert(
mod(8.25, .1)
); // 0.5

That's it, have a nice Sunday

No comments:

Post a Comment