Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Kogan Loloklam]

I'm playing around with haXe and flash in Flash Develop.
I'm trying to learn variables and random generators.
I'm running into an odd situation...

Spoiler: This code (click to show/hide)

Code: [Select]

class Main
{

    static function main()
{
var FoodCheck = ((Std.random(43)+1)/1000);
var FoodCheckBase = 0.042;
trace("Food Check Drop");
trace((FoodCheckBase)+(FoodCheck));
trace("Food Check Percentage");
trace(FoodCheck);
trace("Food Check Base");
trace(FoodCheckBase);
    }

}


Output  "0.07200000000000001" for Food Check Drop, 0.03 for Food Check Percentage, and 0.042 for Food Check base.

Now to say I don't know where it came from is a understatement.  It seems to work normally most of the time. What did I do wrong? Any idea?

The point of doing this is to create a percentage to work with that operates to 6 digits. (xx.xxxx)
This gives me lots of leeway to create a system that is affected by many different small details and check against percentage-based pass/fail checks. While most parts I created of that system tested independently worked fine, This part had such an odd output number that I think something I'm doing is fundamentally wrong.
I ran multiple tests when I discovered this, and it outputted similar results sometimes.

Sometimes it works fine. Here's a few examples of what it spat out when it didn't:

Spoiler (click to show/hide)

FCD: 0.05400000000000006
FCP: 0.012
FCB: 0.042

FCD: 0.08499999999999999
FCP: 0.043
FCB: 0.042

FCD: 0.07100000000000001
FCP: 0.029
FCB: 0.042

FCD: 0.07500000000000001
FCP: 0.033
FCB: 0.042

FCD: 0.05300000000000005
FCP: 0.011
FCB: 0.042

FCD: 0.04300000000000003
FCP: 0.001
FCB: 0.042

FCD: 0.05900000000000004
FCP: 0.017
FCB: 0.042

So yea, I'm pretty darn confused!

[Max White]

Umm, could you give some sample output of what you want it to produce?

[Rysith]

You're almost certainly running into floating-point imprecision issues. See Wikipedia for the technical details.

My advice, if you want absolute precision, would be to do all of your operations in whole numbers representing ten-thousandths of a percent, to give you your 6 digits as xxxxxx (since integers can be represented with infinite precision, and I would hope that Flash represented integer quantities as integers).

[Kogan Loloklam]

Umm, could you give some sample output of what you want it to produce?

xx.xxxx
instead of
xx.xxxxxxxxxxxxxxxxxx

...

Reading that, I think your right. Interesting. I never knew binary wasn't able to represent 0.1 accurately. It's fun to learn limitations! That'll be a headache if I ever wish to show the results in a decimal format though, even if I keep all operations in whole numbers. I imagine I'll find some workaround if I ever have to use it though. Still, outputting information is easier than dealing with data that isn't operated on correctly.