Pow float problem

Wingnut

Well-known member
3rd Party Module Dev
Tutorial Author
Joined
Jan 2, 2020
Messages
1,282
The -1 does not decrease result of Pow(2,32), can anyone explain why this is happening?

Code:
Strict

Function Main:Int()

Local maxunsigned:Float = Pow(2,32) - 1    ' This needs float as it doesn't fit an Int. The -  1 hardly takes effect?
Local maxsigned:Int = Pow(2,32-1) - 1         ' Int works fine
Print maxunsigned                                         ' PRINTS 4294967296.0 (but should be 4294967295)
Print maxsigned                                             ' PRINTS 2147483647 (fine)

Return 0       
End
 

Wingnut

Well-known member
3rd Party Module Dev
Tutorial Author
Joined
Jan 2, 2020
Messages
1,282
I think I understand now, 32-bit on Float as well is used in CX.

I noticed that HTML5 it works (as Number is used as datatype for float, which will give more than single precision, it will give 53bit minimum).
 
Top Bottom