06-03-2009 11:08 AM
06-03-2009 12:06 PM
Hi dragonlord,
you ran into the limits of floating point arithmetics of the CPU...
SGL allows 23 bits mantissa, DBL 52 bits, and EXT 63 bits resolution. This translates to ~7, ~16, and 20 decimal digits, respectively.
If you need more precise arithmetics you can code your own or use one of the available "high precision" software packages
06-03-2009 12:11 PM
I can break this 24 digit number down into smaller numbers and use those. Was just wondering if there was an easier way. Thanks anyways!