LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Unable to convert all digits of a large number string to Decimal

I feel like I am missing something easy here, but it is escaping me at the moment.

 

I have a transaction ID from a SQL database, 1000109720220517184732283.0

 

It is part of a JSON text but whether I use a JSON conversion tool kit or just the standard" fract/exp string to number", I cannot get the value converted correctly.

 

When converted from string to a number, I get 1000109720220517230000000

 

Kenny_K_0-1655167759839.png

 

According to this page, the double should be capable of Maximum positive number: 1.79e+308

 

https://www.ni.com/docs/en-US/bundle/labview/page/numeric-data-types-table.html

 

Kenny

0 Kudos
Message 1 of 8
(2,091 Views)

But DBL is only good for about 15 decimal digits. (range is not the same as precision!) Even U64 is insufficient for your desired precision.

 

 

altenbach_0-1655170852789.png

 

 

Just keep it as decimal string. I doubt you are trying to do any math with it. 😄

0 Kudos
Message 2 of 8
(2,070 Views)

According to this page, the double should be capable of Maximum positive number: 1.79e+308

https://www.ni.com/docs/en-US/bundle/labview/page/numeric-data-types-table.html


Well, that's true but it's not *complete*.   There's also a very thorough wiki, but you can find a more simplified summary here.

 

Summary: double precision has enough bits to represent about 15-16 decimal digits of precision.  So you can represent *all* integers up to roughly 10^15 or so.  Beyond that, only *some* integers can be represented.  For a while you can represent 1/2 of them, then for another while 1/4, then 1/8, etc.

 

 

-Kevin P

ALERT! LabVIEW's subscription-only policy came to an end (finally!). Unfortunately, pricing favors the captured and committed over new adopters -- so tread carefully.
0 Kudos
Message 3 of 8
(2,060 Views)

Out of interest, what is the original datatype in the SQL database?

0 Kudos
Message 4 of 8
(2,044 Views)

The Data type is decimal, with the following properties according to the SQL database

Kenny_K_0-1655914133817.png

 

 

I ended up leaving it as a string, as altenbach suggested.

Kenny

0 Kudos
Message 5 of 8
(1,969 Views)

So that is a type of fixed-point number, but you'd need at least 84 bits and LabVIEW doesn't support that yet. But yes unless you need to do maths on it, a string would work well, such as for a key in a map.

0 Kudos
Message 6 of 8
(1,950 Views)

Actually wouldn't a precision of 30 require at least 100 bits to be able to fully encode such a number?

I think they use internally an int128_t for that except that you need a very recent GCC version to be able to use that.  Since it is a transaction ID I doubt there is ever any mathematical operation needed on it, so treating it as a string seems very reasonable although of course not the most performant solution.

 

Getting it in native format through the ODBC/ADO/DAO or whatever database interface would be most likely the biggest challenge anyhow. Well ODBC knows for instance the SQL_C_NUMERIC type, so that would be in principle possible, but LabVIEW doesn't know an arbitrary sized numeric datatype, so at that point it needs to be converted into something LabVIEW can deal with and the string is the most sensible format for this.

 

Rolf Kalbermatter  My Blog
DEMO, Electronic and Mechanical Support department, room 36.LB00.390
0 Kudos
Message 7 of 8
(1,937 Views)

For 30 digits, yes, but the number in the first post was fewer.

0 Kudos
Message 8 of 8
(1,930 Views)