LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

how do i get more accuracy in converting a string to decimal

I am trying to read an attitude and heading referance sensor. An example of the data output is a string as follows;
I -001.9 +00.0 006.8 +00.0
I -001.9 -00.0 006.8 +00.0
I -001.9 +00.0 006.8 +0 ... etc.
It consists of various columns seperated by spaces. My problem is reading the digit after the decimal point. I have tried in two ways: first with the modified parse string example using the scan string vi and secondly with the spreadsheet string to array function. In both cases, the numbers are rounded of the the nearest integer and any negative signs are ignored. Can anyone suggest any reasons for this and/or how to get round it?

Phil
0 Kudos
Message 1 of 4
(2,916 Views)
First - spreadsheet string to array function gets format string parameter. In your case it "%f".
Second - when you see result array in indicator array or graph set its representation to double and presision to 4 for example (how you need).

Remember that first column you will get always 0 because you have in string first column letter I.

P.S. Change delimeter to space (by default it Tab)
0 Kudos
Message 2 of 4
(2,916 Views)
I have an example attached
A multiline string splitted into an array of strings
processed by scan from string in a for loop.
If you look inside the for you find the solution for one line.
I don't understand why you had the resolution problems but send me the original source and I can find out
greetings from the Netherlands
0 Kudos
Message 3 of 4
(2,916 Views)
Phil2 schrieb:
> I am trying to read an attitude and heading referance sensor. An
> example of the data output is a string as follows;
> I -001.9 +00.0 006.8 +00.0
> I -001.9 -00.0 006.8 +00.0
> I -001.9 +00.0 006.8 +0 ... etc.
> It consists of various columns seperated by spaces. My problem is
> reading the digit after the decimal point. I have tried in two ways:
> first with the modified parse string example using the scan string vi
> and secondly with the spreadsheet string to array function. In both
> cases, the numbers are rounded of the the nearest integer and any
> negative signs are ignored. Can anyone suggest any reasons for this
> and/or how to get round it?
>
> Phil

Hi Phil,

look at the same question from Razmocket today.. I just gave an an
swer.

In your case you have to use folling format string for "Scan From
String" function: %.;I %f %f %f %f
Your fail was not to set the right decimal point. With "%.;" you can
specify using "." as delimiter. For more information look into the faq
of LabVIEW

greets
Henrik
0 Kudos
Message 4 of 4
(2,916 Views)