01-24-2013 02:14 AM
Hello all,
In this app i am calculating total distance for the data base . so i have find the distance between two points . In this process..............If the distance to be calculated between the two points are same means i.e x1 = x2 and y1 = y2 ..i am facing the below problem
In the below attached file the distance calculater vi is some time showing the result as zero and some times as Nan and some times as 9.49353E-5
But when i open the distace cal vi and run it individually but entering the same inputs which are passed to it , it is showing zero . but if i call it as a sub vi and read its value this is showing some times as nan or 9.49353E-5
01-24-2013 07:06 AM
01-24-2013 07:07 AM
It is because floating point is not truely exact. There are rounding errors from your calculations that could cause the results to be off by a bit or more. This is true for any programming language.
01-24-2013 09:00 AM
Hello
GerdW see the attachment in 11 version
01-24-2013 09:01 AM
01-24-2013 09:11 AM
01-24-2013 10:30 PM
01-25-2013 01:54 AM - edited 01-25-2013 01:56 AM
01-25-2013 04:57 AM
Right click on the indicator & select display format, then select floating point. Then make the precision type as digit of precision & in the digit section enter the digit for precision. If you want 2 digit precision then enter 2.
This NaN is actually not a number. Reason for this can be,
Make sure this condition.
Hope this will help. for better help conver the vi to 2011 compatible. Will deal with it
01-25-2013 05:24 AM
Hi Ranjeet,
one more reason for NaN is given in the context help for ACOS - and (I think) that would be the most reasonable here...