07-11-2008 01:39 PM
07-11-2008 01:44 PM
07-11-2008 01:51 PM
I'm not sure how to avoid this problem. Is there a way that I convert my DBL to a 32bit integer, and avoid this problem?
@johnsold wrote:
You stated that you need a DBL for the case selection. The case structure will coerce that to I32, so be careful if your values are not EXACTLY integers. Mathematical errors due to the finite size of the binary representation of the numbers can cause values to be different from what you might expect.
Lynn
07-11-2008 01:57 PM
07-11-2008 02:12 PM
07-11-2008 02:16 PM - edited 07-11-2008 02:16 PM
@johnsold wrote:
You got one more question so now we get one more.
Actually, no one minds more questions when they show that you are making progress and learning something. Even the most experienced users often learn something new, sometimes because of a question.
Where does the DBL come from? Why is it a DBL, in other words?
Lynn
Here is where I express embarrassing facts. Actually, the
DBL is a numerical control all the way from the primary VI (the one that calls
the SubVI that calls this subVI2). I wanted to let the user select between 1,
2, and 3.
Now this is when you tell me that there's a way of doing all of this with an
integer; a way that I don't know!
By the way, is it true that "conversions" in LabView take up plenty
of memory, and it's best to avoid them as much as possible?
07-11-2008 02:24 PM
07-11-2008 02:42 PM
07-11-2008 03:09 PM - edited 07-11-2008 03:10 PM