LabVIEW
Equal numbers greater than each other?
LabVIEWn00b wrote:
I'm not using floating point values, and I've tried changing the precision and the representation. Recall, I said that the same comparison works on it's own in another VI. I don't know if it has something to do with its position in the loop in the IV or what. I've got it working now, had to do some minor conversions. As far as I can see there's something involved somewhere which is introducing FAR more precision than needed, but it's been taken care of.
Form your words, it is clear that you don't fully understand what's going on and it would be important to fully analyze the issue:
"I'm not using floating point values": Was the wire orange or blue?
"I've tried changing the precision": that is just a cosmetic display property and has no influence on the underlying data.
"and the representation": THIS changes the underlying data! What whas the representation before and after that you used?
"Recall, I said that the same comparison works on it's own in another VI": Was it with the same value or with a manually entered value that "looked" the same from the displayed digits?
" I don't know if it has something to do with its position in the loop in the IV.": If the outcome of code were different depending on where it is placed, that would be surprising unless there are race conditions.
"or what": What?
"I've got it working now, had to do some minor conversions": What kind of conversions? What is the difference between a minor and major conversion?
"As far as I can see there's something involved somewhere which is introducing FAR more precision than needed": All numerics of a given representation have the same precision (in terms of e.g. bits/mantissa). Since you said that you were NOT using floating point values, can you explain what you mean by precision? "something" and "somewhere" is not good enough. Can you pinpoint the exact location of the problem?
"but it's been taken care of.": How?
Message Edited by altenbach on 06-30-2008 08:11 AM
Your best bet when attempting to find equality of floats is to use "In Range and Coerce." Subtract one double from the other, and see if it's within -0.00001 and 0.00001 or something. These are two ways to do it (one obviously less efficient, but maybe more intuitive).
Message Edited by JeffOverton on 06-30-2008 01:55 PM
Some of your code is very Rube Goldberg'ish:
If you want to operate with one significant decimal digits, multiply everything by 10 and work entirely with blue integers inside the loop. Divide by 10 after the loop.
You should use an autoindexing FOR loop and stop if the resistance match (Just show the conditional terminal).
Message Edited by altenbach on 06-30-2008 11:15 AM
Here's a simple alternative, assuming you want one decimal digit.
Also: A math loop should NOT have a wait!
Message Edited by altenbach on 06-30-2008 11:31 AM
I have not tested this extensively, but would the following work as well? (Note that I did not put in the check for >10. Also, since I have 8.2 I don't have the conditional stop on the for-loop. 
)
Message Edited by smercurio_fc on 06-30-2008 01:32 PM
)Message Edited by smercurio_fc on 06-30-2008 01:32 PM
Ah, yes. I did not notice the x2 pattern in the resitors.
In this case, things are even simpler!
Message Edited by altenbach on 06-30-2008 11:51 AM
