Less than operator misbehave

[labview-programmer]

The values came from (string to number) pattern match and were misbehaving.

After tinkering with type casting and formats (didn't spend much time though), there was no change in wrong evaluation:

 

Message Edited by labview-programmer on 06-04-2010 10:19 PM

Message Edited by labview-programmer on 06-04-2010 10:21 PM

[Broken_Arrow · Accepted solution]

Sneaky. X is 4.4999999999999998

[Darin.K]

We have our weekly (at least) winner in the mysteries of floating point values contest.

See this thread to start.

http://forums.ni.com/ni/board/message?board.id=170&message.id=41383&query.id=2902191#M41383

[smercurio_fc]

And another nickel to my retirement fund!

[Broken_Arrow]

---

smercurio_fc wrote:
And another nickel to my retirement fund!

---

Better re-check your statement. They are only depositing 4.99999999982 cents.

[Coq_Rouge]

No it works. And this is common problem then working with floating point numbers :smileyvery-happy: The problem are caused by small rounding errors. The number format is the source of the trouble. It has been a lot of postings about this

Use the in range check instead.

[labview-programmer]

Yes, correct.

but least expected this. as it was initialised as fig.1 and incremented as fig.2

 

[PhillipBrooks]

smercurio_fc wrote:

---

Better re-check your statement. They are only depositing 4.99999999982 cents.

---

Hey, don't be stealing my salami! (wikipedia: Salami slicing)

[Broken_Arrow]

---

labview-programmer wrote:

Yes, correct.

but least expected this. as it was initialised as fig.1 and incremented as fig.2

 

---

Aniket, change 'em to DBL's.