Double Value != Double Value

[Madottati]

I have the following signal in a LIN LDF, I want to read the signal and evaluate it:

The LDF signal encoding definition for this signal looks like:

CTR_PO_IFE_SigEncod {
  logical_value, 0, "Richtung vollständig schließen";
  physical_value, 0, 65530, 1, -32765;
  logical_value, 65534, "Richtung vollständig öffnen";
  logical_value, 65535, "Signal ungültig";
 }

The signal has the value 0xFFFF physically on the bus (looking at the bus monitor) --> the XNET read function returns the double value 32770 --> looks OK. BUT! If I compare this number in LabVIEW with 32770 (DBL), the two numbers are NOT equal.

So I type casted both numbers in strings, and I see the following difference:

0x40 E0 00 40 00 00 00 00 --> this is the number I expect = 32770

0x40 E0 00 3F FF FF FF FE --> this is the number XNET Read returns =32769,99999999998544808477

I have two questions:

1. How does XNET calculates from the value 0xFFFF the above mentioned number?

2. What can I do in order to get the comparison result TRUE?

Logically, taking the definition from the LDF, the result should be 65535 - 32765 = 32770, where 65535 is the value on the bus, -32765 is the offset. Or am I wrong? Or does it have something to do with the maximal physical value of 65530?

Any help is appreciated

Madottati

[JefeL]

Upconverting from a 16 bit integer to a 64 bit float and then performing floating point math can result in anomalies such as this. The conversion resulted in a rounding of the lest significant bit of the double which results in the smallest possible difference between our expected value. While the two are extremely close in value they are not technically equal. 

As a general rule you should never compare double values with the equality operator to avoid difficult to catch bugs that result from these minute rounding errors.

Integer data types are well suited for equality conversions. I would recommend casting the values to a U16. 

[Madottati]

Hello Jeff,

this was also my idea, but I have to take care about the scaling of the signal. I can use this option only if the scaling factor of the signal is integer. If the scaling factor is i.e. 0.1, and the signal value is 24.3, I don't want to round it to nearest integer 23...

But I will try to work it around.

Thank you