cancel
Showing results for 
Search instead for 
Did you mean: 

Output error.

SOLVED
167274
Member
Solved!

Output error.

Message contains an attachment

y=0.049~0.51 Output error.

8 REPLIES 8
RavensFan
Knight of NI

Re: Output error.

Please put more words into your post so we know what you are concerned about.

 

All the results are just as I would expect.

 

Keep in mind that when doing math on floating point numbers, the results may not match exactly what you'd expect if you were doing it on paper.  Not all decimal numbers can be represented exactly in the binary world that computers operate in.

altenbach
Knight of NI

Re: Output error.


@167274 wrote:

y=0.049~0.51 Output error.


How can there be an error if you don't even do any error handling.

 

What number do you enter, what do you get, and what do you expect to get instead?

As has already been said, floating point comparison are not very predictable. What are you trying to do?

 

 

jwscs
Active Participant

Re: Output error.

registered 2013, one post, seems fishy


If Tetris has taught me anything, it's errors pile up and accomplishments disappear.
RavensFan
Knight of NI

Re: Output error.

Originally that post was in one of the Idea Exchanges.  I asked the moderator to move it out.

 

I'm not expecting anything fishy other than someone who isn't able to compose a reasonable message to explain their problem.

jwscs
Active Participant

Re: Output error.

when i think about it .. somehow the title seems appropriate Smiley Very Happy


If Tetris has taught me anything, it's errors pile up and accomplishments disappear.
altenbach
Knight of NI

Re: Output error.


@jwscs wrote:

registered 2013, one post, seems fishy


Lurkers are perfectly welcome here, though Smiley Very Happy

Highlighted
167274
Member

Re: Output error.

Message contains an image

The constant(x) compare with a control value(y) the result is x=y. But when use a number(x1) which came from the result of calculator's result, then do comparison with the control value(y). The result is x1<y.The constant(x) compare with a control value(y) the result is x=y. But when use a number(x1) which came from the result of calculator's result, then do comparison with the control value(y). The result is x1<y.

GerdW
Knight of NI
Solution

Re: Output error.

Message contains an image

Hi number,

 

yes, your results are expected and "by design" as has been said in message #2 and #3 one month ago…

 

Please set the formatting of ALL your controls, indicators and constants to show 17 decimal places - and you will see the reason behind your comparison results!

Like I did here:

check.png

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019