04-11-2011 04:40 PM
Hi, I set a limits as GE 11 and LE 14 and Numeric Format is "%.0f". The test report shows "fail" because the measurement is "11"( it is "10.9" when the format changes to "%.1f"). I am confused if Teststand compare the limits value with real measured value with fraction? How to fix such problem? Thanks!
04-12-2011
03:46 PM
- last edited on
10-20-2024
12:10 PM
by
Content Cleaner
Hi,
The numeric format should only affect how the data is diplayed, not what values is used in the comparison, as the TestStand help for the Numeric Limit Test Edit Tabs indicates. I tested this myself, and found that the numeric format did not affect the result of the comparison, although it did show differently in the report. You may want to check that the comparison type you are using is what you need.
Hope this helps,
Jen W
Applications Engineer
National Instruments