Truncating and Rounding problems

[dacad]

This is one I really didn't expect to have problems with.  Running LV 2011 on a Win7 box.  Am taking readings from a flow meter and using the results to pass or fail the unit.  Problem I have is that results are in double precision and so that if the result is even a single value at the 15th decimal place over the max spec, the unit fails.

Technically, that is correct but when you apply statistics, along with the accuracy, resolution and repeatability of the test equipment, you have some rules that can be used to truncate the values so the units that technically fail by .00000000000001 are determined to be real world acceptable.

To that end, I am trying to truncate my results to 3 decimal places, then round to two decimal places to determine my result.   For example:  My upper spec would be 1.1  If my initial result was 1.105xxxxxxx  I want to truncate that to 1.105, then round it to 2 decimal places making it 1.11 (using standard rounding rules, 4 rounds down, 5 rounds up)  This unit would then fail as it is higher than the upper spec.   In the other direction, if my initial result was 1.104xxxxxxx I want to again, truncate to 3 decimal places making it 1.104 then rounding to 2 decimal places making it 1.10, therfore passing.

I had never noticed that LV didn't have basic truncate and round functions (Excel does it quite nicely)  This is basic math and should be easy.

I have tried all sorts of manipulation using conversions, convert representation, convert to string, multiplying, dividing, back and forth and am just missing the mark.

I can achieve the truncated value by multiplying by 1000, Rounding torwards - infinity and dividing by 1000 but then the rounding to 2 decimal places is the sticking point.

Any ideas on this are much appreciated.

Doug

[crossrulz]

Your problem is that you are truncating first.  If you are rounding to 2 decimal places you want x = round(x*100)/100.  It is that simple.

[dacad]

I am truncating first so that numerals past the third decimal place do not affect the rounding.

If I only round to 2 decimal places,  1.1051 and 1.1049 would both round to 1.11     Not what I want.

The LV Round to Nearest Function does not play by standard rounding rules, instead, it rounds to the nearest EVEN integer.  See text below straight from LV help on this function.

"For example, if number is 1.5 or 2.5, nearest integer value is 2."

Doug

[crossrulz]

---

dacad wrote:

If I only round to 2 decimal places,  1.1051 and 1.1049 would both round to 1.11     Not what I want.

---

Not true.  1.1049 will round to 1.10.

As far as the round up with 5, I'll have to play around a bit.

[JÞB]

---

@dacad wrote:

The LV Round to Nearest Function does not play by standard rounding rules, instead, it rounds to the nearest EVEN integer.  See text below straight from LV help on this function.

"For example, if number is 1.5 or 2.5, nearest integer value is 2."

Doug

---

Doug,  That round to even is actually part of  the IEEE 754 standard.  It is correct by definition.

[dacad]

Figured it out.  See attached.  Not the most elegant solution but it serves the purpose.  The multipliers and divisors could be made into controls for more options for other circumstances which I will likely have to do on other projects forthcoming.

I would really like to see the basic mathmatical functions for truncating and rounding (with perhaps a option to select midpoint rounding direction) added to the program.  If MS can do it in Excel, then it should be a non issue for NI to do it.

 

Doug

[crossrulz]

---

dacad wrote:

The LV Round to Nearest Function does not play by standard rounding rules, instead, it rounds to the nearest EVEN integer.

---

As Jeff said, that is the IEEE standard.  If you want the "traditional" rounding, here's the hoops I had to go through.  Also keep in mind that this currently only works for positive numbers.

Again, you do NOT need to truncate first.

[JÞB]

---

@dacad wrote:

Figured it out.  See attached.  Not the most elegant solution but it serves the purpose.  The multipliers and divisors could be made into controls for more options for other circumstances which I will likely have to do on other projects forthcoming.

I would really like to see the basic mathmatical functions for truncating and rounding (with perhaps a option to select midpoint rounding direction) added to the program.  If MS can do it in Excel, then it should be a non issue for NI to do it.

Doug

---

Excel is not IEEE 754 compliant

[dacad]

I do understand the IEEE rounding standard but that technique will not work for my situation.   It is also not what I consider standard street rounding. Granted, I've been out of school a few years but for basic mathematics, I was always taught that 5 rounds up and 4 rounds down.

This is why I state that 1.1049 rounds to 1.1.  Again, perhaps old school but the 9 round the 4 to a five, in turn rounding the 0 to a 1.

Crossrulz,, did more or less the same thing that you posted. The effect of rounding a dbl precision to an integer with the LV functions is essentially the same as truncating.  Just a different way to accomplish the desired outcome.

As a search on the great web will show, there are many methods of rounding available so it seems to be a matter of selecting which one suits the situation.

Interesting that something so seemingly simple could become so very convoluted.  I may have to strike up a conversation with our statistician guru

Appreciate the feedback on this.

Doug

[billko]

---

@dacad wrote:

I do understand the IEEE rounding standard but that technique will not work for my situation.   It is also not what I consider standard street rounding. Granted, I've been out of school a few years but for basic mathematics, I was always taught that 5 rounds up and 4 rounds down.

This is why I state that 1.1049 rounds to 1.1.  Again, perhaps old school but the 9 round the 4 to a five, in turn rounding the 0 to a 1.

Crossrulz,, did more or less the same thing that you posted. The effect of rounding a dbl precision to an integer with the LV functions is essentially the same as truncating.  Just a different way to accomplish the desired outcome.

As a search on the great web will show, there are many methods of rounding available so it seems to be a matter of selecting which one suits the situation.

Interesting that something so seemingly simple could become so very convoluted.  I may have to strike up a conversation with our statistician guru

Appreciate the feedback on this.

Doug

---

The method of rounding that LabVIEW uses is called "banker's rounding" and creates a more statistcally "neutral" result - that is, if you have to do several rounds of calculations and roundings, it comes out statistically closer to the actual number, because you will round down as much as you will round up.  "Normal" rounding is biased towards the higher number.  That being said, everyone who codes in LabVIEW probably had to make a "round normal.vi" similar to the ones shown here to please various powers that be...

Did you know that .NET uses banker's rounding also?