Demonstrating numerical precision

[dan.laks]

I'm running a workshop teaching high school students to use Labview.  For a lot of these kids, this will be the first time they use the program.  One of my favorite programming topics is understanding numerical precision and how important it is to a computer (signed vs. unsigned, integer vs. floating, 8-bit vs. 64-bit, etc).

I'm trying to come up with some simple demonstrations in Labview to drive the point home.  Ideally, I'd assign an seemingly simple math problem to solve.  Maybe something they could do fairly easily in their head or on a calculator.  But when they program it in Labview, it comes up with a completely wrong answer because they didn't properly account for precision.

For example, starting with a U8 integer equal to 1.  Double the number eight times in a loop.  If I ask them to predict the answer, they'll probably say "256".  But, of course, the program will come back with "0" which will force them to think why and figure it out.

I'd welcome any clever suggestions to throw at the kids.  Nothing too complicated - I want them to be able to program it themselves after a introductory lesson on using Labview.  But something that will shock them a little and help wrap their heads around the concepts.

Thanks!

[JÞB]

Congratulations Dan!

Some of the simpler ones I can think of would be:

Add 0.1000 in a loop 10 times (Floating point), predict the result..... (put out flames)

Increment an I8 past 127 (set sign bit)

How math savy are they? can the handle roots on complex numbers?  take the srt of -1 +0i

Add 0 to -0 (Float)

Compare NaN to NaN.

(Play with +-inf too)

Thank for the service to our future

[RavensFan]

Take the sine of a number in a loop and graph the results.  If the number is I8 or U8, it should show some coarseness to the sine wave.  An I16 or U16 should look better.  (of course scale the number so that the max number is equal to 2 pi so that you just see one sine wave.)

[dan.laks]

Thanks for the suggestions, guys!

Jeff, I particuarly like adding 0.1 ten times using double precisison.  That's probably the simplest program a beginner could write and should blow their socks off when the computer comes up with the wrong answer (after you set the display format to 16 digits or more).

[altenbach]

This is similar to my quick demo posted here:

Quick puzzle: What's the value of "Numeric" after the loop stops? 

[JÞB]

---

@dan.laks wrote:

Thanks for the suggestions, guys!

Jeff, I particuarly like adding 0.1 ten times using double precisison.  That's probably the simplest program a beginner could write and should blow their socks off when the computer comes up with the wrong answer (after you set the display format to 16 digits or more).

---

just test for equality to the students expected answer. then show how far off they guessed. THEN change display const

[crossrulz]

Definately make sure they understand that floats are not exact and test for equality just doesn't work!  I'm tired of debugging those kind of errors.  Rollover errrors is the other really common numerical error I see.

[paul_a_cardinale]

Here's my favorite.  Simply coding the solution to a quadratic equation based on the usual algebraic form can result in severe loss of precision if b >> 4ac.

Code it as follows to avoid the problem:

x1 = (-b - sqrt(b^2 - 4*a*c)) / (2*a)

x2 = 2*c / (-b - sqrt(b^2 - 4*a*c))