Controlling actual decimal places of double precision data

[Ecrh]

Hi

I perform a calculation from which I get a double precision number of many decimal places, I have created local variable to read from and use the number later in the vi. When I use it later I woul like to restrict the number to 3sf or 2 decimal places with zeros where applicable. Hoiw can I do this. I do not want to change just the displayed number but the actual number. Cheers

Emily

[jasonhill]

The Numeric Palette does have a rounding function, but it is to the nearest whole interger.  What to do?

How about this?

Message Edited by jasonhill on 04-22-2006 08:49 AM

[shoneill]

Emily,

only do this is there are NO further calculations performed with these numbers, otherwise you're going to get some wierd rounding problems.

As a final result from a previous process, this should be OK.

You could alternatively multiply the value by a fixed amount (say 1000 for 3 decimal places) and actually store the number as an integer (Assuming you don't need such large numbers).  This way you always have the three decimal places present and don't really have to worry about rounding errors, since they'll be rounded automatically.

i.e. 3.147 could be made into 3147, 123.500 made into 123500 and so on.

Hope this helps,

Shane.