LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

rounding in LABVIEW

Solved!
Go to solution

What is the data type of "input"? Can you show us some code instead of long descriptions?

0 Kudos
Message 11 of 12
(854 Views)

@Ahmed11 wrote:

Dear altenbach , i am facing a new problem . As you know , i have used the formula (input*255/5) to convert to digital, then i have used digital to binary avi to convert this digital value to 8 bits.

The problem here is the data type mismatch between my output from the formula, which is a double data type and the input of the avi which is a digital data type, so how to solve this problem ? Thank you in advance


Wrong equasion!  What you are doing is the equivallent of walking up a flight of steps by lifting each foot by exactly the height of the average runner.  Any small deviations in runner height are going to cause you to trip on the step.  In Other words you are trying to predict an ADC conversion without accounting for quantization error 

 

Use Floor((Input+5/511)*255/5)) for 8 bits and a 5V rail

 

 


"Should be" isn't "Is" -Jay
0 Kudos
Message 12 of 12
(837 Views)