how can a 12 bit sample be represented with a 7 digit decimal number

[walter 772]

I used the sample c program DAQsingleBufSync.c to familiarize my self with some of the daq ai functions
and was surprised to see A reading of 2.512992 volts
how is it that a pci 6024E 12 bit bipolor analog in sample 0 - 4096 can be resolved to such a degree of acurace in decimal?I feel as though I need an explanation before I continue with my project
bertleeman@hotmail.com

[Chris Matthews]

Remember, computers communicate in binary. Just because you get 6 digits, doesn't mean that that is the precision of the measurement. That is just the best representation in floating point of the binary transmission. It's not like the readings on either side of this number would be 2.512991 and 2.512992. They would be more like 2.503994 and 2.517332 probably. If you want to check the accuracy of the board figure out the code width you should expect based on range, resolution, and gain, then check the smallest increment you are seeing between measurements.

Best Regards,

Chris Matthews
National Instruments

[aperez]

Walter,

To better understand resolution and precision, I would recommend you read the following DAQ Specifications tutorial:

http://zone.ni.com/devzone/conceptd.nsf/2d17d611efb58b22862567a9006ffe76/45acc30d4a769a3f862568690061d750?OpenDocument

Acquiring data from hardware and representing it in software are two different things. You ask how can a 12-bit sample be represendted with a 7-digit decimal number. I disagree with the question. 2.512992 is a floating number, not a decimal number. A decimal number would be 23 or 34. No fractional part, no decimal point, or in computer terms, no mantissa. If you want to see more digits of precision, the largest data type is double (64-bit number). That should be enough to accurately represent a 12-bit sample.


Regards,
Azucena