DAQ board (PCI-6031E) measurement offset comes and goes

[dabretty]

Hello -

I've been dealing with seemingly noise problems for quite a long time, and am encountering very bizarre offset-measurement behavior now with our PCI-6031E.

To sum up our setup.  I have a PCI-6031E connected to a SCB-100 connector box.  I then have about 10 DAQmx channels configured and scaled, pretty much all of them by a factor of 5X.  (True measured range is approximately 0-40V, which we are then dividing down with resitors to about 0-8V seen at the box, followed by the aforementioned 5X scaling to give us ballpark accuracy)  So our resulting voltages that are returned are 0-40V.

Now, the problem!  We have two batteries that are being measured, nominally in the ballpark of 32V.  Using a voltmeter, we know that they are reading 31.35V.  For a good portion of the time, our DAQmx measurement (whether done in my own Labview program along with other channels, or entirely alone using MAX) reads the correct voltage.  But, maybe 20% of the time, a very large offset occurs of (scaled) +300mV, giving us a reading of approximately 31.65V.  This translates to a real (unscaled) measurement offset (on the 0-8V input) of about +60mV.  Then, while it's sitting and operating in this offset, it does appear more susceptible to noise - but all in the direction of what we know the true measurement should be (31.35V).  Follow me?  It's like there's some charge built up that periodically dissipates, causing the measurement to drop momentarily to the real value before the charge buildup returns.

Attached is a JPEG showing this (in Excel).  The top chart shows thermistors, which I understand have an inverse voltage/temp relationship (hence the drops in temperature).  The bottom chart shows the two battery voltages that I have been discussing (the "true" measurements being the lower ones).  The offset can last for a few seconds, or a few minutes, or (sometimes) into hours.  It happens to all channels at the same time, no matter if they are measured individually (using MAX) or what their ultimate cable run length is.  (We have some long cable runs of ~30ft, as well as some short ones ~5ft).  We have played around with different things like SCB-100 box grounding, to no avail.  We cannot prompt it to occur, nor can we make it go away.  We are also in a relatively electrically-noisy environment (vacuum chambers/pumps, etc) but we cannot tie it to any particular events (we've powered things on and off, to find absolutely no correlation).

Any ideas?

Thanks!

Brett

[ChrisR.]

Hi, Brett. It could be ground related. You don't state if you're measuring single-ended or differentially. Is it possible one or more of your signals is floating out of the common-mode range of the board? Another possibility is that some unused channel is picking up a nearby noisy signal and overloading the input multiplexer. This would corrupt all the channels of the board, even when not scanning. Try grounding unused channels. I realize there are a lot of them... maybe you could try scanning all channels on the board to see if any particular channels look they're exceeding allowable voltages. Of course the act of scanning them could help bring them in line, as the charge from one channel is transferred to the next.

Another possibility is RF rectification in the front end. Are you near any transmitters?

I hope this at least gives you some ideas of how to proceed.
Chris

[Hani R]

Hi Brett,

Thank you for contacting National Instruments.  I would first suggest making sure the signal is properly grounded.  Refer to the Developer Zone article, Field Wiring and Noise Considerations for Analog Signals, for details how to properly ground the signal.

I have a few questions for you about your setup.  What input configuration (differential, referenced single ended, or non-referenced single ended) are you using to measure the signal?  Also, to what channels on the SCB-100 did you connect your signal? 

As mentioned in the Field Wiring article, something to be aware of if you are measuring a differential signal from a floating source (not connected to building ground) is that you will need to connect a resistor (10kΩ to 100kΩ) from AI+ to AIGND and another resistor from AI- to AIGND.

[dabretty]

Thanks for the help so far!  I feel like we're getting somewhere ...

To first answer the questions as to our setup - yes, we're measuring floating signal sources differentially.

I went back (Chris) and used the measurement and automation explorer to attempt to read all of the differential pairs.  Sure enough, the first open channel (differential pair #12, pins 55-to-56) was railing to +10V.  Continuing down the line, most of the remaining pairs were doing the same, with a few floating around 9V.  Even though I am not scanning these open (floating) channels, I understand this is not a good thing!  We are now in the process of taking all unused channels and connecting them to ground.  (Yep, this will take us a little while)

While we are in the process of doing this, I did a quick test (Hani) with taking a single channel's differential pair to ground via 50k resistors, as you guys had outlined in that document you referred to.  It didn't do anything by itself, but I suppose bias currents floating in on any channel that's being read is just as bad as all of them?  While we're already in the painful process of taking all unused channels to ground, I figure we had might as well double the fun by doing this with every AI input pin in the connector block.

So to sum it up we're going to try both things you two suggested (regarding the open inputs and bias current resistors), which will take some time to wire.  I'll get back to you with our results!

Thanks!

Brett

[ChrisR.]

The app note Hani pointed you to recommends resistors to ground on both inputs, but I suspect you only need one, because I imagine your voltage divider is unbalanced. In other words, the low side of the divider probably presents a lower impedance to the DAQ board than the high side. So one resistor per channel should suffice. Furthermore, I don't see any reason that resistor can't be zero ohms. In other words, just directly tie the low side of your divider to the board's AIGND terminal. Referring to Figure 11 in the app note, then, you'll have R1 = infinity and R2 = 0.

The fact that an unused input is railed when measured isn't too alarming. In order to mess up the input mux you have to inject some current (microamps at least?) into it, but it takes only a few picoamps to rail the inputs. Nevertheless, shorting unused inputs can't hurt. Except for the person who has to do it!

If you're scanning, something else you might try is to short just one channel to ground and then put that channel right before all the floating ones in the scanlist. The scan-induces interchannel coupling that is normally such a nuisance in a multiplexed board would work for you in this case by pulling the floating channels to ground. Just a thought.

Chris

[dabretty]

Thanks Chris, Hani ...

Some new info here.  We (I use that term loosely!) completed wiring all of the unused inputs to AI GND.  They are all tied down.  As for the actual input signals, both sides (+ & -) are wired to AI GND via 50k resistors.

Result - the signal seems to be even noisier.

But, the potential good news (or the "a ha!" at least) is that in doing so I found some very peculiar things.  I opened up MAX and looked at an unused channel (now tied down) and saw it oscillating between 0 and 0.1V in a very discrete-looking manner.  Very bizarre, considering they are directly tied together.  So then I went and looked individually (via RSE measurement, using AI GND as the direct reference) at each single-ended input that makes up the pair, and did this for all of our unused (tied down) channels.  All of one side of the differentials (the + side I believe) appear to be solid 0 (okay, maybe it's slightly offset by 1 mV or something).  If I do a RSE measurement on the - side however, I see a discrete switching back and forth between 0 and -0.1V.  This is true for every single differential pair, when each half is measured with reference to AI GND.  And as for the actual signals themselves?  Well we've got to scale the still but they appear to okay so far (this is only after watching for ten minutes, so may be premature).  The other guys are shifting back and forth like usual, but the actual signals are not when measured via RSE measurement.  They do suffer the same noise kinds of issues (peak to peak of about 5 mV), but at this point we are not complaining.

So now this has us considering using SRE measurements on signals that might be traveling via (twisted pair) over very considerable lengths.  Obviously we'd like to use differential, but our DAQ board is not delivering here. 

Does it sound like our board is broken?  Any other ideas of what I could look into?

Thanks so much for all of your help!

Brett

[dabretty]

So I apparently was premature in thinking the single-ended solution might be a fix.  I ran an overnight test, recording at 1 second intervals, of two channels that are ultimately measuring the same thing.  One was a single-ended measurement, the other a differential measurement (and scaled).  In both cases, the signal is going to ground via a 50k resistor - while in the differential case both legs are going to ground via a 50k resistor.

I've attached a screen capture of the readings (again, over a 9+ hour span).  Very bizarre!

Any suggestions?

[ChrisR.]

Can you attach a drawing of how you have everything connected? Or an alternate approach might be to try to simplify the system to the point where it works. For example:

1. Disconnect everything and measure some internal channels (ground, reference, analog output,...). If they measure OK, then...
2. Connect just one signal. Does it measure properly? If not, let's work on that. If it's OK, then...
3. Add more signals until it doesn't work properly any more.

Eventually, we should be able to figure out what's going on.

Chris

[dabretty]

Chris, Hani -

Thanks for all of your help, and sorry I didn't update you guys ...

I logged data at 1 Hz for two days straight, and sat down last Friday to try to further decipher what the cause was.  I wanted to be able to point at some heavy duty building HVAC controls not too far away (that we've always suspected as not being grounded properly).  But in sitting down to try to correlate different events, what I found was quite shocking and made me feel a bit stupid ...

Long story short, it appears that the monitor on our monitoring PC was causing the problem, via grounding loops.  We had two cases that we were fighting before - one a constant shift that appeared to occur, and the other being very ratty readings as it appeared to be temporarily "jumping back" to where it was before the offset.

In the first case, everytime our monitor would cut out into power-saving mode (60 minutes) it would cause a constant offset to form.  When the monitor woke up, it'd jump back.  Unfortunately this PC is used by many people for different purposes, so it was very hard to pinpoint this up front (maybe even the desk was bumped, mouse activated, etc ...).  So I guess when the monitor is in the on state, it caused some grounding loop in the measurement that translated to an offset.

Secondly, I had a rackmount PC (that we barely need to view the display of) with its analog VGA cable connected to the same measurement PC's monitor.  Hardly ever used it, easy to forget about!  After discovering the monitor had very much to do with that offset in the previous case, I was in the process of switching monitors when I discovered that when I would disconnect the monitor's secondary analog input, the signal got rock solid (it stopped trying to make those temporary "jumps").

Over the weekend, I turned the monitor onto an always on mode (feeding it a blank screensaver) and I let this thing sit and monitor some batteries ... and sure enough, it had no issues (offset or the "jumps")!  So I guess we're back in business.

My only concern at this point is, is there something wrong with our board?  While I could see how the rackmount PC being tied into the monitor could create some ground loop issues (it comes through a completely different path back to the same UPS), why the heck would the card be so sensitive as to the PC it's attached to having it's monitor turn on and off?  I tried this with multiple monitors, and they all have the same effect (although slightly different in offset magnitude).  Are we going to point the finger at our video card?

Thanks so much for all of your help ... I guess it just took a little sleuth work in the end to figure out!

Brett

[ChrisR.]

Well, I wouldn't have anticipated your fix. Does turning the monitor on and off affect the measurement even if the second PC isn't plugged into the monitor? Could it be that waking up the monitor enables the connection to the second PC, which completes a ground loop? I will say that it doesn't sound like a problem with the PCI-6031E, but I don't see how the video card could mess things up either. You shouldn't have to disable monitor sleep to make things work. In fact, CRT monitors are notorious noise sources, so letting them sleep is generally a good idea. I suspect some kind of system problem. Are there any other wires connected to your system that you've forgotten about?

Chris