error -200010 during hardware timed single point analog input

[Peter_B]

Some more thought I had on this. 

The PCI-601 and PCI-6036 both only have a FIFO input buffer of 512 bytes. which if being filled up would do so in about 2.5 seconds@200Hz [1]

The PCI-6251 has a 4096 byte FIFO input buffer which should fill up in about 20 seconds at 200Hz, instead it appears not to fill up until about 40seconds.  Once again these numbers would really only make sense if I wasn't doing single point H/W timed AI DAQ as the FIFO buffer should never fill up.

So in summary, the task now at hand is to elucidate the difference between (my PCI-6014s and PCI-6036) and (your PCI-6014/6251 and my PCI-6251)

[1] Just a slight clarification, I have been saying my CLK rate is 200Hz, which is true but my timed loop rate is actually only 100Hz because I have set dt=2 (see prior entries in this thread to understand why).

regards

[Peter_B]

Hi Brooks,

The plot now thickens...

Dmitry (an NI Apps Engineer in OZ) has been using a PCI-6221 and he gets the same behaviour as me with my PCI-6251 whereby after 40seconds error -200714 occurs.  OK so that observation doesn't  explain too much (in relation to error -200010) because of the larger FIFO buffer.  Dmitry then got his colleagues in Sydney to repeat the test using a PCI-6014 and a PCI-6036 which proceeded to give error -200010 after just 2.5 seconds.  Neither of their cards had been calibrated.  THIS MEANS they can repeat my problem !!

Now I wonder if you could double check the fact that you "checked out a PCI-6014 and still haven't been able to reproduce this issue" .  If that really is the case then it means there is a difference between your setup and our setups in OZ.  At least Dmitri can continue trying to find the source of the error in a parallel investigation.

Dmitri and I notice that upon first running the  vi, it will sometimes hang LabVIEW in the first sub-vi (Initialise DAQmx_distilled_v0.7.3.vi).  We circumvent the problem by turning on Highlight Execution for that sub-vi. Once it runs once we can turn off Highlight Execution and it runs fine after that until we close/re-open the vi or (cold?) start the PC.  Perhpas that hanging is another symptom of the problem, perhaps it is urelated.  I just wanted to mention it to you in case you experienced something similar.

Perhaps I am barking up the wrong tree when I suspect EEPROM differences.  The best way to progress now would be if I could swap my PCI-6014 card with  yours and see what happens !!  What do you think we should try now ?

I hope it doesn't take you too long to get up to speed with our progress over the weekend.

regards

[Brooks_C]

Hello Peter,

I had noticed the hang the first time you run the program--since it was corrected as soon as I started troubleshooting with execution highlighting and you hadn't mentioned it I hadn't looked into it.  However,  I think the issue is that the DAQmx Create Timing Source (Control Loop From Task).vi needs to be called before the timing for the analog output is configured--I've attached a version of the Initialize VI that should work every time since the race condition has been eliminated.  I took the brute force approach of configuring all of the AO after the AI task, but I believe you technically only need to call the Start Task on the AO after the Create Timing Source for the AI.

I have been able to reproduce this error now, and I believe you've identified the cause of the error (onboard FIFO overflow).  I'm not sure how I missed the error in my earlier attempts to reproduce this--I'd tried it several dozen times.  I think the problem is that after I disabled the AI for a period of time I never enabled the read again.  Without calling a DAQmx Read that error can't be thrown.  With that being said, I will be escalating this issue appropriately.  I'm not really sure if this is expected behavior or not.  The easiest solution would be to make sure you call at least one read before the FIFO fills up, however, in your case, if Windows is really preventing the loop from running for 2 seconds or more the solution may not be as quick.

If your system requires you to account for 2 second hangs in the software then you may need to consider using a task timing other than hardware timed single point.  From my experimenting it looks like the driver doesn't use DMA (direct memory access) transfers to automatically empty the FIFO with this timing scheme.  I will be checking if this is the case, but I'm fairly certain that it is.  If you were to use a continuous acquisition then DMA transfers would automatically transfer the data from the DAQ card's onboard FIFO to the computers RAM.  You can control the size of the RAM buffer and with a sample rate of 100 Hz you could easily make this FIFO large enough to sustain a 10+ second hang (this buffer can be quite large, you could probably make it 100+ seconds).

On the other hand, while Windows is not deterministic and may have occasional blips in loop rate, a 2 second hang is quite substantial compared to anything I've experienced before.  It may be worth looking for any applications/software you have installed that may be causing this in the background while your program runs.  Your computer's specifications seem more than adequate so there has to be something using all that CPU time.

Thank you for your solid troubleshooting efforts.  I think we've identified the issue, and I will give you an update when I have more information.

Cute dogs!

[Peter_B]

Hi Brooks,
Thanks again for the progress you made.  I tried using DMA as the AI transfer mechanism but error -200734 ensued

"Measurements: Device does not support DMA for the Data Transfer Mechanism when performing non-buffered acquisitions.Set Data Transfer Mechanism toProgrammed I/O. "

(refer to updated atttached project).

The local DAQmx help file says that Programmed I/O transfer mechanism is typically used with Software Timed operations.  I guess atypically Programmed I/O is also used in single point H/W timed acquisition.

"Programmed I/O is a data transfer mechanism in which a buffer is not used and instead the computer reads and writes directly to the device. Software-timed (on-demand) operations typically use programmed I/O."

If I try using interrupts error -200845 is thrown asking me to use DMA !!

Clearly in order to use DMA I need to use finite or continuous acquisition mode so that NI-DAQmx creates a buffer in RAM.  Would that work OK with my current architeecture ?

Re. windows taking control of the CPU for >2 seconds.  It happens very ocassionally and as you say it is rare. However if it ever happens I need to deal with it.   Here are some times when it happens:

• If the screen saver comes on (which locks the PC) then I have to unlock it
• One tab pane in my application has 6 charts with 15 plots distributed in those charts and the chart history buffer length is 1440001 points. Switching to that tab after a time of not viewing it can cause a lengthy redraw (I can probably work around that problem if necessary by using X-scroll bars on the charts).
• Other times are random and I haven't worked out why, perhaps it is the resident corporate antivirus S/W or some other service running in the background
  

If it turns out that DMA transfer does work for me using finite or continuous acquisition then that is good, otherwise I might have to consider purchasing a DAQ card with 4096 FIFO buffer size.

regards

[Brooks_C]

Hi Peter,

Sorry for the delay.  I typed up a response yesterday, but apparently I forgot to submit it. 

From what I've found, this is expected behavior, or at the very least not the intended use for hardware timed single point acquisition.  With hardware timed single point acquisition, it is assumed that the user will be reading every single point--the purpose of this acquisition mode is for control loops where you need to read in every single point as soon as you acquire it.  In that case you wouldn't run into this issue because you would be continually reading. 

There are two ways we can approach this issue; select another acquisition mode (probably continuous sampling) or figure out how to reduce your computers load so that it can sustain operation without experiencing 2 second hangs.  I will discuss both options, but as I would probably switch to continuous sampling.

Continuous Sampling:

By changing the acquisition mode to continuous sampling, automatic DMA transfers will transfer data from the DAQ card's FIFO into a FIFO on RAM.  Since you can choose the size of this buffer you can make sure that you are able to continually sample without erroring out.  This offers a slight benefit over 1 sample (on demand) sampling because continuous sampling will always acquire samples using a hardware clock.  Essentially if you run your loop fast enough to read 1 point each loop iteration with continuous sampling it has the same behavior as hardware timed single point.  In the cases where your loop runs late the DAQmx read will return more than one sample, but you can index out that array and just get a single point from it (or you could decide to keep them all).  The reason buffered acquisition is implemented is so that you can run a hardware timed acquisition reliably on a non-deterministic operating system.  By acquiring samples with a hardware clock and buffering the results you can have reliable data even when your system isn't 100% consistent. 

CPU Load:

To avoid the excessive CPU hangs you may be able to remove the strenuous portions of your code.  Instead of displaying 15 charts with a history of 1440001 points, you may want to consider reducing the history length and writing your data to a file instead.  TDMS files (see example programs) use binary reads and writes so they are quite quick.  Another option would be to use the Defer Front Panel Updates Property to reduce the number of redraws.  Also, I would recommend changing your computers power save settings to avoid screen saver mode.  If your computer goes into power save/screen saver mode while you're acquiring data it isn't all that surprising it had undesirable results. 

While getting another DAQ card with a 4096 buffer may get you around the issue, I think there are other solutions that don't require new hardware.  By utilizing the available functionality of your current hardware and the DAQmx driver you should be able to do what you want. 

Cheers,

Message Edited by Brooks_C on 08-13-2008 10:30 AM

[Peter_B]

Hi Brooks,

Thanks for the summary of your investigations.

My plan is to switch to continuous sampling.  I may not have time to do that straight away as there is a need to validate the S/W in its current state, albeit with a list of known deficiencies.  After that I'll implement continuous sampling.

I know how to deal with the charts slowing things down.  I have actually been pushing the envelope of what I can get away with using my new PC.  There is a benefit to having lots of points on the chart (the user can scroll back through many hours of data for the current test without me having to implement a playback (from HDD) function for now), but certainly not worth the hassle of having long redraw times so I am going to experiment with a trade off of length versus redraw times.  I already stream the data to disk to post processing. Anyway with continuous sampling the redraw delays shouldn't be a problem.  The screensaver is a corporate one that can't be disabled on some PCs (a security feature that locks them after a time of non-use).  If that becomes an issue I can deal with that too by taking the PC off the corporate network and having the screen saver disabled.

 

regards

Peter 

[Brooks_C]

Hello Peter,

That sounds good.  Let me know if there's anything I can help with!

Cheers,