PXI-6624 and PFI's

[SnackNStack]

I've got a PXIe/PXI DAQ containing one PXI-6624 counter/timer card.  The card measures six different incoming frequencies.  Everything works fine, until the frequency on two of the inputs (hardware redundancy) reaches about 6 kHz, when it appears that Ethernet communication between my DAQ and computer breakdown.  I've gone through my wiring, but can essentially remove everything and reproduce this with a function generator wired directly into my SCB100a terminal box. 

Eventually, I noticed in our software setup that the two spares on the pxi-6624 card were set up with the same PFI values as the two that fail at 6kHz.  I changed those to the SOURCE values dictated in the SCB100a user guide (all the others use their associated SOURCE values,) but the DAQ becomes unhappy, telling me those values are already in use.  I ended up changing the two spares to their GATE PFI values, and that let's me get up to 8 or 9 KHz before occasional, one second drop outs occur.  Much better, but I still feel I'm doing something wrong, as this card can supposedly do up to 400 kHz.  

I'm not specifying PFI usage anywhere else in my hardware setup and can't quite tell what's interfering with the pxi-6624. Any suggestions as to how to go about this?  

[Kevin_Price]

Can you post your code, preferably back-saved to LV 2020 or so?   

I'm unclear on why you bring up Ethernet communication -- are you running an RT controller on the PXI system and communicating over Ethernet back to a host PC?   If so, I'd recommend a first step of *simulating* the frequency data while testing whether your communication code can sustain 10 kHz/chan data bandwidth across the Ethernet link.

-Kevin P

[SnackNStack]

Hey Kevin, thanks for responding.  I'm working with a retired engineer's Labview codebase, and it's reached monster size, but I'll see if I can pull out the snippet that's applicable here and post if I can.  To answer your other questions, yes, we have a pc running our Labview application, which is Ethernet networked with a PXIe-1088 chassis with a PXIe-8821RT controller, a couple other cards (6704, 4302, 4300, 4353,4357,) and the PXI-6624.  Everything's been working great for months at lower frequencies until two of our frequency inputs reached up to around 6 kHz, then our software reported Ethernet COM loss.  I could verify this by bringing up the pc's Windows Network traffic graph and watch full communication dropouts for a split second, occuring repeatedly, and more quickly as the frequency rose.  I suspected ground loops, mis-wiring, etc (I'm the hardware engineer on this) but eventually realized that I could repeat the problem with a function generator pumped directly into the card at the SCB100a.  Anyways, we program the cards initially with a config file.  We specifically program each 6624 channel with the following type of block:

#PXI-6624 Card 6, Ch 0:NGG 1
[SCANITEM1]
Channels=PXI1Slot7/ctr0
Type=CI Frequency
CI-Freq-Term=/PXI1Slot7/PFI39
Range=2.0 12000.0
Units=Hz
Index=0
Task=AI3A

I finally noticed that the "PXI1Slot7PFIXX" numbers on the first two channels were re-used on the last two channel spares.  I guess our engineer felt it needed to be sampling something.  Changing the spares numbers solved the COM problem, but I'm confused what else the system is using in terms of PFIs.  The numbers I tried were the SOURCE numbers for channels 6 and 7 out of the PXI-6624 manual and SCB100a guide.  I'll look into simulating with my software engineer tomorrow and post code if I can.    Thank you.

[Kevin_Price]

I'm not a wizard on network communication issues.  Given the dropouts you see, my main thought is that perhaps excess CPU usage on the RT (producer) side is preventing the timely transmission of data across the Ethernet link.  Or maybe the Ethernet comms-related code is being starved of incoming data so it has nothing to transmit.

How much other data is being acquired and transmitted by other devices?  Do ALL of them drop out or only your frequency data?   Can you find places where the RT code might need to perform new memory allocation?

-Kevin P

[SnackNStack]

Good morning, Kevin,

    I'm no network communication specialist, myself, so anything I say should certainly be taken with a grain of salt, but I think the network dropout was a symptom of resource allocation.  After I changed the two 6624's spare channels to not share PFI values with the first two in-use channels, the network problem ceased to be an issue.  We read from all the cards I mentioned yesterday and sample all the channels at a rate of about 100 times a second.  With the Ethernet dropout, everything stopped talking, but now that PFI's are different, everything works fine, it's just that the 6624's couple of channels read most of the time, with a 0 value set every couple seconds for about one second. So, you'll get valid data for a few seconds, then suddenly a string of around 100 zeros, then back to valid data. Our pc still picks up the values, just that it's zero.  So, I'm thinking this is a PFI, what-clock-am-I-using-per 6624 channel issue.  I haven't quite figured out PFIs yet, as the different numbers for each channel seem to dictate whether it's a source, gate clk, or an output.  I use our channels as sources and rely on an internal clock.  I gather there are a few internal clock options, but haven't ascertained how to change that yet.  Thanks again.   

[SnackNStack]

I should also clarify that the patch of zero data also only starts around 8 kHz, and gets significantly worse the higher I go in frequency.  

[Kevin_Price]

From over here, those symptoms sound like coding issues -- namely, inefficient handling of tasks and/or data.  I'd like to see the code for these frequency measurement tasks, including the method for moving that data to the host.  (Please back-save if necessary to LV 2020 or earlier).

-Kevin P

[SnackNStack]

My software engineer passed alone the vi he says is responsible for acquiring the frequency card information.  I asked for the LV 2020 back save, but am not sure if this is or not.  

What he's told me of our data sampling, we request data 100 times per second.  We do that regardless of whether our accelerometers connected to the PXI-6624 get signals at 0 Hz or 10 kHz.

If there's anything else I can provide to help, please let me know.  

Thank you,

  Nick

[Kevin_Price]

It's LV 2022 and the block diagram is password protected.  Nonetheless, I think I have an answer for you after reviewing the spec sheet.

Your device only has 3 DMA channels available for data transfer so only 3 of the tasks get to use (fast, efficient) DMA.  Any additional tasks will have to use interrupts.  And as the incoming frequencies increase, so does the interrupt rate and subsequent demands on the system.

One possible solution is to deploy a 2nd 6624 in your chassis.  Each would have 3 DMA channels so you could do all 6 frequency measurements without bogging down the system with interrupts.   Yes the devices are pricey, but so is time wasted on tests that can't be made reliable.

Barring that, perhaps you could describe the nature of these frequencies you measure.  What do they represent and mean?  Are they all independent or is there some correlation or causation among them?  What range of frequency?  How much quantization error would be acceptable -- 1 part in 100, 1 part in 10000?   What led to the decision to sample at 100 Hz?

Counters are pretty flexible.  Sometimes there are indirect ways to get "good enough" information when direct measurements aren't an option.

-Kevin P

[SnackNStack]

Sorry about the LV and password.  I can try this again tomorrow.  

The DMA element is interesting.  I won't rule out a second card, but since I'm here without one, may as well experiment if I can.  Here's a rundown of the signals we've got in place.  

CH 0 and CH 1: Engine shaft rotational speed sensors, coil and magnet type.  0 - 10 kHz range.  Should report same information.

CH 2 and CH 3: Secondary shaft rotational speed sensors, also coil and magnet type.  0-10 kHz range, though I'm told this shaft never reaches the same speed as the first shaft.  Should report same information.

Ch 4 and CH 5: Two fuel flow sensors plumbed in series.  I assume these signals are electronically generated off flow, since it has local on-board smarts.  As flow increases, so does frequency.  0 - 10 kHz.  Should report same information.

I don't know the original reasons for the 100 Hz sampling, but this project was carried forward from an older SCXI NI system, so older hardware limitations, perhaps?  As for necessity, I don't believe sampling this fast is absolutely necessary. Most data I've seen from them has a lot of repeating values, indicating to me that it samples faster than it actually changes. We record it for a file to view later, display it on a screen, and are capable of averaging values before displaying if we need to. 

The CH 0 and CH 1 are the places I was seeing zero dips, which I believe is the highest frequency inputs among the six.  Do you know of a way to assign DMA specifically to particular channels for priority?   

Thank you very much for your help.  I certainly appreciate it.    

  Nick