Your bandwidth (151 channels at 1.6 kHz) needs aren't trivial but they also seem manageable with some strategic coding changes. Exactly which ones matter most I am not sure, but here are the places I'd explore (in "dataflow" order, not priority order).
1. In your DAQ producer loop, you're reading 101 samples at a time from all tasks. Thus you expect to iterate this loop at about 16 Hz. That's not bad, but I would consider reading *more* samples at a time, reducing the loop rate so that any overhead happens less frequently.
2. Ah, overhead! The use of "Merge Signals" jumps out as a primary suspect. All the forum experts agree that the Dynamic Data wire should be avoided whenever possible. And it's almost always possible. And in your case it's most definitely possible.
I would try other methods to collect all your arrays of waveforms together and try to test which one is most efficient for both CPU (your main issue now) and memory (which can have CPU impact). Here are 3 candidate methods:
- use Build Array in "concatenate" mode (right click the node for this option) to produce a single 1D array of all your waveforms. (My gut feel is that this is likeliest to support the most optimizations downstream).
- use "Bundle by Name" to populate a typedef'ed cluster where each named field is a 1D array of waveforms from a single device.
- use Build Array in regular mode to produce a 2D array of waveforms. (With this method, I'd probably find it a little confusing to keep track of what the rows and columns mean.)
3. A smaller contributor to overhead may be all the queries for "available samples per channel". Even if it's just for troubleshooting purposes, all the tasks should be telling you pretty much the same thing. I would conditionally query only 1 task, I would do the query *after* the read (when the # ought to be quite low), and I would base the condition on something like the Elapsed Time vi with 1 second intervals.
4. In your "Task Starter" vi, your calculation of actual sample rate looks to be probably correct, but it's simpler to query the property SampleClock.Rate which I *know* will be correct. This is just FYI, not part of the problem you're solving.
5. Now let's start looking at your primary consumer, "Slow Write Final". I certainly agree with others to avoid the Express VI, many of them are known to be somewhat to very inefficient, and thus *all* of them fall under immediate suspicion. This will serve the additional purpose of getting rid of the also-troubling Dynamic Data wire.
6. I assume you *want* a fairly low sample rate but your hardware has a 1.6 kHz minimum, thus the inline data reduction where the full-rate data never goes anywhere. To really speed things up, I would use a Decimation function with averaging turned OFF. (You can also experiment with it turned ON, but OFF will be faster.)
7. As suggested earlier, I'd try to make use of For Loop Parallelization to put more CPU cores to work on this data reduction. The independence of each waveform makes this an *excellent* candidate for parallel processing.
8. Your feedback node equality comparison is probably hurting you more than it's helping. With real world signals and double-precision floating point values, it's very unlikely you'll ever get a perfect equality match. And all you stand to save is 1 slow-rate write operation.
It's possible the comparison function has enough smarts to exit early at the first inequality out of the large # of potential comparisons it needs to do. But I don't know that for sure, and I don't think the reward is worth the risk of extra CPU.
I'd reduce the data, probably via decimation without averaging, and send it off to be written every time. This has the further advantage of producing an output file with constant time intervals between samples.
I'd suggest you try ALL these things at once, not just one at a time. Assuming your backlog goes away, you could revisit some of these efficiency enhancements. I think the only one worth messing with is to try decimation *with* averaging, which will have the effect of filtering out some of your signal noise. The other efficiency enhancements shouldn't have any appreciable downside to trade off.
-Kevin P
ALERT! LabVIEW's subscription-only policy came to an end (finally!). Unfortunately, pricing favors the captured and committed over new adopters -- so tread carefully.
