Variable sub sampling of data in RT

[falcon98]

HI,

I am acquiring data from pxi 6259 card in RT. I have 32 channels at 10KSPS.

The requirement is that I need to implement variable averaging of data.

Like channel 1 needs to subsampled at 1KSPS(i.e averaged for 10 pts), channel 2 needs to be at 500 Sps (averaged for 20 pts) etc.

The averaging factor is read from a configuration file vary from 1 to 50. This is being done for removing the noise from signals. 

In the end all the signals of card are a part of single 2D array which can be logged directly. SInce the data lengths will be unequal, 0 can be appended at bottom of each channel to make the array lengths same.

Any idea how can I implement this efficiently.

Thanks

[Bob_Schor]

Let the RT part do what it does best (i.e. "acquire"), and postpone the filtering (averaging as you are doing it is a form of low-pass filtering) until you need to "look at the data".  Acquire all 32 channels at 10 kS/s, take the 1D Array of Waveforms (or the 2D Array of <numeric>) and send it off to the Consumer.  No messing with Arrays, padding with zeros, etc.  I recommend that you carry this through to the "save-to-disk" step.

I should point out that I'm a scientist, not an engineer, so to me, "data are sacred".  Who knows, half-way through the Project someone might say "Too bad you averaged every 10 points, we really should have used a 4th-order Butterworth Filter with a cut-off of 2 kHz ...", but if you have the raw data, you can just substitute the filter and (as we like to say in my family), "Bob's Your Uncle".

Bob Schor

[falcon98]

Thanks Bob for the answer.

I am actually doing the same. IN producer loop I am acquiring data and consumer loop I am logging to the file.

Is there any efficient way to implement variable averaging over 2 D array.

Actually I am Engineer working with Scientists, and this is an experimental requirement. Hope you understand.

The data is quite noisy and averaging is able to solve the noise issues, hence I am acquiring at 10KSPS and then averaging to required sampling rates.

[Bob_Schor]

---

@falcon98 wrote:

Is there any efficient way to implement variable averaging over 2 D array.

---

"Efficiency" comes from delaying the Averaging until the very last moment (as I suggested).  I'm presuming that you are producing displays at various display rates.  Since the requirement seems to vary on a Channel-by-channel basis, the way I'd be tempted to do this is to have a function with three input variables -- an Array of Data, the Sampling Rate, and the desired "Output Rate".

Another thing you need to consider (particularly if you are working with someone else's data) is what do they really want to see in the data.  I once was collecting eye position data at 1 kHz and displaying it at 20 Hz, and was averaging 50 points at a time and displaying the averages.  However, what was of interest wasn't the mean eye position (which I was, of course, computing), but the "steadiness" of the eye, which was better represented by simply choosing a single point (I could have chosen a random point, but I chose the first one) and throwing the other 49 away.  The students I was helping were happier with that display ...

Bob Schor