FFT and too little data / small array for low frequencies

[ArthurPauken]

Hi!

with an Arduino I sample Data at 1 MHz. The ADC data is sent in binary chunks of 256 U16 data (ignoring the fact that I actually transmit 512 U8 chunks plus a little synchronizing overhead).

I "applied" the FFT funtion to read any frequencies applied to the ADC. This  works well for frequencies above approx 4000 Hz, but low frequencies are not shown.

I think what is happening that the FFT uses the data from each chunk of data (256 array only) and thus limits the capabilities to calculate lower frequencies than 3906 Hz.

So I suppose that I should implement something like a circular array of several chunks of data. 

From reading this and that I should probably solve this problem somehow using shift registers, but I have not familiarized me with this functionality yet.

1: Can someone please guide me to a solution. An example based on my code would be awesome!

2: Any hint on my code next to my question would be also appreciated. I don't feel that the structure / performance is too good.

3. Any hint for a good Labview tutorial / book in German (preferred) / English would be also appreciated. Up to now I have figured things out with the help of the Examples, Google, this forum etc... but that is not so efficient and probably leads to ignorant questions from my side...

(sorry for not using more professional terms. I am not much of a programmer...)

Thanks - Arthur

[GerdW]

Hi Arthur,

---

@ArthurPauken wrote:

with an Arduino I sample Data at 1 MHz.

---

Do you really reach that samplerate? (I just found this.)

---

@ArthurPauken wrote:

I "applied" the FFT funtion to read any frequencies applied to the ADC. This  works well for frequencies above approx 4000 Hz, but low frequencies are not shown.

---

What do you expect when applying a FFT on just 256 samples with a samplerate of 1MHz? This results in a df of 1MHz/256 = 3906Hz…

---

@ArthurPauken wrote:

So I suppose that I should implement something like a circular array of several chunks of data. 

---

Yes. The more samples you collect the less is df after FFT…

---

@ArthurPauken wrote:

2: Any hint on my code next to my question would be also appreciated. I don't feel that the structure / performance is too good.

---

• You don't need the BytesAtPort node.
• You don't need to open the COM port after using SerialPortInit…
• Do you really need to sync again with each iteration of the outer loop?
• FFT operations are rather slow, so why not use a producer-consumer scheme to separate serial port handling from data analysis?
• Filling 3 charts/graphs also may take some time. Do you really need all of them?
• What's that formula node good for? Right now all it does is hiding relevant information from the programmer as we cannot see the formula. (I suspect the formula is simple enough to use plain G functions instead…
• Are you sure "t_start" and "t_now" are wired correctly?
• Why are there all those coercion dots?

(Keep in mind: I don't use LV2021 so far, so all items are just from looking at this snippet.)

[ArthurPauken]

Hi Gert,

thank you very much for your reply. I saved the code for you as a LV20 file.

I've also come to the conclusion that I need a longer history array for sampling frequencies as low as 1 Hz. But I am still not sure how to practically implement that. The "Unflatten from String" will result in an array of 256 U16 values. Each value represents one sample per 1 µs. Can you please guide me how to practically create such a "rolling" array 3906 times larger than the original array (that should result in enough data from my understanding for low frequency detection). Or do you happen to have a similar example?

I am using an Arduino DUE (not he UNO) which is capable of sampling at 1 MHz. I verified the speed by comparing the incoming data stream with what a decent Hameg Scope / Signal Generator in parallel shows. The code and the Arduino generally speaking seem to sample well at 1 MHz. The performance of such cheap piece of equipment is an my eyes quite astonishing!

The numerous graphs I am using are only to verify what I am doing. Later on I would remove them. The formula node calculates the sampling frequency of the incoming binary stream. As it calculates to 999,9 kHz this sounds quite plausible to me. Nevertheless, you are right that the formula would be easy enough to use plain code.

The Bytes at Port I use to check if my computer processes the data fast enough. It looks like Windows sometimes does not assign enough resources to Labview and I loose data. Though this only happens if I increase the butterworth filter to like the 64th order which was more of an experiment than necessity.

I'll update the other things you mentioned now.

Thanks - Arthur

PS: Are the coercion dots a significant problem? I assumed that inputting an integer is always more efficient even if that function was expecting a double or whatever as an input? Of course I could easily modify most so that no coercion dots would pop up.

[Bob_Schor]

It is easy (and intuitive) to calculate what you need to calculate the lowest frequency of an FFT (which will also be the "frequency spacing" of your spectrum).  Figure out the period associated with this lowest frequency, and collect sufficient data to equal this period.  Thus if you want to have 1 Hz in your spectrum, you need 1/(1 Hz) = 1 second of samples.  If you sample at 1 kHz, you need 1000 samples to have a lowest frequency of 1 Hz, and if you sample at 1 MHz, you need 1 million samples.

Bob Schor

[ArthurPauken]

Hi Bob,

thanks... I understand how to calculate the required sample size. But I don't understand how to create from my existing array with 256 samples an array with 1.000.000 samples.

Basically I have the question how to put the array resulting from "Unflatten the string" 3906 times in a big circular array which updates itself with each new arriving string. I assume that this resulting array can than be easily be fed to the FTT. In C I would probably just code a circular buffer.

Can you give help me with an example?

I am actually quite confident that my question is for most people not totally new to Labview is rather trivial.

Arthur

[GerdW]

Hi Arthur,

---

@ArthurPauken wrote:

But I don't understand how to create from my existing array with 256 samples an array with 1.000.000 samples.

In C I would probably just code a circular buffer.

---

Then use a circular buffer in LabVIEW too!

Initialize an array of 1M elements. Use ReplaceArraySubset to put those 256 new elements into this array. The next iteration you replace the next block of 256 elements. When the end of the buffer is reached you start over from index 0...

(Or you use the PtByPt-DataQueue function, which basically does the same.)

[Bob_Schor]

Do you know the history and the math behind the FFT (or Fast Fourier Transform)?  What makes it "fast" is when the number of samples can be factored "nicely".  The "nicest" (and hence "fastest") number of samples is a power of 2.  So if you took your 256 samples and acquired 4096 of them, concatenating these arrays to make a single Array of 1,048,576 samples, you would optimize the speed of your FFT.

What would the Frequency range be?  The highest frequency is half the sampling frequency, or 500 kHz, and the lowest is 1/period = 1000000/1048576 = 0.95 (approximately) Hz.

Here are two little VIs that run just a little slower than 1 MHz (probably somewhere around 150 kHz) that will generate a random Big Waveform that you can use to test things out.  The inner one is clocked at 256 points/msec (it seems to take a tad longer) and the outer one concatenates the Samples and builds the Waveform.  They are so simple a Snippet should be adequate, even if you have an older version of LabVIEW ...

DEMO Gather Big WaveformUTIL Simulate 1 MHz Sample

 Build Big Waveform by concatenating Samples     Simulate a small sample every ms

Bob Schor

[JÞB]

---

@GerdW wrote:

Hi Arthur,

---

@ArthurPauken wrote:

But I don't understand how to create from my existing array with 256 samples an array with 1.000.000 samples.

In C I would probably just code a circular buffer.

---

Then use a circular buffer in LabVIEW too!

Initialize an array of 1M elements. Use ReplaceArraySubset to put those 256 new elements into this array. The next iteration you replace the next block of 256 elements. When the end of the buffer is reached you start over from index 0...

(Or you use the PtByPt-DataQueue function, which basically does the same.)

---

Much simpler to initialize a 2D array of 256x4096 with NaN on a SR.  Build Array (Concat) with latest reading on top. Delete from array (defaults to removing the bottom row and returns rest of array up top) 

Place that back on the SR .  Reshape to 2^20x1 and run the FFT.  You could even use a channel wire to plot time and frequency domains concurrently while filtering. 

the first iterations will ignore the NaN values and yield a dF, A/K/A RBW,  that eventually converges to <1Hz at 1Ms/s

What I don't understand is why you need >57dB of dynamic range in the frequency domain.  That's unrealistic.