How full is my DMA FIFO from the FPGA?

[wz2b]

Can you tell how many data elements are in an FPGA fifo by issuing a READ of 0 elements?  And is that horribly inefficient?

My situation is that I want to process this in a sub-vi.  I want to read as many items as possible, but I want to avoid doing it in a loop because doing it in a loop would mean I had to splice (append) arrays on every loop iteration, which sounds like a lot of reallocating and copying would be going on under the hood.

[Bob_Schor · Accepted solution]

I just had the same problem (except I definitely did it in a loop, without having to worry about allocation, explained below).  I don't know whether it is "recommended", but I did exactly what you asked about, namely put a Read of 0 and asked how many elements were remaining.

In my case, the FPGA was executing a loop, producing 4 Arrays of 16 U16s (from a 4-channel A/D chip) , putting each Array on an FPGA FIFO.  My LabVIEW RT loop was running in a nested set of For loops, the outer representing the number of Samples I expected, then the number of Channels (4), and the inner loop asked to read 16 Elements (which was the number put on by the FPGA).

To avoid having to allocate memory, I created a Buffer in memory pointed to by a Data Value Reference (DVR), and used the In-Place Element structure to sock my data (coming in at about 10 kHz for 4 channels) away during acquisition.  This, of course, requires that you know the maximum size of your data sample, and have enough memory to hold it all at once.

Bob Schor 

[wz2b]

Thanks, Bob.  It does seem to work, I just wasn't sure if it was a bad idea for some other reason.  I'm going to look at making a pre-allocated Buffer like you say -- I do know the maximum bounds of the FIFO, at least as defined in the project.  There is something in the documentation that makes it sound like the buffer on the RT side can actually be a different size, which is perplexing.  That might throw a wrench in the works, not sure yet.

[Terry_ALE]

There is no get status function.  Calling Read zero elements never seemed to slow things down for me.  What kind of rates are we talking about?

The "High Throughput Streaming.lvproj" shipping example is capable of transfers of up to 2 GB/sec.

Read Region minimizes data copies https://zone.ni.com/reference/en-XX/help/371599P-01/lvfpgahost/fpga_method_fifo_acqread/

One thing I have been doing lately is having more than one FIFO so that I do not need to de-interleave data on the host side of things.

[wz2b]

Thank you!

My data rate is 51.2 kS/s * 24 bits * 8 channels.  I do not know when you specify an FXP(S,24, n) if in the FIFO it is actually 24 bits or if it 32-bit aligns.  I'm guessing it 32-bit aligns.  So that's 1.6 megabytes per second, a fraction of what you just told me is possible.

At the receiving end, I need to do a little formatting and get this data into a TDMS file.  The trick so far has been figuring out how much to read at a time.

I think I need a better understanding about how these FIFOs work:

A DMA channel consists of two FIFO buffers: one on the host computer and one on the FPGA target. After creating a DMA FIFO, you write block diagram code to write data to, and read data from, the appropriate buffer. For example, if you are transferring data from the FPGA to the host, you write code on the FPGA that writes data to the buffer. You also write code on the host that reads data from the buffer.

Does that mean that once you "start" the FIFO that the transfers from the target buffer to the host buffer are automatic, hardware-triggered transfers, and that I can make the host buffer huge and not have any more problems.  The thing that perplexes me about that is the 'overflow' on the FPGA write fifo side.  It seems like that overflow would not have any idea what's happening on the host buffer side and would not see host buffer overflows.

[Terry_ALE]

Rates depends on hardware and configuration.  Newer chassis/cards can support something like 24 GB/sec.

[Intaris]

---

@wz2b wrote:

Thank you!

My data rate is 51.2 kS/s * 24 bits * 8 channels.  I do not know when you specify an FXP(S,24, n) if in the FIFO it is actually 24 bits or if it 32-bit aligns.  I'm guessing it 32-bit aligns.  So that's 1.6 megabytes per second, a fraction of what you just told me is possible.

At the receiving end, I need to do a little formatting and get this data into a TDMS file.  The trick so far has been figuring out how much to read at a time.

I think I need a better understanding about how these FIFOs work:

A DMA channel consists of two FIFO buffers: one on the host computer and one on the FPGA target. After creating a DMA FIFO, you write block diagram code to write data to, and read data from, the appropriate buffer. For example, if you are transferring data from the FPGA to the host, you write code on the FPGA that writes data to the buffer. You also write code on the host that reads data from the buffer.

Does that mean that once you "start" the FIFO that the transfers from the target buffer to the host buffer are automatic, hardware-triggered transfers, and that I can make the host buffer huge and not have any more problems.

---

Short answer, Yes. It is possible to set the buffer in Fabric to actually be quite small. The data is automatically transferred to the RT-side buffer "in the background" and at a high rate. I have DMA transfers which produce 128bit data at 1MHz (with up to 1 million datapoints, that's 16 Megabytes of data), but my buffer FGA-side is only something like 30 elements. And even that is generous. When testing, I never actually needed more than 6 or 7 elements to guarantee no data loss. The buffer on the RT-side is enough to hold multiple data sets (and is always a precise multiple of my maximum dataset.