FPGA DMA Fifo Null Data Returned

[VicEngineer]

I have a strange problem, I checked the forums but I didn't find anything similar.

Project description:

I have two similar VI's running on two seperate cRio Chassis's cRio 9068 & 9067. The 9067 is the slave chassis running a data acquisition loop on 26 channels passing data via DMA Fifo (Target to Host) to the RT target on the 9068 Chassis. The 9068  is also running a data acquisition loop acquiring 26 channels of data via DMA fifo from the FPGA target on the 9068 chasis. Each cycle the FPGA targets return 1613 samples per channel to the RT target

Problem description:

Both data acquisition loops are running and functioning as expected. I've been working with these VIs for the past few months, I recently added in code to perform shunt calibration for the strain bridge modules of the NI 9237 modules. For all intents and purposes it appears as though the 9068 chasis is running and returning data to the RT target at a rate consistent with previous testing. However all the data returned is null there is no time information there is no data in the returned arrays from the 9068 FPGA. The 9067 chassis is running and returning data as expected. Neither FPGA is throwing any errors, or having DMA fifo over/under flows. I should also note that if I use a breakpoint and a probe in the data acquisition VI the very first cycle returns data and every subsequent cycle returns null data. I know the FPGA data acquisition loop is returning elements because the front panel displays a changing % of RT data buffer, which is consistent with previous testing.

Has anyone run into a similar situation where a data acquisition loop runs but does not return data?

The screenshots are of the two FPGA front panels and the RT front panel.

The zip file attached is the project folder (sorry there may be a VI or two missing, I have a locked VI somewhere in the project and I cannot "Save As" the project in order to relocate it to a new folder)

[speleato]

Hey VicEngineer,

That is indeed a very strange failure. Just a couple questions to start,

- What version of LabVIEW are you using?

- Do you still have the bitfile from before shunt calibration was added?

- Have you re-compiled the bitfile since adding shut calibration?

Thanks!

Sebastian

[VicEngineer]

I'm using the latest vesion of Labview.

Labview myRIO 2014 Version 14.0.1 (32-Bit) with the most recent updates and service packs (I updated the other day in hopes to rectify this problem, it had no effect)

Yes I still have the bitfile and project prior to shunt calibration, it functions as expected.

I have recompiled many times, with several different VI configurations for sequencing the calibration loop and the main data acquisition loop. All in attempts to rectify this strange problem.

I thought it was perhaps an issue with having two loops writing to the same FIFO, although they never run at the same time. So I created two seperate FIFO in the project one for continous data acquisition and one for the calibration loop. This also had no effect on the issue of the data acquisition loop not returning data.

The calibration loop does return data in the proper format with measurements within expected magnitude from the sensors. If the VI data acquisition loops are compared for the Slave FPGA(9067) and the Master (9068 - currently broken) they are virtually identical save for some changes with different module types(But this is the same configuration as the project without strain gauge calibration which functions), but the overall structure of the VI and the Data pipelineing is the same.

I'm at a loss as to why this is occuring, previously with this project if the acquisition was up and running the data was always valid so I'm not sure as to what has changed to cause this issue.

[speleato]

A couple more questions,

By null data, do you mean that the data returned is all zeros. It sounds like elements are moving through the system but they are just zero, is that right?

I took a quick look at your code, I assume on the Master system you are talking about the DataU32 FIFO. Is that right? Does that mean Slave_Data and READ_FIFO are working as expected?

One thing to check, have you verified that the values you are writing in to the DMA FIFO on the FPGA side are what you expect? Maybe put an indicator on the diagram and read it from the host to see if sane looking values are being written. That'd help us know if it's a problem in the FIFO itself or something on the diagram.

Sebastian

[VicEngineer]

By null data, I mean if I put a breakpoint and a probe on the DataU32 Fifo.Read method in the RT code the arrays are empty, the probe watch window show [ ] for the DataU32 array, even though the read function is showing that there is elements being read and elements remaining to be read.

That is correct. The data acquisition FIFO is DataU32 for the DMA Fifo on the 9068 Master Chasis(sorry i need to rename this to SGL now). Yes the Slave_Data FIFO is working as expected returning values to the RT target on the 9068 chassis, this is passed via direct FIFO access over ethernet.

The FPGA is currently compiling, it will be ~50 minutes before this iteration completes. After I check this current version I'll add a numberic display to the output of the data acquisition array in the FPGA to check that the data values are as expected before entering the DMA Fifo. 

I've also tried compiling the FPGA code with different strategies: Reduced compilation time, Performance, Routing Constraints, Power etc... they have the same result on the FPGA execution.

[VicEngineer]

It appears that the data is getting lost somewhere in the RT code. Thanks for the tip about investigating the FPGA with numeric indicators. Still not sure why when I investigated with a probe it shows the Array as null.

The FPGA seems to be functioning properly, the data appears to disappear in the rwfm_AcqRead(Wfm).vi I added numeric indicators after the data is returned from the FPGA. I think it may have something to do with the "First Read" variable and the waveform initialization.  I'll post once I figure out where the program is going wrong.

It seems to be a problem with making the SGL data a waveform signal.