producer/consumer misses data when writing to file

[CFJ]

Hi,

The attached code is intended to log data from a 9237 and 9215 module over a 12 hour test. To limit the output file size and to clear the buffer(?), I set a new file to be opened every 30mins. Due to the data rate limitation of the 9237 I collect at 2000Hz and use "align and resample" to reduce the data to 200Hz. The structure is producer/consumer.

I currently have two problems:

1) If I collect at 2000Hz, I need to set the Samples per Channel to 2000 also, otherwise there is a lag in data written to file, and when the new file is opened, it is writing data from the last period of sampling, not the current one. I'm worried that when setting samples per channel to 2000 (a) there will still be a lag in the data written to file, (b) there may be confusion if the samples per channel "buffer read" gets faster than the sampling rate and hence there isn't a sample ready to be read. Can someone tell me if either of these are a concern, and if so, is there a solution?

2) The file that is written misses a lot of data, I believe this is a problem in the consumer loop rather than the producer loop. I have attached a text file that shows what happens in a 1 minute file write. Is this a problem with how the computer is allocating memory? Can someone suggest a solution to this?

I have read that putting a "wait" of 1 ms in the consumer loop will help speed of writing - will this lead to data ending up in the "wrong" file? This doesn't appear to reduce the amount of data "lost" in my program.

Thanks, as always, for your help.

Claire.

[RavensFan]

You might have an issue with dataflow in the consumer loop.

When the loop iteration starts, part of your code will pause at the dequeue functions waiting for the data to come in.  However, in the lower part of your loop, the elapsed time, comparisons, and open/close file functions will happen immediately.  Those aren't waiting on the dequeue function.  You may want to put of your lower loop in a sequence frame that can't execute until after an item as been dequeued.

[CFJ]

Hi RavensFan,

Thanks for your reply. I've placed the dequeue(1) and the resampling/open (2) and file close (3) in separate sequence frames within the consumer loop (updated VI attached). This doesn't seem to have any effect on the data written - even the space at the top of the file remains, which puzzles me because I thought the sequence structure would at the very least fix that.

I've also just tried removing the Align and Resample block as I thought that might be slowing things down, but it doesn't affect the result.

Any other ideas of things to try?

Thanks,

Claire.

[Hesloppy]

Hi Claire,

Have you tried just using the Export Waveforms to Spreadsheet File without the Producer/Consumer architecture?  What happens when you just acquire a signal and write it to the spreadsheet?

What version of LabVIEW are you using?

Best Regards,

[CFJ]

Hi Bryan, yes I have tried this previously. Using this approach the buffer has overwrite errors, and labview reports that it has run out of memory.

I'm using LV 8.6

Thanks, 

Claire.

[RavensFan]

What have you changed in your Write Waveform vi's that you renamed them _cfj?

[CFJ]

I haven't changed anything (to my knowledge), I just split up the 1D write waveform VI (made subVIs from the standard VI) so I could open, close and write in the different parts of the loops.