Logging data at high sampling rate using producer-consumer loop

[sm81]

I want to achieve a seemingly simple task of acquiring and logging data at sampling rate up to 1MS/s (so quite high). I have experiemented with a few approaches. I have attached two different VIs, both uses a ni USB 5133 scope to acquire data. In one VI, I attempted writing binary file, assuming it is the simplest file type. This results in a file that is not readable at all! 

The second VI uses an express VI block in the consumer loop, saving data in a .lvm file but this is slow enough to tile  blocks of data properly in the file and results in a loss of data between each block (if it acquires 20000 samples then between two blocks of 20K data, there is discrepency).

Could I get some help to correct this last bit, please?

(Both VI uses a subVI which is added below)

[crossrulz]

How would you like to read the data file?

Binary is the smallest file type, but you have to make a separate VI to read the file and display the results.

The LVM is just a text file.  And yes, it will be slow.  But you can read the result in a text editor.

It all depends on your exact requirements.

[sm81]

1. Do you reckon the VI has everything to write into a binary file?

Ideally I would like to be able to use MATLAB to read the measurement file. Although wouldn't mind if I use LabVIEW if it can be done easily.

2. I didn't want to use the lvm format because i know it may be inadequate for fast acquisitions.

I really don't mind as long as I can save and read data reliably. Your inputs will be very useful!

Thanks

[crossrulz]

I would save it off as a binary file (use Write to Binary File).  This will be the fastest way to write the data.  You want this since you have a lot of data that you can't miss.

To make something Matlab can read, you could make a utility VI to convert this binary file to a CSV.  But I prefer just using LabVIEW.

[sm81]

So what about my first question- Does my VI has everything to write into a binary file?

[crossrulz]

Yes.

Looking at the setup a little closer, I would actualy use 2 files, one for each waveform.  All you would need to write is the Y array of each waveform directly to the Write to Binary File and I recommend setting the "Prepend String/Array Size" to FALSE.

[sm81]

Please take a look at this VI, followed from your advice.

There is an example code to read from binary file, I am going to try. If it does, I will be happy.

Thanks for your help so far!

[crossrulz]

I would just use the same consumer loop.  You will need a separate Open File, Write to Binary File, and Close File for each file you are creating.

[sm81]

I almost thought the same as soon as I posted the last message. Thanks eversomuch!

However, I suspect I still have some discripency in the saved data. I am using the following readily available example VI to read my files. If you look at the zoomed in data you can see what I mean. The acquisition loop is taking 2000 samples per iteration which is lot smaller than the sample rate (1MS/s). 

Any suggestion? This should solve this thread once for all.