Help! data manipulation for high speed streaming to disk from multiple boards and multiple channels

[jtsa]

I am using Labview 7.1 and have been trying to capture data from 12 channels simutaneously sampled at 2MS/s each and streaming to disk for up to a minute or more.  The hardware I am using is 2 x PXI 6133 S series boards with a MXI4 link to a Pentium D 2.8 Ghz machine with 2Gb ram.   I have 2 sata drives set up in a raid 0 configuration which should give me hard disk write speed faster or equal to the MXI-4 transfer speed. 

I have first started off by using the example code "multi device sync - analog input- cont acquisition" which has enabled me to sync the two boards and sample at the required speed. 

To stream the data to disk, I have first merged the data from each board  together to save it to one file.  I have tried using the storage vi's but I end up with a Daqmx read error (trying to read data that is no longer available).  I have played around with the read data size to the point that I either get a insufficient memory error, or I get the "trying to read data that is no longer available"  error.  I have also tried using the file IO blocks with some success and have found that I have been able to stream to disk only if I configure the daqmx read block to output the data in "raw 1D I16" format and plugging it into the file-write block.  In doing this, I have noticed that using  multiple channels on one daqmx read task, I will get all the channels in one 1D array rather than a 2D array organized by channels.  This makes it messy to read at the end of this, and I also don't want to write another vi to separate the channels, due to the high chance of getting the data mixed or messed up if I happen to change the number of channels on a board

Is there a cleaner way of streaming this data to disk and keeping the channel data separated from each other?, and/or is there a better way to capture and handle the data I need? 

I have attached the vi which I have got to consistantly work streaming to disk using the raw 1D I16 format.

Thanks in advance to anyone who can help.

[devchander]

Hi,

i can suggest following

1. Refer to an example VI called as "High speed data logger.VI"  in conjunction with "High Speed data logger reader.vi" in Labview examples. Alhrough the logger might be in Tradiditional Daq format, it can be quite easily converted to Daq Mx format to store data in Binary (I32 format) . I have used this for many of my applications and i have found that the data retrieved does not have any "messups".
2. Why not keep a seperate file for each card? This way, you do not have to load your application with extra process. You only have to acquire and save. After saving in Binbary format, you can retrive it offline, convert it to ascii format and merge the data files of various cards to get one consolidated ascii data file.

hope this helps

Regards

Dev

[jtsa]

Hi Dev,

Thanks for the suggestions.

I have managed to find that I can also stream to disk using the unscaled I16 format.  And then merge them together using 'insert into array' this will stream to disk into one file quite well.  The reason i didn't want to put them into two files is that as I understand it, the disk streaming will be a bit slower, due to the writing to two different parts of the disk.

I started off not using the 'high speed datalogger.vi' mainly because I wanted to synchronise the timing of the capture from each board.

-jtsa