moving large data real-time

[DFGray]

Proverbs also states that plans succeed through a wealth of couselors, so brainstorming is recommended by Solomon.

Back to the problem at hand...

Putnam, can you send your data to the DAQ D/A one line at a time and use the hardware buffer to store it before outputting it?  This may be hideously ineffecient, but it may work.  It will also prevent yet another copy of your data as you buffer it before sending it out.

If you need to buffer it locally, you can assemble it in a preallocated shift register, then take an array subset of the appropriate size to send to the D/A.  Unfortunately, I count three copies of data at this point, and there could be more.  Depending on your memory constrainsts, this could be an issue.  You may get lucky and LabVIEW will optimize out one or two of the copies .

[LV_Pro]

Memory shouldn't be an issue, have 1 Gb between two cores. I do have a question, which I'm having a hard time determining from RT execution trace toolkit (way too much info), which is what kind of hit do a get if I pass the entire maximum size array over (using the queue technique suggested by Ben, tnx!) but tell my DAQmx Timing (Sample Clock).vi that there will be "samples per channel" that is equivalent to the actual length of the desired output waveform vs the maximum array size, then pass the "big" array into the DAQmx Write (Analog 2D I16 NChan NSamp).vi? It seems to be reasonably happy, but I ran it late yesterday and am still trying to decipher the trace info. I'm hoping that the later VI loads the full length array, which in some cases will contain a fairly short bit of data of interest followed by a lot of zeros, and only output the data length defined in the previous "sample clock" vi. If this is the case, then I think I have a winner, although there is a ton of other stuff to deal with!

[DFGray]

You have officially moved out of my area of expertise.  I haven't really used DAQ since before I joined NI.  I will see if I can get one of our DAQ gurus to answer your questions.

[BrowningG]

Hello all,

When you are performing a finite generation, there are two options for timing.  One option is to use the Use Waveform instance for the DAQmx Timing VI.  This option automatically uses the length of the waveform attribute Y to determine the total number of sample clock pulses to generate the waveform.  It also uses the attribute dt to determine the frequency of the sample clock.  The other option is to use the Sample Clock instance of the DAQmx Timing VI.  This allows you to independently set the sample rate and number of samples to generate.  If your waveform is of length 500 and you set samples per channel to be 100, the task will only generate 100 sample clock pulses.  This will only generate the first 100 points of the waveform sent to the DAQmx Write VI. 

During a continuous output task, the samples per channel input does not affect buffer allocation or the number of samples that are generated since the output clock is continuous.  The Use Waveform and Sample Clock instances of the timing VI are used to set the output frequency.  For both finite and continuous generation, the buffer is configured when the DAQmx Write VI is called.  More detail on this topic can be found in the section How Is Buffer Size Determined? in the NI DAQmx Help. 

Message Edited by BrowningG on 12-26-2007 04:44 PM

[Kevin_Price]

Coming a bit late to the party, but FWIW...

LV_Pro: have read through the thread quickly and don't have time to fully digest details now.  Hope I'm not way off track here due to my negligence.  Sounds like you've got a good handle on all the in-place array operations as you perform calcs and accumulate data.  That part sounds like it's ok and the problem comes when you want the other CPU core to start working on the very large accumulated data set.

I *think* the best way to use a Queue or RTFIFO is the following:

1. Send large data array wire into Enqueue function.  Wire should terminate there with no other branches and as a result, your shift-register storage will get reset to an empty array (or perhaps some pre-allocated constant array of appropriate size).  By terminating the wire at the Enqueue function, LabVIEW gives ownership of the memory pointer to the queue without the need for copying the data values.

2. Dequeue from the code running on your DAQ-dedicated CPU core.  The Dequeue operation should now transfer ownership of the memory pointer to the other core, allowing access to data without copying all the memory.

Caveats:  Don't know if you can live with the need to terminate the large dataset at time of Enqueue.  Idea is based on what I *expect* to happen based on single-core non-RT experience.  I haven't actually done this on a multi-core CPU or under RT.

-Kevin P.