how to optimize the memory for a large labview array?

[altenbach]

You are not quite there yet. The code does not really improve any memory allocations, because you are still no treating the 3D array "in place" (see below in red).

Just glancing at the code, here are a few things I noticed:

• Frame 0: The single sequence frame has no purpose.
• Frame 1: This is a CPU burner loop, spinning millions of times per second calculating the same thing over and over again. Spin the while loop once and then only if "scan params" or "scan" changes value. Use an event structure. You have a lot of duplicate code, for example the loop contains 3 diagram constants of 1000". One if probably sufficient. 🙂 instead of reading from the local variable of  "scan params", just read from the terminal. It is more efficient and more predictable.
• Frame 2: This can be merged with frame 1 so you can wire directly to the parameters coming out of the while loop. No locals needed. Execution is fully determined by dataflow.
• Frame 3: can also be merged with the previous frames. Place the "mikeZ, um" terminal instead of the write local and replace the two locals of it with a wire. The data dependency leading out of the loop eliminates the need for the sequence frames. Same if you place the TRUE leading to "camera" inside the loop. It will not go on until the loop is finished. Read "scan Params" via a wire. There is no need for tic0 indicator, just wire to the place where it is needed in the next frame.
• Frame 4: Instead of a huge diagram constant, use "initialize array". Make it SGL instead of DBL, half the memory! (your original data is only U8, so SGL should be plenty. You already know "Nframes", so you can allocate accurately and don't need to trim in the inner frame 2 and obtain the result of the 3D array out of the right shift register. All the code in the lower right corner of the FOR loop is not needed because all outputs are discarded anyway. You are still slicing and dicing your 3D array like a sushi chef. You are constantly recopying buffers. Keep the 3D array exclusively in the shift register. Do you really need to update the HH indicator with every iteration or would it be sufficient to update it after the FOR loop? Do you even need that indicator at all??
• Frame 5: don't read scan params from local variables. Use a wire to the original control or you get race conditions. For example if the user would modify the control during the run, you'll use "scan params out" that have no relation to the data actually acquired.
• Frame 6: The inner sequence has no purpose. Read from terminals instead of local variables. The "stop" loop needs a wait, it is consuming 100% of the CPU doing nothing. It is very important to eliminate the local variable for HHH 3D". It forces an additional copy of the huge cluster.

[altenbach]

Of course in this particular case, you can create the 3D array by aoutinidexing the 2D arrays at the right FOR loop boundary. This is also very efficient, because the compiler knows the full size before the loop starts.

[8139741635]

Hi,

I attached three vi's: hrec_before is the original scanning vi, hrec_no array initialization is the modified hrec_before without initializing the 3D array, and hrec_array initialization is the modified hrec_before with the 3D array initialization. Also, the zip file contains a *.doc file with the memory performance.

Once again I want to build up a 3D array of N 2D arrays (holograms). N is the number of wavelengths which is equal to the number of holograms. What I have noticed? When I save the data file (3D array) there is not a significant difference in the file size (about 13.0 MB). The array is made of 50x256x256 for 13.0 MB (N=50). It's any way to further improve the program to get a small file size or that is it?Thanks.

[Ben]

Arrays of more than 2D make me nervous since they can grow by a cube factor. Also singel arrays require contiguous memory.

60 single elements queues each containing a 2d array could be used to hold the data. Read the queue when you want that slice, write it back to the same queue after crunching.

[Edit]

Hold it! DF already said that! I guess I owe him a Kudos.

Ben

Message Edited by Ben on 10-10-2008 07:48 AM