Too much memory & Time used during data save.

[djs_at_eaton]

The attached code is designed to run in a system with:
  1) a 6528 isolated DIO card with a hardware watchdog AND
  2) a 6221 DAQ card.  Among other things.

The real program runs two machines, acquiring data from both and monitoring the output comands to both.  Periodically both machines save a data history.  Since it takes some time to format the data, machine 1 will "snapshot" its data and inform machine 2 that it has done so.  When machine 2 gets to a point in its operation where it can tolerate a long interruption, it signals the "data manager" that it's safe to save the snapshot to disk.  At that time, the watchdog is reset to allow a much longer timeout, the waveform data is appended, the command data interleaved, and the whole batch saved.  After that, the data monitors are reinitialized and the watchdog set to a normal interval.  If necessary, the DAQ resets as well.

Interleaving the data is required because the DAQ runs at 200Hz and the commands are updated at ~50Hz (nominally).

If you don't have the cards in your system, this test has been designed to run with a simulator.  The data isn't pretty because most of the control system isn't here, but it works.

The problem is that sucessive data saves tend to take longer and longer.  Sometimes the OS (W2k) complains it's out of memory and shuts LabVIEW down.  I think there's a memory leak somewhere, but it's not consistent.  It also seems to work differently on WxP.  WxP complains about increasing the page size of virtual memory.  Troubleshooting so far shows most of the time is being used in state "Format WF".  In the real system, without simulating, the operation can take from 4-5 seconds to over 2 minutes.  The ERA section seems to have more trouble than HA, although state "Format WF" is the same for both.

Anyone care to comment?

[devchander]

Guess u missed out attaching your code!

[djs_at_eaton]

Sorry, attached file this time I hope.

[altenbach]

What is the name of the toplevel VI?

[johnsold]

One thing which may help would be to put the data acquisition in separate loops from the processing and saving functions. As your program is structured, the next "Accum" state must wait until the "Save" state is finished. The "Snapshot" state has a subVI which also does both DAQ and Save. So it may take some restructuring of your entire program to achieve this type of parallelism.

I would put all related DAQ tasks into one loop (which could be a subVI of course). Pass commands to the loop and return data (and status) from the loop via queues (or LV2 style functional globals). Put the formatting and array manipulations in a "Processing" loop. Put the user interface into a "GUI" loop with an event structure. Pass commands, parameters, data, and status among the loops via queues. Look at the Producer/Consumer examples.

Lynn

[djs_at_eaton]

Test Data Save 1.vi

I saved the ZIP file to preserve the folder structure.

I think you will also need the attached text file in the same folder as BCU Sim Config 8.vi

This post is now mostly of academic interest.  I've solved, or at least worked around the problem by saving 4 data files fo "ER" and 3 for "HA."  I will write another VI to join the files later.  The newer VIs still take 3-5 mS to run (accumulate), which is marginal for the application.  I don't see how they can be made faster though.  The cycle rate could probably be lowered, and the data rate taken to 100Hz, but I'm fighting response time and some noise filtering issues too.  Higher sample / cycle rates help.

I've got a test program that will run all weekend trying to fill up a hard drive.  If it survives then I'll call it good to install in the real program.