Limit in waveform data size?

[Armánd]

Here is the situation:
I have to process files which contains records.
These records contain data of the waveform datatype.
I need to append the record to eachother, using a (while)loop, appending the Y values only as the rest of the attributes and values are the same.

The problem:
If i try to process large files like say 670Mb the appending works for a while but stops at some point with the remark "Not enough memory to complete operation"
The icon is flashing where i append the Y array starts blinking (the append array icon form tha array tool pallet).
Files with a size up to 200Mb do not generate this error, i do not have any files with a size in between.

I also tried to reserve space up front based on the number of records.
I used the 'intialise array' for this but it also failed with the same remark


Question:
- Is there a limit in the amount of data that can be but into wavevorm data type instance
- Is there a limit in the append function
- Am i getting stuck on limitations on memory in Labview or windows

I use a P4 machine with 1Gb of physical RAM

[Greg_McKaskle2]

> Question:
> - Is there a limit in the amount of data that can be but into wavevorm
> data type instance
> - Is there a limit in the append function
> - Am i getting stuck on limitations on memory in Labview or windows
>

You probably want to use the task manager or other Windows tools to see
just how much memory is being used. My guess is that you get the
message when the process hits 2G of virtual memory, and that is the
limit imposed by the OS. Keep in mind that when you are manipulating
large arrays that copies can be made, and to display them copies must be
made. These can be minimized to a point, but at some point the
convenience of dealing with a simple arrays takes too much memory and
you need to operate in blocks or on decimated data. T
here are some
pretty good articles on dealing with large arrays on the NI web site.

Greg McKaskle

[chutla]

Dear Carnivorus:

I would venture to say that perhaps there is not enough memory to complete the operation! 🙂 All snide aside, working with large arrays takes some serious number crunching. You might try twiddling your swap file size (on the Advanced properties tab of My Computer in win2K and later). However, you're probably bumping up against the absolute limit of the OS.
If there is ANY way of dividing and conquering your data so as to not have to deal with such a large array, I think you should invistigate it. YOu may find you don't need anywhere near the samples you think you do. (voice of experience,here!) Or perhaps you don't need the dynamic range. (On most DAQ cards, you can't change the number of bits, but you might be able to use
a card with fewer than you're currently using)

Eric