Memory full at the initialization of a large array (above 200MBytes)

[xred]

Hello,

I get the "memory is full" error message when I try to initialize a large array (but whose memory footprint is much smaller than the amount of available memory).

See the example attached. With this simple VI, I get the "memory is full" error with "dimension size"=200e6, that is to say, with an array of 200e6 integers I16, that is, 400Mbytes. The computer has 4GB of memory, so Labview should be able to use at least 2GB (=typical limit on 32 bits system).

What happens? Is there something I can do in the configuration of Labview?

Thanks a lot.

[Ben]

Memory buffers have to be contiguous physical memory. What does your free memory look like when you have this issue?

Does it have thsi probelme after a fresh re-start of LV?

Ben

[demux]

Greetings xred,

The problem is probably, as Ben described above, that LabVIEW does not have enough contiguous memory available to be able to allocate the array.

By default, LabVIEW is able to access 2GB of memory there is a way to expand this to 3GB (on a 32-bit OS). LabVIEW is Large Address Aware as of version 8.5, earlier version will be fixed to the 2GB limit. More recent versions can break the 2GB barrier. 

This is taken from the LabVIEW 8.6 help file. More background information can be found here. 

The article will describe what to change in the boot.ini file, be sure to take a complete backup of your system before you run the test!

However this will make chances more probable to find a contiguous memory block, it might still not be sufficient depending on how fragmented your memory is. Please go through the Managing Large Data sets in LabVIEW article to get acquainted with the best practices. It will show you how to avoid LabVIEW taking additional copies of your data and describes the practice of chunking.

Kind Regards,

Wouter

National Instruments

Applications Engineering, Northern Region

[xred]

Hello Ben, Demux,

Thank you for your inputs. Unfortunately, I was already aware of these issues:

• I had already tested whether the problem persists after a fresh restart of both Windows and Labview. And it persists.
• I have absolutely no additional copy of my data: I'm testing here a single VI (now attached).

I have now attached the VI example I used. On my machine with Labview 8.6.1, it raises an error with input = 150000000.

Could you please test it?

Thanks a lot for your feedbacks,

Kind regards.

[Jörn]

If your data array has got mutiple dimension, e. g. a series of images can be a 3D-array, you might be able to intruduce a cluster. In the example the image series would become an array of a cluster containing a 2D-array of image data. Now LabView is not looking for continous space for all images at once but only for one image. The individual images of the array can be spread over the none continous available space.

I had this problem. Furthermore I store my images (600 VGA images) in a while loop and access the image data only imagewise. If you would wire such a large array to a SubVI LV would allocate memory for a copy of the data and your application can easily eat up all availabe memory.

[xred]

Dear Jörn,

In my case, I would like to store in RAM a long signal acquisition. The simplest option would be a large 1-D array of I16, as in the example I posted. But in this example, I don't understand how I can be limited to 200MB. I'm pretty convinced it is not related to contiguous memory: the computer has 4GB of memory and the problem occurs (even after a fresh restart of the computer) with the simple VI I posted. 

Of course, I could cut my large array into smaller ones, as you suggested. But, still, the question remains: how large can be the smaller arrays? 200MB seems a bit low with today's computers...

Cheers.

xred

[xred]

Dear Ben,

I have already replied to your answer. But it seems I have replied to the thread and not to your specific answer. 

Thank you for your input. Unfortunately, I was already aware of these issues: I had already tested whether the problem persists after a fresh restart of both Windows and Labview. And it persists.

I have now attached the VI example I used. On my machine with Labview 8.6.1, it raises an error with input = 150000000.

Could you please test it?

Thanks a lot for your feedbacks,

Kind regards.

[xred]

Dear Demux,

I have already replied to your answer. But it seems I have replied to the entire thread instead of replying to your specific answer.

Thank you for your input. Unfortunately, I was already aware of these issues: I have absolutely no additional copy of my data: I'm testing here a single VI (now attached).

I have now attached the VI example I used. On my machine with Labview 8.6.1, it raises an error with input = 150000000.

Could you please test it?

Thanks a lot for your feedbacks,

Kind regards.

[demux]

Greetings xred,

Did you have the chance to adapt the boot.ini file on your system so LabVIEW will access 3GB (instead of the default 2GB)? (be sure to backup the system!)

I cannot run the test since I don't have a system with more then 2GB of RAM at my immediate disposal.

As expected your example also errors out on my personal system since in my case we run into the physical 2GB barrier. On your system however, I think you are running into the software barrier, which you can remove by adapting the boot.ini file.

Kind Regards,

Wouter

National Instruments

[Jörn]

With Windows XP 32bit you cannot use more than about 3 GB of the installed 4 GB. If you have many drivers and Windows stuff running in background from start on I could imagine that this splits the memory.