allocating extremely large arrays (>1GB)

[mkdieric]

Hi,

we have an application that needs an extremely large array of singles in memory (array sizes up to 1.6 GB, preferably more, size varies)
With LabVIEW we can fill the memory only up to 1 GB (appr. 750 MB array, rest = OS) even though 4 GB RAM is present.
With C we can go up to the Windows defined limit of 2 GB (appr. 1.8 GB array, rest = OS), so RAM fragmentation is not the cause of the 1 GB limit.

Question 1 : does anyone know why the 1 GB limit exists in LabVIEW?
Question 2 : does anyone know a way around this
Question 3 : could we do the allocation in C and then let LabVIEW access this memory space?

Thanks in advance,

Manu.

[chrisger]

hi manuel

you could use a binary file pointer instead of an array, but navigating through the data would then become much slower.

best regards
chris

[mkdieric]

Thanks for the suggestion, but as you say, via harddisk is much, much slower, too slow for our application.
We really wonder why the 1 GB limit is there, allocating the space in C works fine until 2 GB.

[altenbach]

Manuel,

You should make sure that your code does not genereate extra data copies. For a detailed discussion and guidelines, have a look at Managing Large Data Sets in LabVIEW, especially the section to reduce data copies.

Shrink your data to 10%, then carefully monitor memory usage and buffer allocations. See how many data copies there are in memory.

[chrisger]

hi manuel

what datatypes do you use? how about single precision/short integer instead of double precision/long integer?

best regards
chris

[mkdieric]

Hi,

we made sure that no extra copies are generated. We regularly have an array with 700 MB, and it is clear from the task mamager that this is allocated only once. In response to the other reply, the datatype is a single precision float (SGL). We need the float notation.

Manu