write to binary file

[J.Mamakos]

I have a relatively large 3D array (Phase Settling), which I want to store in a Binary File. The array is a 15000 point sweep (after decimation), repeated up to 48 by 48 times (in two FOR loops, one inside the other), meaning it can end up having as many as 34,560,000 data points. At the moment, I am attempting to dump that 3D array into a Binary File in one go, using the following VI. There are two possile versions of this array (both the same size as each other) and, depending on which has just been tested, they will be written to their own individual file. Once both tests have been run and their results stored, the idea was to open both files, subtract one array from the other, and store the new 3D array to another Binary File whilst also displaying this latest data elsewhere in a higher-level VI.

After its first trial run, LabVIEW presents me with an error message saying something along the lines of "LabVIEW had run out of memory". I assume it would be to do with this VI, since that's where the first error code appeared (Error Code: 2), apparently originating in a 'Write to Binary File' function.

Any ideas as to how I could get round this first hurdle, or perhaps an alternative method I could try?

Thank you.

- James

Message Edited by James Mamakos on 04-08-2010 10:07 AM

[Norbert_B]

Hm, you say that you have about 34M data values of doubles, so 8 Byte each. So a single 3D array takes up 272MB of RAM. You have at least 5 copies in your code (True Case), so this will eat up more than 1GB.

Since arrays need to be in RAM contiguous and RAM is most often fragmented by default, you simply run out of available junks of appropriate size......

The only thing i am not sure about: Why should Write to Binary create error 2? Is it always creating the error or only if you execute the VI more than once after loading?

Norbert

[J.Mamakos]

Well, as shown in the attached screen-shot, the error indicator shows an error code of 2, and that the error occurred in one of the Write to Binary File functions.

However, when trying to save this particular VI after a couple of tweaks, I was presented with the following pop-up alert message:

LabVIEW:  Memory is full.

Cannot save VI "Compare Amp - calculate.vi".
LabVIEW Save error code 10: Default data space.

Anyway, returning back to the main problem at hand, I've though of a couple of possible solutions...

1. We don't really need a particularly high resolution to the data points for the comparison, so I could convert the array doubles into an array of singles to save space.
2. I should be able to implement the 'Write to Binary File' for the initial arrays into the outer FOR loop, thereby reducing the amount data written in each instance by a factor of up to 48. The only problem I can foresee in this is that I'm not sure how to append 2D arrays into a binary file such that, when the file is read, they would appear as (or be easily converted into) a 3D array.

As for the 'True' case in my case structure, am I being grossly inefficient in my passing of data from one place to the next? I know there are a lot of branches (and buffers?) of 3D arrays; is there a decent way to reduce this overhead by tweaking the data paths or something?

- James

P.S. The computer I'm using has 1GB RAM.

Message Edited by James Mamakos on 04-08-2010 12:39 PM

[J.Mamakos]

---

James Mamakos wrote:

1. I should be able to implement the 'Write to Binary File' for the initial arrays into the outer FOR loop, thereby reducing the amount data written in each instance by a factor of up to 48. The only problem I can foresee in this is that I'm not sure how to append 2Darrays into a binary file such that, when the file is read, they would appear as (or be easily converted into) a 3D array.

---

Anybody have any ideas for addressing this problem, or if it's even worth implementing?

Thank you.

- James

P.S. Sorry if I appear rude or impatient; just that my lack of foresight in this is pulling me behind schedule...!

[Norbert_B]

Norbert