binary file memory usage issue

[Tom WP]

Hi,

I am having major memory problems with the write to binary file VIs.  If I write a large arrays to a large number of files and keep the refnums open LabVIEW seems to keep copies of all the data in memory.  My PC quickly runs out of memory.

I tracked the problem down to the byte order that the data is written in, if it is the native byte order everything is fine, if not, it uses massive amounts of memory.  I understand that LabVIEW has to keep the data in memory while it converts it for writing, but after writing surely it should discard the memory.  This is really poor behaviour!  In the attached example memory_usage.vi, the memory used on my PC (Windows XP) goes from 34MB (native format) to over 180MB (network format).

Having isolated this problem I applied it to my application, only to find it still used all my memory!  This time I tracked it down to writing the data to the file in a cluster, see memory_usage_2.vi.  The write to binary file seems very very ropey!!

Be aware if you run my examples they will use alot of memory.

Has anyone seen this before, or suggest an reason why this is valid behaviour - before I submit it as a bug?

Tom

[Evan]

Tom I agree that what you are seeing doesn't seem right.  I have filed a report so that it can be looked at by the File IO guys.

There is a "workaround" though I must admit I never thought I would suggest this.  Right after you do your write you can close and then re-open the file.  That seems to keep the memory where we expect it.  It doesn't really slow you down too much because in this case it works, rather than taking the computer to a crawl due to no memory.

Thanks for the report.

[Albert.Geven]

Hi Tom

it sure is a bug make an entry in the breakpoint monthly bugs (april 2006) section

http://forums.ni.com/ni/board/message?board.id=BreakPoint&message.id=2033

[chrisger]

hello tom,

can you please post the VIs in 7.1?

[Tom WP]

Same VIs in LabVIEW 7.1 attached....

[chrisger]

hm, to bad, can't run them with 7.1. some missing VIs and connectors... sorry i can't help.

good luck! 

[Tom WP]

Sorry, that was a bit of a red-herring.  The first VI won't have any effect in LabVIEW 7.1 as there was no variable for byte order!

The second VI does have an effect in 7.1, but unfortunately there are memory leaks in both modes, just the rate of memory leakage (?) changes.

Tom.

[Albert.Geven]

Beautiful ICON Chris a pity that this is not allowed in labview
wire is a member of a cycle.

[altenbach]

Somehow, I wonder if the current behavior is actually by design to improve performance under certain usage patterns?

Has anyone tried these on a big endian platform (Linux, Mac)? 

---

Albert Geven wrote: Beautiful ICON Chris a pity that this is not allowed in labview wire is a member of a cycle.

---

You just need to insert a feedback node ;). Or use a shift registers like in the example I made over a year ago:

[johnsold]

Someone from NI posted a comment a while back that the "endian" functions do not work right on Mac OS X. I ran the VIs on LV 8.0.0 and Mac OS X 10.4.6. One version was much slower (virtual memory swapping, I think) and used much more memory. The Profiler did not show the extent of the memory usage. One VI was about 800 KB and the other 2400 KB, while the Activity Monitor showed tens and hundreds of MB while the VIs ran. Does the Profiler only show code size?

Lynn