labview memory

[smercurio_fc]

I'm only going to say this once, so pay attention.....

DO NOT POST BITMAPS TO THE FORUM!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

DO NOT CHANGE THE EXTENSION OF THE FILENAME TO GET AROUND THE BAN!!!!!!!!!!!!!!!!!!!!!!

[smercurio_fc]

---

@otiose wrote:

Hi,

Just to put my problem in the picture. The memory issues causes the PF usage to build up. Please check the attached the picture. 

This was the result of running the VI. I never get back the 11 MB difference. this difference increases when bigger files are used.

---

You are looking at the overall usage. You should really be looking at the memory usage of the LabVIEW process.

Your .csv format makes absolutely no sense to me. You're prepending the single element of "Data label 2" to the first row of the first page of "Output Spectra", and then it appeared that you were prepending the elements of "Data label values 2" to the other rows of Output Spectra, but then I just got lost. It was just a big jumbled mess. Please explain the format of the file more clearly.

Also, your filepath control for the file to write restricts you to existing files, so the way the VI is coded right now, you will always be appending to the file, even if it exists. This will create larger and large .csv files. Are you sure this is what you want?

[otiose]

HI smercurio_fc ..

thank you for looking into the problem.

the change of file extension was required because data is only collected in .bin format and NI Forum does not allow to upload .bin file. I will try to find the right way for posting such files.

Memory Issue

The problem with memory comes when I start to use larger data files. the data file could be as big as 1gb to 2gb in future. this fills up the pf usage memory and the computer becomes very slow. I also can't run any other programs then as ram goes low. using deallocation vi only gives a bit memory back but not all it. is there a way to recover this memory?

Could you please explaing what do you mean by labview processes and how to optimize the vi accordingly?

CSV format 

I have to convert the .bin data file into csv file using this software. the .bin is written as an array of cluster. this is the csv format required at the moment. this cluster has 3 elements. i will try to explain the csv format based on these three elements.

Each data file has only one data label. after that I have an array of about 100(in this case) data labels 2. the output spectra is 3d array which has as many elements as the data labels. each element in the 3d array is an array of 7(1x512 elements).

or I will try the other way around, which ever helps. i collect data as an array of 512 elements(1x512). for a complete single data capture i need to collect 7 such arrays. this forms an element of 2d array( 7x512). then i keep on adding these 2d elements which gives me a 3d array(100x7x512 for the data file attached). this forms the output spectra.  these 2d elements are captured at specifc intervals as represented by the data labels 2( array of 100x1). so there are as many elements in the 3d array as the number of elements in data labels 2. when writing the csv file i have attach each data label 2 to its 3d element.

i hope this helps.

filepath control

this is just to select a single csv file of choice at the moment. each such csv file will contain only one .bin data file conversion.

Thank you once again

[altenbach]

---

@otiose wrote:

the change of file extension was required because data is only collected in .bin format and NI Forum does not allow to upload .bin file. I will try to find the right way for posting such files.

---

Easiest would be to place everything into a zip file before attaching. This also make life easier for us, because things are kept lgically together. Now we already have the file and instructions how tho rename it so no further action is required at the moment.

[Mark_Yedinak]

Might I suggest that you read the file in chunks. LabVIEW does not do very well when reading very large files into memory in one shot. This is both from a memory perspective and a performance perspective. Since you are splitting the file read/write it in chunks in more reasonable sizes. When you reach the size threshold for your new smaller file simply close that one and create a new one. You will find that the code will run faster and use less memory.

[smercurio_fc]

---

@otiose wrote:

HI smercurio_fc ..

thank you for looking into the problem.

the change of file extension was required because data is only collected in .bin format and NI Forum does not allow to upload .bin file. I will try to find the right way for posting such files.

---

I was referring to changing the extension on your bitmap to .jpg just so you can get around the ban on uploading bitmaps. There is a reason for the ban on bitmaps. They are HUGE files whereas a png is but a fraction of a bitmap.

[otiose]

hi Mark Yedinak,

thank you for looking in the problem.

i agree with labview not reading large files properly. im my case the data collection and writing to file has to be at very high speeds, infact highest possible. currently we achieved it by using clusters to bundle the data and storing it using write as binary file. writing to binary was the fastest in this case. i would still try to use this suggestion and make changes. thanks

on the other front, if i try to read/write data in chunks, it does use smaller memory but it never returns the memory back. recovery of this memory is the important issue for me as it will allow me to use the software again and again instead of using it once and closing it down to make labview release the memory.

thanks 

[otiose]

---

@altenbach wrote:

---

@otiose wrote:

the change of file extension was required because data is only collected in .bin format and NI Forum does not allow to upload .bin file. I will try to find the right way for posting such files.

---

Easiest would be to place everything into a zip file before attaching. This also make life easier for us, because things are kept lgically together. Now we already have the file and instructions how tho rename it so no further action is required at the moment.

i will keep that in mind! 🙂

---

[otiose]

---

@smercurio_fc wrote:

---

@otiose wrote:

HI smercurio_fc ..

thank you for looking into the problem.

the change of file extension was required because data is only collected in .bin format and NI Forum does not allow to upload .bin file. I will try to find the right way for posting such files.

---

I was referring to changing the extension on your bitmap to .jpg just so you can get around the ban on uploading bitmaps. There is a reason for the ban on bitmaps. They are HUGE files whereas a png is but a fraction of a bitmap.

---

i will keep a check on that. 🙂

[smercurio_fc]

cross-posted to LAVA: http://lavag.org/topic/14244-labview-doesnt-release-memory/page__gopid__85825#entry85825