Saving large amount of txt image file - getting slower as the program runs longer

[sp@davis]

I run a image acquisition vi in which the camera acquires hundreds of images (with ring buffer pre-allocated) within another for loop which runs for hundreds of times. The loop time was constantly monitored and it seems that the loop time increased gradually, might be up to 30-40%. Anyone has similar experience who could enlighten me on the problem? I am not sure if Window tends to write data slower as the number of item in the folder growing bigger? The comp we are using have 32GB ram.

[altenbach]

Is this LabVIEW 64bit?

Does the memory used by LabVIEW increase over time (task manager)?

Is the HD very full or heavily fragmented?

Does it get fast again if you restart the program?

What are "txt" image files? Is there formatting involved?

 How are you measuring the speed?

[Prickly_Pear]

I suggest that you monitor your PC resources in Windows Task Manager while you run your VI. That should give you some more hints about what is going wrong.

Jeremy P.

[Hooovahh]

---

@Prickly_Pear wrote:

I suggest that you monitor your PC resources in Windows Task Manager while you run your VI. That should give you some more hints about what is going wrong.

---

Performance Monitor (included with Windows 7) does the same thing as Task Manager but has alot more things it can monitor, and you can set it up to log changes over time.  Sure leaving task manager open for a day and see how it increases but this way you can see if it is non-linear.

[sp@davis]

A quick reply before I check out the task management, could I tweak the workstation (8 cores; 32GB RAM) such that LabVIEW processes are always given the highest priority or maximum resources when being run? It is worth to mention that each of my image acquired is about 2MB and saving at 125 KHz over minutes will generate huge amount of data.

[altenbach]

I don't think tweaking priorities will gain you anything. Most likely the resource use of all other processes is insignificant. Windows and LabVIEW do a good job managing resources fairly.

The bottleneck is most likely writing to the HD and everything else is infinitely fast in comparison. Maybe you could upgrade to faster storage devices (SSD, RAID 0, etc.).

Trying to write 2MB at a 125kHz rate (250 GB/s???) exceeds all current hardware, I think. Are you sure your units are correct?

[sp@davis]

Apologies my bad for the confusion of the frame rate: it is more about 500MB/s data acquisition. Here is the calculation: a line camera running at 125KHz, thus grabbing 512 lines to form an image equivalent to less than 250 fps.

I have the buffer pre-allocated (exact amount of images), in each loop, the same buffer was used to acquire 100 of frames at close to 250 fps,  then the whole buffer was extracted and saved using a par-for loop. The data were written to RAID (mainly) or SSD; before starting another loop to acquire and write another set of data of 100 frames. 

More information 

Is this LabVIEW 64bit? No, 32bit.

Does the memory used by LabVIEW increase over time (task manager)? Not really, memory goes up from 9GB to 11GB (stay consistent) when the program is running.

Is the HD very full or heavily fragmented? Not really. 

Does it get fast again if you restart the program? Usually the program run fast at the beginning, and get slower over time.

What are "txt" image files? Is there formatting involved? Basically just reformat image file to text file, proposed to be a faster way to write image file.

 How are you measuring the speed? Time count function to monitor the loop time.

CPU Usage is constantly less than 12%; around 1-2% when the labview is not running. Also I suspected the transfer between the camera and framegrabber is not always max out; I am using PCIe-1433 frame grabber. Is it possible that the transfer speed drop over time? I am running the recommended maximum possible frame rate of the camera.

[altenbach]

LabVIEW 32bit cannot use more than 4GB of memory.

12%cpu use looks like you keep exactly 1core busy.

What is "reformat"? Just a typecast?

[bsvare]

If you consider the absolute ideal transfer speed of 6.0Gb/s is only 750MB/s. And that's ideal, IE, all one large file, not MFT writing, no mechanical movement.

The Average Sustained write speed is on the order of 50-75MB/s. This considers smaller files, mechanical movement, MFT writing, etc.

This all depends on your hard drive. You can get better or worse performance from this, but this is likely an average.

There's no way any current top of the line hard drive system can sustain 500MB/s without complete RAID.

Though it sounds like this is offloaded outside your datacaputre, it'll still take many seconds to write that data external to your collection.

As for your code, if you're jumping by 2GB over the course of running your code, then you are not preallocating all the images into ram. You are probably running into a significant performance reduction reallocating the structure or copying it. I'd recommend running DETT to see where your memory is being allowcated.

Also, as already pointed out, a 32bit LabVIEW is not capable of handling more then 4 GB of RAM. You're running at about 100% single CPU (assuming 8 logical cores), so there's definitately code performance issues to look into.

[sp@davis]

I used IMAQ Flatten Image to String.vi before writing them into binary file.