File write/File flush

[Serge67]

In my data acquisition system I am getting large number of data points every second and writing them to the file once a second. I looked at all Write functions in LV 2009 and it appears that all of them will write data and flush the file at the same time. I don't really want to add overhead of flushing the file every second, I would rather do it once every 5 or 10 seconds. Also, saving data in the buffer for a few seconds and then writing once in a while doesn't seem to be a good idea either. I can't use TDMS format for my project because the output has to be in ASCII.

Does any one know if there is any low level hidden Write function in LabVIEW that will not flush the file automatically?

Thanks,

Serge 

[RavensFan]

I don't understand.  You want to write the data out to the file every second, but you don't want to actually write the data to the file every second.  But you don't think storing several seconds worth of data in a buffer is a good idea before writing out to a file?  Why not?  I'm guessing you are afraid of losing data.  But how would having the data in a limbo state by writing to a file every second but not actually flushing it out to a file every second be any better?  You are still at the mercy of something going wrong in the computer causing you to lose several seconds worth of data.

[altenbach]

Can you show us how you are writing to the file? My impressions is that you don't necessarily flush the file unless you call "flush file" explicitly.

 Also, what is your OS? Any custom settings? For example is "write caching" enabled for the volume you are using?

[rolfk]

Last time I checked (definitely not LabVIEW 2009) LabVIEW did not do a flush explicitedly at every write but simply used the Windows File IO API to write the data and let Windows handle the flushing. Why do you think your LabVIEW program is doing some explicit flushing behind the scenes without your use of the Flush function?

What functions are you using? If you open the file and then write, leaving it open then there should be no flushing. Of course when you close the file Windows will quite quickly flush the data anyhow and LabVIEW might even force a flush there, but that last one I'm not sure about. Maybe you use the Write File function without explicit File Open, just passing the file name to it. I believe LabVIEW will simply perform a File Close at the end in this case if the file refnum output is not wired.

Message Edited by rolfk on 03-03-2010 09:51 AM

[Serge67]

Thanks everyone for your replies.  Ravens Fan - I do want to write data to disk once a second, but not to flush it every second because flush is a relatively time consuming operation. As a trade-off, I would flush the disk once every 5 seconds or so realizing that if the system dies for whatever reason, I would loose the most recent 5 seconds worth of data. I am using LV 2009 on WIndows XP, nothing fancy, except for aggressive requirements for data acquisition and processing. I need to acquire data for 64 channels at approx 30k samples/sec rate, which is 8MB of data every second, I have to do some data analysis, data reduction, maintain two circular buffers for recording pre-trigger events, monitor alarms, and record 1000 of single precision floats per channel for all channels every second in ASCII format. So, I am trying to be efficient and lean in terms of CPU and memory usage. I was also under impression that unless you flush or close a file handle, the file will not be finalized (completely saved) on the disk. But I guess the OS takes care of file flushing because if you write something to a file using Write To Binary File function and then abruptly stop the vi without letting it close the file handle using Close File function, the file still will be saved on the disk and auto-magically it will contain all written data. That's why I figured it must have been flushing after every write. I just wish documentation for Write To Binary File was more detailed in this regard.

Serge 

[rolfk]

A LabVIEW file refnum is another resource where LabVIEW does garbage collection. That means when you abort your VI or when the hierarchy in which the refnum was opened goes idle LabVIEW will close the refnum automatically, so in effect issuing the Close File function anyhow.

That is not to encourage to never close a file, in fact rather the opposite as such lazyness always comes around later to bite you.

[Ben]

---

Serge67 wrote:

...

I just wish documentation for Write To Binary File was more detailed in this regard.

Serge 

---

Write to binary depends on the Operating System to schedule and execute the write to disk. The actual time the data is writen depends on may factors including other pending I/O's to the disk etc. Flush is a special request to the OS to write it now.

So its not the details of "Write Binary File" that get involved in writting to disk but rather the OS.

Ben