Prefered method of storing data to text file

[KSU Flyer]

Hello all.  As the subject line suggests, I'm curious as to the prefered method(s) of storing data to a text file.  Right or wrong, I suppose what has driven me to ask this question is that there seems to be many ways to accomplish the collection of data, then store it to a file.  In my case, I've got a pretty simple example of using a thermocouple to store a single temperature measurement at some rate, but what if I want to sample 5 or 10 thermocouples, resulting in a lot more information?  Also, would change your answer if I wanted to store all the temperature data sampled at 1000Hz, 100Hz, 10Hz, 1Hz, 0.5Hz, etc.?

Additionally, I'm thinking I want this sort of activity to run on a computer already running a labview generated executable, so having this application run in the background raises some concern about harming the performance of the application already running.  Is it wise to write the data in an array, then every so often write to a file?  ...or just write straight to a file and don't worry about it?  ...and so on.

-Mike

[SteveChandler]

You are right, there are lots of ways to do it. If you are storing data at a high frequency make sure you store it as binary and not text. Also take a look at TDMS. Go to help/find examples and search for TDMS. I have not personally used it but I have heard that it is very high performance.

[Ray.R]

As you stated, there are many ways to tackle this.

TMDS is an excellent suggestion.  Very efficient.  I should use it more often.. 😉

Alternatively, you could initialize an array to store the data and replace its elements to avoid resizing the array. 

You could have a state machine (producer / consumer) where the data storage takes place (to the array & to the file).

A shift register would hold the values.  Although a buffer (queue / notifier) could also do the trick, I think there may be some additional overhead as compared to a wire & shift register.

You could set writing to the file to occur on a timely basis, based on the worst case scenario (1kHz sampling rate).

You could append to file, only the new data that is collected.  Just make sure you keep track of how many new elements were written to the array.