Writing to Multiple Data Files Only When selected?

[NT_Engineer]

I am building a system that is measuring the CO2 content of 12 vessels but only 1 CO2 gas analyzer. So, my system has a series of valves that only lets one valve open at time so that only one vessel is being measured at a time.  What I would like to is have a measurement file for each vessel so I don't have to worry about determining which data came from each vessel. Also, I only want to write measurements to each file whenever its corresponding valve is turned on. The third thing I would like to do is to be able to read the data file and plot the trend data as the test is running.  These test will run for 14 days.  So what would you recommend to do this? Would you recommend using a Write to Measurement File Express VI, Write to Spreadsheet, Write to Text File, Write to XML? 

Requirment Summary

1) File for each vessel

2) Only write data to file when corresponding valve is selected

3) Ability to read the data files and plot the trend data over time for the 14-day test.

I am using a Consumer/Producer design with two consumer loops one for processing and writing the data and the other to control a pump for a waterbath that was made inhouse.

Thanks!

[Robbob]

This is such an open ended question...but I will give some advice.

If you plan on reading and writing to a file, dont use the write to measurement, you want to use the open/create/replace and close file functions.  The particular file you write to can be either of those listed, but the text file will require all numeric values to be changed to strings, which is just more overhead to take care of.  If I could recommend a type, it would be datalog files.  They are great to organize your data.  I don't see any reason to use XML.  Do you want to be able to open and review this data later? Do Post-Analysis in Excel? Maybe write to spreadsheet is what you are looking for then.

Also, how fast are you gathering data?  At high speed, maybe you want to use binary files.  Downside is that you wouldn't be able to easily open the file for viewing outside of LabVIEW.  For a 14 day test, make sure you have enough resources for your sampling rate, as any ASCII files, such as .txt, can grow large very fast depending on how you sample and how many digits you store.

For example, if we only store a 5 digit sample from one channel at 10 hz for 14 days:

5bytes/sample*10Samples/s*60s/min*60min/hour*24hour/day*14days =  60,480,000 bytes, or 60MB.

Are you wanting to show all of this data on a chart at once, and update live? This will use a lot of resources(RAM), but should be doable.  I would obviously have to know more about your engineering requirements to make any more suggestions, just wanted to give you a feel for the ups and downs of some of your options.  If you have the LabVIEW DSC module, then Citadel would be great for logging this data for you.

[Robbob]

I just realized you posted code as well, and here are some thoughts:

-Why are you doing finite measurements when you plan on running this for 14 days?

-The cluster of 13*13 booleans is...interesting.  I am sure there are easier ways to increment which valve is open, such as a bit shift on a single number and number to boolean array/boolean array to #, but this isn't a huge deal.

-Do averaging of your 3 RTD's before you pass them through the queue, thats 1/3 the amount of data you have to pass.

-100ms Wait in every loop is probably overkill

-You never close your file reference(what version of LV are you using? these are very old file vi's)

-The producer/consumer is definitely the correct route to go for your file writing, but I would definitely look into the datalog files.  You could use one file to store all 13 channels.  Take a look at the datalog examples, datalog is a format not many people know about, and are usually happy with once they learn about them.

[NT_Engineer]

Thanks for the replies. I would like to update the data live if possible. This way the operator can see the results as the test is running and not have to wait 14 days and hope the data is useful. Also, I will do post-analysis using either Excel or Matlab. So, I need the data to be written in such a manner that it will be relatively simple to write a Matlab or Excel program to do my statistical analysis.

I've tried to run the Write Datalog File Example.vi, but when I try to open the .dat file in wordpad, it is just junk symbols.  I changed the data file name to a .txt, and I get the same result.  Is there a way to use the Datalog functions to do tab delimited, and how can I use just one datalog file to separate the measurements of each vessel?

[Robbob]

Yeah, that is the 'downside' of the datalog files - they are in binary format (http://digital.ni.com/public.nsf/allkb/747B83B6C8E4A6218625669100563B61?OpenDocument).  I recommended them for their write/read speed and ability to organize data easily.  If you want to do post analysis without converting the binary file to an ASCII file, write to spreadsheet.vi can do tab delimited, and so can the write to measurement file.

Message Edited by Robbob on 05-29-2009 10:23 AM

[NT_Engineer]

Okay, I've gotten the program almost to where I want it to be, but I am having one problem. Whenever I start running the program, the prompt asks the user for what folder to store the data into, which is correct, but then it asks for a file name, which is not correct. I have already named each of the 12 files, and I want the user only to choose the folder the data is stored into. I have opened up the menu for the Prompt Express VI, and made sure that the folder option is the only thing selected. What is causing this problem? Also, I would like to name the columns in the data files, only once at the top, so that it is easier for the person doing the data analysis.  How do I do this with the Concatenate Strings?

[Robbob]

The File Dialog seems to work fine for me.  I ran it and got this:

It says shoose or enter path of folder. It will not let me select files. There is a file name and 'save' button, which would make new files, but there is also a button that says 'Current Folder'

A better way to do this would be to use the 'Current VI's Path' VI, use a strip path to take the name of the VI off of it, and then programmatically build the path to the folder you want to write in, since you stated that you have the files already created.