Efficient method to read a Setup file ? Config VIs ?

[winterishere8]

Hello All:

I am developing a project with a large setup file which is linked to a global variable in my code. Each section in the setup file maps to a cluster in my global variable. 

Typically I use config VIs to read and modify the setup file from my code. Of late I've been using the openG config VIs since they are easy to work with clusters. Since speed is a major concern, is there an efficient way to do this OR an alternative to config VIs ?

[Andy F.]

Maximus00,

If you want your program to be as efficient as possible (and thus as fast) I have a couple of suggestions for you.  The first would be to not use global variables, since this is basically like reading and writing to file, which is time consuming.  Another suggestion would be to read and write using a binary file instead of ASCII, which would also save some time.  If you use binary files you can define any structure that you would like, just make sure that you make a type def of that structure which would allow you to read that file as well. 

Here is my suggestion, create a config file writer that writes the information you want to file using the binary file writing scheme (if you would like examples theres a great one in the Example Finder).  The reading operation will be much quicker using binary.  Once you have the reading and writing down, you should just use the data that comes out of the file read functions instead of writing them to file.  It may clean things up, but you will save in performance if you don't.  Plus you will be able to follow the flow of the data better. 

Let me know if you have any questions!
Andy F.

[winterishere8]

Hello Andy

Thanks for the suggestion. Couple of questions. I have about 100 variables currently in my global variables categorized as clusters.

I am planning to implement your suggestion and this is how I would do it. Please do correct me if i am wrong. I would have a VI dedicated to communicating the data between my VI and the file using binary r/w. After the file is updated, I would read this file to extract the values whenever I need to use it in my program.Is that right ?

Also what would be the case if I need to use these variables in subVIs ? Do I have to read it from the file again ? or use globals or shared variables ?

Thanks

Sridhar.

[WNM]

Your routine that writes the data to the file and reads the data from the file should contain an uninitialized shift register in a while loop (like a "LV2 global vi") that carries all the data in the file.

When you tell the routine to write to the file, it updates the file and the data in the shift register.

When you tell the routine to read from the file it only reads from the file and updates the shift register if it is the first time that it has been called, otherwise it does not read from the file and instead just returns the data contained in the shift register (since it exactly matches what is in the file).

[winterishere8]

Hey Andy

Would it be possible for you to put a small example ? R/W binary files and also to create type defs ?

[Andy F.]

The way I was thinking was to have read the VI once during initialization.  Once you have those values you can use them throughout your entire project, but that is just one method.  The file writer VI would only be for creating the config file.  If you would like some examples for reading and writing binary files launch the NI Example Finder by selecting Help >> Find Examples, then browse Fundamentals >> File Input and Output.  There should be a Read and Write Binary File VI.  The main thing to remember is you must keep the data types the same to allow you to read the data back in.  Creating type defs are very easy.  If you open the LabVIEW Help and search for type def you should bring up a how-to page about how to create them.  One tip would be to create the constant and then right click to create the control.  Once you have it on the block diagram you can follow the steps in the help file.  Let me know if you want further clarification.

Hope this helps!
Andy F.

[winterishere8]

Andy

I've coded some VIs using your idea. I've hit anothe roadblock. I have a VI which has read all the values from the binary file. Now to use them throughout the project I've gotta use global or shared variables or pass them as variables into subVIs. I am not very comfortable passing them as variables to subVIs. I was thinking that inside each subVI I can read the binary file. Would that be fastest or what would you recommend ?

[Neil.Pate]

Maximus,

using file access throughout the whole project is a big no no I think (seems very inefficient to read/write from permanent storage at every single VI). Its quite easy to get the global variable method to work, its what I used in my first large project. This method means you dont have to pass any variables throughout all your VIs etc, but it is generally frowned upon by all the LabVIEW gurus as it breaks the dataflow and can cause race conditions. Also, with 100 separate global variables things get quite hairy, and its very easy to accidentaly read/write the wrong variable, which can cause headaches when debugging. If you do go with globals, consider making global clusters (of similar parameters) rather than individual global variables. In my experience though, if your project is mostly sequential, then global variables should work just fine!

However, the recommended method as far as I can tell (am self-taught, so don't take this as gospel!), is to use dataflow programming wherever possible. For a large project, if you are not comfortable passing single variables to VIs then you should probably stick to the global variable method, or more sensibly, take some time to get your head around how variables are passed in and out of VIs.

The problem with a large project is that if you are like me, and don't *fully* define the problem upfront, then you will constantly be adding variables you need that you didnt think of in the beginning. If you have a single wire (probably a cluster of clusters), then adding a single extra variable will break all your VIs! The easiest way around this is to use the typedef (as suggested by a previous post) which you define somewhere else (can be saved as a custom control), then you make all your additions to the typedef and this can be automatically propogated throughout the project.
 
I hope this helps

Neil

[winterishere8]

Yeah, that's exactly what I have now. I have about 100 variables pre-defined and grouped into 7 clusters. Yes, the project is sequential. I have about 7-10 VIs which will read from the global variable. I guess I would use binary R/W to communicate with the config file but use global variable in my project. Thanks all for the help !

[DFGray]

The real answer to this problem is to use reference objects - objects you can get to anywhere that contain arbitrary data that can be easily modified.  I can recommend two - LV2 globals and single element queues.  Both are discussed, with code samples, in this thread.  I have also attached a short and amusing tutorial on large program development which addresses many of the issues you are seeing (LV 7.1 and 7.0 formats).

For complex configuration files, you can't beat one of the free generic hierarchical file systems.  I usually use HDF5, but there are others.  You can find a LabVIEW API for an older version of HDF5 here.  Note that the learning curve is fairly steep and the VIs are not multi-thread safe, so don't try to use them in two places at once.  If you do, you will get errors at best and corrupt your file at worst.

Let us know if you have any more problems.