How to handle VIs without treating them as files?

[CoastalMaineBird]

My app uses "plug-ins", i.e. dynamically called stand-alone VI files in a certain folder.

To guarantee that the VI that produced the data is unchanged and traceable, the VI is included in each and every data file.

I do this by writing two strings for each VI, the Name of the VI, and the VI itself.   I just open the VI file and read it as a string.

When the user opens the data file later for viewing, I take those two strings and write the VI into a sandbox folder, where I can call it again (the user can change scale factors, etc., and re-process the recorded data, so I need to be able to re-run the plug-ins).

When I update the data files (which are hybrid datalog + binary files) from one version of my program to another, I've always just read the string from the old file and written it to the new one.  No problem.

However now, we're moving from LV 8.6 to LV 2010, and the VIs need to be re-compiled to work with the new executable.

The brute-force way to do the upgrade is to read the name string and the VI string, write the VI string to a temp file, load the temp as a VI, save the VI as a temp, read the temp file as a string, and write the string to the destination data file.

Is there a better way?  My client has thousands of data files, and the process is already slow enough.  

What I'd like is a function ConvertStringToVIReference(), and ConvertVIReferenceToString ( ), where it all happens in RAM without involving the disk.

---

SEPARATE QUESTION:

Is there a way to do the recompiling part from an EXECUTABLE?  Currently, these files are on 12-15 machine with only an EXE, not the LabVIEW DevSys.  As I understand it, no compiler is available in an EXE (makes sense).  

Ideas?

[CoastalMaineBird]

I suppose that it's impossible to have a VI that exists outside a file.  A VI contains relative paths to subVIs that it calls, and there's no way to get a path from RAM to a disk file.

As feared, the execution time of the brute-force method is atrocious (2 sec per file, compared to 0.2 sec per file, without the recompile).

[nathand]

In the development environment it's possible to run a VI that hasn't been saved to disk, although I have no idea how you would duplicate that programmatically.

Do many of your data files use the same VI?  If so, consider this: each time you read a data file and its associated VI, calculate a checksum for that VI.  Maintain a lookup table of the VIs that you've already recompiled.  If you calculate a value that you've already seen, load the matching recompiled VI instead of recompiling again.

[CoastalMaineBird]

each time you read a data file and its associated VI, calculate a checksum for that VI.  Maintain a lookup table of the VIs that you've already recompiled.  If you calculate a value that you've already seen, load the matching recompiled VI instead of recompiling again.

That's a good thought.  I already have a hash mechanism, so I can just run a hash on the VI as a string, look up the hash in a table, and use the precompiled version from the table, if the hash matches.

That should get the same result with less recompiling.  There is a total of about 35-40 VIs that are in use, some data files have 1-2, some have 15-20. So your idea should save a boatload of time.

[Ben]

---

@nathand wrote:

In the development environment it's possible to run a VI that hasn't been saved to disk, although I have no idea how you would duplicate that programmatically.

Do many of your data files use the same VI?  If so, consider this: each time you read a data file and its associated VI, calculate a checksum for that VI.  Maintain a lookup table of the VIs that you've already recompiled.  If you calculate a value that you've already seen, load the matching recompiled VI instead of recompiling again.

---

Just another crazy idea to ignore...

It is possilble to imbed a VI in a data file as the raw binary of the VI itself. It the binary is pulled out of the file, saved as A.VI then accessed using VI server, it would be effectivley packing the decoder ring inside the cracker jacks.

Resume ignoring now,

Ben

[CoastalMaineBird]

It is possilble to imbed a VI in a data file as the raw binary of the VI itself. It the binary is pulled out of the file, saved as A.VI then accessed using VI server, it would be effectivley packing the decoder ring inside the cracker jacks.

Perhaps you're misunderstanding something, because that's exactly what I've been doing on this project, for several years now.  The VI is embedded (along with it's name) in the data file itself.  I pull it out, save it as X.vi, and then execute it when necessary.

The problem is that I need to RECOMPILE said VIs, because of the change from 8.6 to LV 2010 on the part of the host.  So I'm looking for a better way than 

Extract text

Write text to VI file

Open VI file

Save. Instrument

Close VI file

Open VI file as text

Read text

Re-insert  text into DATA file.

If I could tell it that "this chunk of text is a VI", and have it believe me, that would be ideal.  But I don't see how.

[CoastalMaineBird]

nathand:

FWIW, it took 1063.7 seconds to update a batch of data files via the brute-force way.

Re-doing the same batch with the hash idea of yours took 230.8 seconds.

Among the first few files, you could see the list indicator stall as it came across a file containing a vi it hadn't seen before. Eventually it saw them all and went a lot faster.

Thanks for your idea.

[tst]

---

@CoastalMaineBird wrote:

What I'd like is a function ... ConvertVIReferenceToString ( ), where it all happens in RAM without involving the disk.

---

I believe there is actually such a method (a private one) and it's quite possible that the opposite exists as well. I don't remember the name offhand, but I could look tomorrow.

[Jarrod_S.]

I like the hash idea. It was my first thought.

What types of calculations are these? Any chance they could be saved as string formulas and parsed/evaluated using the Formula Node VIs? I'm guessing not if you've been doing this so long, but that would make this whole problem trivial.

Also you might be interested to know that TDMS added similar functionality that allows you to embed scale information in your files. It's mostly designed to facilitate quicker streaming and smaller files because for most DAQ applications you can save I16s instead of doubles.

[tst]

---

@tst wrote:

---

@CoastalMaineBird wrote:

What I'd like is a function ... ConvertVIReferenceToString ( ), where it all happens in RAM without involving the disk.

---

I believe there is actually such a method (a private one) and it's quite possible that the opposite exists as well. I don't remember the name offhand, but I could look tomorrow.

---

It's called Save:To Buffer, but since you got it working with the hash method, I suggest you continue with that, as private methods like this are not the sort of thing you want to use in a customer's app.