Need ideas for plug-in re-design

[CoastalMaineBird]

My DAQ app uses "Plugins".   A Plugin is a VI which conforms to a particular terminal configuration.

Here is the complete diagram of a simple one:

 

The idea is that a plugin is another channel: This one takes the value of TORQUE (in N-m) and the value of SPEED (in RPM), performs some computation, and produces a result POWER (in kW).

The user can then think of this as just another channel - it shows up in the datastream and the user can plot it, record it, set alarms on it, convert it to an output and send it out the CAN bus, anything he can do with a real channel, he can do with this plugin channel.

So the above is exactly equivalent to having a POWER transducer.

They can get much more complex than this.

My code organizes these so that the prerequisite channels are run first, and then the dependent ones. (a prerequisite channel could also be a plug-in).

If plugin B depends on plugin A as a prerequisite, I make sure that A runs first, regardless of which comes first in the CONFIG file.

So the actual DAQ code samples the hardware channels, and then the plugins in a particular order.

The DAQ is sampling at 10 Hz, and so these plugins run at that rate too.  There may be 100 different plugins, maybe 50 at a time in one test.

The DAQ runs on a PXI box, running RTOS.  The host is an ordinary PC, running Windows.

Even though the PXI is running a compiled RTEXE, it still loads the source-code VIs and runs them.

My data file includes all the recorded data, all the config stuff used to set that up, AND THE PLUGIN VIs THEMSELVES.

The plugin VI files are read as strings and stored in the datafile as strings, along with their names.

When reading a data file, I read this string, and store it in a "sandbox" folder with a .VI extension.

This guarantees that the version I have is the version the data was originally recorded with.

The user can EDIT the data files, for example changing the scale factor on the SPEED channel from 307.3 RPM/V to 308.5 RPM/V.

That causes a re-calculation of the SPEED channel.

It ALSO causes a re-calculation of the POWER channel: My code detects that POWER depends on SPEED and since SPEED changed, I have to run every sample back through the POWER VI to get a new value. (Remember they're not all as simple as this example).

THAT is why the VIs are stored in the data file.

ALL OF THAT WORKS. 

100%

Like a dream.

For 8-9 years now.

EXCEPT...

When it comes time to change LABVIEW versions.

MY client has 10,000+ data files, each with 5-50 VIs embedded. Files are spread among 30+ machines, with a central server backup.

We need to hang on to this data.

I have a File Updater, which will take datafiles and update them (they are a DataLog file).

THE TROUBLE IS:

If I take a recent datafile, recorded with LV2010 (where we've been for 3+ years) and open it with the EXE built with LV2013, there's a problem if it tries to reprocess it.  The  error comes out as 

Error Code 1126 occurred at Open VI Reference in HDT Open PlugIn.vi->HDT Find Calc Prerequisites.vi->HDT DataFile Prerequisite Manager.vi->HDT DataFile Viewer.vi->HDT.vi<APPEND>

An error occurred loading VI 'BSFC.vi'.

LabVIEW load error code 10: VI version (10.0) is too old to convert to the current LabVIEW version (13.0).

The EXE code, having no COMPILER handy, cannot use the old VI.

My FILE UPDATER, has a recompiler built into it (it reads the VI string, saves it as a VI file, opens it as a VI, saves it as a VI, then reads it as a string and puts it back where it belongs. 

But if THAT is built into an EXE, it doesn't work either. Same problem, same error.

If I run the FILE UPDATER from the LV2013 DevSys, then it's OK.  The re-compile process works fine, and then the EXE can read it and use the VI.

BUT:

1... That's painfully slow.  I have a cache handler where I recompile each VI only once and recognize it if I see it again, but still it's a long process.

2... That doesn't work from an EXE.  My client has 2-3 DevSystems and 30+ computers with just the EXE. But that's where the data files are.

3... Backup up Data Files on CD or DVD doesn't work.

SO............  TO make a long story even longer.......

I'm looking for an alternative.

REQUIREMENTS:

1... String-based, I suppose.  I want to type in "Power = Torque * Speed / 9545" and have the code know to fetch the value of  "Power", the value of "Torque", do the math, and produce the result.

2... Has to execute quickly.  I have a zillion things to do at 10 Hz.

3... Has to be able to re-order itself.  If I have ANOTHER plug-in channel called Fuel per Watt, and it calculates "FuelPerWatt = FuelFlow / Power * 10.34", then it has to know to execute POWER before it executes FUELPERWATT.

4... Musr work from an EXE and an RTEXE.

I've looked at a thing called MathToG, and it works OK for creating a VI from an equation, but it needs VI scripting, and therefore won't work from an EXE

Any ideas?

[DFGray]

I have been researching a similar issue.  In particular, we do not want to be dependent upon LabVIEW versions.  Note that I have only prototyped this, so do not know how it would scale.  The solution I plan to use is as follows:

1. Create a class which is your interface.  In your case, this would be a parent class with an execute VI and a GetDependencies VI which is called by the execute VI.
2. Create child classes for each measurement.  The GetDependencies would return an array of child classes of other measurements (but would actually be an array of the parent class).
3. For each child class, instead of implementing the functionality directly in the class, create a LabVIEW DLL to do it.  Call this DLL with the class.

This gives you a LabVIEW specific abstraction layer that can be easily up and down compiled, and a LabVIEW specific functionality layer that "only" needs a runtime.  You can update this as you need to, or use a different language, if that is appropriate.  Once the runtimes are loaded, this method has good execution speed.  Nothing is recompiled and everything is running at native speed.  The performance hits for dynamic dispatching and calling the DLL are pretty small.

If you need more info or try it and run into issues, let me know.

[CoastalMaineBird]

Create child classes for each measurement. 

I don't know what you mean by that.  Do you mean for each channel?

Another requirement I should have mentioned, is that I don't (rather, my client doesn't) want to require a programmer to create these things. If I can't fix the recompile issue, then ideally, an engineer could create them, without resorting to calling in a programmer.  That's where I'm thinking "typing".

But I still want to explore your idea.  How would it apply to my example "Power" case. Would there be a "power" class to do the calculations?

[DFGray]

Starting at the top, create a class which has no data and two methods - GetPrerequisites and Execute.  Both are dynamic dispatch (use the right-click option in the New menu in the project to create them).

GetPrerequisites - has the input and output of the parent class so it is dynamic dispatch, and has a single output which is an array of the same class.  You may want error I/O as well.

Execute - has the input and output of the parent class so it is dynamic dispatch, and has Result and Units outputs.  The prerequisites output is redundant with the previous function, but can be added.  Again, you may want error I/O.

In the parent class, both of these functions are essentially empty and return default data.  The parent class is used for a template and to provide a parent for the function classes.  As such, set the properties for each of the methods so that children must override them (do this in the class properties dialog box).  Set these up the way you want them up front, since the children must use the exact same connector pane.

The functions themselves are all created as follows:

1. Create a class
2. Change its inheritance so it inherits from the parent class - it is now a child
3. Use the new menu on the child class to create functions to override the originals
4. Delete all code but the terminals in the child functions
5. Change the GetPrerequisites code so it returns a constant array of the prerequisite objects.  These will be parallel children of the original parent class.  For your power class, this would be an array containing the torque and speed classes.  The easiest way to create this array is to drag a copy of the torque and speed classes onto the block diagram and use build array.  They will be "coerced" to the parent class, but LabVIEW still knows what they are.
6. Change the Execute code so that it does whatever it needs to do.  In general, use the GetPrerequisites function to fetch the prerequisites.  Use a FOR loop to run the parent Execute function on each of these prerequisites.  Dynamic dispatching will execute the correct code.  You now have an array of the results.  Use this, with whatever other functionality you need, to calculate your final values.

The prerequisites can be dynamically loaded so you do not have a static link to the prerequisite classes.  If you would like to have a text interface, you can dynamically load the individual functions, then use their values in a LabVIEW formula VI for the final calculation.

If this is still Greek, you may want to walk through the examples of LabVIEW classes and dynamic dispatch to get more background on how to use classes.

If you would prefer a text string for your units so that you have more flexibility, let me know.  I have an SI unit string validation VI that may be of use.  It works for most unit combinations, provided they are not nested too deep.

[CoastalMaineBird]

OK, I'm an expert at object-oriented programming in other languages, and I've played around enough with LabVIEW OOP to know my way around, so what you say is not Greek to me.

What I don't understand is how this changes things, with respect to LV version changes. If I embed one of these objects into my data file, it's still a VI, version LV2013 or whatever, so if I move to LV 2014 next year, the EXE that I build then will complain about the LV2013 VI, won't it?

And if I don't embed them in my data file, then I'm subject to the problem that my current method solves, namely guaranteeing that the version I use in the RE-processing of a data file is the same version I originally ran.

[Matthew_Kelton]

Instead of embedding code into your data files, why can't you have a repository of plugins (with versions)?  I don't necessarily mean a subversion/git repo, but just a place on the network with Power_1.0.0.vi and Power_1.0.1.vi.  In your data file, you just put the name of the VI to use.  When you upgrade LabVIEW, you upgrade your plugins directory on the development system and send them an updated plugin group.

This way, you have one copy of your plugins that is much more easily maintainable, it can be upgraded, etc.  You do have to be disciplined about versioning your plugins.

[CoastalMaineBird]

 In your data file, you just put the name of the VI to use. 

That's a good idea, except I would put the name AND VERSION into the data file.

When you upgrade LV, you upgrade ALL the VIs, even older,-no-longer-used ones. 

If they open an old file, it can look for the old version.

That means they'd have to upgrade the whole site (30+) machines at one time.  Somebody won't like that, I suppose.

Maybe if we incorporate the LV Version as well as the plugin version into the name, or the directory structure.

If cell A runs LV2012, that would be in the file, and when read, it would look in the LV2012 folder for "Power 1.0.0.vi"

If celll B runs  LV2010, that would be in the file, and when read, it would look in the LV2010 folder for Power 1.0.0.vi"

When we start with LV2013, we mass compile all the VIs into a LV2013 folder.

More thinking required, but that's a good idea.

Thanks!

[Matthew_Kelton]

Why would you embed the LV version in the data file?  You would still be in the same situation.  Whatever software is generating/modifying the data file would determine what LV version you should use, not the data file.  Otherwise, you would end up trying to load 2012 plugins in 2013 and end up in the same situation you are in now.

[CoastalMaineBird]

I didn't mean embed the LV version into the data file, I meant incorporate it into either the directory structure or the plugin file Name.

That way a machine running LV 2010 could look in Plugins\LV2010\Power 100.vi

and a machine running LV 2013 could read the EXACT SAME FILE and look in Plugins\LV2013\Power 100.vi

[DFGray]

It does not fully solve the problem.  You still have a set of shell code that will need to be recompiled for every version of LabVIEW.  However, the core functionality code in the DLL will not change, reducing the amount of recompiling you will need to do.  The recompile is limited to the calling shell code.  I would love to come up with something better, but have not figured anything out yet.