Unit Test Framework and dynamic dispatch

[Daklu]

As I've been exploring the Unit Test Framework, one thing I've been very excited about is the ability to test indicator values that are not part of the connector pane.  In reusable instrument class methods where I take the user's inputs and create a string or byte sequence to send to an instrument, my plan was to drop an indicator right before sending the command to the instrument.  This lets me test code without having an instrument connected and helps ensure future developers don't accidentally break existing client code.

Unfortunately, after attempting to run my unit tests I discovered the UTF can't "Include controls and indicators from front panel" when the VI under test is dynamic dispatch.  Two questions:

1. Is this limitation likely to be removed, or is it based on something within Labview that is very hard to change?
2. Are there reasonable work arounds that would provide me with some way to test the commands?  (I don't want to put the commands on the connector pane as that exposes internal functionality to the user.)

Thanks,

Dave

[Will_S.]

Hi Dave,

The reason why running a Unit Test on Dynamic Dispatch VIs is the lack of differentiation between the multiple copies that could be available. Since Dynamic Dispatch VIs allow you to have multiple VIs of the same name where the only similarity is the connector pane, there is no way of knowing that LabVIEW is always running the test on the correct code in memory.  This could give incorrect results when running the tests. However, R&D is currently looking into this issue and will hopefully have a solution in future versions of LabVIEW.

In the meantime, you can copy the code in your Dynamic Dispatch VIs into Static Dispatch VIs and run Unit Tests on those. This way the code can still access the class information.

Regards, 

[Jim_Kring]

Hi Dave,

At the expense of sounding like an infomercial (by not actually helping answer your specific question)...

VI Tester is great for testing methods of LVClasses.  VI Tester is object-oriented, internally, and works great for creating mock objects to pass into your methods under test -- there's even a great article on Using Mock Object with VI Tester by Ben Hysell.

Cheers,

Message Edited by Jim Kring on 03-23-2010 12:32 PM

[Neil_Li]

Hi Dave,

Thanks for providing feedback. When you include controls and indicators not in connector pane (CP) into a test case, UTF will execute test in this process: set values to controls not in CP, then run the tested VI with inputs passed through CP, get values returned from indicators on CP, grab values from FP for indicators outside CP, check values and give result. But you can not tell which dynamic dispatched VI will be really executed before running it, the parent class method or child class method. So UTF can't know which VI to set control values outside connector pane. Indicator has another story. In case your parent class method has an indicator outside CP, but child class method doesn't, you create a test case for parent but pass in child ref, UTF can't find the target control specified in test case on this child method, then it has to return an error. This is why dynamic dispatched VI doesn't support "Include controls and indicators from front panel".

From your feedback, we aware the importance of this feature for dynamic dispatched VIs and is looking at it to find possible solutions. But currently, you have to add your indicator into CP. Sorry for the inconvenience.

[Herbert Engels]

You could also use user-defined tests for this, where you create a new VI that executes whatever your test is and then returns the result to the Unit Test Framework. Being able to configure a test in a purely dialog-based environment however would require the controls/indicators to be on the connector pane. Both would be valid workarounds until we come up with a better solution (which we are looking into right now). In your test report, the results of configuration-based and user-defined tests will show up the same way.

Thanks,

Herbert

[Daklu]

Will/Neil,

I had thought about the problem of inputting a child class object and calling an overriding method that does not contain the disconnected control/indicator.  However, the UTF appears to be designed to test a specific vi, not a specific vi plus all the overriding methods.  In other words, it seems to encourage one unit test for Parent:MyMethod and another unit test for Child:MyMethod.  I could create a setup vi that inserts a child object on the class input, but I don't see any advantages in doing that versus calling the child class' method directly.  Am I missing a valid use case?

Jim, 

I've used JKI's VI Tester framework and I think it's extremely well done.  It's very powerful, very flexible, and best of all... very free!  (I use it for my personal projects.)  In my organization I'm trying to introduce both unit testing and OOP.  Using a unit test framework that relies on objects is likely to meet more resistance.  The Labview UTF also provides a little more structure that (IMO) helps guide users through the process of creating and organizing unit tests.

[Daklu]

Herbert,

I don't think user defined tests would help in my case, or at least it's not clear to me how I would use them in my situation.  Here's one of the vis I'd like to unit test.

The class is designed to control an Ontrak ADU218 usb relay box.  This vi sets/resets a relay by creating a command string based on desired relay state and which of the 8 relays is selected.  Then it sends the command string to a protected method that encapsulates the actual dll call.  The unit test for this method has 7 test cases defining it's behaviors under various input condition.  Those conditions that are expected to work read UtCommandString; those that are expected to fail read the error out indicator.

Without being able to read UtCommandString I don't really have a way to verify this method is coded correctly.  I can't put UtCommandString on the conpane without exposing the method's internals and (unintentionally) making the command string part of the public api.

Creating a private sub-vi that does the actual string building and then unit testing that vi instead is an incomplete work around.  I really want to unit test my public methods as much as possible.

I thought about saving the command string as part of the class data and reading it out in a tear down vi, but that's an unpleasant solution too. Carrying around unit test data and methods at runtime that don't have any pratical runtime value feels like a sloppy hack.

None of the examples in example finder provided any clues on how to deal with this.  Any ideas?

---------------------------------------------

Regarding the UTF examples in example finder, two of them left me scratching my head.

1. Basic Functions - Any particular reason Divide by Zero is a separate unit test and not included as part of the Divide unit test?  I can see a single vi having multiple unit tests if they have different setup or teardown requirements.  These don't, so...?
2. LV Class - Sure enough, this example has a dynamic dispatch unit test complete with a setup vi that allows the user to create test cases where the parent method is called rather than the child method.  I'll reiterate my question from the post above... why?  I would think it is poor practice to create a unit test for a child method that also conducts tests on a parent method.

[Neil_Li]

Dave,

After a second look at Herbert's workaround, I think User-defined  test VI can work for your case, but need more coding work. In your case, you are testing the child class method directly instead of letting UTF select the target method dynamically, then you can use LabVIEW Scripting in user-defined VI to open the method VI and get value from UtCommandString. Although this isn't a perfect solution as you have to hard code the VI Path to open and get UtCommandString, but as you don't want UTF select tested method VI dynamically, this can work for you.

LabVIEW Scripting is a NILab product and here is the link. http://decibel.ni.com/content/docs/DOC-4973

[Neil_Li]

Good to hear your opinion on dynamic dispatch testing. Once a test is create, what UTF can do is to pass outputs from setup VI to tested VI. User can specify whatever inputs he want in setup VI and we don't know until runtime. That's to say, this is controlled by user, while not UTF. UTF can know which VI is called in runtime, but what should it do? to give an error says your parameter is invalid seems a bug for customer who create a simple test to assure Dynamic dispatch is really enabled for his classes.

We are continue collecting user feedback on this tool to improve its design and usability. We are looking at your feedback seriously which is valuable and helpful.

[Jim_Kring]

---

Daklu wrote:

Jim, 

I've used JKI's VI Tester framework and I think it's extremely well done.  It's very powerful, very flexible, and best of all... very free!  (I use it for my personal projects.)  In my organization I'm trying to introduce both unit testing and OOP.  Using a unit test framework that relies on objects is likely to meet more resistance.  The Labview UTF also provides a little more structure that (IMO) helps guide users through the process of creating and organizing unit tests.

---

OK, then after everyone learns OOP then they won't be afraid of VI Tester.  But, actually, there's no reason to need to know OOP to use VI Tester.  It only use LVClasses like an LVLib to organize similar tests.  And then when you want them all to share common data, you can put the data inside the object data.  Nothing scary there.  You might even say that it's a way to get people to start doing OOP without them even knowing their are using OOP.

BTW, we've got a new version of VI Tester coming out soon, hopefully, that will have some nice improvements.

Cheers,