BUG: VI unloads from subpanel when InsertedVI property is called from an asynchronous process

[fabric]

I use subpanels a lot, and they generally behave well for me if I treat them right... but not always!

I often create a top-level UI with a big subpanel in the middle for displaying dynamic content. For many years I have been using AF actors (or similar asynchronous processes) to run the subpanel content. Recently, I had an application where the content actors were accessing the InsertedVI property of the main subpanel. I noticed that sometimes the content VI would freeze or vanish soon after the InsertedVI property was executed. The conditions for this to happen were a little strange but I was able to create a minimal example to demonstrate the issue. Attached code is in LV2020.

The basic setup is something like this:

The main UI contains a subpanel. The buttons on the left switch between various content panels by loading different actors. (The attached example uses AF, but any asynchronous call seems to trigger the bug.)

The load sequence for each panel is identical:

We first stop* any existing panel, then pass the subpanel reference to the new panel, then launch the new panel. When each new panel actor starts running, it inserts itself into the main subpanel, then waits in its message handling loop until stopped. (See screenshots below for Panel A, B and C.)

* It is important to note that the stop request runs asynchronously, so depending on the shutdown behaviour of the existing panel, the new panel may start running before the old panel has actually stopped. (This will be important later!)

There are three different versions of the content panel which all begin the same way: Insert self into the main subpanel, then run Actor Core. For debug purposes we also display the VI name, launch time, and a blinking indicator to show when the actor is running. Panels A, B and C differ only in their shutdown behaviour.

Panel A has no special shutdown behaviour, so it generally stops completely before any new panel is launched. In practice this is not very interesting. Issues only show up when the shutdown of one process overlaps with the launch of the next.

Test sequence A:

1. Run main.
2. Load Panel A. Observe clone name and launch time.
3. Reload. Observe that launch time changes but clone name remains the same. (This is because the old instance of Panel A stops completely before the new instance launches.)

Panel B includes a small shutdown delay, then removes itself from the main subpanel. This is where the fun begins...

• I added the call to RemoveVI to ensure that Panel B would never be left inside the main subpanel after it stopped running. (This made more sense in my original application where panels could sometimes stop without being replaced by a new panel.)
• This caused a problem though: Because stopping a panel is asynchronous, if the shutdown takes long enough then the main subpanel may already contain a new panel. In this case the call to RemoveVI will incorrectly remove the new panel by the time the old panel eventually stops.

Test sequence B:

1. Run main.
2. Load Panel A.
3. Load Panel B. Observe that new panel loads normally.
4. Load Panel A again. Observe that new panel loads, and then vanishes after 1 second.
5. Load Panel B again. Observe clone name and launch time.
6. Reload Panel B. Observe that both clone name and launch time change, and that the new instance of Panel B vanishes after 1 second.

So how can we fix the vanishing panel issue? Maybe we can check to make sure we are removing the correct panel! Let's see how that goes...

Panel C is just like Panel B, but includes an additional check using the InsertedVI property.

• RemoveVI is only called if InsertedVI returns the current panel (in this case Panel C). If a different panel is already loaded, then we do nothing.
• This is where the bug occurs: Once Panel C completes shutdown and goes out of memory, the VI returned by InsertedVI is unloaded from the subpanel and cannot be reloaded.

Test sequence C:

1. Run main.
2. Load Panel A.
3. Load Panel C. Observe that new panel loads normally.
4. Load Panel A again. Observe that the new panel loads, and then freezes after 1 second! (If the main VI is resized, the subpanel will redraw and the frozen image of Panel A will vanish...)
5. Attempt to reload Panel A. Observe that it does not render correctly. (It seems to have become corrupted...)
6. Attempt to load Panel B, or some other uncorrupted panel. Observe that it loads correctly.

Conclusion

My example suggests that the reference management of the subpanel is buggy. When the InsertedVI property is called, it seems that the ownership of the reference of the inserted VI is transferred to the VI calling the InsertedVI property. Then, when that VI leaves memory, the reference is cleaned up and the referenced VI is dumped unceremoniously out of the subpanel. Interestingly, the subpanel remains usable, but the affected VI is corrupted.

[fabric]

Cross-posted to the AF group: https://forums.ni.com/t5/Actor-Framework-Discussions/BUG-VI-unloads-from-subpanel-when-InsertedVI-pr...

[drjdpowell]

---

@fabric wrote:

So how can we fix the vanishing panel issue? Maybe we can check to make sure we are removing the correct panel! Let's see how that goes...

---

You went wrong here.  You should have asked the question: "who is in control of this subpanel?"  The answer is "multiple actor are fighting over it, and colliding with each other, because noone is actually in control of this subpanel".  Then the fix is to make someone in charge of the subpanel; why isn't your "top-level UI" controlling use of the subpanel, and inserting and removing VIs?

[tst]

A couple of comments:

1. Yes, the InsertedVI property seems to have some unexpected behaviors and I don't remember them. Try searching around to see what people came up against.
2. As already said, you may want to change the way you handle this.
3. You still have a race condition in the following image, because you check for a VI and then remove it. In between, the VI could be replaced by the new VI and then you're removing that one. This is a narrow window, but it's still two separate operations. Again, changing the way you manage this can help.

---

[Yamaeda]

I don't like the idea of a Panel inserting and removing itself, but then i've never touched AF.

[fabric]

---

@drjdpowell wrote:

---

@fabric wrote:

So how can we fix the vanishing panel issue? Maybe we can check to make sure we are removing the correct panel! Let's see how that goes...

---

You went wrong here.  You should have asked the question: "who is in control of this subpanel?"  The answer is "multiple actor are fighting over it, and colliding with each other, because noone is actually in control of this subpanel".  Then the fix is to make someone in charge of the subpanel; why isn't your "top-level UI" controlling use of the subpanel, and inserting and removing VIs?

---

James: You are correct that centralising the insert/remove operations to the owner of the subpanel avoids the bug. I agree that it is a cleaner design with much better lines of responsibility.

There are a few reasons why I didn't implement things that way in the original application:

1. Simplicity: Moving the subpanel handling from the callees (panels) to the main UI (subpanel owner) requires additional messages in the main UI for Insert and Remove. (>> Not too much of a big deal, just more code)
2. Scalability: My original application actually has multiple subpanels. I want the subpanel handling to be generic (i.e. implemented in common code, rather than specific to any one application). Adding the ability to manage multiple panels makes my base UI actor pretty complicated... (>> Solvable, but not very clean)
3. Code uniformity: I have one case where the top-level VI containing the subpanel is NOT an actor and has no messaging infrastructure. (>> That special case would require a different implementation from the other subpanels)

Obviously none of the above are showstoppers, just reasons why I chose the path I did. And I got bitten by a big fat bug for my choice!

All that said, the specifics of my application don't mean the bug shouldn't be patched! 😁

[fabric]

---

@tst wrote:

Yes, the InsertedVI property seems to have some unexpected behaviors and I don't remember them. Try searching around to see what people came up against.

Unfortunately I couldn't find anything useful. That's what drove me to post this.

You still have a race condition in the following image, because you check for a VI and then remove it. In between, the VI could be replaced by the new VI and then you're removing that one. This is a narrow window, but it's still two separate operations. Again, changing the way you manage this can help.

Yeah, that is the problem with minimal examples - they are never very polished...

In the original application the subpanel handling is actually centralised in a base panel actor, and the Insert and Remove are always run as atomic methods. (Well spotted though!)

[fabric]

---

@Yamaeda wrote:

I don't like the idea of a Panel inserting and removing itself, but then i've never touched AF.

---

The bug is nothing to do with AF. It is to do with how LV manages a VI reference when the code that accessed that reference goes out of memory. AF was just a convenient way to demonstrate the issue.

[Jay14159265]

---

@fabric wrote:

...

My example suggests that the reference management of the subpanel is buggy. 

---

I agree with this statement, or at least with the notion that the subpanel management methods do not always act as expected. 

I don't know if you have gotten into docking and re-sizing many VI FPs yet but as soon as you do it will be apparent that you cant easily manage subpanels by using the InsertedVI field. As you have demonstrated, the VIs need to be able to run independently of the UI (subpanel) operations. You run into issues when you are trying to use sequentially loaded VIs to manage the subpanel itself but the lifetime of the VIs are not deterministic, yet you require the subpanel operations to be deterministic. To make this work correctly, you need a new thread (VI) that manages the subpanel and this subpanel manager VI must span the lifetime of all VIs inserted into the subpanel. One solution is to have a "Window Manger" subsystem that takes in refences to windows, subpanels, and VIs and manages what is visible in any window and/or subpanel at any given time. 

[drjdpowell]

I vaguely recalled trouble with InsertedVI and looked up the code and found this reference:  https://forums.ni.com/t5/Actor-Framework-Discussions/MGI-Panel-Actor-potential-crash/m-p/4344991#M77...