DQMH Consortium Toolkits Feature Requests

Credit goes to doyles for initially coming up with this idea.

Go here for previous discussions:

https://forums.ni.com/t5/Delacor-Toolkits-Discussions/Feature-request-Automate-the-Helper-Loop-creat...

1. Include a "checkbox" onto the Add New DQMH Module dialog panel that is labelled "Include a Helper Loop"
   
    

   

    

2. If the user checks this checkbox, a helper loop is automatically added to the Main.vi
3. The helper loop would not be a sub-VI, but simply a third loop on the main.vi block diagram.
4. A Wake up Helper Loop request is automatically created and included in a Private Requests virtual folder
5. Make the helper loop generic as per Sam's suggestion. My suggestion is to have three user events: 1) Timeout 2) <Stop Module> 3) <Wakeup Helper Loop>
6. Label the additional "Register for Events" node something different from the other "Register for Events" node, ie. DQMH_REG_EVENTS_HELPER_LOOP (so that the Validate tool does not raise it as an issue)
7. When generating a helper loop for new cloneable modules, ensure that in the "Wakeup Helper Loop" and "Stop Module" user events, that the Addressed to this Module.vi is used.

This feature request is adding to the already existing "Private Request" and "Helper Loop Creation" requests (or not?):

When working with helper loops in cloneable modules, it would be nice to have a way for sending messages from the MHL to the helper loop which doesn't need the module ID, and which doesn't interfere with other clones' helper loop timings. I'd still like this new mechanism to look and feel like the regular events, with all the scripting and other goodness.

Maybe instead of calling them private events, a better name would be "local" events (which would, of course, be set to private scope).

Many DQMH developers (including myself!) encounter the stumbling block of firing requests before a module is "ready". We assume that the following code is sufficient for starting and synchronizing a module:



The 'Synchronize Module Events' VI ensures that request event references are valid, but it doesn't do anything about ensuring that the module is actually ready. One very common scenario is a module Main VI that includes code in the 'Initialize' frame of the MHL that fires private events to initialize Helper Loops. If the VI that called Start Module isn't waiting on the Module Did Init broadcast, then it could immediately fire a request after calling Synchronize Module Events, and if a helper loop is registered for that event, then helper loop could run the event frame for that fired event before the MHL has had a chance to fire the private event to initialize the helper loop. There are many other possible pitfalls, but this is one of the more common ones I've seen.

To solve this issue, I think DQMH should ship with a Public API VI that is a wrapper for Start Module + Synchronize Module Events, but includes the extra step of waiting for 'Module Did Init'. The module author needs to ensure that the Module Did Init broadcast is called after all necessary initialization code... if they do so, then it is "safe" to begin calling events externally.

Here's my proposed interface for the Start Module and Wait for Initialization VI:

And my proposed implementation:

If this VI shipped with the DQMH singleton and cloneable module templates, it would help us avoid module synchronization + initialization issues like what I describe above.

We have identified that, if multiple clonable module instances are executed and a specific module (e.g. the first launched module) is stopped (no waiting) and afterward all open modules are stopped at once (incl. wait) the "stop module.vi" an error 1 is returning.

Steps to reproduce

1. Create a project and add a new clonable module
2. Create a tester VI and implement the code above 
3. Run the VI and see error 1 at second Stop Module.

What is happening in the Background

Situation 1 - "Stall Data Flow" = 0

1. "Stop Module.vi" 2 runs into "Wait on Stop Sync.vi" and synchronizes stop over rendezvous.
   The acquired rendezvous size is 3 (Module 1 which is at stopping, Module 2 and Stop Module)
2. Module 1 is waiting in "Safe to Destroy Refnums.vi"
3. Module 2 runs into "Close Module.vi"
   

    

   1. Last clone instance is fire at (1)
   2. Releasing the Semaphore (2) will wake up module 1 that it is now safe to destroy refnums now.
      Module 2 runs into "Wait on Stop Sync.vi" (3) and synchronizes over rendezvous.
   3. "Stop Module.vi" and Module 2 waiting for a third participation to join the rendezvous.
   4. Module executes case to destroy Master reference.... and executes "Wait on Stop Sync.vi" (3) with no synchronization as the boolean "Wait for Module to stop?" is on false.
   5. Module 1 executes "Destroy Sync Refnums.vi" (4) and is destroying the rendezvous.
   6. Module 2 and "Stop Module.vi" will be release from the waiting of the rendezvous as the reference is now invalid and returning error 1.

Situation 2 - "Stall Data Flow" = e.g. 2000ms

1. In compare to the situation 1 the first module is already removed here. The obtained rendezvous has the expected size of 2.
2. When module 2 enters rendezvous synchronization in "Wait on Stop Sync.vi" (3) the expected amount of participant is reached, and the execution can continue.
3. In most of our tests, this situation worked fine and did not create an error.
4. For some situation, we had the behavior that shutdown of the first module seems to be faster as the wake-up from rendezvous of the second module. The module main of module 2 opened and showed error 1. Module 1 seems to destroy the references to early. 

Situation 3 - First Module will be stopped with last "Stop Module.vi" call

The shutdown of a module is for this scenario delayed (add a wait 1000ms to exit case of the module)

1. Stop Module 2 waits for 11 rendezvous participations. (10 module and itself)
2. One of the previously closed module will destroy the synchronization events and makes the rendezvous reference invalid. => Error 1 at "Stop Module.vi"

Potential Fix

Spoiler

The following screenshots are showing an extension of the "Stop Module.vi" and the "Close Module.vi".

The idea is to use a single element queue (SEQ) containing a map of sets. The key of the map refers to a "Stop Module.vi" which waits for stopping all module at the time when the "Stop Module.vi" is executed. The Set contains all Module ID's which should be stopped. Each module checks in its close condition if the SEQ is existing. If so, the module ID will be removed from the sets which containing the module ID. An empty set refers to all required modules have stopped and a notifier which is used for synchronization will be fired.
Close Module.vi extention
Stop Module.vi extention

 

With those extensions, all three described scenarios should be fixed. In addition, should it be possible to stop all module and launch in the background new ones, the stop and wait will wait until all those modules ID run at the stop execution are finished.

 

I added the project which the extensions and tests to the post.

Addition

I'm not sure, but I think that with the described change, destroy of the Module's Semaphore (1) should be done with the boolean condition of the First & Last Instance (2). (Red line)

Please let me know if you need any additional information and details.

The problem I want to fix:

I want to be able to try to fix or poke the code more when an error occurs in the API Tester. The opportunity of going to the block diagram or attempting to send a different request goes away because the API Tester, by default, closes when an error occurs.

How I propose fixing it:

The first thing that I change on an API Tester at the first error is to remove the OR connected to the stop terminal in the loop and add an error indicator at the end instead of the Simple Error Handler and connect a local variable for the error. Would you please vote to have DQMH do this by default and have a validator to modify existing API Testers?

Current API Tester:

Proposed change:

Thanks,

Fab (yes, being the DQMH Lead Architect does not guarantee that all my wishes are turned into reality 😉 )

Please create an additional toolbar named "Delacor" that has at least 2 buttons: one to create a new module, one to create a new event.

It's too tedious to go everytime via menu Tools, Delacor, DQMH, bla bla

In this picture you can see on the right the JKI tester toolbar, and also the NI Unit Test Framework toolbar.

thanks

I would like a right-click plugin where I could right-click a DQMH broadcast subVI and "Find Event Frames". This would search all VIs in my current project for event structures that are registered for the broadcast event fired by the subVI I clicked on, and show me a list of results that I could double-click and be shown the event frames one at a time. Or I guess it could just open all the diagrams for me, since I probably want to walk through all of them anyway.

The operation could take a while, but would be worth it. I often find myself wondering where all the places are in my code that are registered for a particular broadcast.

(idea originally posted here)

When I create a new Broadcast, here's what the new frame in the API Tester looks like:

I almost always end up adding the following code by hand (usually by copying it from another frame):

It seems to me that most of this code could be scripted. Maybe everything except the value of the first format string?

(idea originally posted here)

When creating a new Event, it would be empowering to be able to automatically call a VI immediately after the scripting completes. The idea is based on the behavior of the Pre- and Post-Build VI calling ability of the application build process. The implementation should provide an option to specify a VI to call and an option to generate a VI with the proper controls already on the connector pane. A proposed set of such controls would be VI references or paths to the Event VI or VIs (Broadcast or Request... or both for Round Trip) and VI references or paths to the Tester VI and RT Tester VI (if there is one).

Adding the feature would allow developers using DQMH to create their own extensions to the built-in scripting. For example, if the other feature request here is not accepted, this one would allow a developer to implement it on our own.

In our current practice, we encapsulate the content of DQMH MHL events within subVIs to maintain a tidy diagram and especially to facilitate the seamless propagation of changes to similar projects that use the same implementation. 

I think It would greatly benefit the DQMH framework if it could autonomously generate these subVIs as part of its MHL event scripting process.

Current situation:

Expected situation:

Today DQMH scripters add comments with specific hashtags. But some of these hashtags names are very generic (for example #CodeNeeded or #CodeRecommended).

Adding a 'DQMH_' prefix would allow a better isolation of DQMH related hashtags in the bookmark manager (ie. #DQMH_CodeNeeded).

Actually some DQMH tags already have this prefix (#DQMH_HowTo).

When adding an helper loop from the scripter, add a checkbox that would allow creating a subVI that contains the helper loop.

This checkbox should be unchecked by default.

If checked, a (private) VI is create and contains the helper loop.

The subVI is then dropped inside the Main.vi where the helper loop would have been created.

When creating new module, I'd like a way to add a text explaining its responsibility. It will reinforce good conception practices and allow Antidoc to retrieve information to generate a valuable documentation.

Note: IMO, this content should be added to the module lvlib description.

If this field could optionally mandatory to create the module, I would find this great 🙂

When the DQMH Consortium introduced the support for private requests with DQMH 7.0, a conscious decision was made not to support synchronous requests with reply, to avoid dead-lock scenarios.

With the way we at HSE start additional DQMH modules from any given DQMH module (by sending a request from the MHL to the EHL to start and register for modules, then reply back to the calling MHL case, so the whole process becomes synchronous), it would be super helpful if we could script those private "start submodule" requests the exact same way as all other private requests.

TL;DR: Add support for private Request and Wait for Reply.

Edit: After internal discussion, let me add that I know how to make a regular request a private one by setting the access scope. I don’t want to do that - I want the private Request with Reply to be the same as all other private events, and not a modified public request.

Edit2: If you scroll down, you'll see that Manu added a vital piece of information. We usually communicate with one of the EHLs/Helper Loops, not the MHL. So scripting should not create another case in the MHL but keep the reply in the targeted EHL/Helper Loop.

Once a Broadcast happens, only modules that are registered for it can access the value contained in the Broadcast. If a module starts execution and registers for Broadcasts from another module after that module has sent a Broadcast, the registering module must wait until the Broadcast happens again before it has the latest value. We have encountered some use cases where it is beneficial to know the most recent value of a particular Broadcast. In those cases, we have created workarounds such as a "Refresh" Request that can force a module to repeat all or a subset of its Broadcasts. Of course, that requires the addition of a Request and the code to handle it, etc.

The feature request is to have the Broadcast scripting create two additional VIs for each Broadcast. One VI would be Private, and it would be a data store for the value of the Broadcast event (message). The other VI would be Public, and it would be a Read Accessor for that value. I am picturing the Public VI calling the Private one, and the Private one is a Functional Global. In the Broadcast event VI, the value would get written to the Private VI's Functional Global.

Having a Public VI that contains the most recent value for the Broadcast make it easy to write small state machines or action engines that can access Broadcast value data without Event Structures.

There are other messaging architectures that implement the "Read Global" ability, and it has proven to have value for us in development of some systems.

For Cloneable modules, the "Read Accessor" could take a Module ID as an input.

IMHO, it would make more sense to have the Stop Module request at the same level than Start and Sync functions.

Original idea from Matthias Baudot in https://anchor.fm/wired-in-software/episodes/Episode-6---Matthias-Baudot-from-Studio-Bods-emtpti

When you have lots of modules (20+), initializing module selector control tends to take seconds that can be annoying when you need to use the scripting tools.

Finding a way to remove this init time would greatly improve the user experience in large application development.