validate modules in parallel

[AlexElb]

Running the validator on bigger projects takes quite long.

Would it be possible to speed up the process by running the validation of the modules in parallel?

Additionally: Also in CI/CD we know if the module has changed at all. It would be nice if the cli-module-validation could be called with a .lvlib instead of a .lvproj. Therefore, it could be handy to have a VI which lists all dqmh modules in a .lvproj.

[Darren]

All of the project VI Server code that runs when you're validating modules runs in the UI thread. Parallelization would have no noticeable benefit. There are definitely optimizations that could be made in that code, however. So any sort of performance improvement to the validation code would come from optimizing the scripting code itself, not from parallelization.

[AlexElb]

Thank you, Darren.

I guess what would already help speeding up our pipeline would be:

• Having a VI which only detects all DQMH modules in a .lvproj.
• Having a VI which takes an array of DQMH modules and only validated those

Often our DQMH modules are already validated, because e.g. they are released versions in a sub-repository, or because they have not been changed since the last release. We could then just skip validation on like 85% of our modules.

If I could help to implement the changes, please reach out to me.

Edit: the VI which detects all modules in a project already exists. https://wiki.dqmh.org/_media/marketing/what_s_new_in_dqmh_6.1.pdf

[Darren]

There's a lot of library item parsing that each individual validation test currently does, and I think instead that parsing could be done at the framework level. As a general rule, opening library/VI references is the slowest part of any scripting app, so being able to do that only once at the beginning should speed up the whole validation process a great deal.

[mbaudot]

Hey Alex, thank you so much for your trust in DQMH and for taking time out of your day to share your idea with us.

Based on our review, parallelization would have no noticeable benefit.

So, we decline this request as it is written, at least for the next DQMH version.

We will however consider speeding up the validation process and being able to pass a list of Modules to validate.

Again, thank you for your input; it is most appreciated. Please keep those ideas coming!

[Olivier-JOURDAN]

Available in DQMH 7.1