topic Re: LabVIEW Continuous Integration License? in Continuous Integration

LabVIEW Continuous Integration License?

Sam_Sharp — Wed, 07 Jul 2021 17:14:28 GMT

Has anyone bought/tried the new NI Continuous Integration license mentioned here? https://forums.ni.com/t5/LabVIEW-Idea-Exchange/Dedicated-License-for-using-LabVIEW-in-Continuous-Integration/idi-p/3708841?profile.language=en

I cannot find any information about this license on NIs website and the only references I can find to it are on Farnell/Newark etc. (e.g. https://www.newark.com/ni/786474-35/labview-continuous-integration/dp/15AJ1672)

We're looking at moving our current build setup to one or more AWS VMs. Ideally we'd like to have a master image with LabVIEW installed that we can clone as/when needed for new projects.

Some questions:

How does this license work? Do we have to have one license per AWS VM instance (what if we take an image and clone it)? Can we 'pool' the licenses together on our VLM installation (i.e. so if we had 2 licenses we could only build 2 projects simultaneously)?
Does it only work with specific/newer versions of LabVIEW or can we use it with older projects in LabVIEW 2013/2017?
Are there any practical limitations in terms of the IDE...can we still automate unit tests / VI analyser etc.? If we need to debug a build step, can we open it up in the IDE.

Re: LabVIEW Continuous Integration License?

Jens_S — Fri, 09 Jul 2021 10:25:16 GMT

I have been working a lot with LabVIEW CI/CD lately and the licence model would also interest me. It's funny that there doesn't seem to be any public information about this.

Re: LabVIEW Continuous Integration License?

EricR — Fri, 09 Jul 2021 19:34:20 GMT

@Sam_Sharp

Hi. I'm going to provide answers to your questions here but please DM me for follow up on this.

The CI license was developed as purely a license. When you buy it, NI ships you an activation code and nothing else. This license assumes you already own development copies of LabVIEW and simply provide a license key to use your existing LabVIEW on a CI/CD machine, specifically for automated build/test scenarios, not development. The license doesn't really get into "VM" scenarios but the way it was designed, as long as you are only spinning up 1 CI machine at a time, you can use it for as many images as you need. Internal to NI we constantly spin up and destroy images and the software we use for building/testing has similar licenses.

Having said that, this is an area I would love to have feedback on. We have been evaluating multiple different approaches to these CI/CD workflows and I'm actively working on modifying our current approach. I'd love to know what other tools you are using on those CI/CD workflows and how they interact with VM images.

The LabVIEW 2021 release (being released in the very near future) has several changes related to this, including being able to be fully automated for the installation so you no longer need to have the software activated on the images you spin up, you can make the activation part of the image spin-up and do it all through command-line calls. There are other changes I'm working on as well so please email/DM me to talk about this more.

Re: LabVIEW Continuous Integration License?

Taggart — Fri, 09 Jul 2021 20:13:57 GMT

Eric,

I'm not sure how feasible this is, but in my mind the ultimate solution would be for NI to maintain of set LabVIEW Docker images. You just pull one and spin it up as needed. Somehow just limit access to people with a current SSP, or maybe some additional addon that cost a few extra bucks.

Sam

Re: LabVIEW Continuous Integration License?

joerg.hampel — Fri, 09 Jul 2021 21:17:33 GMT

For how we and our customers use server-side automation*, installing LabVIEW for each pipeline (or even for each job) doesn’t sound feasible. If it’s more or less a regular LV install process, that would take way too much time I think. We‘re already looking at ways to streamline processes like VI Analyzer or generation of documentation. Adding a full installation would be just the opposite.

* We’re not really „integrating continuously“ (although we use the GitLab CI engine, Jenkins or Azure Pipelines to trigger automated processes) and I bet most people here don’t either. Or do you?

Re: LabVIEW Continuous Integration License?

Taggart — Fri, 09 Jul 2021 21:17:35 GMT

I agree the install would kill one of the main advantages of CI which is rapid feedback.

A Docker Image or Vagrant Box with LabVIEW preinstalled would solve a lot of that. You still might have to install a few addons, but you could build your own Docker with the LabVIEW one as the base. That would be a one-time hit.

Re: LabVIEW Continuous Integration License?

Taggart — Fri, 09 Jul 2021 21:20:43 GMT

@joerg.hampel wrote:

* We’re not really „integrating continuously“ (although we use the GitLab CI engine, Jenkins or Azure Pipelines to trigger automated processes) and I bet most people here don’t either. Or do you?

I certainly try to. I can't say that I succeed on every project. Although can you really call it CI if you are a one man shop? Who am I integrating my changes with? myself?

Re: LabVIEW Continuous Integration License?

cbutcher — Sat, 10 Jul 2021 11:16:54 GMT

@joerg.hampel wrote:

For how we and our customers use server-side automation*, installing LabVIEW for each pipeline (or even for each job) doesn’t sound feasible. If it’s more or less a regular LV install process, that would take way too much time I think. We‘re already looking at ways to streamline processes like VI Analyzer or generation of documentation. Adding a full installation would be just the opposite.

Perhaps I misread Eric's comment, but I got the impression that the goal with 2021 (I'm guessing that's the comment you're posting in relation to here?) is to allow installation in some image/VM/container/blob file on disk (whatever), but not activate it.

Then when you run the image/VM/container/blob-file-on-disk, you'd activate the license as part of the process (and presumably then deactivate it after finishing).

My hopeful reading of this is that it will simplify the relationship between (number of VMs/containers) and (number of licenses) when containers/VMs/images are not in continuous use...

If we get really lucky(?), then it might make it so we can license containers (not images, here describing Docker but similar ideas probably apply elsewhere) as we use them, and that will (perhaps?) remove some of the difficulties when taking an image created on one machine (e.g. developer PC, maybe using a different host license) and creating containers from it on another (e.g. build machine, maybe using the CI license - although my CI runs with a dev license).

@joerg.hampel wrote:

* We’re not really „integrating continuously“ (although we use the GitLab CI engine, Jenkins or Azure Pipelines to trigger automated processes) and I bet most people here don’t either. Or do you?

I had to go read a definition of "Continuous Integration" again to try figure out what you mean, but I'm still not certain. The definition I read does make it seem like Sam's comment on single-person setups falls away from CI, but for me, my Jenkins server has a few pipelines and one of those builds all of my PPLs for multiple targets. When I want to update code, I use a specific tag in my git commits and then it triggers builds via a GitHub webhook notification.

If I can (find time to, and then succeed at) implement these build processes instead in Docker containers (currently they run on the VM running the Jenkins instance, i.e. the root/master node) then I will presumably cut build times significantly (because I can run debug/release and Win32/64+cRIO in parallel, a 6x improvement in an ideal case, although some steps would need to be duplicated in each case (cloning repos, etc) so it wouldn't be quite that good...)

The stumbling block isn't actually with NI - although an easier time creating/managing/licensing images and containers would certainly help - for me my issues are rather more mundane and relate to nested virtualization being unavailable on my current host... I also want to move to NIPKGs perhaps but there are a few details I haven't made my mind up about and I had hoped that I could rearrange things to improve debugging PPLs on cRIO, but apparently that's a "wont-fix" bug.

Re: LabVIEW Continuous Integration License?

EricR — Sat, 10 Jul 2021 14:56:50 GMT

@Sam Taggart,

If you show me how to get Docker to install DRIVERS as well, I'm all in on providing Docker Images as an alternative installation. I'd love to do a whole brainstorming on just that one topic because I really love the workflow of a basically-zero-install for all kinds of software. I just can't get the virtualizing "layers" that exist today to handle all of the software I need for a good T&M application. There is always some part that they want me to just include in the base install layer.

The main thing we added in 2021 is the ability to leave the LabVIEW layer unactivated (since you aren't sure of which computer you are spinning the image up on) and just calling the activation as a setup step using command lines/scripts. You couldn't do that before so the activation step was always a pain.

Re: LabVIEW Continuous Integration License?

cbutcher — Sat, 10 Jul 2021 15:13:04 GMT

@EricR wrote:

If you show me how to get Docker to install DRIVERS as well, I'm all in on providing Docker Images as an alternative installation. I'd love to do a whole brainstorming on just that one topic because I really love the workflow of a basically-zero-install for all kinds of software. I just can't get the virtualizing "layers" that exist today to handle all of the software I need for a good T&M application. There is always some part that they want me to just include in the base install layer.

Do you want the drivers to work? As in, do you want your actual T&M application to run from a Docker container? Or are you instead wanting to build something (perhaps exe) in a Docker container that uses the drivers (e.g. DAQmx)?

I demoed using Docker to build an application for cRIO at last year's GLA summit, if the latter (including running it successfully on actual hardware), and separately I believe I had installable DAQmx builds, although my demo didn't include any DAQmx code...

For the former, I'm not so sure - but I think NI/you could perhaps make it possible... If you DM me or send an email, I can give you more information about the specific nipkg files that I had problems with installing inside a Docker container - you might be able to indicate if you believe these are true blockers or not...

@EricR wrote:
The main thing we added in 2021 is the ability to leave the LabVIEW layer unactivated (since you aren't sure of which computer you are spinning the image up on) and just calling the activation as a setup step using command lines/scripts. You couldn't do that before so the activation step was always a pain.

I'm hopeful that this is great news. Will it work with all licenses? (I believe VLMs already had an easier time with Docker...)

What are the restrictions? (Or will this information be part of NI-Connect and not yet publicly shared?)

Re: LabVIEW Continuous Integration License?

Taggart — Sat, 10 Jul 2021 15:25:05 GMT

I was about to suggest you talk to Christian, but he beat me to it.

Re: LabVIEW Continuous Integration License?

Taggart — Sat, 10 Jul 2021 15:27:29 GMT

The seperate activation is awesome, because now you could use Vagrant with VMware for Dev systems. Just store the license key in a environment variable and have vagrant register it when it starts up. So just setup one Vagrant box for a project with all the drivers you need and then you can distribute it to the whole team.

Re: LabVIEW Continuous Integration License?

joerg.hampel — Sat, 10 Jul 2021 18:46:05 GMT

@cbutcher wrote:

Perhaps I misread Eric's comment, but I got the impression that the goal with 2021 (I'm guessing that's the comment you're posting in relation to here?) is to allow installation in some image/VM/container/blob file on disk (whatever), but not activate it.

You're right, Christian, I misread Eric's comment (and he clarified it in the meantime).

Our own build machines are VMWare virtual machines hosted on an ESXi server running vSphere. These VMs are "always-on". The way we do things, they need to be alway on because they are running the runners/agents which receive and execute jobs directly on those VMs. FYI, our "CI/CD" customers mostly run physical build servers and don't use virtualisation yet.

Looking at our current situation, and how we use SCC-triggered server-side automation, I prefer having those build VMs running continuously. I can do without the time it takes for a windows machine to boot up - as I mentioned, we're constantly trying to optimise steps in our toolchain to make pipelines execute faster.

Another point for us is that we reuse build VMs across projects where possible. We usually have one VM per LabVIEW version, as most of our projects are on RT/FPGA and don't involve drivers. So we don't have to create new VMs very often.

Still, I'm very curious about the new feature in 2021. Regardless of whether the VM/image/container is running continuously or not, we could still enable/disable the license with each job and end up with a pretty fast system.

@EricR, do you have a ballpark number for how much time the activation of a license would take?

Re: LabVIEW Continuous Integration License?

joerg.hampel — Sat, 10 Jul 2021 19:13:15 GMT

@cbutcher wrote:

I had to go read a definition of "Continuous Integration" again to try figure out what you mean, but I'm still not certain. The definition I read does make it seem like Sam's comment on single-person setups falls away from CI, but for me, my Jenkins server has a few pipelines and one of those builds all of my PPLs for multiple targets. When I want to update code, I use a specific tag in my git commits and then it triggers builds via a GitHub webhook notification.

I really don't want to derail this thread, so I created a new one for this discussion.

Re: LabVIEW Continuous Integration License?

joerg.hampel — Sun, 11 Jul 2021 18:51:06 GMT

@joerg.hampel wrote:

Looking at our current situation, and how we use SCC-triggered server-side automation, I prefer having those build VMs running continuously. I can do without the time it takes for a windows machine to boot up - as I mentioned, we're constantly trying to optimise steps in our toolchain to make pipelines execute faster.

I want to change my position here a bit. Disk space is much cheaper than RAM, and it would indeed be handy to have some rarely used VMs stored on the build server and not always on. If they would spin up just in those rare cases where we need to build some exotic project, and if that would happen automatically, that would be handy.

Re: LabVIEW Continuous Integration License?

cbutcher — Mon, 16 Aug 2021 09:02:01 GMT

@EricR wrote:

The main thing we added in 2021 is the ability to leave the LabVIEW layer unactivated (since you aren't sure of which computer you are spinning the image up on) and just calling the activation as a setup step using command lines/scripts. You couldn't do that before so the activation step was always a pain.

So I'm really excited about this, but so far I didn't see anything about it in Readmes for LabVIEW, NILicenseManager or the /help argument to NiLicensingCmd dot exe (all the 2021 versions, weird formatting in the last bit to avoid a forum block).

Obviously it's only been ~1 day since these things were made available (at least that I noticed), but is there some information about using these options? (please let them not have been cut... :s )

This command:

NILicensingCmd#exe /activateall /serialnumber <my serial number>

still popped up a GUI for me on my Windows dev machine, which makes me less confident it is the intended method.

(It also prompted me with the message "ERROR: The name is necessary for the activation of the product".)

NILicensingCmd#exe /silent /activateall /serialnumber <my serial number>

Adding the silent flag prevented the message (as expected) and also the popup (not explicit in the description), so perhaps this is the required call?

Using the GUI for NILM to deactivate LabVIEW 2021 (and a few, but not all modules) , closing the GUI, and then running the same command, and then reopening the GUI shows LabVIEW 2021 as "Unlicensed", which I guess means it didn't work?

I'm currently trying a few variations on the single /activate option, but the documentation here: ni-license-manager/21.0/manual/automate-activation makes it look like I might struggle in the case that I'm not logged in already, e.g. in a Docker container (but testing will tell, I suppose).

.\NILicensingCmd#exe /silent /activate /type package /name "LabVIEW_PDS_PKG" /version 21.0000 /serialnumber <yay serial>

This command successfully activated LabVIEW Prof. Dev System, and seemingly did so without popping up any windows. I was, however, already previously logged in to NI-Auth.

I'll test it in Docker and post again, hopefully soon(ish, read days not hours but hopefully not weeks...)

Re: LabVIEW Continuous Integration License?

felipefoz — Sun, 22 Aug 2021 08:35:35 GMT

@EricR wrote:

Having said that, this is an area I would love to have feedback on. We have been evaluating multiple different approaches to these CI/CD workflows and I'm actively working on modifying our current approach. I'd love to know what other tools you are using on those CI/CD workflows and how they interact with VM images.

I've been for a while running successfully some pipelines with docker images, not very complicated pipelines (like FPGA, RT), with simple images that allowed me to run Unit Tests, VI Analyzer, perform some builds, etc...

Here are some examples:

https://gitlab.com/felipe_public/gitlab-tools-labview/build-cli-operation/-/pipelines/276011891

https://gitlab.com/felipe_public/labview-shared-libraries/broker_af/broker/-/pipelines/279521632

I am already trying to foment this idea of Docker images through this Idea Exchange Post, any new kudos would help it out, wouldn't it?

https://forums.ni.com/t5/LabVIEW-Idea-Exchange/Official-LabVIEW-Docker-Images/idi-p/4158952

I already have some "recipes" for building the most common LabVIEW images, which work, but, it could use some optimization from NI, especially in shrinking size.

https://gitlab.com/felipe_public/labview-docker-files

About licensing, after reading Christian's post apparently nothing has changed since LV 2020. In my case, the only feasible way was using NI VLM Server with named or on-demand licenses, but this has a limitation for some people because it requires at least 5 licenses to allow this kind of management.

Anyhow, I am also available to help in any improvements, feel free to contact me @EricR.

Re: LabVIEW Continuous Integration License?

cbutcher — Mon, 23 Aug 2021 08:03:35 GMT

Initial attempts with /silent /activate in Docker were unsuccessful. Will keep trying but not a great start... Maybe some different combination of package and family options?

Re: LabVIEW Continuous Integration License?

JeKo — Mon, 23 Aug 2021 09:32:46 GMT

We use:
"C:\Program Files (x86)\National Instruments\Shared\License Manager\Bin\nilmUtil.exe" -s -activate "$productcode" -serialnumber "$serial" -firstname "$first_name" -lastname "$last_name" -organizationname "$org"

where $productcode is a code like:
"LabVIEW_PDS_PKG 17.0001" or "LabVIEW_FPGA_PKG 17.0001"
I didn't find a good list of the product codes, I read them from the license files on disk.

Re: LabVIEW Continuous Integration License?

cbutcher — Mon, 23 Aug 2021 16:16:48 GMT

@JeKo wrote:

We use:
"C:\Program Files (x86)\National Instruments\Shared\License Manager\Bin\nilmUtil.exe" -s -activate "$productcode" -serialnumber "$serial" -firstname "$first_name" -lastname "$last_name" -organizationname "$org"

where $productcode is a code like:
"LabVIEW_PDS_PKG 17.0001" or "LabVIEW_FPGA_PKG 17.0001"
I didn't find a good list of the product codes, I read them from the license files on disk.

I couldn't quite make this work at first, but either it worked and I was impatient, or it worked changing the product code to "LabVIEW_PDS_PKG" and adding -version "21.0000".

In any case, this looks good and seems to have worked. A quick check with G-CLI looks promising.