NI Package Builder CLI Error 16038 in DevOps pipeline

[Stinus Olsen]

Hi,
I'm trying to call the NI Package Builder CLI in a DevOps pipeline, but has so far not had any success.
The call always fails with an error -16038: An error occurred while loading the solution: 'C:\DevOpsWorkspace\10\s\trunk\SW\Package Builder\xxxxxxxx.pbs'.

This is very odd, because the file does indeed exist - and if I invoke the CLI from a command (or PowerShell) prompt, on the machine that contains the local Azure build agent setup, everything works just fine.

The NIPB solution also builds flawlessly on my development machine.

The solution does contain a lot of relative paths, but these doesn't point to any network paths or removable drives, only files in the local hierarchy.
I've also checked that the account the Azure build agent uses (NETWORK SERVICE) has full access to all files referenced by the solution.

Am I missing something?

NI Package Builder info from CLI:

NIPB version 22.5.1.49816

NIPB bitness 64-bit

NIPM version 23.0.0.49275

Operating System Windows 10 64-bit

PS: As seen in the Azure Build Agent Output image, I temporarily tried appending the "--teststand-env-do-not-use-for-....." flags to the command line, but this had no effect on the outcome.

[Scott_Richardson]

Hi Stinus Olsen, NI Package Builder writes to a log file in the same directory as the solution, and there is a more detailed log file created in the "%localappdata%\National Instruments\NI Package Builder\Logs\" directory. Curious, is the directory of the solution writable?

I would recommend reviewing the more details file to see if there is more information. You can share a redacted version here or if you want to share more details with me directly in a private chat.

[Stinus Olsen]

Hi Scott,
Thanks for answering this fast.

Sorry for leaving out that piece of information in my initial post, but no logs are actually generated.
Neither in the solution folder, nor in the AppData folder.

I do see the log files on my development system, and when invoking the CLI manually on the build server - but thought that the missing files when running through the pipeline were due to the error being generated so early.

Am I struggling with a more fundamental access issue here?

[Scott_Richardson]

I am not sure of what the underlying issue is here. I verified that if the directory where the solution is located is read-only the solution opens successfully but the opening of the log file fails with "Error -16062: Unable to open the solution log file", so this is not your error.

1) Have you tried to just open a simple solution file, i.e. maybe one that is empty or just builds an empty package? Maybe skip the --build command and the --testStand commands? If that works, try the same with your solution.

2) What "user" is running the service to execute the pipeline? Is it an interactive user account or a maybe a service account?

3) Can you look at Windows Events to see if there is anything logged for the NIPB process?

[Stinus Olsen]

So, I have some more information on this issue.

I created a very small .NET console application and named it the same as the NIPB CLI executable.
Then I temporarily replaced the NI executable with my own, on the build server.

The sole purpose of the console application is to dump which account it is executed by, and dump the rights to the file in the "-o|open" argument.

The output from that step can be seen in the attached image (note the Windows SIDs - info on these can be found here: Well Known Windows SIDs).

The SID "S-1-5-21-3721774868-2935199446-3188169372-1001" is the ID for the local Azure build agent group, which has NETWORK SERVICE as the only member.

As can be seen from the output, the executable is running as the NETWORK SERVICE user, which is also the user the Azure build agent service is started as.
I can also see that the solution file itself is owned by that same user (since that user created it on disk in the first place).
Also, that user has full Read and Write access to the file (and the whole containing folder hierarchy).

I attached a snippet of the Advanced File Rights window, for the solution file.

Regarding the Windows Event log - nothing special of interest is generated, except that the build failed (see image).

I guess the next step will be to try your first suggestion; a small-scale solution file..

Stay tuned 🙂

[Scott_Richardson]

Stinus, I discovered that for service accounts, application logs will show up under %systemroot%\ServiceProfiles, so you might check the following:

• %systemroot%\ServiceProfiles\LocalService\AppData\Local\National Instruments\NI Package Builder\Logs\

or maybe,

• %systemroot%\ServiceProfiles\NetworkService\AppData\Local\National Instruments\NI Package Builder\Logs\

[Stinus Olsen · Accepted solution]

So, for everyone reading this thread - I ended up talking to Scott in a PM thread.

It seems the culprit is with the rights that the user group, that is created when setting up the local Azure Agent, has assigned.

Apparently it lacks some access rights for some of the files that are used by the NI Package Builder.

Which files that is, I unfortunately haven't had time to really dig into.

My solution was to change the Log On setting for the Azure Agent service, to use the 'Local System account' instead of the special Azure Agent group - that immediately fixed my issues.

A big thank you goes out to Scott for taking his time with this 👌