I can't get Engine.NewExecution to run my sequence

[AliasRole]

I'm trying to use the 'NationalInstruments.TestStand.Interop.API' to run a sequence from Visual Studio 2017 using some C#.  What I want to do is write some code that when executed will do act exactly the TestStand application does when I load a sequence and execute a single pass test.

As I understand it, I should be able to create an engine object, use 'GetSequenceFileEx' to get a sequence object and then use the 'NewExecution' method to run the sequence. Here's the code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using NationalInstruments.TestStand.Interop.API;

namespace TestStandBridge
{
    public class TSBridge
    {
        public TSBridge()
        {
            // Instantiate the Engine for TS
            engine = new Engine();
        }

        ~TSBridge()
        {
            GC.Collect();
        }

        internal Engine engine;

        public Execution StartSequence(string sequenceFilename, string resultDir)
        {
            Execution exec = null;

            try
            {
                User admin = engine.GetUser("administrator");
                engine.CurrentUser = admin;
                
                SequenceFile seqFileModel = null;
                
                int flags = 107; 
                SequenceFile seqFile = engine.GetSequenceFileEx(sequenceFilename, flags, TypeConflictHandlerTypes.ConflictHandler_Error);

                bool modelExists;
                string modelPath = seqFile.GetModelAbsolutePath(out modelExists);
                seqFileModel = engine.GetSequenceFileEx(modelPath, GetSeqFileOptions.GetSeqFile_NoOptions, TypeConflictHandlerTypes.ConflictHandler_Error);
                                
                exec = engine.NewExecution(seqFile, "Single Pass", seqFileModel, false, 0);
                exec.Report.Location = resultDir;
                                            
                engine.ReleaseSequenceFileEx(seqFile);
                engine.ReleaseSequenceFileEx(seqFileModel);
            }
            catch (Exception)
            {
                throw;
            }

            return exec;
        }
    }
}

However, when I instantiate this class from a console application, my sequence isn't executed. I don't get any errors thrown and when I debug it, I can see that it spends ~20seconds doing the 'NewExecution' method and then just continues. I'm fairly sure my sequence isn't actually executing as it should take several minutes and it's not interacting with any of the hardware it normally would do when I run a single pass from TestStand.

The SequenceFile & seqFileModel do go to valid file locations, if I change them then the code will throw an error. As it's not throwing an error, I'm not sure what I'm doing wrong. Any ideas?

Thanks very much.

[RoxtheFox]

Hey, 

I tested your code with a TestStand shipping example, and I was able to execute the sequence file. I also added your function to our shipping C# UI and added a button that would execute the function as a callback and saw the sequence file execute in the sequenceview manager. What have you done for debugging? 

If you test it with any other sequence file, do those execute? 

Thanks

[AliasRole]

Hi, thanks for the reply and testing the code on your machine.

Well first I wanted to see if the execution had finished without performing the sequence actions or if it was getting stuck in the executing state. I added the following statements after starting the new execution as shown below:

                exec = engine.NewExecution(seqFile, "Single Pass", seqFileModel, false, 0);
                exec.Report.Location = resultDir;

                exec.GetStates(out ExecutionRunStates runState, out ExecutionTerminationStates termState);
                exec.WaitForEndEx(-1)

I put a break point on the second bold line. When the code broke, the value for runState was 'ExecRunState_Running' and termState was 'ExecTermState_Normal'. When I continue, it will just hang on that second line indefinitely without executing the sequence. So it looks like the execution is in progress, it just doesn't do anything. It got strange when I changed it to this :

                exec = engine.NewExecution(seqFile, "Single Pass", seqFileModel, false, 0);
                exec.Report.Location = resultDir;

		exec.RestartEx(0);

Then I get the error message in the screenshot below, but my sequence executes! I even get the xml report as normal, but after I close the popup, it throws an error. I assume it's not happy because I'm restarting an execution that hasn't finished yet, but the new execution actually does the job. After I close the error message, an exception occurs and goes to the catch block. I mean this isn't really workable since I need it to fall through the code gracefully but I'm a bit non-plussed as to why the first execution is behaving as it is. I'll try it with a generic sequence file and see if it behaves any differently.

Thanks again.

[AliasRole · Accepted solution]

I think I've got to the bottom of it. You're right, when I used another sequence with no steps, it executed fine. I think difference is that my sequence takes a long time to preload modules so with no 'WaitForEndEx(-1);' statement it was dropping out of the code and disposing of the execution object before anything could happen. When I tried debugging with that statement, I wasn't waiting long enough for the modules to be loaded so I thought it was hanging. If I leave that 'WaitForEndEx(-1);' in and wait for a couple of minutes, it will execute and run normally.

A mix of me not understanding how the execution runs in parallel and not appreciating the time for preloading modules. Doh. Thanks very much for your help.

[SimonDeacon]

I think this problem could have been avoided if the documentation provided with TestStand was any good. Generally we have run in to a lot of problems that have you scratching your head because the documentation is severly lacking, and the limited amount of sample code provided is another source of irritation. It's only thanks to helpful users on forums that things get solved in a timely manner.

[RoxtheFox]

I do want to point out that WaitForEndEx() can cause a hang in your console application because it cannot process UI messages that can be sent by the engine. In the help, it states that, "This method is not meant to be used by a user interface or sequence editor, as it does not process UIMessages. Instead, use this method from a step to synchronize with another execution." It is recommended that you terminate the state of the execution to monitor the execution using the GetStates function (which you have implemented). 

If we can better improve our documentation in any way or if something is unclear, please let us know. Pointing to something specific will help us make our documentation better for all users. 

[SimonDeacon]

I think the problem is that we have a quote stating in the remarks not to use it for a certain purpose, without pointing to what we should use for that purpose. I think also the remark talks specifically about a user interface or sequence editor - the console application in question (which isn't clear to me fulfills the crtieria as a user interface or sequence editor - so perhaps there is a clarity issue) just wants to execute a sequence file then terminate. My gut feeling is that this should be exceedingly simple to acheive, but hasn't proved to be the case. Note that this isn't a NI specific problem, I see it from various tool vendors that provide complex tool chains. Perhaps I have unrealistic expectations.

The API documentation that Microsoft supply, for example, seems good in terms of organisation, and is littered with small, digestible code samples and links to various other documents that walk developers through a process. I think this is a useful approach for users, especially when there is a large amount of complex functionality to be dealt with.

Perhaps we are attempting a use case for which TestStand isn't really designed, perhaps the correct way is to ditch TestStand and create a simple sequencer in LabVIEW that can be called as an exe. We like the integration of TestStand when we are using it to run tests interactively, but it seems like then trying to execute those tests as part of a CI process is perhaps not a supported use case (again, I could be missing something).