Crash in application by using TestStand 4.0 and CVI labWindows 8.01

[sundog]

Our application often crashes in two areas:

1) Often crash in the case:

    During running, if there is poop, like error message, you want to debug by opening other programs,like notepad. 

    It shows error signature:

    AppName: operator Interface.exe     AppVer. 4.0.0.326, ModName  cvirte.dll

    ModVer: 8.1.0.271         Offset : 000b9c8a

2) Often crash at the end of test, when generating TSV file.

     AppName: operator Interface.exe     AppVer. 4.0.0.326, ModName  msvcr80.dll

    ModVer: 8.0.40727.762         Offset : 00008a8c

We use TestStand 4.0 and CVI LabWindows 8.01. The program is running in our customized operator interface.

But I tried using the default TestStand operator interface, it crashed too.

Also we found it is related PC too. One PC in our lab crashed often, but it actually newer PC with bigger RAM. (Pentium(R) 4 CPU 3.2GHz), 3.19GHz, 0.99 GB of RAM)

The one crashed Less often is Pentium(R) 4 CPU 2.80GHz, 2.79 GHZ 504 MB of RAM.

Hope you can help us to find out root cause.

Thanks 

[herosphro]

sundog,

Unfortunately there isn't a whole lot of helpful information in the error codes you posted.  This happens when we run into crashes without the development environments to help catch the errors. 

There are a couple things you could do to help diagnose the issue.  Can you run this program in the TestStand Sequence Editor (not just the default OI, the full TestStand development environment)? If so does the error still occur? If it does occur, does it give you any more information, perhaps error messages that describe the problem?  Could you describe more about what the sequence is doing? Is it using any protocols to talk to instruments (serial, gpib, etc)? What types of code modules are you using?  Does the crash that happens during the sequence happen at the same place every time?  If so, does this step always fail or does it pass sometimes too?

If you wanted to post some screenshots of the error, and perhaps even the sequence file (if its not to complicated) this would also help troubleshoot this problem.

Message Edited by herosphro on 08-18-2008 01:26 PM

[sundog]

Hi, John:

Thanks for your information. I will try "full TestStand development environment" to debug it first. 

And I will let you know later. May I know your email address and I directly write an email to you? I am not sure if I should post my company's stuff here. 

But the crash doesn't happen all the time, at the same place. 

Thanks

Sundog 

[herosphro]

sundog,

If you want to correspond by email, which I understand, the best way is to create an email service request from this link.  This way it can go through the proper channels to track the issue.  Simply follow the link, select the 'Email NI' option, fill out the boxes (I would file this issue under TestStand as the software since its the main offender), click 'Solutions' at the bottom, and on the next screen click 'Email NI.'  After you create this we will be in contact with you shortly to help further troubleshoot, and via this method you won't have to post any sensitive information on the forums.

[Amit Ahuja]

Hi

We are also facing the same problem like after running for some time the test sequence 

crashes with the same error as told in the first message in the post.

Can anybody tell if some solution has been found for the same?

It would be really helpful.

Please find attached the snapshot for the same.

Regards

Amit Ahuja

[herosphro]

Amit,

Unfortunately there is not set procedure or steps to fix this issue.  The previous problem was resolved but it involved a great deal of troubleshooting and testing.  Because of the nature of this problem it is something that will probably require a unique solution for your situation also.

For a start you could answer the following quetions:

With the previous issue on this forum, they were using an Operator Interface, are you also?  If so, do you still get the error when you run it in the sequence editor?  Does this occur on other systems or just one PC? If this error does appear on more than one PC, can you post a small piece of sample code that reproduces the error and we can take a look? Is there a particular sequence that causes the error?  If so, does the error happen only at one step or does it vary?

Message Edited by herosphro on 06-01-2009 08:30 AM

[RayFarmer]

Hi Amit,

Looks like you were running some sort of loop and had done at least one iteration, how many times had you been round the loop before the crash?

Regards

Ray Farmer

[kbsave]

Hi John, Ray,

Thanks for your reply.

The problem we are facing (Amit and myself) is random. It is not specific to any particular sequence.

John:

1. We are not using Operator Interface. We are running the sequence in TestStand Sequence Editor.

2. This problem is not specific to any PC. It happens on multiple PCs with same environment.

3.  We are yet to pinpoint any particular piece of code which causes this issue. Actually we have observed that when TS crashed, the sequence execution was on various different steps (not any particular part of sequence). Yet another onbervation is, many times when it crashed, it had called a VI in labview which waits for a notifier. This VI was successfully running(waiting for notification) but the sequence editor crashed.

Ray:

1. Yes, we were running loops  at the time sequence editor crashed. The loop iterations varry. Sometimes it crashes after more than 100 iterations and some times it crashes just after 5/6 iterations. Apart from the notifier step mentioned above, the another step on which we observed the crash, was a small while loop, having only a wait statement of 1 sec. It keeps iterating this while loop and suddenly crashes some times. This loops iterates till a variable is updated.

We tried to reproduce this issue by creating similar steps in a dummy sequence but it didn't crashed.

We are using TestStand version 4.1 and LabVIEW version 8.6

We thought that this might be an issue because of memory leaks or excessive CPU loading. So we monitored the system resources (CPU usage, memory, threads, handles , GDI objects, etc) but it didn't seem to be a problem. Even when the sequence execution completes, the resources come back to almost their original values (before start of execution).

Please let us know how to troubleshoot this problem. We need a direction for analyzing this issue and to find out if it is related to our coding.

Thanks in advance,

-Kaustubh.

[dug9000]

Kaustubh,

Are there any dll calls either from teststand or from a labview VI? Are you calling labview using the labview development environment or the labview runtime engine?

This sounds like memory is getting corrupted in the process somehow. Since, with memory corruption, the error often happens well after the code containing the bug has already executed, it can be tricky to diagnose and narrow down. If you are only using labVIEW and not dll or CVI calls, I'd recommend setting the adapter to use the labview development environment if you are not already doing so. Doing this can help narrow down whether the memory corruption is occurring in the labview code or some other code (i.e. if it's in labview code then labview.exe will crash rather than seqedit.exe). Similary, if you are using the CVI adapter you can set the adapter to run code modules in an external instance of CVI.

Another idea for narrowing this down is to try to eliminate steps from your sequence and see if you can still reproduce the problem. Basically, the idea is to try to come up with the smallest amount of code/steps necessary to reproduce the problem.

Also, I noticed you mentioned this seem to be related to  waiting on a notifier. Is it a labview notifier or a TestStand notification? If possible could you attach the vi or a screen shot of the code involved?

Hope this helps,

-Doug

Message Edited by dug9000 on 06-08-2009 10:57 AM

[kbsave]

Hi Doug,

Thanks for your reply.

Answers to your queries:

1. We are not calling any DLLs from our sequences.

2. We are only using LabVIEW modules and the VIs are called under LabVIEW development environment and not using Run Time engine.

3.  In our case it is always seqedit.exe which crashes. LabVIEW never crashed. Hence I think its TestStand whose memory is getting corrupted.

We  are trying to narrow down the issue as suggested by you.

By the time, we have observed a strange thing while the sequence editor crashed.

In our sequence we call a thread which calls a VI to run continuously and collect the data from other modules.

What we observed is that, when the sequence editor crashed (the popup appeared), it was still executing the sequence (running a loop). I was able to break the execution and debug it. All the variables were also having proper values.

But the VI which was running in a separate thread was in surprising state. That VI was showing as executing but actually it was not running any of its code. I tried to put probes inside that VI but they were not updated (meaning that part of code was not runnning). I tried to stop the VI using the abort button but no help. It was not having any effect. I also clicked on the Pause button. The button turned red but the VI didn't halt at any part of code (actually the VI was not at all executing). An other VI which was called by another thread was working properly. I could see it by putting probes in that VI.

If your thought about the memory corruption is correct, is it possible that this VI is somehow corrupting the sequence editor memory?

I am not sure, but I think wherever the sequence execution may be, but the above mentioned VI is always in this state at the time of crash.

I tried to reproduce this scenario locally but it ran without problem for hours 😞

Probably I am still not able to properly reproduce it. I'll work on it further and let you know the outcome.

Regarding your last question, We are using LabVIEW notifier. A VI is called which waits on LabVIEW notifier from another VI. That code is pretty simple it just obtains a notifier and waits on it indefinitely.

Please let me know your views on the above mentioned issue.

Thanks,

- Kaustubh