EvaluatingTteststand 4.2.1: Null pointer access violation (error 17502) when attempting to use a mixed-mode std::map / boost shared_ptr class contained in a DLL through an interface from a different DLL

[AndyWatt]

Hi there.

I have been working on a test system which will potentially have TestStand at its core, and downloaded/installed the 4.2.1 evaluation. Current details from the information dialog are:

TestStand Engine version 4.2.1 (4.2.1.83)

Sequence Editor Verison 4.2.1.83

The system will drive mobile radio devices using RS232 serial interfaces, usnig a variety of serial protocols (Hayes, and a couple of proprietary binary protocols for other work).

We already have in existence a soak tester, written in VS2005 with MFC unmanaged DLLs and also managed DLLs, which are mixed up together to provide different aspects of the functionality required for the binary protocols.

I have created:

1. a "master wrapper" DLL for the constellation of other DLLs - this offers up a simple C-API interface which I hope TestStand will use. It's called "SerialSessionMgr.dll""

2. a mixed managed / unmanaged "IJW" (it just works) DLL which wraps up the binary logging protocol control interfaces, which are entirely in managed C++/CLI: the "master wrapper" DLL calls into this through an unmanaged interface to create instances of the CLI-compiled class which interfaces to the underlying managed DLLs imlpementing the logging protocol. This is called "WinTseInterface.dll".

Before I started evaluating TestStand, I knocked up a test harness using CPPUnit / MFC, with which I was able to test SerialSessionMgr.dll and its' use of WinTseInterface.dll and verify they worked. The harness loads the DLL dynamically at run-time using AfxLoadLibrary() to try to emulate the way TestStand uses it.

When we open a TSE session:

1. The SerialSessionManager makes a call to the WinTseInterface.dll to "create" a logging instance. The instance is created in a std::map which is held by the CWinApp derived class, this map contains boost::shared_ptr of the class which works with a managed class. So we have a call trace: Client app->SerialSessionManager{TseStartSession()}->WinTseInterface{WINTSE_GetWinTseBridge()} - WINTSE_GetWinTseBridge() just creates a new instance of the unmanaged class which interfaces through to the managed CLI/C++ class beneath, and returns the address to the SerialSessionManager, which stores it.
2. The SerialSessioManager actually initialises the WinTseBridge using a method of a C++ pure virtual interface class to the class in the std::map in the WinTseInterface DLL implements. The initialise call passes miscellaneous details and actually calls "gcnew" to create the managed class which interfaces to the underlying managed DLLs which implement the logging protocol.

My logs of the DLL loading as seen in VS2005 debug output are attached this message as PDFs

While trying to fix this, I have tried:

• Enhancing the WinTseInterface DLL so it manually loads all the other assemblies in the logging system, including the managed portion of WinTseInterface itself (no effect)
• All the different options for the way the C DLL adapter loads and unloads the DLL in the DLL adapter steptype in TestStand
• I've even tried calling directly into the WinTseInterface DLL before trying any SerialSessionManager DLL calls to try to load the WinTseInterface DLL completely beforehand

and I cannot stop this exception.

It seems like the object is created in the WinTseInterface DLL OK (I have debugged the creation step which runs for WINTSE_GetWinTseBridge() and the object is created OK) when the SerialSessionManager makes the call, but when it calls the first managed function, which creates the managed object, the DLL is reloaded as managed - I suspect this causes the object to be destroyed or overwritten, and thus the call to initialise bombs with null pointer violation.

• Have I done something obviously wrong, and what other approaches can I take? 
• Does TestStand 4.2.1 have issues with driving DLLs which use managed/unmanaged DLLs beneath them?

My nuclear option is to throw the WinTseInterface DLL out completely and wrap up the managed C++ CLI code in a new interface, entirely managed, and use the TestStand ".Net" adaptor to drive that, creating my step library to run to that directly. This isn't ideal as I have some co-ordination of the protocols which is implemented in the SerialSessionMgr DLL.

I hope someone can help. This one's low level, dirty and a stumper to me.

Many thanks to the NI community. This is my first posting!

[AndyWatt]

I should have pointed out in the body of my question - my CPPUnit harness works fine, but TestStand crashes. My original post contained all the information in the attachments, and the abridged version got a little "clipped".

[RayFarmer]

Andy,

What Adapter type are you using in TestStand to invoke your code?

[AndyWatt]

Hi there Ray.

First - many thanks for engaging with this thorny problem. Love the avatar BTW.

I've been using the C/C++ DLL adapter: the original premise of the Serial Session Manager DLL was to provide a simple C type functional interface, using simple data types, so that TestStand would be able to drive the underlying functionality easily through a single DLL.

Seeing as you've been kind enough to respond, I should explain the makeup of the DLLs a little more, I think.

The SerialSessionManager (SSMgr) DLL uses C++ and MFC internally. This is the DLL which TestStand calls into directly through the C/C++ adapter. This DLL does have one odd feature though - because the it uses the Windows Multimedia Timer (implemented in WinMM.dll) I had to artificially create a scenario where it would load the .Net mscoree DLL _before_ it loads WinMM.dll - (http://connect.microsoft.com/VisualStudio/feedback/details/329819/freelibrary-in-the-winmm-dlls-modu...), basically WinMM.dll's init code gets rather upset when .Net DLLs load, which causes the entire executable to be unloaded with subsequent crashes.

To work around this crash, the SSMgr has a single, completely empty, file which is compiled for CLR, effectively forcing a dependency into the DLL on the .Net subsystems and turning the DLL into a "IJW" binary. The resulting DLL doesn't actually have any executable managed .Net assemblies, but I'm guessing that it does contain enough .Net related stubs and information that when the DLL is loaded, mscoree.dll loads before WinMM.dll - and this means the system doesn't crash.

Now, on to the WinTseInterface DLL. My initial description of this little software system was done in something of a hurry.

This DLL is the C++/CLI "IJW" DLL which actually contains managed and unmanaged code. Underneath the WinTseInterface DLL is another system of (this time exclusively C++/CLI) DLL assemblies which implement the TSE protocol interpretation system.

When the SSMgr DLL needs to start a TSE protocol session, it calls a C-API function in the WinTseInterface which creates a new CWinTseBridge class. Here's the steps in a little more detail:

1. SSMgr DLL calls WINTSE_GetWinTseBridge(), passing in the name of the device which needs it (this is used to put the new WinTseBridge into a std::map in the WinTseInterface DLL) - this call runs just fine, and returns the address of the CWinTseBridge C++ object. This is stored in the SSMgr, because the CWinTseBridge class implements a pure virtual interface which is used by the SSMgr as a way of accessing the CWinTseBridge methods. 
2. We now use the address from (1) to make a call to CWinTseBridge::Initialise(), passing some basic configuration details.

I've run VS2005 debug on the Sequence Editor when running the test, and done F11 step-throughs of the code loading each parameter in the Initialise() call onto the stack, and what appears to happen is that when the call is made, the whole lot steps off into oblivion (a reference to 0x00000000 is made, I think).

Do you think it's worth trying to get this approach to work? It would be useful to retain the control surface in the top of the SSMgr as there is co-ordination of the protocols within the logic of that component. If I move the Wintse control out into a separate managed DLL I may not have that co-ordination (this might be overcome with some design, as I've confirmed that it is possible to "reach into" the WinTseInterface DLL with TestStand and create and use managed objects from assemblies contained within).

Are there any known issues with this version of TestStand using C/C++ CLI IJW DLLs? I haven't found much out there on the web.

[RayFarmer]

Hi Andy,

Is there any reason why you didn't go for a dot NET appoarch as this seems to be an interface for managed / unmanaged code?

Also, as you are evaluating TestStand 4.2.1 does this mean you already have a licenced copy of this version which is why you are not evaluating withTestStand 2010. TS2010 has improved handling of dot Net ( if you were going down this route).

Have your checked out Chapter 5 of the Reference Manual regards Adapters encase they are any issues highlighted with regards to using VS 2005.

[AndyWatt]

I've posted my responses in-line.

Is there any reason why you didn't go for a dot NET appoarch as this seems to be an interface for managed / unmanaged code?

The serial session manager was designed to be reusable (I should have mentioned earlier) as a standard interface for applications to use the serial protocols it controls - not just TestStand - I chose a simple C interface so that a variety of languages would be able to use it fairly easily (including things like Delphi, etc).

Also, as you are evaluating TestStand 4.2.1 does this mean you already have a licenced copy of this version which is why you are not evaluating withTestStand 2010. TS2010 has improved

handling of dot Net ( if you were going down this route).

Almost - the test tool we're considering use of (the ServiceForce "Test 'n Ready" system) is using a similar version of TestStand and has shown this exception when attempting to use the DLL to start a WinTse session. I reproduced it in the evaluation copy I have and am using this as a testbed to look at possible solutions which will also work on the test tool.

I do have the installer for the TS2010 trial but the test tool won't be running that version so I'm trying to remain reasonably close to the tool.

Have your checked out Chapter 5 of the Reference Manual regards Adapters encase they are any issues highlighted with regards to using VS 2005.

I've been over chapter five a couple of times and I suspect this one's not something mentioned in there. There is a mention about using only native debugging when working on C++ code modules, but I have to used mixed mode debugging as the WinTseInterface DLL is so built...

If only the .Net adapter could call C++ code directly and not just work with assemblies... I think i may have to consider that the pure C++/CLI assembly route is the way to go, unless this mixed-mode problem can be fixed somehow.

[dug9000]

1) There is nothing TestStand related that should keep you from being able to call a mixed-mode assembly from TestStand

2) From your debug output it looks like the code you are calling is generating an unhandled exception. When debugging in Visual Studio have you tried turning on the feature under Debug->Exceptions... so that the debugger will stop when an exception is thrown? Make sure you are doing mixed mode debugging and stopping when exceptions are thrown and you will likely see the root cause of the exception in the debugger.

Hope this helps,

-Doug

[AndyWatt]

Thanks for those Doug - 

1) I figured things were OK here because we do make a call into the WinTseInterface DLL to create the unmanaged class instance we use to control the managed code from.

2) Way ahead of you (:-)) - I've already run debug with all exceptions trapped for break and there's no useful information because the debugger reckons there's no source code for the "current location". Looking at the stack trace when we break on the exception, we're waist-deep in mscorwks calls so I'm thinking it's got to do with the preamble before the function call is made. This is what makes me think I've got some kind of low-level .Net interop issue with the way the DLL is being driven from TestStand: the same binaries operate flawlessly if driven by a MFC-only CPPUnit harness framework (in debug or release targets).

I'll keep looking to see if there's something specific about the makeup of the WinTseInterface DLL which causes this issue. I'm still wondering if it's got something to do with the Serial Session Manager using a pure virtual interface to call through - it's this call (to Initialise()) which causes the WinTseInterface DLL to be loaded.

And the difference (see my original debug output window logs) between the way the CPPUnit test harness launches the WinTseInterface DLL - only one load, no (managed) - and the way TestStand loads the DLL, where it appears to load the _assemblies_...

I wonder if that second load _is_the problem... could it be that because TestStand sees there are assemblies inside the WinTseInterface DLL (I proved that TestStand can actually create and work with managed objects from the embedded assemblies inside the WintseInterface DLL) it actually calls _CorDLLMain and this completely messes up the CWintseBridge instance which we had created in the previous step?

I may delve into this idea further (I'm not even sure if _CorDLLMain exists for mixed-mode IJW DLLs) - I'm new to this low-level area of mixed-mode DLLs.

Do you guys think this sounds feasible, though? Surely this would happen all the time though - you'd call something which created stateful unmanaged objects inside a mixed-mode DLL, then make a call to anything managed, and it'd cause TestStand to reload the DLL for (managed) and nuke your stateful objects...

[RayFarmer]

Hi Andy,

I have just run the example from here from TestStand and it ran with out any problems (correct Std Output text was captured). I am using TestStand 2010 ( as it's what I have active at the moment) and also I am just calling the exe via the Call Executable step type.

I'll rebuild the project and change it so I can use the DLL adapter.

[dug9000]

Hi Andy,

A couple more things

1) When debugging

     a) Are you using the Microsoft Symbol Server? If not, please do as you will then get better callstacks with more info (names of entrypoints and more correct callstack)

     b) Please copy and paste the callstack into a text file and attach to this thread.

     c) Also look at callstack and see what the first function is that is in non-Microsoft code.

     d) Also when it breaks on the exception can you see what the type of exeception is? Is there any specific information about the exception.

     e) Also Micrsoft DLLs sometimes have expected exceptions so the first one you break on might not really be the one causing the problem since it might actually be a caught before returning to your code. You might have to continue a few times to get to the exception that is really causing the problem. NOTE: This isn't very likely though, just mentioning it just in case.

2) TestStand does not do anything so elaborate as what you are saying when it loads a DLL. Really all TestStand is doing is calling LoadLibraryEx (Windows SDK API):

http://msdn.microsoft.com/en-us/library/ms684179%28v=vs.85%29.aspx

TestStand is also using the LOAD_WITH_ALTERED_SEARCH_PATH option with that API.

Hope this helps,

-Doug