Intermittent problem with TestStand calling CVI DLL calling MSCV DLL

[Joe Roberts]

Sorry about cross post, but I am not sure which group is best to address
this issue.

Setup:
Windows 2000 SP1
TestStand 2.01f
CVI 6.0
MSVC++ 6.0 SP5

Problem:
I have a CVI Test Library DLL that contains test functions called by
TestStand using the C/CVI adapter. The CVI Test Library DLL in turn makes
several calls to another DLL written in MSVC++. I am experiencing an
intermittent problem with one of the MSVC functions. The problem ~appears~
to be stack related, but I am not sure. Among other things, this MSVC
function accepts a const char * argument that is a TestStand lookup string.
The function uses this string to access the TestStand API.

What happens is this: Everything works fine. I then recompile the CVI DLL
after making some mod, then run. The MSVC++ DLL asserts that the const char
* arg passed by the CVI DLL is NULL. However, this is not the case if I
single step through the CVI code. It has happened both with passing
variables as the const char * argument and as hardcoded strings literals, so
its not that I am actually passing NULL. The other argument to this
function is the TestStand sequence context dispatch pointer (LPDISPATCH
pobjSequenceContextDisp) and it always ~appears~ to be passed correctly.

The problem is frustrating and hard to debug because I can not
deterministically reproduce it. The problem ~never~ appears when I debug my
MSVC++ DLL in Visual Studio. And it only occasionally appears otherwise.

The problem, when it appears, always appears on the first run after
recompiling the CVI DLL, though the problem does not happen after ~every~
recompilation. I'd say it happens 1 in 6 times after a recompile.
Recompiling the exact same code does not always make the problem disappear.
If I change the CVI code (code that has nothing to do with the argument
itself though) and recompile the problem almost always goes away. Selecting
'Mark all for compilation' and rebuilding does not make the problem go away.
Only tweaking the CVI code and recompiling does (usually).

Whats more, the problem appears:
* With the CVI DLL built as Debug or Release mode.
* With the CVI default calling convention set to __stdcall or __cdecl.
* With the C/CVI TestStand adapter set to run in-process or external
instance of CVI.

The problem appears to be some sort of stack or argument passing problem
between CVI and MSVC, though thats just a guess based on the symptoms. I
have quadruple checked the calling conventions of all declared functions.
The CVI DLL functions all use TX_TEST (which resolves to __cdecl). The MSVC
DLL functions all explicitly use __stdcall. Is there a problem with calling
__stdcall MSVC functions from a __cdecl CVI function?

I can find no other memory leaks or indications of memory corruption
elsewhere in either the CVI or MSVC DLLs. Its only this one function that
exhibits this strange 'null const char *' problem.

Can anyone offer any ideas about what may be causing this problem? Anything
else I should check/verify?

Regards,

---
Joe

[Elaine_R.]

Hello Joe -

From the sound of it, what you are seeing isn't a behavior we've encountered before with LabWindows/CVI DLLs calling VC DLLs... One thing you could try would be to call your CVI functions with the __stdcall convention (which is perfectly valid) and see if behavior improves.

We would be interested looking over your code if you could somehow reduce the problem down to a pair of small DLLs/ functions that demonstrate the issue. The process of creating a minimal example may also help you troubleshoot your code as well.

-Regards

Elaine R.
National Instruments
http://www.ni.com/support

[silvius]

Joe,

Although I'm not sure if any of the following are the real cause of your problem, I have the following suggestions:
1. There could be a problem with calling
__stdcall MSVC functions from a __cdecl CVI function. As a workaround wrap the _cdecl call inside a _stdcall call that is exposed or exported to TestStand or vice-versa. This can be a problem because if _cdecl is used, the calling function is responsible for cleaning up the stack and if _stdcall is used, the called function is responsible for cleaning up the stack.

2.Don't mix Debug version of one DLL with the Release version of the other DLL. I had some bad experiences doing this and both DLLs where developed in MSVC. Allways use either Debug either Release versions of DLLs.

Hope this helps,
Silvius

[Joe Roberts]

Silvius,

> Although I'm not sure if any of the following are the real cause of
> your problem, I have the following suggestions:


Thanks for the reply. At this point any and all suggestions are welcome...

> 1. There could be a problem with calling
> __stdcall MSVC functions from a __cdecl CVI function. As a workaround
> wrap the _cdecl call inside a _stdcall call that is exposed or
> exported to TestStand or vice-versa. This can be a problem because if
> _cdecl is used, the calling function is responsible for cleaning up
> the stack and if _stdcall is used, the called function is responsible
> for cleaning up the stack.

I was under the impression that as long as everything was explicitly and
consistently declared, you could safely mix cdecl and stdcall f
unction
calls. Is it bad to do this? Is this a known issue with the CVI compiler?
I've never seen a problem with doing this under MSVC++.

I'll try wrapping them in cdecl calls for CVI - though we have a
depressingly large number of stdcall functions in the MSVC DLL ;-). They
need to remain stdcall in the DLL because we also call them from Visual
Basic.

> 2.Don't mix Debug version of one DLL with the Release version of the
> other DLL. I had some bad experiences doing this and both DLLs where
> developed in MSVC. Allways use either Debug either Release versions of
> DLLs.

I verified that the MSVC DLLs were either ALL Debug or Release. I too have
seen nasty problems when MSVC Debug and Release is mixed.

One thing I had not thought of until your reply: What about mixing CVI Debug
DLLs with MSVC Release DLLs? Have you ever seen issues with doing this?

Thanks!

---
Joe

[Joe Roberts]

> From the sound of it, what you are seeing isn't a behavior we've
> encountered before with LabWindows/CVI DLLs calling VC DLLs... One
> thing you could try would be to call your CVI functions with the
> __stdcall convention (which is perfectly valid) and see if behavior
> improves.

I thought CVI Test Library functions required cdecl convention? The TX_TEST
macro in the CVI DLL sample code w/ TestStand is on every function and
resolves to cdecl. Is TX_TEST not necessary? All of the NI function
pointers in stdtst.h use TX_CDECL convention.

I will try replacing TX_TEST with __stdcall. How does TestStand know
whether to call a CVI function using cdecl or stdcall. I can't find any
such setting. Is there any other change necessary to tell TestStand to NO
T
clean up the stack after calling CVI test function? Without this, won't
there be probems if TestStand tries to clean up stack data that has already
been cleaned up by the function?

> We would be interested looking over your code if you could somehow
> reduce the problem down to a pair of small DLLs/ functions that
> demonstrate the issue. The process of creating a minimal example may
> also help you troubleshoot your code as well.

~If~ I can get the problem to be repeatable and boil down to small code, I
will send it.

Thanks,

---
Joe