Hi Mkehret,
Does your dialog have ActiveX controls on it? If your DLL dialog uses ActiveX controls and is not programmed in a certain manner it can hang because of a conflict in the threading model used by the TestStand Engine that is calling into your DLL and the model that is required by the MFC ActiveX Container. If you are using ActiveX controls in your dialog, you need to make special considerations for the threading model MFC dialogs that use ActiveX controls need in order to work properly. The example under \Examples\MFCActiveXContainerDlg illustrates how to appropriately handle this situation as well as explaining why it is necessary.
Note: The above example displays the dialog as modal, but this is irrelevant to the problem I am describing.
Again for information on properly creating a dialog class object and displaying it as non-modal refer to the information on MSDN that I referred you to in this post:
http://exchange.ni.com/servlet/ProcessRequest?RHIVEID=101&RPAGEID=135&HOID=506500000008000000E98A0000&USEARCHCONTEXT_CATEGORY_0=_8_&USEARCHCONTEXT_CATEGORY_S=0&UCATEGORY_0=_8_&UCATEGORY_S=0
If your dialog does not use ActiveX controls or after trying the programming method illustrated in the "MFCActiveXContainerDlg" example TestStand still hangs when calling your DLL, try calling your DLL from another environment just to make sure whether the DLL works at all. If the DLL works properly when called by another environment (e.g. VB, VC++, LabVIEW, or CVI), please submit a MINIMAL and REPRODUCEABLE example source project for your DLL for us to look at (please exclude all unnecessary code not required to reproduce the specific problem of displaying the dialog, and include all files necessary to build your dll with any
specific instructions needed to do so).
Jason F.
Applications Engineer
National Instruments
www.ni.com/ask