Reasons why ibnotify might never return

[Colin at Gould Nicolet]

I have been using asynchronous reads and writes in a C++ class in our application for several years on a PC platform with NI 488.2 version 1.7 installed. Recently we have found that installing NI 488.2 version 2 or later causes the application to hang. Investigation in the Visual C++ debugger has shown that during our application's shutdown a call to ibnotify never returns if NI 488.2 version 2 is installed. We have verified that uninstalling version 2 and reinstalling version 1.7 allows the ibnotify function call to complete successfully.

Can you give me some clues as to why the call I make to stop any further asynchronous activity, ibnotify(m_GPIBBoardID, 0, NULL, NULL), might not return? I am willing to believe that somethin
g in the way the C++ class handles the asynchronous reads and writes is incorrect but since the code has worked for some years with version 1.7 it must be something subtle.

[NI-488.2 master]

Could you possibly attach NI-Spy files of the working and non-working cases? We may have introduced an unexpected behavior in the 2.x series of drivers. You say that you've tried this with version 2 and later. I'm assuming that you have tried version 2.1?

What interface type are you using? What operating system?

[Colin at Gould Nicolet]

Attached are three NI Spy logs that each represent a single starup and shutdown of our application. The v1.7 log shows

steps 1-3 the main thread (3D0) sets up the card

steps 4-5 a different thread (418) starts an asynchronous read

some time passes during which nothing happens then I shut the application down

steps 6-8 the main thread calls ibnotify to cancel any pending asynchronous opearations, calls ibstop (though from my reading of the 488.2 reference material this may be superfluous)

steps 9-10 the other thread responds to the aborting of the asychronous operations by requesting another (it doesn't realise the system is shutting down yet). this may not be optimal behaviour but it hasn't caused a problem be
fore because ...

steps 11-13 the main thread does some final tidying up again cancelling any pending asynchronous operations. again the ibstop may be superflous.

step 14 take the board offline.

You will see that the same code run on v2.1 gets no further than step 6.

However, I have also found that reversing the order of the ibstop and ibnotify calls involved has a different result as can be seen from the final log. There's no hang with then in this order so that gives me a workaround.

My conclusions so far are that either
- my original code only works with v1.7 by luck, possibly because the timing of calls on my two threads and with the 488.2 driver is such that it just happens to work.
- or that my original code is flawed and/or naive and that the ibstop before ibnotify is the correct order. I'm have not been able to find any reference in the documentation for ibstop and ibnotify to their interdependence or a suggested execution order.

[NI-488.2 master]

Thank you for posting the NI-Spy files. It helped quite a bit to see the ordering of execution.

In a nutshell, the ibnotify(gpib0, 0) call hangs, because the 2.1 driver explicity disallows all calls into the driver except from the thread that started the IO. One exception to that is that ibstop is allowed to resynchronize the IO from any thread. Your thought that ibnotify(0) will resynchronize the driver is partially incorrect. ibnotify(0) will resynchronize the IO only if the IO is complete. In your application, if your ibnotify callback is invoked because of a timeout condition, you will still need to resynchronize the IO using ibstop.

The 1.7 NT/2000 driver happened to allow this behavior, but the 1.7 Win98 driver does not a
llow this behavior. We did much work in the 2.x series of drivers to reconcile these different behaviors to make for a more stable and maintainable driver. I don't see us changing the behavior of the driver to allow the ibnotify(gpib0, 0) call to work from a different thread while IO is in progress.

I would recommend that you use the workaround that you have already discovered, and first resynchronize the IO using ibstop.

I hope this explaination of things helps out.

[Colin at Gould Nicolet]

Thanks for the prompt answer. Now that I understand the reason I am happier.

I will make do with the workaround until I have a some spare time to re-code the classes and threads concerned.

Thanks again.

[ndiamond]

I will quote two parts of one of your sentences in reverse order:

"the 2.1 driver explicity disallows all calls into the driver except from the thread that started the IO" (with one very fortunate exception). OK, fine, it is OK to disallow the call. But how to do the disallowance?

"the ibnotify(gpib0, 0) call hangs" is surely an unacceptable misdesign. The call should return an error code.

I think my application didn't suffer this particular hang, but still. It seems my application was just lucky, in addition to the exception that driver 2.1 allows any thread to call ibstop(), my application didn't call ibnotify() again until ... hmm, I'm not sure if my application is lucky enough. The driver might hang me someday too.

W
hen you have a perfectly fine reason to reject a call, surely your driver must return an error code and not hang. Even if the printable .pdf documents will be updated to give the same information that your answer gave here, surely you still must return an error code instead of hanging.

[NI-488.2 master]

To be more clear, a call to ibnotify or any other call will not hang indefinitely. Once the IO has completed, either successfully synchronized using ibwait or ibnotify, or aborted using ibstop, the ibnotify call in the other thread will continue down into the driver. In this situation, ibnotify is treated much like any other driver call. I can think of a few situations in which returning an error in this situation would not be ideal for the user.

Imagine that somebody has a multi-threaded application where each thread is responsible for communicating with a different instrument. A user could use device-level calls, and not really have to worry about what the other threads are doing. This is because the NI-488.2 dr
iver handles the synchronization of the two threads for the user. If the driver had returned an error in this situation, the programmer would have to write code to handle the error condition and resubmit the call. The user could potentially eat up a lot of processor overhead by continuously resubmitting the call to the driver while IO is in progress in another thread.

Making such changes to our driver at this point in time could cause quite severe consequences for existing applications. As it is, we've attempted to take the best behavior from our previous drivers in order to provide the highest level of backwards compatibility. We provide an API that is well over ten years old, so changing it can be quite a difficult task. Unfortunately, sometimes the slightest changes can cause big problems. Making drastic changes to such an old API could very easily break applications that have run fine for many years.

We appreciate the feedback, and will look into how we can provide more
in-depth documentation on our multi-process and multi-threaded driver behavior.

[ndiamond]

I see your point. I understand very well the importance of not breaking existing behavior. Programmers very likely experimented to find out the existing behavior. It's not exactly wrong to depend on such observations when there is no other way to get their products working.

However, I think the call to ibnotify() can hang indefinitely. You say it will not, but if I understand the rest of your message then you only mean there are situations where the call will not hang indefinitely. Consider a different situation, where a thread has disabled timeout and started an async read or write. We don't definitely know when the user will fix a misconfigured device or invoke a call to ibstop().

Unfortunately EOIP isn't d
ocumented as a possible return from ibnotify(), and ECAP is documented with meanings that don't exactly match this situation, but still this kind of thing would be very meaningful.

So I would suggest, in addition to adding documentation, please add another optional setting. Let the default be the existing behavior. Let the new setting force ibnotify() and anything else in a similar situation return immediately with EOIP instead of hanging.