Multithreading with COM port

[TonyG-614]

I want to setup a separate thread for a COM callback function, but I'm not sure about some of the details.  Ideally I would open the COM port and transmit commands in the main thread, and have the COM call back run in a separate thread.  The COM callback will store data in a thread safe queue.  The main thread will use a timer to check the thread safe queue and process the data.  I'm just not sure about some of the details on setting this up.

One complication is that my device could be on any COM port, so I have to loop through all available COM ports.  If a COM port opens, then I need to do the following:

1) Install the COM callback

2) Send a command to the device

3) Wait for the expected response

4) If I don't receive the expected response, close the COM port and go to the next one

I have this working right now all in the main thread, and I'm already using thread safe queues to pass data between the COM callback and the rest of the code.  But I'm not sure how to set up the COM callback in a separate thread.  I think what I need to do is in step 1 setup a new thread and have that function install the COM callback.  Does the thread function need an infinite while loop, or can it just return once the callback is installed?  And if the COM port I just opened doesn't work, can I uninstall the COM callback from the main thread and kill the thread I just created, or does the uninstall have to be done in the secondary thread?  Maybe the secondary thread should install the callback, go into an infinite loop that breaks when a thread safe variable is set to TRUE, and then uninstall the callback?  Then if my main thread doesn't get the correct response, it can set that thread safe variable to TRUE.  But then how does my my main thread know when the secondary thread has exited so it can close the COM port?

Also, is it a problem to call ComWrt from a different thread than the COM callback?  Will the COM port drive correctly manage to different threads trying to use the COM port at the same time?  If I have to issue every ComWrt call from the secondary thread, that's going to get pretty ugly.

Any thoughts, suggestions, tips, sample code, etc. would be appreciated.

[ShineBolt]

While I'm fairly new to Lab Windows, or anything NI related. Over the past few months, I've been fixing up a program's Wi-Fi connections and can talk a bit about how it's done there.

"Does the thread function need an infinite while loop, or can it just return once the callback is installed?"

That depends. Do you want to keep sending messages on the thread? If the answer is yes, then it might be a good idea to put all the logic in a loop as was done with my project. While it's set to continue, each iteration runs a switch statement to execute code depending on what's happening with the Wi-Fi: mainly whether or not it is supposed to be sending or receiving data, but there are also cases for if an error occurs, and special code for retrying the previous operation. Currently, it never calls sleep, and spends a lot of time running in an idle state, so optimizations could probably be made to the design. I also don't like how the loop needs two or more iterations just to receive data. I think you could probably get away with using a single if statement to check if flags have been set to send or receive data and go from there.
However, if you are just checking if a connection is possible, then there is no need to have a loop in your function, just return the thread to the pool when you're done.

"And if the COM port I just opened doesn't work, can I uninstall the COM callback from the main thread and kill the thread I just created, or does the uninstall have to be done in the secondary thread?"

So as I said I'm kind of new to Lab Windows myself and don't exactly know what you mean by installing and uninstalling a callback. Do you mean you're adding the function to System Events? If that's the case then you really shouldn't need to put everything in the thread function into a loop, since ProcessSystemEvents already gets called in so many loops, or at least I think that's how it's supposed to be used. Either way, in my program, Wi-Fi communication is terminated with closesocket(s) from winsock.h, and USB communication is stopped with CloseCom(portNo) from rs232.h. In both cases, the communications are ENTIRELY independent from the threads and can be done in any thread at any time. So if the port doesn't work you can definitely close it on the main thread either before or after terminating the thread that handles communications, or even not terminating it.

"But then how does my my main thread know when the secondary thread has exited so it can close the COM port?"

Simple, use a variable. My project has a boolean variable for if a COM is closed, if a COM should be closed, if there is data to be sent, if there's data to be received, and that's not to mention the data stored for the threads themselves. You can pass a pointer to a structure containing all these variables to a new thread using the functionData parameter like so:

CmtScheduleThreadPoolFunction(thread_pool_handle, ThreadFunction, &aforementioned_struct, &thread_id);//Note: this is C code, perhaps this doesn't apply to whatever you're working in.
Or if funtionData is already being used, you could always make it global. Especially, if you're working in plain C, your options are very limited in how threads can communicate. And while there is probably some snazzy way to avoid a global variable (such as making them static within the function and letting the main thread enter it), all too often it doesn't actually make the code any more maintainable. Make it global if you have to, and if you want just call a Getter when you need to read from it.

"Also, is it a problem to call ComWrt from a different thread than the COM callback?"

It shouldn't be.

[RobertoBozzolo]

Here you can find some informations that may help you to clarify some of your doubts.

There is normally not a problem in opening a port in a thread and accessing it in other threads in the same application; I suppose you can even close the port in a thread different from the one you opened it, but I've never tried it so I cannot be sure.

But I would suggest you to move to a different, event-drive approach that may be more efficient and clear. I seem to understand that you are testing some USB devices that install a virtual com port when connected (I see no other possibility to have serial ports appearing in the system!). If this is so, you could react to device-arrival and removal system message and when a new port appears you  could spawn a new thread that handles all related stuff (open the port, install the com callback, test the device, close the port and exit, sending results to the main thread in between). This could also make COM callback and TSQ useless: you can simply wait some fixed amount of time for device response without need to create a thread-safe queue since your whole communications resolves into a single thread.

It's not clear to me what you indent for a port that doesn't work, but if you succeed in opening the port you should be able to also close it (this should be the case where is the attached device that doesn't work). If on the contrary you receive an error while opening the port, then you cannot install a callback as well, so...

BTW, closing the port automatically clears any callback that was installed on it so there is no need to explicitly discard the function.

As per getting the system message on device insertion and removal, I am attaching a small sample project I made to test this scenario: it recognizes the insertion of my FTDI usb-to-serial cable so I'm confident it will be able to handle your device too.

[TonyG-614]

Thanks ShineBolt and Robert.  I experimented a bit and developed what seems to be a working solution.  Before I explain what I did, I'll provide some more details on the use case.

This is for a production test program that is connected to a uint under test (UUT) via an FTDI COM-to-UART driver.  Only one UUT is connected at a time, but I don't know in advance what COM port will be used.  The test program can send messages which return replies.  It can also receive asynchronous messages at any time.  My program first searches through all available COM ports.  If a COM port can be opened, it sends a message with a known response.  If the correct response is not received, then it closes that COM port and moves to the next one.  If the correct response is received, it leaves the COM port open and continues with the test.

At this point the test program sends lots of synchronous messages which have expected replies.  It also expects occasional asynchronous messages.  The COM port callback is essentially a state machine that reads one character at a time (because messages can be different lengths).  Once it receives and verifies a complete message, it loads the message into a thread safe queue.  An asynchronous timer checks the queue every few milliseconds and processes any messages it finds.

What I did that seems to be working is, once the COM port is opened, the main thread creates a new thread.  That thread function installs the COM callback, then enters an infinite loop which calls ProcessSystemMessages.  When the infinite loop terminates (more on that below), the thread function uninstalls the COM callback and returns.

I had to add two Boolean variables to manage the handshaking between the threads.  The first variable is set by the COM callback thread after the callback is installed.  The main thread monitors this variable and continues on when the callback thread sets it to TRUE.  That allows the main thread to send a message to the device only after the callback has been installed.  The second Boolean is used by the main thread to terminate the infinite loop in the callback thread when the program is done with the COM port.

As I said, this seems to be working just fine.  It also cut my test time to almost a third of what is was originally.  The only caveat is that I'm using non-thread safe Boolean variables for the two cross thread handshakes.  I was going to use thread safe variables, but the function panel for the thread safe variable functions all say to use macros like "DefineThreadSafeScalerVar" instead of the functions.  The problem is that those macros don't exist.  The hyperlinks in the function help don't work, those macros aren't in the help file, and I couldn't find a single topic on the discussion forum that mentions them.  I decided that since both variables are Boolean and should have atomic access, and since I only change each of them once per COM link, it's highly unlikely that I'll have a problem.

The only issue is that if the user disconnects the USB cable while my program is running, it either hangs or crashes.  I looked into setting up a callback for USB insertion/removal several years ago, but I could never get it to work reliably.  I'll take a look at the sample program Robert provided to see if I can use that. 

Thanks!

[RobertoBozzolo]

The macros you are referring to are declared in utility.h file. You should be able to read the help for those macros inside your CVI IDE, but in case you can't you can access the help for the macros from this link: CmtNewTSV 

What I was aiming to was to avoid a continuous scan of all ports trying to get the correct one: moving to a event-driven approach could simplify the code. In the same direction, you could install a callback on the TSQ and have it fired automatically when you post to the queue without need for a timer that polls queue size.

In the direction of a clean code, I would personally avoid all that back-and-forth between threads and try handling all the communications in a single thread from opening the port and up to closing it clearing all associated resources.

But I understand that this could imply rewriting a possibly relevant part of the code which can be inconvenient for you, so please consider this simply as a clarification of my previous post.