ServerTCPread is not obeying SetBreakOnLibraryErrors(0);

[RadioAct]

We are writing a TCP server for some automated testing we are doing and everything is working ok except for this debug runtime break that we keep getting.

What happens is when reading data from the TCP stack I do like the example code does, keep reading chunks of data until the read times out signifying the buffer is empty. At first I was using 'DisableBreakOnLibraryErrors' and 'EnableBreakOnLibraryErrors' to stop the debugger from breaking on the expected error and that worked fine for the longest time. However once we started stress testing the code with multiple clients then we started getting debugger breaks even with the DisableBreakOnLibraryErrors. I dug though the help and found 'SetBreakOnLibraryErrors' so I tried to use that instead of the older function and found that this had no effect.

So I decied to turn off break on library erorr all together in the debugger options and once again I kept getting the errors. I thought my project might be messed up so I rebuit it and still I kept getting the errors. I even went so far as to try a release version with the breaks disabled, and you guessed it I am getting non-fatail run time errors.

I did manage to get one version working with out the breaks, what I ended up doing is 'DisableBreakOnLibraryErrors' at the start of the program and then calling it again before the ServerTCPread and not calling 'EnableBreakOnLibraryErrors' to re-enable the erors. This is a hack fix and when writing a large application like this one those error messages are very useful for catching problems with the dynamic arrays we are using.

This is getting very confusing and I can't explain why it is happening with all the checks disabled. Has anyone else had this sort of issue? 

[RadioAct]

Iv looked though the code again and the only thing I could come up with is that the non-fatal runtime error is being buffered to when I re-enable the error checking or that CVI is missing a call to 'SetBreakOnLibraryErrors' and that is causing the error to occur. The strange thing is that when I have only one or two clients running traffic though the server the problem does not happen but when I have 6 clients all running traffic it happens.

Is it possible I am calling 'SetBreakOnLibraryErrors' to offten or too fast? I was thinking that CVI could handle this but it looks like when there is a large number of calls that it misses a 'SetBreakOnLibraryErrors(0)' one in a while which results in me getting the error message. I would estimate that my server is calling 'SetBreakOnLibraryErrors' about 400-600 times a second.

I will keep trying a few different things and will see what happens.

[Travis_M.]

Hello,

Thank you for posting your questions to the NI Discussion Forums!  Which example are you referring to in your post?  I am unclear about what exactly you are attempting to do.  It seems like you just wish to perform a TCP read of the entire TCP buffer contents.  I may be missing something, but why are you using the “break on library errors” functions to do this?  I would recommend starting by looking at the TCP server example from the CVI Example Finder (Help->Find Examples, search for TCP).  In this example, the TCP callback function “ServerTCPCB” is called for events, if the event corresponds to data being ready, the ServerTCPRead function is called to read a max of “datasize” characters from the buffer.  If you suspect that more data may be in the port, you can always perform multiple reads in a loop.  The function will return a negative number corresponding to an error code if a timeout occurs but should not create any sort of library error.

From the Help for the ServerTCPRead function:

“Return value indicating whether the function was successful. Unless otherwise stated, zero represents successful execution and a negative number represents the error code.

You can call the GetTCPSystemErrorString function to obtain a system message that describes the error. The system messages can be more descriptive than the TCP Library error codes. To obtain the correct system error message, you must call GetTCPSystemErrorString immediately after calling the TCP Library function that failed.

For functions that read or write data (ClientTCPRead, ClientTCPWrite, ServerTCPRead, ServerTCPWrite), if the function was successful, the return value is the number of bytes transferred.

You can have a maximum of 255 concurrent conversations and up to 1,024 connections. If you exceed this limit, -kTCP_TooManyConnections will be returned. You may not be able to open the maximum number of connections allowed by LabWindows/CVI because of limitations imposed by the operating system.”

So, I would start the debugging with determining what is causing the runtime errors.  No need to ignore errors once the errors have been eliminated.  Again, I may be missing something here – please let me know if you can provide any further clarification.

I’m happy to help – let me know how I can,

[RadioAct]

Seems that I misscommunicated my problem.

The example I used was the "MultiClientServer" example and the code I was referring to is:

/*---------------------------------------------------------------------------*/
/* Receives data from client.                                                */
/*---------------------------------------------------------------------------*/
static void CVICALLBACK DeferredReceive (void *data)
{
    ClientInfoPtr   clientInfoPtr = (ClientInfoPtr) data;
    char            dataBuf[256];
    int             bytesRead;
   
    assert (clientInfoPtr->readingData == 1);
   
    /*
     * Disable library error checking as we are going to read until
     * there is no more data left (read call times out).
     */
    DisableBreakOnLibraryErrors ();
    while (1)
        {
        bytesRead = ServerTCPRead (clientInfoPtr->handle,
            dataBuf, sizeof (dataBuf) - 1, 100);
        if (bytesRead > 0)
            {
            /* Update user interface with the new data. */
            dataBuf [bytesRead] = '\0';
            SetCtrlVal (clientInfoPtr->panel, REC_PANEL_DATA, dataBuf);
            }
        else
            {
            /* No more data to read. Update flag, and exit loop. */
            clientInfoPtr->readingData = 0;
            break;
            }
        }
    /* Enable library error checking. */
    EnableBreakOnLibraryErrors ();
}

The library error I was referring to is the ServerTCPRead returning -11 to indicate the read timed out because there is no longer any data in the TCP buffer; I read it all out in a loop similar to this one. The problem I am having is even with the "DisableBreakOnLibraryErrors ();" before the loop and "EnableBreakOnLibraryErrors ();" after the loop, I sill get a debugger break and popup on the non fatal error of ServerTCPRead returning -11.

Here is the catch, if I am running a small amount of traffic though the server I never get the debugger break and popup. The server reads the data out from the client just fine and everything works ok. However if I have the clients run traffic at a faster rate the every so often I the program breaks and the little popup informing me of a non-fatal runtime erorr of ServerTCPRead returning -11 is displayed. If I click continue then the server continues, never having missed any of data from the clients, until 15-20s later when I get the same popup and debugger break.

..Continued below

[RadioAct]

Here is the code I am running:

int TCPServerServiceRx(unsigned int Handle){ 
 unsigned char smallbuf[RXBUFFERSIZE+1];
 unsigned char *tempchar, *tempchar2;
 unsigned short int burstlen;
 int size, temp, retest;
 int client;
 
 //Disable breaks on errors since we will deliberatly be causing one.
 SetBreakOnLibraryErrors(0);
 
 //First look up the client index
 client = TCPServerClientIDLookUp(Handle);
           
 //First step is to pull data out of TCP and append it to the TCPBuffer
 do{
  //Error will happen here
  size = ServerTCPRead(Handle, &smallbuf, RXBUFFERSIZE, 100);
  //If size is less than zero then we got all the data in the tcp buffer
  if(size <= 0){
   //Break out of the while loop
   break;
  }
  //Make a new TCPBuffer that can hold the old data and the new data.
  tempchar = malloc(sizeof(char)*(size+TCPServer.Client[client]->TCPBufferSize));
  //Copy over the old data first
  memcpy(tempchar, TCPServer.Client[client]->TCPBuffer, TCPServer.Client[client]->TCPBufferSize);
  //Now append the new data
  memcpy(&tempchar[TCPServer.Client[client]->TCPBufferSize], smallbuf, size);
  TCPServer.Client[client]->TCPBufferSize += size;
  //free the old buffer
  free(TCPServer.Client[client]->TCPBuffer);
  TCPServer.Client[client]->TCPBuffer = tempchar;
 }while(1);

//Re-enable breaks on errors
SetBreakOnLibraryErrors(1);

....Continues on to process the received data

 Iv tried encasing the while loop in "DisableBreakOnLibraryErrors ();" and "EnableBreakOnLibraryErrors ();" as well as trying "SetBreakOnLibraryErrors(0);" and "SetBreakOnLibraryErrors(1);" and they both produced the same effect.

I might be able to explain this behaviour if when I call "ServerTCPRead" it does a "ProcessTCPEvents". If it does then the possibility exists that the code would jump to another callback to service a different client before finishing this code which would look similar to this:

DisableBreakOnLibraryErrors
ServerTCPRead ->before returning here it generates a callback which does:
                DisableBreakOnLibraryErrors
                ServerTCPRead
                EnableBreakOnLibraryErrors
ServerTCPRead <-then goes to return from the first call and since EnableBreakOnLibraryErrors was called before it returned, the break occurs.

This is a bit hard to follow but Im not sure how else to explain it here.

Let me know if you need more clarification.
Sorry for the long winded post...

Tyler

Message Edited by RadioAct on 04-27-2006 04:13 PM

[Travis_M.]

Humm….. I think I see now.  Thanks for the detailed explanation.  The events could be processed in the TCPRead, but I’m betting that the problem is due to the multithreaded nature of this program.  If 2 of these reading functions are happening concurrently (say for multiple connections) I see at least one possible way you could get this error.  See the following execution possibility:

1) Thread A – disable break on error

2) Thread A – start reading

3) Thread B – disable break on error

4) Thread B – and complete reading

5) Thread B – enable break on library error

6) Thread A – continuing to read, encounters the timeout error

7) Thread A – you press <continue>

😎 Thread A – enable break on library error

I’m not completely certain this is what’s happening, but it sounds likely enough.  Fortunately though, if you are planning on distributing this application in release mode the enable, and disable break statements don’t matter since they never break on the errors in release mode. 

Does this sound like a possibility to you also?

[RadioAct]

Thanks for your response.

But here is the kicker, this is not a multi-threaded application, I use only one thread to service all the clients in a first come first server manner. That is why I don't understand why this is happening only under heavy load, it should happen all the time or not at all. The server will be threaded in the future but to get the project off the gound a singe callback was used. For now Im relying on CVI to queue data received from the TCP stack until I can get around to servicing it.

I agree that building a release should not cause a problem, and since that development is done on this block it has been built as a release, PV has reported no issues. It was just a bit annoying during development when trying to watch for other problems with the code under a heavy client load.

As I mentioned before, Im still thinking that my code is being interrupted after it enters the loop, which is causing it to perform the same loop with a different client handle and then returning to finish the first loop. I don't have a very good understanding of what conditions will allow a callback to execute, do they wait for ProcessSystemEvents or ProcessTCPEvents, or does it interrupt currently running code? For now I will assume that TCPread will do a processTCPevents and I will work around it. I might be able to log the order of events to capture this case.

In any case thanks for the info

Tyler

Message Edited by RadioAct on 04-28-2006 06:08 PM

[Travis_M.]

Humm……

If you are using the example this application is multithreaded.  See the documentation: 

“/* FILE:    MultiClientServer.c                                              */

/*                                                                           */

/* PURPOSE: This TCP server program allows multiple TCP clients to connect   */

/*          to it. The program allows the user to write to any of the        */

/*          connected clients. The program receives and displays any data    */

/*          sent by the clients. The program uses worker threads to manage   */

/*          reading and writing to each client. You can use the TCP client   */

/*          sample program to interact with this server.  “

The multithreading is buried in the other callback functions:

[Travis_M.]

/*---------------------------------------------------------------------------*/

/* Worker thread function for clients.                                       */

/*---------------------------------------------------------------------------*/

static int CVICALLBACK ClientThreadFunction (void *data)

{

    ClientInfoPtr clientInfoPtr = (ClientInfoPtr) data;

    clientInfoPtr->threadId = CmtGetCurrentThreadID ();

    /* Load a new panel for this client message. */

    clientInfoPtr->panel = LoadPanel (0, "MultiClientServer.uir", REC_PANEL);

    SetPanelAttribute (clientInfoPtr->panel, ATTR_CLOSE_ITEM_VISIBLE, 0);

    /* Set the client's name in the panel. */

    SetCtrlVal (clientInfoPtr->panel, REC_PANEL_CLIENT_NAME,

        clientInfoPtr->name);

    DisplayPanel (clientInfoPtr->panel);

    /* Process user-interface, TCP and other system events. */

    while (!clientInfoPtr->stopFlag)

        {

        ProcessSystemEvents ();

        }

    /* Discard the client panel. */

    DiscardPanel (clientInfoPtr->panel);

    return 0;

}

[Travis_M.]

static int ConnectClient (unsigned int handle)

{

    int             tcpErr = 0;

    ClientInfoPtr   clientInfoPtr = 0;

    char            peerName[128], peerAddress[128];

    /* Create client information data-structure. */

    clientInfoPtr = calloc (1, sizeof (ClientInfo));

    if (clientInfoPtr == NULL)

        return -1;

    clientInfoPtr->handle = handle;

    /* Get descriptive name for client. */

    tcpChk (GetTCPPeerName (handle, peerName, sizeof (peerName)));

    tcpChk (GetTCPPeerAddr (handle, peerAddress, sizeof (peerAddress)));

    sprintf (clientInfoPtr->name, "Client name: %s, address: %s",

        peerName, peerAddress);

    /* Create worker thread for this client. */

    CmtScheduleThreadPoolFunction (DEFAULT_THREAD_POOL_HANDLE,

        ClientThreadFunction, clientInfoPtr, &clientInfoPtr->threadFuncId);

    /* Add the client to the list. */

    ListInsertItem (gClientList, &clientInfoPtr, END_OF_LIST);

    /* Add client to user interface and update disabled controls. */

    InsertListItem (gPanel, PANEL_CLIENT_LIST, -1, clientInfoPtr->name,

        (int)clientInfoPtr);

    SetCtrlAttribute (gPanel, PANEL_DISCONNECT, ATTR_DIMMED, 0);

    SetCtrlAttribute (gPanel, PANEL_SEND, ATTR_DIMMED, 0);

    SetCtrlAttribute (gPanel, PANEL_DATA, ATTR_DIMMED, 0);

Done:  

    return 0;

}