efficiently add to queue from dll

[Dixie]

Hi, is there an efficient way to add an array to a queue from a C++ dll? I need to transfer approximately 500 MB/s to labview and would prefer to directly enqueue the data from the C++ code over polling a circular buffer from Labview.

[rolfk]

There isn't any way I would know of, which I would think to mean, there is no way outside of possibly some inside knowledge only available to the LabVIEW developers. But it may be truely impossible without direct access to the LabVIEW source code.

[Yamaeda]

Maybe your C++ code can create Active X callbacks or something sending chunks of data. You can register for catching those in LV.

If it's enough, though ... *shrugs* 

/Y 

[Ben]

---

@rolfk wrote:

There isn't any way I would know of, which I would think to mean, there is no way outside of possibly some inside knowledge only available to the LabVIEW developers. But it may be truely impossible without direct access to the LabVIEW source code.

---

Rolf,

I agree there would have to be at least one buffer copy but what if an Action Engine that exposed action like;

Init - Accept a queue reference

Set - Accepts the array and does an Enqueue using the queue ref

was wrapped up as a dll the C++ code could call.

The calling LV app would create the queue and call the dll with the "Init" action.

The C++ code could then call the dll with the "Set" action.

The LV code could watch for entries in the queue.

Do you think that could work?

Curious,

Ben

[rolfk]

Ben it could but I would actually think that there could be a more direct way, but it is hard to envision a system setup without knowing more details about the actual use. I could envision some sort of DLL call were LabVIEW passes in a buffer handle that the DLL can then resize and fill in to pass it back at the next call by exchanging it with the new handle passed in from LabVIEW. This would likely reduce the data copy to one copy in the DLL itself. The key point is to use completely LabVIEW native array buffers even in the DLL.

[Dixie]

Ben, that is indeed what I had in mind. I think it would definetely have to involve a data copy, but when I use an intermediate circular buffer in the dll I need two data copies. I know that that can still provide the necessary speed, but it is a hassle since it requires manual synchronization.

[Ben]

---

@Dixie wrote:

Ben, that is indeed what I had in mind. I think it would definetely have to involve a data copy, but when I use an intermediate circular buffer in the dll I need two data copies. I know that that can still provide the necessary speed, but it is a hassle since it requires manual synchronization.

---

Then give a whirl and let us know if it works. Queues will generally keep the data in the buffer it was in when the enqueue operation occured so it it may only require a sinlge buffer copy.

If that does not work you may want to concider using TCP/IP to move the data but that would get the OS and the TCP/IP stack involved.

Still thinking out loud...

Any way to set up a Ramdrive tha can be accessed from both?

I do not pretend to know the answer. I am only throwing out ideas.

Ben

[Dixie]

Hi Rolf, the system setup is as follows: I have a data acquisition card that spits out data buffers of a fixed size. Every time a buffer is ready a callback function is called. I have access to the sourcecode of the callback function and can adapt it to my wishes. Until now the way I have dealt with the system is that I copy buffers into a circular buffer in the callback function. The circular buffer contains both the data buffers and a flag for each buffer that holds the state (read or unread). Access to the circular buffer is protected with critical sections to prevent race conditions. In labview I have a loop that calls a dll function which looks in the circular buffer for unread data. If there is none it sleeps until a new buffer is pushed by the callback function. Otherwise it copies the data into a labview array. This works, but I would be interested in any easier or more efficient/elegant way.

Thanks.

[rolfk]

If you maintain in your circular buffer not just the C pointer but actual LabVIEW array handles you could swap out those handles in your data collection call. Something like this:

typedef struct {

     int32 numElm;

     uInt8 elm[];

} LabVIEWArray, **LabVIEWArrayHdl;

typedef struct {

     ......

    LabVIEWArrayHdl handles[];

    .......

} CircularBuffer;

MgErr GetNextBuffer(LabVIEWArrayHdl *array)

{

      // Determine if there are any elements in the circular buffer

      int32 idx = NextDataAvailableIndex();

      if (idx >= 0)

      {

            // Copy next available buffer handle

            LabVIEWArrayHdl temp = CircularBuf->handles[idx];

            // Store the passed in handle in its place

            CircularBuf->handles[idx] = *array;

            // pass the handle back to LabVIEW

            *array = temp;

            return noErr;

      }

      return noDataErr;

}

And in your callback you do something like this

callback(...., void *data, int len, .....)

{

      MgErr err;

      int32 idx = NextDataInsertIndex();

      if (idx < 0)

           Buffer full!!!!!   

      LabVIEWArrayHdl handle = CircularBuf->handles[idx];

      err = NumericArrayResize(uB, 1, &handle, len);

      if (!err)

      {

            MoveBlock(data, (*handle)->elm, len);

            (*handle)->numElm = len;

            CircularBuf->handles[idx] = handle;

      }

}

This does not contain the code about initialization of the circular buffer and maybe almost as importantly about deallocation. For initialization you simply should make sure that the array of handles is all initialized to NULL. NumericArrayResize() is smart enough to allocate a new handle for NULL values, and resizes non NULL handles. For deallocation you need to walk the array and for any element that is not NULL call the DSDisposeHandle() memory manager function. It also does not show the critical section handling that is clearly needed here in order to synchonize the data get and the callback function.

[GregFreeman]

---

@rolfk wrote:

Something like this

...

callback(...., void *data, int len, .....)

{

      MgErr err;

      int32 idx = NextDataInsertIndex();

      if (idx < 0)

           Buffer full!!!!!   

      LabVIEWArrayHdl handle = CircularBuf->handles[idx];

      err = NumericArrayResize(uB, 1, &handle, len);

      if (!err)

      {

            MoveBlock(data, (*handle)->elm, len);

            (*handle)->numElm = len;

            CircularBuf->handles[idx] = handle;

      }

}

....

---