buffer array passed by pointer to DLL: how to preserve across many calls

[Neal Crocker]

I have a DLL which uses an array passed by pointer from LV as a buffer to return data. The DLL has a routine to pass the a pointer for the array and a second routine which, when called, actually fills the array. The second routine doesn't take any reference to the array as an arguemnt. Rather, it expects that a pointer to the array has already been passed to the DLL by a call to the first routine and that the pointer still points to allocated memory. My question is two-fold: first, I know how to pass an array by pointer with the "Call Library Function" node in the first call, but how do I guarantee that the pointer passed will still point to the array when I make the second call.? Second, how
to I access the array in a diagram after the second call? Ideally, for reason I won't go into here, I would like to make the two calls in separate subVIs if I can figure out how to safeguard the array between calls and access its data after the second call. I should mention that the first routine can accept buffer arrays of a variety of different atomic types (e.g. int16 and single) depending on how the "Call to Library" node is configured. It actually treats the array pointers as (void *). All type dependent processing occurs in the second routine.

[Greg_McKaskle2]

...The DLL has a routine to pass the a pointer for the
> array and a second routine which, when called, actually fills the
> array... it expects that a pointer to the array has
> already been passed to the DLL by a call to the first routine
> ...how do I guarantee that
> the pointer passed will still point to the array when I make the
> second call.? Second, how to I access the array in a diagram after
> the second call? Ideally, for reason I won't go into here, I would
> like to make the two calls in separate subVIs if I can figure out how
> to safeguard the array between calls and access its data after the
> second call. I should mention that the first routine can accept
> buffer arrays of a variety of different atomic types (e.g. int16 and
> single) depending on how the "Call to Library" node is configured. It
> actually treats the array pointers as (void *). All type dependent
> processing occurs in the second routine.

There are several things that seem inflexible about this interface, and
I'd encourage you not to keep the same interface and pass it on to your
diagram, or your customer's diagram. Instead, make two subVIs, one that
takes the parameters for the acquisition and returns on int16s, and
another that takes the parameters and returns floats. Each of these
will call the various entrypoints in the DLL and return the data all in
one step.

The first step that has to happen is determine exactly how the buffer is
allocated and sized. At this point I still don't see how this interface
would be used from C code. If the first call is given a pointer and
expected to resize it, then the original pointer is not guaranteed to be
valid -- this would require a handle or pointer to pointer type of
structure. If the first call allocates the buffer on its own, then this
pointer has to be returned on either the first or second call. So the
first thing to do is draw a simple bubble diagram of how the buffer is
created and how each entity know how to access the buffer.

To answer your direct questions, the DLL can be a passed an array in
several ways. The one called Array Data Pointer points into the LV
handle, past the int32 size and to the first data element. This is
normally accompanied by another parameter that tells the DLL how many
elements it can access. The DLL needs to be careful not to access
memory past end of the array pointer, and it cannot reallocate the data
as it only contains a pointer and has no way to tell the caller about
the new pointer.

The other way to pass the data is in the LV handle or pointer to the LV
handle. A LV handle is a pointer to a structure of (sizes,data). For a
1D array, there is only one size, then some alignment depending upon
platform, then the data elements. The handle can be resized using the
appropriate LV helper functions.

Whenever you pass either of these into the DLL as a parameter, you can
wire the array out of the right side. If do wind up putting the various
DLL calls into different subVIs, you cannot guarantee that the LV array
will not be copied due to a branch in the wire. The contents of all
copies will be the same, but a pointer into the array is not guaranteed
to be valid.

So, I'm not sure how much this helps, but the first thing to do is
determine how this API would be used from C. Once you know that, it is
relatively straightforward to ma it over to LV. I'll answer the
typecast question in the other posting.

Greg McKaskle

[wiebe@CARYA]

Hi,

Another option is to use windows heap functions. Using CreateHeap and
AllocHeap, you can reserve memory. You can use istrcpyn function to copy the
type casted data to the reserved memory.

I've used a construction like this to call the GetOpenfileName api, which
requires a structure with pointers to strings.

It's far from ideal, but it works... The hardest thing to do is the
istrcpyn, because it stops copiing after a NULL is encountered. This can be
solved by increasing the destination pointer with one, and copy the remained
until everything is copied.

Regards,

Wiebe.


"Greg McKaskle" wrote in message
news:3E87067E.8020905@austin.rr.com...
> ..The DLL has a routine to pass the a pointer for the
> > array and a second routine which, when called, actually fills the
> > array... it expects that a pointer to the array has
> > already been passed to the DLL by a call to the first routine
> > ...how do I guarantee that
> > the pointer passed will still point to the array when I make the
> > second call.? Second, how to I access the array in a diagram after
> > the second call? Ideally, for reason I won't go into here, I would
> > like to make the two calls in separate subVIs if I can figure out how
> > to safeguard the array between calls and access its data after the
> > second call. I should mention that the first routine can accept
> > buffer arrays of a variety of different atomic types (e.g. int16 and
> > single) depending on how the "Call to Library" node is configured. It
> > actually treats the array pointers as (void *). All type dependent
> > processing occurs in the second routine.
>
> There are several things that seem inflexible about this interface, and
> I'd encourage you not to keep the same interface and pass it on to your
> diagram, or your customer's diagram. Instead, make two subVIs, one that
> takes the parameters for the acquisition and returns on int16s, and
> another that takes the parameters and returns floats. Each of these
> will call the various entrypoints in the DLL and return the data all in
> one step.
>
> The first step that has to happen is determine exactly how the buffer is
> allocated and sized. At this point I still don't see how this interface
> would be used from C code. If the first call is given a pointer and
> expected to resize it, then the original pointer is not guaranteed to be
> valid -- this would require a handle or pointer to pointer type of
> structure. If the first call allocates the buffer on its own, then this
> pointer has to be returned on either the first or second call. So the
> first thing to do is draw a simple bubble diagram of how the buffer is
> created and how each entity know how to access the buffer.
>
> To answer your direct questions, the DLL can be a passed an array in
> several ways. The one called Array Data Pointer points into the LV
> handle, past the int32 size and to the first data element. This is
> normally accompanied by another parameter that tells the DLL how many
> elements it can access. The DLL needs to be careful not to access
> memory past end of the array pointer, and it cannot reallocate the data
> as it only contains a pointer and has no way to tell the caller about
> the new pointer.
>
> The other way to pass the data is in the LV handle or pointer to the LV
> handle. A LV handle is a pointer to a structure of (sizes,data). For a
> 1D array, there is only one size, then some alignment depending upon
> platform, then the data elements. The handle can be resized using the
> appropriate LV helper functions.
>
> Whenever you pass either of these into the DLL as a parameter, you can
> wire the array out of the right side. If do wind up putting the various
> DLL calls into different subVIs, you cannot guarantee that the LV array
> will not be copied due to a branch in the wire. The contents of all
> copies will be the same, but a pointer into the array is not guaranteed
> to be valid.
>
> So, I'm not sure how much this helps, but the first thing to do is
> determine how this API would be used from C. Once you know that, it is
> relatively straightforward to ma it over to LV. I'll answer the
> typecast question in the other posting.
>
> Greg McKaskle
>

[Neal Crocker]

Aside from making a CIN, is there no way in LV to guarantee that a pointer remains valid across two Calls while retaining some means in LV of access the data pointed to? What if I put both Calls in the same diagram? I was hoping there was a pure LV solution to this problem that doesn't require any C coding. I would like to avoid any C coding if I can, because I've never done C coding on the Win2K platform and my experience with C coding on other platforms suggests there's a significant investment of effort involved in just getting started. For instance, my understanding of the use of CIN nodes is that I will have to find a C compiler for my platform to use one. After that, there are whole host of other possibly trivial but time consuming is
sues I'll have to address.

[Greg_McKaskle2]

> Aside from making a CIN, is there no way in LV to guarantee that a
> pointer remains valid across two Calls while retaining some means in
> LV of access the data pointed to? What if I put both Calls in the
> same diagram?

In my previous post, I asked if you can explain better how the DLL is
used from C code. If you can arrange for the DLLs to use the LV array's
pointer, and the array doesn't need to be resized, then this should be
pretty straightforward.

Greg McKaskle

[Neal Crocker]

>> If you can arrange for the DLLs to use the LV array's
pointer, and the array doesn't need to be resized, then this should be pretty straightforward.

This is how the DLL is intended to work. When LV calls the first routine, it should pass an array by pointer. The array should be sized appropriately to hold the data that the second routine will write. The second routine will write to the memory addressed by the array pointer the first routine has received. This pointer is stored by by the DLL between the two calls. The second routine uses this stored pointer. In case I haven't mentioned it yet, the first routine expects the pointer to the array data to be passed by value or, in LV-speak, the Call node for the first routine should be configured so the buffer paremeter is Array Passed by Pointer. Any code which uses the DLL is expected to insure that the memory region pointed to is already allocated and is of sufficient extent for the second routine to write to.

Based on a previous statement you made, I get the sense that when the first routine is called, the wire coming out of the Call node from array parameter's output terminal retains it association with the memory referenced by the pointer that the first routine received. Presumably, if I connect this wire to anything in my diagram that can receive it, the receiver will get the contents of the buffer array. What I'm trying to understand is how to control time of execution of the call to the second routine so that the receiver of this array wire actually gets the data that the second routine writes to the array's memory space. A necessary prerequisite to accomplishing this coordination is having some way of guaranteeing that when the second routine writes to memory, the buffer its writing to hasn't been deallocated and the buffer is still equivalent to the data space of the array wire. You say that this is straitforward, so how do I accomplish it? If I, for instance, if I use a sequence structure to control the timing of the calls to the first and second routine, presumably, the array wire will have to pass through a tunnel to get out of the sequence. Are there any conditions under which the array wire is guaranteed to be associated with the same memory space from the momement is exits the first Call node until to the moment it exits the sequence? That would insure that when the second routine tries to write data to the array buffer, everything will go as expected, wouldn't it?

Thanks,
Neal.

[wiebe@CARYA]

Neal,

If you allocate memory using the heap functions, you get a pointer from
those routines (a simple I32). you can pass this number to any api that
expects a pointer to an array.

The allocated memory can be modified by the api, and untill the user closes
the heap (or the process is destroyed), the memory remains valid.

The pointer can be passed between subvi's without any problem. It is memory
reserved by the LV process, not by any specific VI.

In other words, the pointer does stay valid...

Regards,

Wiebe.


"Neal Crocker" wrote in message
news:5065000000050000008CE10000-1042324653000@exchange.ni.com...
> Aside from making a CIN, is there no way in LV to guarantee that a
> pointer remains valid across two Calls wh
ile retaining some means in
> LV of access the data pointed to? What if I put both Calls in the
> same diagram? I was hoping there was a pure LV solution to this
> problem that doesn't require any C coding. I would like to avoid any
> C coding if I can, because I've never done C coding on the Win2K
> platform and my experience with C coding on other platforms suggests
> there's a significant investment of effort involved in just getting
> started. For instance, my understanding of the use of CIN nodes is
> that I will have to find a C compiler for my platform to use one.
> After that, there are whole host of other possibly trivial but time
> consuming issues I'll have to address.

[Greg_McKaskle2]

> This is how the DLL is intended to work. When LV calls the first
> routine, it should pass an array by pointer. The array should be
> sized appropriately to hold the data that the second routine will
> write. The second routine will write to the memory addressed by the
> array pointer the first routine has received. This pointer is stored

You've answered the big question that was remaining, so I think that
this will be straightforward. Let me describe it back to you, and if it
sounds like I've misunderstood something about the DLL, then ignore my
advice and correct the misunderstanding.

The diagram calling the first DLL function needs to have already
allocated a buffer. The first DLL function doesn't resize the buffer or
write to it, but stores it waiting for a second DLL function to be
called. The second DLL function doesn't resize the buffer, but does
write the data to it. You need to make sure nobody access the data in
the buffer until the second call completes.

If this is correct, you can resize the initial array on the diagram
using the Reshape Array node or Initialize Array node. Pass this array
along with the other parameters to the DLL node. It sounds like your
DLL configuration is correct. Place the second DLL node in a single
frame sequence and configure it appropriately. Wire any outputs needed
from the first DLL call through the sequence and to the second DLL call,
and wire the array through the sequence as well. Make sure the wire
goes through and not beneath the sequence and has black tunnels on both
input and output edges. Even if it isn't modified by the sequence, the
fact that the array passes through the input and output tunnel will
ensure that no other node can access or modify the data until the second
DLL node finishes and the seqence produces its data. Be careful not to
place any functions on the array wire between the first and second DLL
call. If you branch the wire between the first and second call or
insert a node to search, sort, or otherwise access the array, all bets
are off.

If you take my advice about making the pair of DLL calls into a subVI,
then you can forego the sequence. Tou can wire the output array
directly to the output array terminal of the subVI. Since the subVI
will not finish and produce data until the second DLL finishes, this
acts much like the sequence and synchronizes the data propagation to the
completion of the second DLL call.


Greg McKaskle

[wiebe@CARYA]

Hi,

What is also very important, is that the dlls have to be in UI Thread. If
you put it in reentrant, each instance will have it's own data, and the dll
will not buffer the pointer.

Regards,

Wiebe.



"Greg McKaskle" wrote in message
news:3E899E91.3040400@austin.rr.com...
> > This is how the DLL is intended to work. When LV calls the first
> > routine, it should pass an array by pointer. The array should be
> > sized appropriately to hold the data that the second routine will
> > write. The second routine will write to the memory addressed by the
> > array pointer the first routine has received. This pointer is stored
>
> You've answered the big question that was remaining, so I think that
> this will be straightforward. Let me describe it back to you, and if it
> sounds like I've misunderstood something about the DLL, then ignore my
> advice and correct the misunderstanding.
>
> The diagram calling the first DLL function needs to have already
> allocated a buffer. The first DLL function doesn't resize the buffer or
> write to it, but stores it waiting for a second DLL function to be
> called. The second DLL function doesn't resize the buffer, but does
> write the data to it. You need to make sure nobody access the data in
> the buffer until the second call completes.
>
> If this is correct, you can resize the initial array on the diagram
> using the Reshape Array node or Initialize Array node. Pass this array
> along with the other parameters to the DLL node. It sounds like your
> DLL configuration is correct. Place the second DLL node in a single
> frame sequence and configure it appropriately. Wire any outputs needed
> from the first DLL call through the sequence and to the second DLL call,
> and wire the array through the sequence as well. Make sure the wire
> goes through and not beneath the sequence and has black tunnels on both
> input and output edges. Even if it isn't modified by the sequence, the
> fact that the array passes through the input and output tunnel will
> ensure that no other node can access or modify the data until the second
> DLL node finishes and the seqence produces its data. Be careful not to
> place any functions on the array wire between the first and second DLL
> call. If you branch the wire between the first and second call or
> insert a node to search, sort, or otherwise access the array, all bets
> are off.
>
> If you take my advice about making the pair of DLL calls into a subVI,
> then you can forego the sequence. Tou can wire the output array
> directly to the output array terminal of the subVI. Since the subVI
> will not finish and produce data until the second DLL finishes, this
> acts much like the sequence and synchronizes the data propagation to the
> completion of the second DLL call.
>
>
> Greg McKaskle
>

[Neal Crocker]

Thanks. I'll make sure this is so.