read buffered network published shared variable

[LabBEAN]

Is a buffered shared variable intended to hold it's buffered values in the NIVE (NI Variable Engine) between VI runs?  It appears to only hold the last element...

I wrote this code to fill up the buffer:

So these values were written:

 

Then, I emptied the buffer before exercising it with new elements with this code:

 Notice that there is only one cleared item (1009 is there twice because it times out and exits the loop with the second read):

Why don't we see 10 elements being cleared?

Distributed System Manager only ever shows one element... No way to see buffered items is there?:

Thanks Mark...

[mishkin]

It seems that we need to look into the client side buffer lifetime (when they get created/destroyed) when accessing shared variables via datasocket. With shared variables nodes, the buffer lifetime is explained in this section of Using the LabVIEW Shared Variable article. In this case, the client-side buffers get created when the VI containing the shared variable node runs.Therefore it should not be retaining buffered values between runs (except for the last value).

With datasocket, it seems to be related to datasocket open/close VIs. The reason I say this is that when put both parts of the code from the previous post in one VI, I still see the same behavior - only one element, 1009, is cleared from the buffer (meaning the client-side buffer gets destroyed somewhere in the code even though we do not stop/start the VI). When I removed the Close Datasocket at the end of the first part of the code and the Open Datasocket in the beginning of the second part of the code (essentially connecting the Datasocket connection ID wire directly between the two parts), I see the behavior you were probably expecting:

 To me, that seems to imply that the client side buffers get created and destroyed when we call Open Datasocket and Close Datasocket VIs. 

Also, unfortunately there is no way to see the buffered  values in the Distributed System Manager - it will only show us the current value of the shared variables.

[LabBEAN]

---

mishkin wrote:

With datasocket, [when the client side buffer gets created/destroyed] seems to be related to datasocket open/close VIs.

---

This is good info and begins to explain the differences seen between the DataSocket API and Shared Variable Nodes.

Here are my outstanding questions (for BLAQmx, mishkin, et al):

In my example above (Message 15), "104" is the first element read with the first two reads, but "0" is the first element read with the last.  Can you explain?

When you set BufferMaxPackets with a DataSocket property node, how does this correspond with "elements" in the shared variable buffer?  How does this work for string shared variables where you configure "Number of strings" and "Characters per string"?  And, how about variant shared variables where you configure "Buffer size (bytes)"?

What happens when the shared variable is configured to have a different number of elements? Can you elaborate on what's happening with the two buffers (one configured with DataSocket property node and the other configured when creating the Shared Variable)?

How and when do we use "Buffer Maximum Bytes" (e.g. with Variant Shared Variables only?)?

When accessing a buffered shared variable via DataSocket, is the BufferMaxPackets property node always necessary after an Open?:

[MarkBlack]

---

LabBEAN wrote:

In my example above (Message 15), "104" is thefirst element read with the first two reads, but "0" is the firstelement read with the last.  Can you explain? 

---

 

I am not sure.  When I ran that same example my outputs were more as I would expect.

Clear Buffer = 104

Shared Variable Data = 104,0,1,2,3

Shared Variable Data 2 = 104, 1,2,3

Its possible there is another writer we are not accounting for.

---

LabBEAN wrote:

When youset BufferMaxPackets with a DataSocket property node, how does thiscorrespond with "elements" in the shared variable buffer?  How does this work forstring shared variables where you configure "Number of strings" and"Characters per string"?  And, how about variant shared variables whereyou configure "Buffer size (bytes)"? 

---

BufferMaxPackets and Shared Variable "elements" are synonymous for data types for all data types except though that do not support the BufferMaxPackets property (variants and large clusters).  Therefore variants buffer size (bytes) only corresponds to BufferMaxBytes.  I believe BufferMaxElements maps to Number of Strings and Characters per strings in this way:  Number of strings corresponds directly to BufferMaxPackets, while Characters per string defines how many characters make up a single element/single packet.

---

LabBEAN wrote:

Whathappens when the shared variable is configured to have a differentnumber of elements? Can you elaborate on what's happening with the twobuffers (one configured with DataSocket property node and the otherconfigured when creating the Shared Variable)?

---

 This question is explained by understanding the fact that although Datasocket can access PSP URLs the implementation is slightly different.  When we are setting the buffer on a Shared Variable we are defining the server and client side buffers for the variable.  These buffers are the same size, but since the server side buffer is cleared very quickly we usually only care about the value being stored in the client side buffer.  The client side buffer has been the buffer of discussion throughout most of this thread.    When we configure the buffer with the Datasocket property we are essentially letting Datasocket know that this buffer exists, how big it is, and it can place values in it.  If we don't set this property Datasocket has no knowledge that it is even there.  If we configure a Shared Variable to have a buffer of 50 and tell DataSocket that its BufferMaxPackets = 40 then a Datasocket Write we only be able to put 40 elements in the buffer.  

---

LabBEAN wrote:

How and when do we use "Buffer Maximum Bytes" (e.g. with Variant Shared Variables only?)?

---

 As I state above variants and clusters are only applicable to describing the buffer with the BuffferMaxBytes property.

---

LabBEAN wrote:

When accessing a buffered shared variable via DataSocket, is the BufferMaxPackets property node always necessary after an Open?:

---

 If you intend on access the PSP client buffer, then yes it is required.  If you only care about the last value written to the Shared Variable then it is not required.  

Cheers! 

[LabBEAN]

---

BLAQmx wrote:

I am not sure.  When I ran that same example my outputs were more as I would expect.

Clear Buffer = 104

Shared Variable Data = 104,0,1,2,3

Shared Variable Data 2 = 104, 1,2,3

---

Would it be possible for you to:

-Download the ZIP I attached in Message 15

-Remove your TestVars process using Distributed System Manager 2009

-Extract the ZIP and open the project

-Deploy the TestVars library from the project

-Run SV Node Buffer Issue VI (twice)

The first time I got:

 The second time I got:

"104" is the first element read with the first two reads, but "0" is the first element read with the last.  Why?

Here is that diagram again, for reference:

---

BLAQmx wrote:

I believe BufferMaxElements maps to Number of Strings and Characters per strings in this way:  Number of strings corresponds directly to BufferMaxPackets, while Characters per string defines how many characters make up a single element/single packet.

---

In that statement, you didn't mention how BufferMaxElements maps.  Can you re-explain how to set up "Number of strings" and "Characters per string" with the DataSocket properties?

[LabBEAN]

Thanks in advance for answering those questions...

[MarkBlack]

Hi LabBEAN,

Sorry for not posting back sooner.  We have actually taken a lot of interest in your thread and your feedback is most definitely effecting the design and functionality of these APIs in the next version of LabVIEW. That being said I've been putting off your questions for long enough, so here we go...

---

LabBEAN wrote:
"104" is the first element read with the first two reads, but "0" is the first element read with the last.  Why?

---

The explination for this lies in the nature of the SV buffer.  When we add more values to the Shared Variable buffer than is allowed we have to push the oldest value (which is the same as the first value in the FIFO) out of the buffer.  In your case the SV has a buffer of 10.  If we have already run this piece of code once the last value written to the SV is 104; therefore this will be the first value read from the SV FIFO and this value will remain the FIFO if there are no other values in the FIFO.  If we write 10 new values to the SV (as we do in this example) we would have 11 values in the FIFO, but because the FIFO has a cap of 10 elements we have to throw out the oldest element (104).  On the next read we will read out the new first value in the FIFO, 0.

SV Value/Buffer at Clear buffer indicator = 104

SV Value/Buffer at first For Loop (Write) = 104,0,1,2,3,4  (notice there are now a total of 6 elements in our 10 element buffer)

SV Value/Buffer at second For Loop (Read)=  104,0,1,2,4,4  (because there are still more SV clients waiting to read all of these values remain in the buffer even though they have been read)

SV Value/Buffer at third For Loop (Write) = 0,1,2,3,4,100,101,102,103,104 (notice there are 10 elements in our buffer and the 104 is missing.  This is because our buffer can have a max of 10 values in it and the oldest value is thrown out when the buffer is full and we write an additional value to it.

SV Value/Buffer at forth For Loop (Read) =  0,1,2,3,4,100,101,102,103,104

SV Value/Buffer after VI leaves memory = 104 (the reason there is the rest of the buffer is emptied in this example is because all SV clients accessing this buffer leave memory when the VI leaves memory)

---

LabBEAN wrote:
In that statement, you didn't mention how BufferMaxElements maps.  Can you re-explain how to set up "Number of strings" and "Characters per string" with the DataSocket properties? 

---

After talking to some of our developers there really isn't a one to one correlation to what we can describe in DataSocket and what we describe in the Shared Variable dialog.   The take-away here is that the buffer as described by the DataSocket API will be what ever value gives the smallest buffer; BufferMaxElements or BufferMaxBytes.  Therefore its very difficult to find out how many string characters will be added to a single DS packet.  I known this is really a non-answer, but its the best I can give you without writing some tests.  

[LabBEAN]

Hi Mark,

Glad you've found the comments constructive.  *Would you consider "tak[ing] interest" even after reading this post ?* I had originally planned on attending the CLA Summit in a couple of weeks, where I signed up to talk about Shared Variables with R&D.  Maybe next time...  I have a lot to learn.

The SV node buffer function you described, while not intuitive, is at least explainable.  Could you elaborate as to when an API (e.g. SV node, DataSocket Read) removes elements from the buffer upon reading them (i.e. traditional FIFO) and when the API leaves elements in the buffer, in which case older elements are removed as new elements are added to a full buffer (i.e. cache)?

Under "Buffer versus cache" here, Wikipedia explains:

 "A cache acts often also as a buffer, and vice versa. However, cache operates on the premise that the same data will be read from it multiple times..."

After your explantion, I had thought the answer to the question was:

DataSocket API = traditional FIFO buffer

SV Node = FIFO cache

... until I wrote some code to utilize one SV Node for reads and one SV Node for writes.  The following code, like before, reads the SV once, writes 0,1,2,3,4, reads 5 elements, writes 100,101,102,103,104, and reads 5 elements:

This time, I observed the same "traditional FIFO" results as obtained through the DataSocket example (with no errors):

Can you explain?

The goal in all of this is to be able to read a network-published, buffered, shared variable (e.g. string type) remotely.  There are obvious benefits of reading via URL vs hard coding a node.  Is any of this affected by binding local SVs to remote SVs (where a new buffer is created for each client SV) or enabling RT FIFO?  Is any of this documented (in addition to what is found in Using the LabVIEW Shared Variable)?

It just seems like there are several things playing together, and I'm not fully wrapping my mind around all of them:

- function changes between APIs (e.g. DataSocket, SV Node (multiple nodes, same variable, same VI), SV Node (single node), new Variable API)

- function changes when accessing a remote SV directly versus accessing a local SV bound to a remote SV

- function changes (?) when enabling RT FIFO

Thanks for your continued support in this, Mark.

[LabBEAN]

I've done some experiments with multiple DataSocket connections that I'd like to share.  Page 3, like others in this thread, has gotten sort of long, so...

[LabBEAN]

... in the interest of space I'm going to post those lengthy results (including 4 front panels and 4 snippets) at the beginning of Page 4 -->