Network stream for array of clusters

[daqrookie15]

I am working on a data acquisition application and part of it requires me to read CAN signals using an XNET session and send them from the target VI to the UI VI using a network stream. Since I want to log the timestamp for when each signal is received, I am using an XNET session in Signal Input XY Mode.

The problem I'm running into is that Read Signal XY returns an array of clusters. Each cluster element contains an array of timestamps and an array of doubles. When I try to deploy a VI to the target that contains a network stream of this data type, it crashes the entire application. I've attached a screenshot of the relevant block diagram.

Is there a better way to create this network stream? Is it even possible to have a network stream of an array of clusters?

[Bob_Schor]

I think the issue may be the inherently variable and "unknown until transmission" size of the data format.  You want to send an Array of Clusters (no problem so far), but the Clusters themselves contain arrays of unknown size.

You say that the Cluster contains an Array of Timestamps and an Array of Doubles.  Are these, perhaps, "pairs"?  That is, could you define a Cluster that contained a single Timestamp and Double, and instead of a Cluster of Arrays, go with an Array of Clusters?

You should be able send such an Array of Clusters via Network Streams, and on the receiving end, you could (if you wanted) transform them back to a Cluster of Arrays.  This still leaves Signal XY which is an Array of either my Array of Clusters or your Cluster of Arrays.  However, if Signal XY is of a fixed size (say 6), your Network Stream protocol could be "Transmit 6 Arrays of Clusters" and, as they are received, assemble these Arrays into Signal XY.

Bob Schor 

[daqrookie15]

Thank you for the answer.

You're correct that the array of doubles and the array of timestamps represent pairs of signals and the time at which they were received. Every time Read Signal XY is called, it returns all of the signals that had been received by the hardware since its last call. So for my application, there are 120 signals coming in from 120 different sensors. 

Each element of the cluster array corresponds to a specific sensor. And within each element, the size of the timestamp/signal arrays depends on how many times the sensor had transmitted since the last time Read Signal XY was called. So, if a sensor had sent 10 messages, the arrays will be of length 10 in the corresponding cluster element. If the sensor had not sent any messages in that period, the length of the arrays will be 0. This happens a lot because the arbitration within the CAN bus prevents the frequencies of the messages from remaining completely consistent.  

With this randomness, do you think its possible to rearrange the data structure and guarantee that no data will be lost?

[daqrookie15]

Just to give a quick update of where I'm at. I tried running an application containing just the logic I showed in my screenshot. With this, I had no problem sending the data from the target to the host via the network stream. However, when I try to run my main application with the same feature included, the program runs briefly before stopping. I then get a message that connection to the target has been lost, and then an error message saying that the SVE on the target could not be found or did not start. 

I've tried reducing the buffer sizes of all of my network streams and eliminating all unnecessary shared variables, but the only thing that fixes the problem is either removing the CAN network stream, or changing its data type.

[Bob_Schor]

You have 120 sensors that may or may not have data to send.  The data consist of readings and times-of-readings.  Some questions:

• Are the readings at "random times"?  Or (we should only be so lucky!) are all the readings of the form "N Samples, starting at Time T0, with some fixed time-between-readings of dt"?  Note that for any one "set of readings", N (# samples), T0 (starting time) and dt (essentially the reciprocal of the sampling frequency) can vary, but are fixed for the specific reading.
• Alternatively, if the readings are not sampled at a fixed interval but at "semi-random" time points, do we at least have N, number of Samples, array D, N data points and array T, N timestamps?
• Can you describe the data structure that "originates" and "will contain" these data?  Because the number of points and the number of sampled channels seems to vary, I'm guessing that this is being saved as a file rather than an N-channel Array, which would imply that all N channels had the same number of data points, which you said was not the case.

It seems to me that (depending on the specifics of the answers to these questions) you should be able to do something like the following:

• Ask the Remote to send 120 channels of data, one channel at a time.
• As each Channel comes in (with potentially varying amounts of data, including nothing at all), save it as appropriate.
• When all the Channels have reported, you are done until the next time you need data.

Network Streams can easily handle this one-channel-at-a-time situation.  The questions I asked first were part of an attempt to simplify the Channel traffic -- a Waveform for a fixed sampling frequency but potentially differing # of points, or an Array of Clusters for Value/Time pairs.

Bob Schor

[Bob_Schor]

You have 120 sensors that may or may not have data to send.  The data consist of readings and times-of-readings.  Some questions:

• Are the readings at "random times"?  Or (we should only be so lucky!) are all the readings of the form "N Samples, starting at Time T0, with some fixed time-between-readings of dt"?  Note that for any one "set of readings", N (# samples), T0 (starting time) and dt (essentially the reciprocal of the sampling frequency) can vary, but are fixed for the specific reading.
• Alternatively, if the readings are not sampled at a fixed interval but at "semi-random" time points, do we at least have N, number of Samples, array D, N data points and array T, N timestamps?
• Can you describe the data structure that "originates" and "will contain" these data?  Because the number of points and the number of sampled channels seems to vary, I'm guessing that this is being saved as a file rather than an N-channel Array, which would imply that all N channels had the same number of data points, which you said was not the case.

It seems to me that (depending on the specifics of the answers to these questions) you should be able to do something like the following:

• Ask the Remote to send 120 channels of data, one channel at a time.
• As each Channel comes in (with potentially varying amounts of data, including nothing at all), save it as appropriate.
• When all the Channels have reported, you are done until the next time you need data.

Network Streams can easily handle this one-channel-at-a-time situation.  The questions I asked first were part of an attempt to simplify the Channel traffic -- a Waveform for a fixed sampling frequency but potentially differing # of points, or an Array of Clusters for Value/Time pairs.

Bob Schor

P.S. -- while I've worked with Network Streams, I've never worked with CAN.  I tend to avoid the Shared Variable Engine, since (in my hands) it appears much less robust than Network Streams.  I can tell you that I've used 4 Network Streams in a LabVIEW Real-Time Project -- two Streams provided bi-directional Message Queues between Message Handler State Machines on the Host and Remote, one Stream was a Data Stream continuously sending 24 channels of 16-bit data at 1000 Hz to the Host (where it was displayed and streamed to disk), and the final Stream was "Point-in-Time", or Event, data, such as State Machine transitions or Digital inputs and outputs.

[daqrookie15]

So far, I have not had any luck with your suggestions. When I run this part of the code in its own VI, there are no issues, but putting it in the main application does not work. It seems that whenever I try to stream to anything more complex than a 1D array of doubles, starting the application crashes the target.

I've attached the VI that is running on the target, showing what I had originally attempted. The network stream giving me trouble is the one named "/data/CANwriter", which is written to in the CAN loop. Is it possible that I'm overloading the target? If so, what can I do to alleviate this?

[Bob_Schor]

My Network Stream Names seem to be a bit more complex than yours, but I think the most obvious problem is that you are still trying to send "an array of arrays", the data structure for the CANwriter, which I would expect to cause trouble because of the uncertainty of the data size.  I'm prepared to be proven wrong by this.

I've already suggested that instead of sending an Array of Clusters whose elements are Arrays (presumably of exactly the same size, but much messier if the sizes can differ), you should transform the Cluster of Arrays into an Array of Clusters and send that.  Alternatively, create two Network Streams, one for the TimeStamp Arrays and one for the Dbl Arrays, split the data and send them separately on the Target, reassembling them (if necessary) on the Host.

Bob Schor