Data Transfer Suggestion

[ashm01]

Hi,

I am looking towards optimizing the data transfer of a 2D String Array between an RT Controller and the host. My Previous solution was to have a Datasocket connection which would embedd a 2D string array in a cluster and then transfer it over DS having a hand shaking between the host and the RT.

This seems to put a huge overhead in between them and could be slow at time if the size of the data exceeds more than 1000 elements in the array.

I am looking at possible alternatives in acheiving the same.

-One option was to create an File with those values and then FTP it into the RT and then have the RT read the values and process accordingly

OR

Open to any other suggestions,

Regards,

Ashm01

[devchander]

hi,

Try using simple TCP/IP  client-server.

in should work well for your kind of aplication

regards

Dev

[ashm01]

Hi,

Initially I did try this approach, but noticed a few aspects which were annoying and it prompted me to drop this method.

1) I was noticing few packets being dropped in the communication and the data can be corrupted.

2)The TCP Connection handling between the host and the RT is a major deterent in this approach. I would be getting a connection error every few minutes since the connection would be opened for data transfer and closed after receipt repeatedly.

Are there any proven examples available? BTW, the data can be huge at times. An Array of 15*1000 is the norm, anything beyond this causes problems.

Still open to suggestions.

Ashm01

[Dynamik]

Hi Ashmo1,

Should it be assumed that data-transfer must occur over an ethernet connection, or are you open to other communication links between the RT Controller and host?

---

I am looking towards optimizing the data transfer of a 2D String Array between an RT Controller and the host. My Previous solution was to have a Datasocket connection which would embedd a 2D string array in a cluster and then transfer it over DS having a hand shaking between the host and the RT.

This seems to put a huge overhead in between them and could be slow at time if the size of the data exceeds more than 1000 elements in the array.

I am looking at possible alternatives in acheiving the same.

-One option was to create an File with those values and then FTP it into the RT and then have the RT read the values and process accordingly

OR

Open to any other suggestions,

Regards,

Ashm01

---

[TiTou]

Hi,

I'm not sure wether the method I am using is the best but it work quite well in a big application that transfers a lot of data between a host and 2 RT targets.
On the RT target, I have a vi that put a boolean to TRUE when a new chunk of data is ready. This same vi is called dynamically by the host PC, the host checks the boolean peridically, if it's TRUE reads the chunk data.

I am also interested to learn other ideas, thanks for sharing

[ashm01]

I have somewhat adhered  to TiTou 's logic, However, there must be a grander way to share data between the host and the RT (Yes using an Ethernet connection).  One of them could be an FTP also and the host/RT polling if the File is new.

"there's more than one way to skin a cat", lets hear a few more options... 

[TiTou]

Ashm01,

I am still using LV 7.1 and will continue for at leat 6 month ahead, that's why I am very interested in learning any ideas on this thread.

Nevertheless, I keep an eye on LV 8 new features. It seems that "Shared Variables" do what we are talking about wihtout having to deal with the transfer.
1. Does anyone know how these shared variables are implemented in LV (FTP, ...) ??
2. Does anyone now if "Shared Variables" in LV 8 is a good way to transfer large amounts of data ??

"Share what you know, learn what you can"

[ashm01]

Hi TiTou,

I will be taking the plunge to LV8 and migrating my 7.1 projects to 8.0. I did install LV8 on a regular PC and tried the shared variables out. They did seem transparent and looked capable to transfer data by creating a "Custom Type" control and then binding it.

However, I cannot comment on the RT side because I cannot revert back to 7.1 if things go sour. I will know in a 3-4 weeks after I start the migration task. Till then, I am considering other possibilities.

Ashm01

[.avi file]

Greetings!

Shared variables are your best option towards optimization and transfer of a large amount of data. Also look at the shared variable demonstration.

http://www.ni.com/realtime/software.htm

The following links touches on all the communication mechanisms which you all have considered.

http://zone.ni.com/devzone/conceptd.nsf/webmain/614fe7a8bb91b16c86256e5c00574a80

Thanks much and hope this helps

Best regards
Avi Harjani

[Doug M]

Shared Variables are a great new technology in LabVIEW and make shared communication much easier and transparent.  You can enable a Shared Variable to be also have a RT FIFO for Real-Time applications, however you cannot use this option for the Custom (cluster) data type, only simple datatypes.  Nevertheless, this shouldn't matter if you are using the Shared Variables in a Communications loop rather than your Time-critical loop.

Also, a previous post mentioned trouble using TCP because of opening and closing connections.  You should be able to open a TCP connection once outside of a while loop and leave the connection open as long as the VI runs, writing to it inside the loop, then closing the connection when the VI finishes.  You should be able to transfer large amounts of data using TCP assuming the communications loop is getting adequate processor time.

As far as how Shared Variables are implemented -- one of the computers acts as the server and will host the Shared Variables in a Shared Variable Engine.  For optimization this should be the host PC, not the RT target, but you can host them on the RT target as well.  The protocol is called NI-PSP, and is built on UDP.