topic Re: TCP/IP transfer efficiency in LabVIEW

tcp/ip transfer efficiency

przemmo — Mon, 15 Aug 2011 16:50:26 GMT

Hello,

I have cRIO-9075 with 4 modules the NI-9148 expansion chassis and host PC. As the 9075 is not too fast (small DMA channels, small Spartan) I want to transfer all data, read by cRIO from C Series modules, to the host PC and process them there. The modules are:

9237 4 channels, bridge input module, 50 kSps x 4= 200 kSps

9201 voltage input module 500 kSps

9211 Thermocouple input module, 4 channels, data rate below 20 Sps

9411 digital input module user for triggering the other modules

the expansion chassis will home 9203 200 kSps current input module and, (user can remove the 9411 and 9211 from cRIO and install them in the expansion chassis) 9411 + 9211.

The total data throughput is:

500 kSps x 2Bytes = 200 kSps x 2 Bytes + 200 kSps x 3 Bytes= 2 MSps (this is the max transfer rate).

On the CRIO I have two loops:

time critical, high priority to read data from modules and put them to FIFO

non-time critical for transferring data to the host PC.

To transfer data to Host I want to use TCP/IP.

My question is how big should be the blocks of data I write to the "TCP Read" function and how often should I put them there?

Please find attached example transfer loop. This piece of program transfers an array of 10 000 elements (uint32) to the host and handels TCP errors.

Best regards,

Przemmo

Re: tcp/ip transfer efficiency

przemmo — Mon, 15 Aug 2011 20:31:44 GMT

Hi,

I made something more, please find attached two files- these are the previously mentioned loops.

The time critical only reads data from modules and saves to queue

Not- time critical receives data from queue, processes and sends to TCP Write function.

How long the timeouts should be? Maybe the streaming functions will be better for these purpose?

Re: TCP/IP transfer efficiency

Himanshu_Goyal — Tue, 16 Aug 2011 18:35:26 GMT

Hi,

The TCP/IP data transfer rate is depend on your network properties. Generally in network we use 10/100 MBPS line so it indicates your transfer rate.

In your recent system there is some system with 1 GBPS feature.

The network transfer data rate is totally depend on your hardware specification.

Thanks and Regards

Himanshu Goyal

Re: TCP/IP transfer efficiency

Mark_Yedinak — Tue, 16 Aug 2011 19:15:19 GMT

@Himanshu Goyal wrote:

Hi,

The TCP/IP data transfer rate is depend on your network properties. Generally in network we use 10/100 MBPS line so it indicates your transfer rate.

In your recent system there is some system with 1 GBPS feature.

The network transfer data rate is totally depend on your hardware specification.

Thanks and Regards

Himanshu Goyal

In addition the ultimate throughput will also be dependent on what other traffic is on the network.

Re: TCP/IP transfer efficiency

przemmo — Tue, 16 Aug 2011 20:03:58 GMT

Hello!

Thank for reply.

Yes, these are basic infirmation concerning TCP. Both Hosta and cRIO have 100Mbps ethernet cards. But my question is how to organize the pransfer as it doesn't work. Maybe I can set buffer size in Windows 7 to bigger size? I haven't found any information about buffer size in cRIO 9075 (?). What else can I do?

Przemek

Re: TCP/IP transfer efficiency

Mark_Yedinak — Tue, 16 Aug 2011 20:22:25 GMT

I have seen some strange things with TCP and Win7. Others have reported issues too on LAVA. Microsoft changed their stack and it seems to behave differently. Are you able to try your program on an XP machine?

Re: TCP/IP transfer efficiency

przemmo — Wed, 17 Aug 2011 06:55:35 GMT

Will try to run my program on XP.

For now I wrote a very simple program for the cRIO- it sends an array 400 long to the Host. It works correctly. But when I incerase the size of the array the time necessary to send it increases twice (can be seen on both windows 7 and cRIO ). When I increase the size to 1600 Bytes the time increases.

It looks like the transfer limit- but somehow I do not believe.

Re: TCP/IP transfer efficiency

Gandalf_il_bianco — Wed, 17 Aug 2011 07:22:16 GMT

Using tcp u have to consider the data maximum data size. about 1500 bytes (?) not sure.... when you go over that size tcp use fragmentation of the data to send them.

This means that u double the time to send them when u oversize the limit even of one byte......

Often the use of tcp is a wrong choice... may be the better choice is udp, udp have bigger data size and do not nead the round back time of the acknwledge.....

So is faster then tcp.... U can easily insert a packet number in your datagram to check the consistency of your data and eventualy work around a lost of a packet......

have a look to tcp/udp datagram on the net.....

bye

Re: TCP/IP transfer efficiency

przemmo — Wed, 17 Aug 2011 12:52:24 GMT

Hello Gandalf,

It is not a problem- the packet size. The TCP Read function handles it for you.

Most probably there is a problem with Ethernet card settings under Windows 7.

I think Polish Support Enginner helped me to solve the issue. Will report.

Thank you,

Przemek

Re: TCP/IP transfer efficiency

DominicRoberton — Wed, 17 Aug 2011 13:22:04 GMT

Hi przemmo,

An alternative would be to simply use the data streams method, I have been investigating this with my system (2 cRIO's generating 169KB/s), my thread is here

The data streams are good because you can put any data type in, and as long as you know how to define it at the other end you will get the data out without any further manipulation.

Dom

Re: TCP/IP transfer efficiency

Gandalf_il_bianco — Wed, 17 Aug 2011 14:25:28 GMT

hi of course the function handle it for you, fragmentation of the packets are managed by tcp ip.

To verify just try an ethernet analyzer like wire shark......

You will see with your eye....

Re: TCP/IP transfer efficiency

Mark_Yedinak — Wed, 17 Aug 2011 14:32:03 GMT

@Gandalf il bianco wrote:

hi of course the function handle it for you, fragmentation of the packets are managed by tcp ip.

To verify just try an ethernet analyzer like wire shark......

You will see with your eye....

You will not see any significant benefit from fragmentation or the lack of it when using UDP over TCP. Both are constrained by the packet size of Ethernet. UDP is faster since there is less overhead but you are also subject to potential data loss. With TCP you will not lose data. If you are sending large data sets TCP and UDP will be very similar in terms of packet fragmentation as both will send maximum size Ethernet packets. Again, UDP has less over head so it can stream data faster than TCP as long as you can be tolerant of potential data loss.