FTP Alternative for downloading 50mb files

[rex1030]

I need to transfer 50mb binary data files from a crio to a pc over a celluar modem. I have implemented the FTP file transfer but for some reason FTP is moving really slow. Whether its the service provider throttling the transfer rate on the port, we are not sure.

I was wondering if there are any other ways to send files with LabVIEW. I don't have much experience with HTTP functions like HTTP GET  and I am not sure if that is even applicable here. The internet tool kit doesnt seem to have an HTTP Get function like it does for FTP. I have heard rumors with an OpenG solution but I have not been able to locate it and I would rather use labVIEW functions/toolkits first before going that route (for collaborative maintainability of code).

If there are other solutions, how do they work? Are they easy to set up? Do you have an example VI?

Thanks in advance.

[PhillipBrooks]

The speed of any data transfer will be dictated by the speed of the slowest part of the link. The cellular portion is going to be slower, regardless of FTP, HTTP, etc...

I don't know if it can be configured on the CRIO side, but the best you can usually do is to tweak the MTU size to match the network path...

Message Edited by Phillip Brooks on 10-06-2009 04:40 PM

[rex1030]

How would I implement a Path MTU Discovery on the network? How would I change the MTU settings of the crio/ftp?

[Ben_S]

rex1030,

According to this article, Do National Instruments Products equipped with Ethernet support Jumbo Frames (custom MTU)?, you cannot alter the MTU settings on the cRIO.

Check out these other documents that you might find helpful for your application. 

• Optimizing File I/O in LabVIEW and LabVIEW RT
• FTP Options with LabVIEW Real-Time
• Real-Time VI to Host VI Communication Methods

[rex1030]

Great post Ben. Very useful info. I had read a lot of it already a while ago but I had never really looked into the TCP stuff before.

Anyone have an example vi of sending/recieving a file using the TCP method described here?

http://digital.ni.com/public.nsf/allkb/48D244EC86971D3986256BD4005CCC28?OpenDocument

I guess I am not sure about modifing the data client.vi and data server.vi TCP example VI's to sending a binary file. I guess for the server I would read from binary file and type cast it to string. Would I read it and send it in 1500 byte chunks or would I just read the whole 50mb file? And then on the recieving side how would i type cast that to binary to write to binary file. I would assume i would be writing to a queue on the client side in one loop and using another loop to write it to file in chunks.

I know people in the community have some better examples of this than NI is providing. Worse comes to worse I'll just use shared variables but I wouldn't mind the bit of boost in transfer rate. Every bit of time I save sending/recieving the data files is more time I could be recording more data with the device

[battler.]

That's plausible and easy to do.  However, I don't see how it will speed up the transfer rate if that's what you are trying to do.  The FTP VIs use TCP.

Yes, just modify the shipped LV example client and server VIs.

Why would you need to send in chunks?  TCP/IP is not lossy (is guaranteed) or it will produce a detectable error.  I didn't know that you could Typecast to/from binary.

Take a look at the SMTP email VIs, which also use TCP.  If I recall correctly there is an option to transmit as Raw Binary.  You can open the email VI to see how transmitting binary over TCP is possible.

Cheers,

Battler.

[rolfk]

What kind of cellular modem do you have? What is the data service your provider allows? There are various systems, network provider plans and more.

As I don't know where you are I can't give you specifics but here in Europe you have basically at least 4 different levels of mobile network data communication speed:

GPRS (GSM): (CS1: 9kb - CS4:21.4kb/s)

EDGE (GSM): (theoretically up to 128kb/s)

UMTS (384 Kb/s, and a maximum of 2Mb/s for a single application only)

HSDPA (UMTS): (3.6 and 7.2 Mb/s)

LTE (long term evolution and as the name indicates still mostly just vaporware, theoretically up to 100Mb/s)

Note that these datarates are all in bits/per seconds, and usually not fully usable due to network congestion. And they are also raw channel throughput. Any protocol such as TCP/IP does have a considerable overhead that will reduce the maximum possible netto data throughput further.

FTP is indeed not an ideal protocol to transmit a large number of files efficiently through limited throughput channels, but for a single file transfer the additional overhead should be not to bad.

But the first question is to find out what data rate service and plan you are using. Most industrial cellular modems nowadays still work on the GSM or equivalent level and therefore use data services such as GPRS and there you simply can't go higher than about 2 k Bytes per second.

Higher throughput services such as EDGE are not supported on all GSM networks and not by all modems. And if you want to go even higher than that, you need an entirely new type of modem and network based on the still not widely used UMTS or equivalent technology.

Rolf Kalbermatter

Message Edited by rolfk on 10-09-2009 09:06 AM