I was wondering if anyone had advice on how to speed up operations using these functions. Generally, I'm using them to build up text files with columns of data, like: for(i=0;i<numitems;i++) FmtFile(file_handle, "%s<%f%c%f%c%f%c", dataarray1[i], 9, dataarray2[i], 9, dataarray3[i], 10); This produces a very nicely formatted file that can be read with Excel and other applications. The file can be read back by the application in a similar fashion: for(i=0;i<numitems;i++) ScanFile(file_handle, "%s>%f[d]%f[d]%f[d]", &dataarray1[i], &dataarray2[i], &dataarray3[i]); This works great for small data files. However, if the number of items gets large, or if the file is on a network share or USB drive this is rather slow. It seems that this method causes lots of small file I/O operations that slow things down, especially if the file is not on the local harddrive. I've tried thinking of a few ways to speed things up, including: 1. Witing the arrays all at once with one call to FmtFile instead of a loop, such as FmtFile(file_handle, "%s<%*f%*f%*f", numitems, dataarray1[i], numitems, dataarray2[i], numitems, dataarray3[i]);. The problem with this is the file wouldn't have the nice columns in it anymore which would make it a lot harder to import into other applications like Excel. 2. Using a temporary interleaved array. Reading in the whole array at once could be done like this: ScanFile(file_handle, "%s>%*f", 3*numitems, tempdataarray); Then seperate tempdataarray into their respective arrays in a loop. This seems like a good solution for reading, as I don't think the different seperators would conduse the ScanFile function, but I can't think of a way to write the file in a similar way and get the nice columns. Any other ideas? I think what would be ideal would be a version of Fmt and Scan for operating on large buffers, where there is an internal pointer that moves with each call in the same way as FmtFile and ScanFile works with a file pointer. Then a large temporay buffer could be used to hold the file contents in memory, and the buffer could be formatted and scanned usning loops exactly the same as if using FmtFile and ScanFile. Thanks.

LabWindows/CVI

Speed of FmtFile and ScanFile

Author

Well, I would hate to give up the convenience of the CVI functions, but if the C functions are twice as fast, I might have to brush up on my knowledge of the standard C library functions. In any case, I have a few things to try. Thanks for your replies!

Message 11 of 11

(649 Views)

LabWindows/CVI

Speed of FmtFile and ScanFile

Rif.: Speed of FmtFile and ScanFile