Reading 2 GB binary text files

[csgolden]

Hello,

I have several data files that are around 2.1 GB. They were recorded in binary and the vi I use to read them seems to be having trouble with files this large. When I try to read any of these files I receive the following message: "Error 1 occurred at Close File in filename.vi" I do not have this problem for 1 GB files. I'm guessing labview is having trouble reading in this much data. What is the best way I can go about resolving this?

Any recommendations for programs that can split the file up into smaller sizes? I've been playing around with text editors that are capable of opening files this large, but progress is slow.

I'm running labview 8.2 on windows vista with 4 GB of ram with a 1.86 GHz processor. 

Thanks.

-chris

[Captain_Kirby]

Hi Chris,

Error 1 is not normally the error we see when dealing with large file sizes that cross over certain limits. In all honesty, error 1 often encompasses a fairly broad range of errors. The most common error that I see related to error 1 comes from incorrect characters in file names, though that doesn't seem to make sense for you since you are receiving the error after you have opened the file successfully. In either case, this document might provide some inspiration as to where the problem stems from.

Error 1 or Error 6 Occurred During File I/O - Input Parameter is Invalid

In any case, if you could post a small screenshot of your block diagram code section responsible for the effected file IO, it will greatly help us determine if any small and easily remedied error has occurred.

Beyond that, there are a few file limit sizes, but the only ones imposed by your system are related to the operating systems single process memory usage. LabVIEW itself uses a 64 bit data type to address the storage space within the files it creates. This will allow for a file size of up to 9.2 exabytes, so I don't think your running into any LabVIEW imposed limit. See the following document for more details.

Can LabVIEW Read Data Files That are Greater Than 2 GB in Size?

If the file size is the issue at all, it is more likely that the issue stems from LabVIEW possibly reading this file on disk into its application memory space. On a 32 bit operating system such as Vista 32-bit (I am assuming your reference to Vista is for the 32-bit version,) each process is allowed only 2 GB of physical memory usage without some OS trickery. This same rule applies for a 64-bit version of Vista if you are still using a 32-bit version of LabVIEW, which LabVIEW 8.2 is. But in this case, the error encountered is usually during the read and write operations, and comes up as a specific OS error, or a LabVIEW error without an error code. In either case, this is unlikely with the supplied Error code 1. But in any case, have a look at this article I found on MSDN listing out the memory usage and limits of the various Windows operating systems. It should provide a deeper understanding of the above topics.

Memory Limits for Windows Releases

Hopefully that helps you out some and gets you started down the right path. If your still having trouble, please post a screenshot of your block diagram code that handles the file I/O in question.

[rager]

Thank you for the response and your help. Unfortunately I haven't made any progress. Maybe the block diagram will help. Thanks for all the help.

-chris

[Captain_Kirby]

Hi Chris,

I'm having difficulty reading your screenshot. I am interested in the logic your using to generate the file name used by your File IO functions. Try generating this file name from a simple constant, does the error continue to occour?

Further, can you confirm that the error is coming from the Close File VI? The error is more often seen on the Open File VI instead.

[csgolden]

Hi Captain Kirby,

We have actually just resolved our problem. There was a glitch on our end on how we were reading in the file that caused the file size to be corrupted with files larger than 2 GB. Thank you for all your help though.

-chris

[Captain_Kirby]

Hi Chris,

Glad to hear you got it all worked out!