How to read blocks of 25,000 rows from a textfile and save them separately?

[JackyCola]

Hi,

i have the following problem:

I have a saved textfile with approx. 3,000,000 rows of measured values (time, voltage, current).

I need to read blocks of 25,000 rows out of this textfile, generate histograms from voltage and current and and finally save each pair of histogram in a separate file.

The problem is, i didn't find a way to read the blocks of 25,000 values out of the large textfile.

Thx for any help!

[IanW]

Hi Jacky

Use the Read Lines From File vi with the numer of lines set to 25000. After reading the first data count number of characters and use this for the read offset for the next resd. Add the offsets each time round the loop.

Hope this helps

Ian

[tst]

The Read File VI (and some more top-level VIs as well) includes a Count input and an offset input. You can use this in a loop with a shift register to read a chunk of the file each time. You can just use the shift register to remember where the last place you read was. There is probably at least one example showing something like this in the example finder.

[JackyCola]

Thanks for your help guys but i can't find either the read from lines VI nor the read file VI in Labview 8.0.
They are available in 7.1 but I have to use 8.0.
Can you tell me a way to use one of them in 8.0 or do you have another idea?

[tst]

I only had the evaluation version and it expired, so I can only say that I remember they made some changes to the icons of the file VIs. The functionality should still be the same, however, and the appropriate VI should probably be in the File palette. In the worst case, you can save the VI in 7.1 and open it in 8.0 to see. You can also try looking at the examples or searching the palettes.

[DavidJCrawford]

You are right the Read Lines From File isn't there in the File I/O palettes but its still there in file.llb

David

Message Edited by David Crawford on 01-17-2006 01:36 AM

[JackyCola]

I used the way posted bei tst and saved the VI in 7.1 then re-opened it in 8.0 and used copy&paste. This worked. I only experienced some problems with offset but

we'll see.

[altenbach]

You should "read lines from file" only for very short files, because it is not very efficient. You'll end up doing 25000 read operations on the file. 😞

If your file is well formatted (and hopefully it is), you have a fixed number of characters/line and you can calculate the offset to read your 25000 lines in one swoop.Then you can dissect it in memory.

Please attach a sample file with about 30 representative lines and we figure out the best way to do this. 🙂

[JackyCola]

I will attach a sample file tomorrow.
In the meantime i tried the Read From Spreadsheet VI but had the problem that it gives out only a single-precision array and i need a precision of at least 3 digits.

[tst]

If you will look inside that VI you will see it simply uses one of the string functions to convert the array. By changing the type input to DBL, it will work.

As Altenbach mentioned, however, this is even more inefficient since it calls the Read Lines From File VI and adds even more overhead. As he suggested, it would be best if you could use the Read File primitive to read a specific number of bytes from the file.