LabVIEW for ARM 2009 Read from text file bug

[w0utje]

Hello,

If you use the read from text file vi for reading text files from a sdcard there is a bug when you select the option "read lines"

you cannot select how many lines you want to read, it always reads the whole file, which cause a memory fault if you read big files!

I fixed this in the code (but the software doesn't recognize a EOF anymore..) in CCGByteStreamFileSupport.c

at row 709 the memory is allocated but it tries to allocate to much (since u only want to read lines).

looking at the codes it looks like it supposed to allocated 256 for a string:

Boolean bReadEntireLine = (linemode && (cnt == 0)); 

if(bReadEntireLine && !cnt) {
  cnt = BUFINCR;    //BUFINCR=256
  }

but cnt is never false since if you select read lines this is the size of the file!

the variable linemode is also the size of the file.. STRANGE!

my solution:

Boolean bReadEntireLine = (linemode && (cnt > 0));  // ==

 if(bReadEntireLine) {    //if(bReadEntireLine && !cnt) {
  cnt = BUFINCR;
  }

and now the read line option does work, and reads one line until he sees CR or LF or if the count of 256 is done.

maybe the code is good but the data link of the vi's to the variables may be not, (cnt and linemode are the size of the file!)

count should be the number of lines, like chars in char mode.

linemode should be 0 or 1.

Hope someone can fix this in the new version!

greets,

Wouter

[w0utje]

I have another solution, the EOF works with this one.

the cnt is the bytes that are not read yet, so the first time it tries to read (and allocate 4 MB).

you only want to say that if it's in line mode and cnt > 256 (BUFINCR) cnt = BUFINCR

the next time cnt is the value of the bytes that are not read yet, so the old value minus the line (until CR LF) or if cnt (256) is reached.

with this solution the program does not try to allocate the whole file but for the max of 256.

in CCGByteStreamFileSupprt.c row 705

 if(linemode && (cnt>BUFINCR)){
   cnt = BUFINCR;
 }

don't use the count input when using the vi in line mode. count does not make sense, cnt will be the total file size. also the output will be an array.

linemode seems to be the value of the file size but I checked this and it is just 0 or 1, so this is good 🙂

update: damn it doesn't work!

[w0utje]

the read from text file vi only checks for an EOF if you add a variable to count, then is the output a string array.

 if (textOutType != StringDataType)
  bReadLines = true; // string array output

otherwise it will just do the function with the code I changed because of the memory bug.

the funny part here is, there are 2 readlines variable's, the second goes with the function and the first in an if case.

but this one is checked on the data type? why not just that other variable..

I will test this further

[BasvanDijke]

Hi Wouter,

My colleague in Amerika has tested this with the attached code. With this code, reading lines works good. Could you please test this code?

Regards,

Bas van Dijke
National Instruments Netherlands Support

[w0utje]

I have tested it and it works, I read 3 lines of a file that has 5 lines of text.

BUT, when I try to read 3 lines of a file that has many lines of text ( I have log files of 4 MB)

it doesn't work!

it tries to allocate the WHOLE file (4MB!!) , so it crashes.

with my solution in CCGByteFileStreamSupport.c is does work.

change the following on line 703:

 if(bReadEntireLine && !cnt) {  
  cnt = BUFINCR;
  }

replace with :

 if((linemode && (cnt>BUFINCR))||bReadEntireLine){
   cnt = BUFINCR;
 }

[BasvanDijke]

Hi Wouter,

It is possible that it has to do with how the file is build up. 

Of course, there's going to be a problem if all file data is on one line in the file, and that one line is read into LV. The amount of data capable of being read into LV is dependent upon how much of the heap is available for use. Could you please post a sample file and your code?

Regards,

Bas

[w0utje]

Hi Bas,

My log files have many lines. Labview for ARM reads a line in maximum parts of 256 bytes, you can see that in the code (BUFINCR).

So, I have many lines and my log file is like 4 MB (or bigger). Read lines tries to allocate the whole file and thats the fault in LabVIEW for ARM.

the problem isn't how the log file is build or how you LabVIEW code is made, but the C code that's made of the LabVIEW code.

You can try it yourself in the LabVIEW code you placed in this topic (from your colleage in USA).

place a file (with lines) like 80 chars per line & 76800 lines.

your ARM will crash because the C code allocate the whole (76800x80 bytes=6 MB) file.

Greetings Wouter