How read just a part of a binary file?

[madness]

I want to read only a part of a big binary file.
I have made a piece of labview code that works well, but every time I try to read only a part of the file, I have an error.
I try to search where I have make a mistake in the code, but dont find.

Just for information : at the end of the code, I just display 1 data of 5.

Thanks in advance.

[Ray.R]

Good morning madness,

Unfortunately, I do not have LV installed on this PC, so I cannot look at your code.
However, (from memory) you should be able to wire a number representing the amount of data you want to read after opening the file. There is another "trick" if this gives you a problem, simply read the whole file then truncate the data you need. Nevertheless, you shouldn't have a problem to simply read the correct amount of data in the first place. Could it be that you are trying to read beyong the size of the file? How big is the binary file and what value is wired to the READ FILE? (Sorry, as I said, no LV at this PC).

Context Help is very useful when wiring to the FILE vi's.

Since I can't see the code: Are you trying to read a portion of the bin file, starting at point "x" & ending at point "y"?

Also, you may want to post the error message that you obtain.

Sorry that I could not open your vi (No LV on this PC). Check the values wired to the size of file, offset, and lenght of data to be read. You can wire in the size of the file by simply getting the size from the file info vi. The offset is where you want to start reading the data from and the length is the size to where you want to stop reading data.

Hope this helps.

-JLV-

[madness]

Joelabview.

So. I understand that you'r srry to dont have labview on your PC 😉
I make a screen capture to allow you to see the code.
The problem is that my binary file is about 50Mo/file !
The idea is to just read the interresting part of the datas. To be sure of the memory allocation, I create a table with the size of all the data and the I fill it with the datas. By this way, I'm sure there is no problem of memory allocation (I have the method from Ni support).
Then when I lunch my program, the error code is 116 (I try to translate from french...)

"Error 116 occurs in "read a file in LECTURE-ZOOM.vi
Possible reasons :
Labview : an operation on flattened datas or on a byte structured file is not working because datas are corrupted, unatented or cut"

I try to use the help as you tell me, but, it seems not to be helpfull : I have written the code after reading the labview help :-S.

Then, what can I do??? 🙂

[Ray.R]

Thanks madness,

The screen capture was a good idea... there are no limits to providing help 😉

I do have a question: why do you read the file within the loop?

Is the data obtained dynamically? (No LV, so I'm going from memory on the vi description)
What I mean is: before reading the file, do you know where the block of data resides within the file, as in where it starts and how long it is?

If so, you may want to simply open the file and read that portion of the file. Then pass the desired data to the loop for data manipulation.

JLV

[madness]

So...

For the loop, It's because the memory have to be managed correctly (If I understand well NI technical support).
The loop is just to read datas one by one, because when I was reading the data in one time,
labview take more then 1Go RAM ! So labview advise me to do this way : I do this way 😉

Then. Yes I can know how long is my file, but, it's not very easy. But I can know how many points are stored in the file. The problem is the following : If I read the file, then cut the part I'm not interrested in, the memory can "bug". When you read a file in labview, the datas are stored in memory, the if you display the datas, they are again stored in memory. If you read all, and the just take what you want, the memory is full off "all data+ data wanted." Thus, to prevent the memory "overflow", the less data read the best.

The question is : how read a file why my vi, from an offset given in front panel, to an other offset? I don't understand why I have this error.
Perhaps the wires are wrongly connected. If yes, I dont understand why I read the datas from an offset to...the end of the file... Indeed, I don't know realy how to read from one point to another correctly. 😞

Can you help me whith those points???

Thank you.

[johnsold]

I think there may be two points to be addressed.

1) If you wish to read successive segments of the file, wire the "offset" output of the file read to a shift register. Connect the initial offset "Debut lecture" to initialize the shift register. Connect the left terminal of the shift register to the "pos offset" input of the file read.

2) The error message you get ssems to suggest that the amount of data you are reading does not represent exactly the right number of bytes to produce the datatype you expect for the output (2-D array of SGL). If you dig into the help files, they describe the representation of various datatypes in memory. I think that array types include information about the size of the array. If your data is stored as one large array, the size information will not be present in each segment read from the file (only at the beginning of the file). Perhaps reading the data as individual numbers of type SGL and then creating the array external to the file read might work. I do not have any suitable data with which to try this, so it may not work.

Lynn

[madness]

I have wire the vi as you tell me, but the error still there.
I think the problem is what you explain.
Perhaps reading the data as individual numbers of type SGL and then creating the array external to the file read might work.

In reality, I have as you tell me : One big array.
So perhaps, you'r true whan you say that the data definition is in the file header, and that's why it doesn't work.
But the problem still there :if I read the datas with a single sgl read (with loop I think) and then create an array, I think that the memory allocation will change during the time, and then the memory problem will appears.
I will try to program this way, and tell you the result.
Because I'm in france, perhaps you will only have the answer tomorrow (the time I code+other work + sleeping 🙂 = tomorrow...)

Then If you can try do do a piece of code on a computer with labview, it will be very nice/helpfull !

Thank you for your very good help.
See you in an hour or tomorrow.

regards

[johnsold]

Here is a sample VI. The LECTURE-ZOOM.2.vi is a modification of your VI to show the principle. The arraytest generator.vi is a simple VI I wrote to produce a file to test the reading. Both will give file dialogs rather than requiring the user to enter a path.

Look carefully at the help files for file read VIs and at the datastrg.pdf file (in the manuals directory) which contains details about how data is stored in memory and in files. It appears that the array is stored without length information in the file.

Also the read count input to the read file VI may be useful in reading a row at a time of your array. I did not test this.

Lynn

[johnsold]

Here is the arraytest generator.vi

[madness]

Johnsold.
Thank you for your answer.

I have tested you vi LECTURE-ZOOM.2.vi, and I think there is some problems with my application.
First, let me transduce the vi into words : if columns index is more than the max columns value, then you stop, and the same think with the rows index.
With the sift register you make a memory allocation for the array that changes every loop.

By doing this way, I have two problem.
When I read a big file, the memory allocation takes a lot of memory, because the array allocation in memory changes every loop.
Besides, when I run the vi, the columns and rows are completly wrong, because, the command "chanel number is not respected : if I put 7 chanels, I have 5 chanels on the array.
Then, When I have it, I compare the "sortie" array versus "entrée" array, and I seems that AFTER decimation (1 point taken on 5), I have more points at the end than at the begining.
I give you a sample of a data file (compressed in zip...) to test your vi.
It's a 6 chanels data file, puted in a array.
When I try to run the vi you modify for me, I seems that it doexn't work because the result was totaly wrong... 😞

I think that I make a mistake/error in the vi understanding, because I try to change some parts of your vi, to adapt it to my application, but it doesn't work.
What do you think about my way of read the datas?
How can I change the vi to obtain the same array than in mine.

To see what I obtain, just lunch the vi in my next post with the data file in this post.