Read From Measurement File... removes X Values of first column?

[rex1030]

During one of our tests, two instruments were switched at the terminal by accident. I need to read in the massive lvm files, remove the wrong scaling and apply the correct scaling, while switching the values in the columns and right it all to new files. Simple right?

I wanted to use the Read from Measurement File.vi to make things easier, because the files are very large and I would like to analyze them 100 rows at a time. Some of the the files are around 1.5 Gigs in size so I need to read them in chunks.

The read from measurement file keeps removing the first column from the data! It outputs the data as a signal (dynamic data) and I have to use the dynamic to numeric array express vi. For some reason, before I even get to that point, the first column is not in the data.

No matter what settings I pick on the read from measurement express vi, the time column is removed from the data. I have checked/unchecked the "first row is channel names" and "first column is time channel" to no avail. The odd thing is that in the preview it shows the first column, as if it will read it properly... but it doesnt. Nothing I change in the settings can seems to make a difference in getting the first column, the x values, out of the file.

Below you can see the first column completely removed from the data.

This is extremely frustrating. By probing the signal out I can see the dynamic data attributes and the time column has already been removed, so I don't think that the signal to double array express vi is the problem, but I am not sure.

I am attaching my VI and a small data file to be analyzed. You can see what I mean.

The alternatives seem less than adequate. The read from spreadsheet file vi wants an offset of a specific number of characters not rows. The problem is that this is not constant between rows for some reason, when hidden characters are taken into account so I cant just set the number of characters in 100 rows and increment the offset in a loop... like I normally would. That means I might miss data or get a corrupted row.

This means that I have to use the read from text file, read how ever many characters I think a row is (over estimating a bit) then search for the newline character, find out how many chars are in that set and then offset that for the next loop iteration, all while converting each string number to a double. Talk about slow.

I have searched around and found that I am not the only one that has had this issue. This is a common thing, but no one seems to have the answer. Why can't the read from measurement file VI read all of the numbers in every row? Why cant I tell it I want a 2d array of doubles out and not a dynamic data type? It has to be something I am doing wrong.

Attached is a zip file with my VI and two data files. The "S19_A_DSI_detensioning_c.lvm" is the one generated by my VI (_c meaning corrected). "S19_A_DSI_detensioning.lvm" is the original measurement file. I hope you will pardon my messy VI, it's a quicky.

Any help you guys can give would be much appreciated.

[rex1030]

Here are my current "Read from Measurement File.vi" Express VI Settings. Like I said, changing the settings around doesnt seem to fix the problem.

[TLE]

Use the "Read from Text File" primitive.  Right click on it and select "Read Lines".  You can then read a specific number of lines.  Use a for loop to set however many lines to read at a time.

Does this make sense?  If not I can probably send you a quick example.

[rex1030]

That is a great workaround. The help talked about putting a check next to "read lines" but for the life of me I couldn't find where to do that. I wonder what other VIs have mystery check options in the right mouse click menu. I mean normally options like those are inputs, I thought. I'm going to start right clicking on every VI I drop to see if there are options there I never realized.

I would still have to use the set file position VI and specify the byte offset right? How would I know where that is? I guess each character is a byte and I would count the characters in the string retrieved and then offset by that amount on the next iteration using a shift register?

While waiting for help, I ended up using the read from text file and using the match string to look for the new line character, and using the spreadsheet string to array vi, analyzed the files line by line. Thats just because I couldn't easily come up with a regular expression to get 100 lines. It was slow but it worked.

However, that still really doesn't answer the question of why it is impossible to get the first column with the read from measurement file express VI. Does anyone know? Is this a known bug?