conditionally select rows from 2D array

[Sam Hutton]

I have ascii files that contain lots (up to 500,000) rows of data. The first item in each row can either be a number or text. I'm trying to write a VI that reads the asii file into an array (using Read from Spreadsheet FIle.vi) and then "parses" the resulting 2D array so that all the rows which start with a number go to one sub-array, and all the rows starting with text go to another. So in the snippet below I would end up with one 4c by 10r array with all the "numeric" rows, and a 11c by 4r array with all the "text rows" in it.

15764226      242.5      285.3     1379.0  
15764230      242.6      285.5     1379.0  
15764234      242.7      284.0     1379.0  
15764238      243.1      282.5     1378.0  
EFIX R   15764010    15764238    232      239.1      282.7       1381
SSACC R  15764242
15764242      243.3      278.9     1378.0  
15764246      247.3      272.6     1375.0  
ESACC R  15764242    15764246    8      243.3      278.9      247.3      272.6       0.25         47
SFIX R   15764250
15764250      247.1      272.0     1373.0  
15764254      247.0      271.7     1361.0  
15764258      246.7      271.6     1357.0  
15764262      246.6      271.6     1354.0  

I've made some progress - the vi below reads the data in and then spits out all of the numbers from the first column into a subarray. I just can't work out how to extract all of the data for each row, not just the value in the first column. I've attached a massively trunacted example ascii file.

Any pointers much appreciated!

[SteveChandler]

First of all you do not want to use build array. That will cause the array to be copied at each iteration. (LabVIEW is pretty smart so it might make guesses to avoid a copy each iteration, but performance will really suffer anyway). What you want to do is initialize each of the subarrays to the number of rows you read, then after the loop get the subset containing the data.

Something like this. [Edit: Attaching file in LV8.2]

[Alexander_Sobolev]

Steve, why do not you autoindex original array?

One more note (Sam): If one row in 2D array has 11 columns, then all the array has 11 columns - you waist a lot of memory.

I think it is better to analyze and sort array while reading from file row by row. Read from text file.vi can read single line.

[nyc_(is_out_of_here)]

I would immediately connect the output of the Read From Spreadsheet File VI to a For Loop and use its auto-indexing capability.

Get the first element then get the first character of that element. Decimal Digit? function on that first character. Use the output of that for a Case statement. Have two shift registers to build the two different 2D arrays you want.

[SteveChandler · Accepted solution]

Good point Alexander, I thought he was only interested in the first column.

That is a simple modification. Just change the two array shift registers to 2D

[SteveChandler]

---

@Alexander Sobolev wrote:

Steve, why do not you autoindex original array?

In this case I don't think it makes a difference. I am always interested to hear about performance gains by doing something one way vs another way.

The point I wanted to make is that using build array inside of a loop will absolutely kill performance.

[Sam Hutton]

Wow thank you so much for your rapid replies - I will work through your solutions and try to understand them!