Save me from myself (i.e. my rube)

[GregFreeman]

I didn't really know how to put a topic on this that would be anything meaningful, but it's been a while since I've asked someone to save me from a rube. I keep looking at this code and thinking there must be a better way. So, as per usual, there probably is. 

I am reading a bunch of data files and concatenating the 2D arrays. These files will all have the same number of columns, but could have varying number of rows. In my example I've posted, I've just used a 1D array, but the logic should be the same. Basically, if a condition is met, I want to notify the user which file has the bad data. But, because all the data from all the fiels has been concatenated this is difficult. So I have done the following: When loading the files, create an array of file names along with an array of the particular files lengths. Find the bad index in the file data array then determine which file the bad index is in by doing the following (easier to show the snippet than describe). The intial for loop is just to create some dummy data

[Darin.K]

I would be inclined to maintain a cumulative array, for example, each element contains the start index for the file's data in the concatenated array.  In this case it would be 0,3,7,12...  Then a simple Threshold 1D array followed by taking the floor of the fractional index gets you the desired file index.

[GregFreeman]

---

@Darin.K wrote:

I would be inclined to maintain a cumulative array, for example, each element contains the start index for the file's data in the concatenated array.  In this case it would be 0,3,7,12...  Then a simple Threshold 1D array followed by taking the floor of the fractional index gets you the desired file index.

---

Not a bad idea. Easy enough, just keep a counter which keeps track of the sum of all the file lengths loaded, as they are loaded, and uses autoindexing to build up the cumulative array.

[GregFreeman]

Darin's method attached

[MarcoMauri]

HI!

Two proposals in the opposite directions:

1) Tag each line with an idex identifying the file it comes from. Basically you need to add a column to your 2D array.

This allows you to sort the lines, for instance, before the validation.

The cost for this is, let's say, 1 byte per line.

2) Validate the line before concatenating the arrays. But I suspect this is not an option....

Marco