LabVIEW
Most suitable method to analyze data?
Solved!
You don't have a database, so a database library won't do much good. To fill the database, you'd still need to parse the file.
200-300 lines with 28 numbers per line is not that much. 400 MB is a lot... Although of course it depends on the processing time you need... Or refactor to ditch the file altogether and put the result in a database.
EDIT: Don't assume it will be slow. Test it, and then optimize\refactor only if needed. The simplest working solution is the best solution.
Simply use Spreadsheet String To Array, regular expressions, and\or File Read (maybe one line at the time, although it seems overkill) and parse the data to an internal structure of your choice. An array with a cluster, a cluster with arrays, a 2D array, or a set (or variant attribute set) would work... Then make VIs to support basic functionality on that (type def'd) structure.
Of course, putting it in a class would make sense to me, but that's just icing on the cake...
There are other pragmatic solutions, like make a new file with the ID, and put the one line in it. Then you can find the ID from the file name.
It's your program, choose what suits you the best.
Search LabVIEW like a graph! 
Imagine the steps you take in Excel and then automate them in LV.
Read file (You already have an easy to access text file, use that)
Sort by ID (Technically not necessary)
Calculate error frequency by ID
Report.
/Y
Spreadsheet String To Array is incredibly fast. So it's a good choice. Doing the same manually is often 1000X slower.
If you'd put the file in a class's private data, you can use whatever you like in there. The rest of the code won't get more cluttered if you have for instance an array of ID's and an array of clusters. On top of that you can change the entire implementation (e.g. from array of clusters to separate arrays, variant attributes or whatever). The rest of the program won't even notice, as long as the contract (interface) doesn't change. The implementation is encapsulated...
If you're at all interested in learning OO (if you're new to it) this would be a nice introduction.
@Haddock01 wrote:
I have realized that big programs in LV needs either OO or much maintenance. I'm using "much maintenance" method.
Arguably, but no argument from me.
@Haddock01 wrote:
Classes and so on are above my level of knowledge and perhaps I don't need to reach that level.
Which is a bit funny, as I use OO now because it's easier (after 15 years of non-OO).
There is a learning curve, but more then that a certain required mindset.
Starting out with suitable and small problems is crucial, and this would be one.
Haven't checked, but this seems to be 'a lot of information' (not sure about the quality):
Most important thing is to get started after reading about the very basics.
Like SCC, you'll get benefits very fast. But it's easy to get lost in all the discussions about fancy stuff. Fancy stuff you don't need when you start. Fancy stuff is a topic with OO, but note that this fancy stuff would be near impossible in plain LV (except by making your own OO-ish framework).
