@Tarnarmour wrote:
First of all thanks for all the helpful comments and explanations. I guess I was naively hoping there was some magical way to just find locations in a file but it makes sense that to find a specific substring in a text file you would need to read out the file. The data here is not crazy large but it's probably big enough to make it not worth doing that. I understand a lot more now the use of things like database structures, this has been informative.
I'm curious about the TDMS format. I thought to use text files mostly for ease of access; the raw test data being stored is more of a redundancy. The main use of these files is for other engineers to be able to verify that a product validation test was carried out and that the product passed the test when they were manufactured. I'll do some reading about the TDMS format, but do you think it's easy enough to access for people who may not have LabVIEW?
That is EXACTLY why TDMS was invented. So engineering can know more about the data collection to determine have better viability to "Process" and greater confidence in results while asking "Test Developers " to quickly expose data of interest that may influence some production control matters.
Dig out the example finder. And Read THIS!
Don't get me wrong, The TDMS format is not the best option for every instance! But, I will be willing to bet a real Magic 8-Ball (tm) that your Engineering clients will adopt it quickly. The rare exception being a guy that loves regexes.
CA and tst both alluded to a possibility of the problem that the file space will either be bloated or become fragmented. In the case of TDMS the file is binary (less bloated than text) but can become fragmented. Tools exist to defragment the file since the manner that fragments are caused is known up front ( and kept in a safe file)
"Should be" isn't "Is" -Jay
