I am attempting to read in a large CSV file that contains a days worth of 1hz data (500 columns, 86400 rows, about 450MB). In my program, I am reading in multiple files and plotting certain columns for multiple weeks worth of data at a time. The data used to come in 1hr files (around 14MB), and I could manage it, but now it will be coming in 24hr files instead.
I only need about 100 columns of this data and I only need 1 out of every 300 rows (I would like to set this as a variable later). When I use the 'Read from Text File' function, I get a memory error: 'Not enough memory to complete this operation'.
Is there a way to only read a certain line (row) from a text file? Right now, it seems like I have to read the entire text file before performing any additional operations on it. I have tried playing with the 'Read Lines' and 'Convert EOL' settings, I've tried using the 'Read from Spreadsheet File' vi, and I have read through many different postings on this site but I can't figure out a way to do this.
If there are any other ways I might be able to read in this large data file, I am interested in those also. Thanks for your help.
