LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Reducing data rate written to a LVM file

I have a LabVIEW 7.1 question regarding data logging. I have designed a system that reads the pressure of 6 4-20Ma pressure transducers. This data is then filtered to remove any noise, and displayed to the user on the computer's screen.
I am currently reading the data at 1000HZ using the DAQ Assistant, which seems to give a clear picture of the pressure changes. My supervisor has asked me to allow the system to log the results for what may be 3 or 4 days. As you ca imagine, 48 hours at 1000 samples a second is a lot of data! I need some method to reduce the data rate written to the file. Ideally, I would like to take one second of data, find the highest reading, and then file a single entry and a timestamp interval of 1 second. So, for each hour, there would be 3600 rows with 7 columns (1 time stamp and 6 sensors). It would be nice if the 1 second average could be adjustable, so that if long term logging is to be done (a week, for example), the interval can be changed to 5 seconds or whatever.

I have tried reducing the data read rate as much as possible, but it seems that 100hz is the smallest that is possible. That is still too high... as the hard drive would fill up in no time.

Any help in this area would be appreciated. I have tried a few things, and all to no success. I have included a copy of the code for anyone to review.

Hardware:
- 1 X P3 Laptop running LabVIEW 7.1 data acquisition software
- 1 X NI DAQCard-6062E interface PCard
- 1 X NI SC-2345 data acquisition hardware with
- 3 X SCC-CI20 current input module
- 6 X Omega PX 0-600PSIA 4-20mA pressure transducer

thanks so much!

Andrew
0 Kudos
Message 1 of 6
(3,316 Views)
You would have to talk with your supervisor first to determine what he intends to do with the log data and what degree of resolution he actually needs first. You probably want the 1000Hz Sampling rate so you can do decent filtering on the signal (hopefully your pressures aren't actually changing at a faster rate than that). I'm assuming you are returning a single result for those 1000 readings for each sensor. Specify some file logging duration of n-seconds (or minutes or whatever). Between file writes, pop each filtered measurement into an array (either 1 array of 6 dimensions or 6 1-dimension arrays). After n-seconds have passed, determine the min, avg, and max values from each sensor (array) and log those with your timestamp. So if you set your log timer for 1 minute, you would log a single min, max, and average reading of 60 readings for 6 sensors (this would only require 1 row with say a timestamp and 3x6 (18) columns for each sensor's min, max, average data). After a 24hour period you will have logged 1440 rows of data. In 3 days that would only be 4320 rows of data. All that is as easy as using a timer and a case structure around your logging function which would be triggered every n-seconds. Everything else you're doing would be the same. None of this really has much to do with labview as it is more of a logical explanation of how and when to acquire and log your data. What method are you using for storing your data? CSV, BIN, etc.. If you also want to display the data in a chart, I would recommend charting the same data you're logging, otherwise your chart will probably crash your system at 1000samps/second for 60 hours.. Once again, it depends on how your supervisor is analyzing the logged data. Make your log duration programmable and change it until he is happy. If he's(she's?) using Excel, your maximum log timer would be 9 seconds (Equates to ~6.67 Logs per Minute, ~400 Logs per Hour, ~9600 Logs Per Day, for a total of 28,800 Logs(rows) for 3 days -- Excel is limited to 32000 rows).
0 Kudos
Message 2 of 6
(3,306 Views)
Your "Minimum" Log frequency would be 9 seconds (Not your maximum).. Duh..
0 Kudos
Message 3 of 6
(3,305 Views)
Thank you for posting to the National Instruments Discussion Forums.

Regarding your question, I believe you have at least two different options for reducing your data to store it to disk. As mentioned in another post, one option is to save only analysis and statistical information about your data. This is most likely be the smallest way to store the information, but likewise will lose any detail about the shape of your data.

If the only issue is that you are unable to reduce the sampling rate given by your hardware, there are options to further reduce the data in LabVIEW. Attached to this post is an example of how this may be done in LV 7.1 with an express VI. The sine wave in this example is generated at 1kHz and displayed in the first waveform graph. The second graph displays the data at the new sampling rate determined by the control labeled "New
Sampling Frequency".

Please feel free to reply to this post if you have any additional questions.

Have a great day!

Scott Yellig
Applications Engineer
National Instruments
Message 4 of 6
(3,287 Views)
Thanks for the resample.vi! I have incorporated it into my program, and it performs excellent! I do have one more question. The values that are output from the resample, are they the average, maximum, or minimum? I am sort of looking for the Maximum values to see where the sensor readings peak over the resample interval. For example, if the resample is going from 1000HZ(in from DAQ) to 1hz (to file to conserve space), then what is the value written? Our machine is running at a 0-400psi cycle at 3 hz. We would like to find the maximum value of the pressure over this interval so that we can trend the pressures inside the machine.

Thanks for your patience and all the help I have received so far.

Andrew
0 Kudos
Message 5 of 6
(3,280 Views)
Andrew,

The output from the resample VI is not really an minimum, maximum, or average. The resamples VI treats your signal as if it was actually sampled at a lower frequency.

For example, let us say that we sampled a signal at 10 Hz and resample at 5Hz, the VI would keep only the samples that would exist if we samples at 5Hz originally. If the original frequency is not an even multiple of the resampling frequency the VI will interpolate the necessary points based on the "Interpolation Mode" option in the configuration dialog box.

There is an express VI which is capable of doing statistical analysis. It can be found on the "Signal Analysis" express VI menu and is called "Statistics" This VI will do analysis on the an entire set of data, and if you would like to look at only a subset of the data you will need to write a VI to break the data apart using, for example, the index array VI.

For your application I believe you will need to first break the data into smaller sections, and then determine the peak value for each section before writing it to disk.

Thank you, and have a great day.

Scott Yellig
Applications Engineer
National Instruments
0 Kudos
Message 6 of 6
(3,265 Views)