Data Logging challenge

[maxidivine]

I have designed the VI attached to achieve a certain task. I have been on it since yesterday but decided to take another approach.

I want to save data to the log.xls file every five seconds (or as the timer would specify), continuously. When the inner loop counter gets to 2 the inner loop ends. That means there will always be three records in the log file at any time.

On clicking the button, a 'snap shot' of the log.xls file should be taken, meaning all the contents of the log.xls file should be stored in the snapshot.xls file. This should happen once only and the inner loop should continue running afterwards (i.e continue saving data to the log.xls file). It must be ensured that the snapshot.xls file is not overridded with new data from the log.xls file.

In effect the snap shot file would contain everything the log.xls file contains at the point the inner loop ends after the button is pressed.

I ran the code several times but there was nothing in the snapshot.xls file.

Where do I go from here please?

[RavensFan]

Numerous problems.

I just fixed your file so it would work and scattered comments throughout it.

[maxidivine]

Ravens Fan,

That was wonderful. Its amazing how the errors were spotted and corrected. Thanks a lot. I did a little modification to your software and the reason is this: The big picture for this software is that in reality it would hold data consisting of about 30 colums and 17280 rows (which corresponds to the number of 5 seconds in 24 hours).

So in effect we have roughly 17280 rows of data collected in a day (once every 5 seconds). I put space consideration thats why I had to replace the data continually because there is a limit to how much data excel can take and also because the most important data would be the data that has been collected when the button is pressed and the snapshot taken.

What do you think of my adjustment to your code in view of the requirements stated?

Thanks again for your wonderful piece of code.

[maxidivine]

Thanks for all the assiatance. Some ammendements to the VI requirements are: data is to be logged every 50ms, data is logged on a 24 hour basis, with the 'cyclic' log file being overriden every 24 hours to avoid the file increasing out of proportion over time.

At the iinstant the trip button is clicked, the time instant should be noted and registered, data 10 seconds before pressing the button and 10 seconds after should be stored in a log file as a snap shot.

The challenge I have is with the attached VI, the snapshot is taken alright but the file log starts from the instant the button is pressed.

Is there a way to retrieve data at least ten seconds before as well, because if the button is pressed at 23:59 for instance, it would not be possible to get data 10 seconds later and if the button is pressed 00:01 for instance it would not be possible to get data 10 seconds earlier.

[RavensFan]

If you are looking at 50 msec timing interval, I would recommend changing the architecture of the code.  Is the real data coming from data acquistion hardware like an NI card or module that uses DAQmx?  Somewhere else?

At that kind of rate, I would set up DAQmx to acquire at a 20Hz rate and acquire 20 samples.  So 1 seconds worth of data.  Do that in its own loop.  Pass the data to another parallel while loop using a queue in the producer/consumer architecture.

Set up the other loop to log data to a file.  You could even accumulate the data and store 10 seconds or more at a time to minimize file writes.  Let that loop do the file writing and management, so that after 24 hours it will start a new file or overwrite the existing file.

For the +/-10 second snapshot of writing to a file, I think I would do that in yet another parallel loop.  Send the same data to that loop using a second queue.  That loop could use some implementation of a circular buffer so that it always maintains at least the last 10 seconds of data.  If it gets the trigger signal, then it will write the previous 10 seconds to a file and also sets a flag that will tell itself to write the next 10 seconds to the file as well.

With a 2 file writing loops, each one can hold on to its data and handle it in its own way.  The second one won't care that the first file just got overwritten.  It keeps its own history of past data.

[Ben]

---

Ravens Fan wrote:

If you are looking at 50 msec timing interval, ...

At that kind of rate, I would set up DAQmx to acquire at a 20Hz rate and acquire 20 samples.  So 1 seconds worth of data.  Do that in its own loop.  Pass the data to another parallel while loop using a queue in the producer/consumer architecture.

Set up the other loop to log data to a file.  You could even accumulate the data and store 10 seconds or more at a time to minimize file writes.  Let that loop do the file writing and management, so that after 24 hours it will start a new file or overwrite the existing file.

For the +/-10 second snapshot of writing to a file, I think I would do that in yet another parallel loop.  Send the same data to that loop using a second queue.  That loop could use some implementation of a circular buffer so that it always maintains at least the last 10 seconds of data.  If it gets the trigger signal, then it will write the previous 10 seconds to a file and also sets a flag that will tell itself to write the next 10 seconds to the file as well.

With a 2 file writing loops, each one can hold on to its data and handle it in its own way.  The second one won't care that the first file just got overwritten.  It keeps its own history of past data.

---

Simple and elegant!

 

Ben

[maxidivine]

Thanks a lot Ravens,

The data is not from any DAQ. The data is purely simulated data. Sorry, how are parallel loops implemented? Do I write two different loops side by side with each doing its separate task?

Thanks a lot.

[RavensFan]

Yes.

Go to File / New.... and look for producer/consumer architecture (Data) for a reference.  Also search the forums for examples of Producer/Consumer Architectures.

[maxidivine]

I have been battling with the producer/consumer architecture as advised. I came up with this VI and I am stuck because the second loop does not run. Please SOS, it's giving me sleepless nights.

[RavensFan]

Here is closer to what you want.  The idea of the queue is to pass the data.  No reason to reread the file when the data is passed by way of the queue.

This doesn't handle the +/- 10 seconds of data like you said you wanted, but at least it is using the queue correctly.