How to store data conveniently in labview

[REDJ]

When testing hardware devices or performing real-time control, it is usually necessary to store the data of interest for offline analysis. But when I use TDMS and text to store data in labview, it doesn't feel very convenient. So I would like to ask what data storage method is convenient in Labview

storage requirements：

1. Not only include the original data of the AD card, but also store any data of interest in the VI.

2.Data storage operations do not affect the timing accuracy of the loop. For example, the execution time of each loop needs to be fixed at 0.5ms, and about 20 double-precision floating-point data needs to be stored.

3.Storage of data in different loops, and these loops have different timing periods.

4.Data need to record time, variable name.

I have used dSPACE's controldesk software, the data storage process is simple and the data logging is rich. How to implement data storage with the above requirements in Labview.

[altenbach]

LabVIEW is a very rich programming language and you can do whatever you want with just a little bit of programming. 😄

Can you explain what you mean by "doesn't feel very convenient"? It is difficult to read "feelings" out of a posts. 😄

• Convenient to program?
• Convenient to use?
• Convenient for post processing?
• ...

Is this running on a real-time system or on a PC? How do you ensure the 0.5ms loop timings? (impossible on windows!)

What is the "original data"? What is the "data of interest"? Does the software for offline analysis require a specific format or is it your own program? First you say that all loops are at 0.5ms, but later you say they have different timing periods.

Maybe it would help to take a step back and explain the experiment. Does each loop produce it's own file? Does one file collect all data from all loops (interlaced? different columns? something else?). Doe the data need to be streamed to disk or is it sufficient to accumulate in memory and write in chunks?

So. Many. Questions.

[RTSLVU]

---

@REDJ wrote:

When testing hardware devices or performing real-time control, it is usually necessary to store the data of interest for offline analysis. But when I use TDMS and text to store data in labview, it doesn't feel very convenient. So I would like to ask what data storage method is convenient in Labview

storage requirements：

1. Not only include the original data of the AD card, but also store any data of interest in the VI.

2.Data storage operations do not affect the timing accuracy of the loop. For example, the execution time of each loop needs to be fixed at 0.5ms, and about 20 double-precision floating-point data needs to be stored.

3.Storage of data in different loops, and these loops have different timing periods.

4.Data need to record time, variable name.

I have used dSPACE's controldesk software, the data storage process is simple and the data logging is rich. How to implement data storage with the above requirements in Labview.

---

Well first off doing this for a long time and I have found no need for anything more complex than a plain text data file. Even for long term tests that were logging >60 data points at 1 second intervals and ran for years!... I can open a plan text file in any program for analysis but we mainly use Excel.

1. You can store any data type you want in a text file
   1. Worst case you have convert it to a string first.
2. That's what a Producer/Consumer architecture is for.
3. With a Prod/Con the timing of the loops is independent of each other.
   
   1. For instance you can be monitoring at a 10mS rate in the Producer loop and logging at a 1 minute rate in the Consumer loop.
4. Time stamping data is simple, just save the time stamp. Labeling column headers is simple too.
   1. Open a file
   2. Write your column headers to the file
   3. Acquire data
   4. Write a line of data to the file
   5. Goto 3 until done
   6. Close your file

[REDJ]

Thank you for your reply, sorry for not describing my problem clearly.

“doesn't feel very convenient” mainly mean “Convenient to program”.

I wonder if it is possible to achieve something like in matlab:
1. Add a data management module to the labview project;

2. Through a special oscilloscope module, the data of the oscilloscope is automatically stored in the data management module when vi is running;

3. The data in the data management module can be exported.

This is the way I expect data to be stored。

-----------------------------------------------------------------------------------------

about “0.5ms loop timings”

Yes you are right, loop timing less than 1ms cannot be achieved under windows. 

The doubt I want to express is: Is it possible that the time required for the data store operation will exceed 0.5ms, so that the timing of the loop will exceed 0.5ms?

-------------------------------------------------------------------------------------------

"original data"  and  "data of interest"

"original data" is the original value of the sensor read by the acquisition module or serial port module.

"data of interest" is the data generated in the calculation process of labview, such as the control voltage of the actuator, etc.

The stored data can be imported into data analysis software, such as matlab.

-------------------------------------------------------------------------------------------

different timing periods.

What I want to express is that the data that needs to be stored comes from different loops, and the timing periods of the loops are different.

[REDJ]

Thank you very much for your reply, I will try to log the data using a text file. Have you ever tried to record 500 columns x 10 rows of data in 1s time? Will he reduce the loop execution time?

Problem with using producer-consumer loops is that producer loops are multiple loops with different timing periods.

[crossrulz]

---

@REDJ wrote:

Problem with using producer-consumer loops is that producer loops are multiple loops with different timing periods.

---

What really matters is the sampling rate.  But if those are different, then TDMS is really what you should be using, having a different Group for each producer loop.  You can still use a single consumer loop with all of your other loops using the same queue to send the data to the logging loop.  Part of the data in the queue element should be the group and/or channels the data belongs to (use a cluster for the queue data type).

[johntrich1971]

---

@REDJ wrote:

Problem with using producer-consumer loops is that producer loops are multiple loops with different timing periods.

---

That's why a producer/consumer approach would be ideal. The consumer loop is your data storage module. You send data to it via messages. My message format is usually a cluster containing a message name (string or enum) and a variant for data. The consumer would receive the message and, based on the message name retrieve the data and handle it appropriately. 

[RTSLVU]

---

@REDJ wrote:

Problem with using producer-consumer loops is that producer loops are multiple loops with different timing periods.

---

That's not a bug, it's a feature...

The Producer loop will acquire data at a high rate and put it in a Queue. (or Messenger Channel)

The Consumer loop can then take it's own sweet time dequeuing, analyzing, displaying, and storing the acquired data without effecting the Producer loop timing.

[billko]

---

@RTSLVU wrote:

---

@REDJ wrote:

Problem with using producer-consumer loops is that producer loops are multiple loops with different timing periods.

---

That's not a bug, it's a feature...

The Producer loop will acquire data at a high rate and put it in a Queue. (or Messenger Channel)

The Consumer loop can then take it's own sweet time dequeuing, analyzing, displaying, and storing the acquired data without effecting the Producer loop timing.

---

Well, if the consumer is too slow, you may run out of memory.  😄

[johntrich1971]

---

@GertrudeGrove wrote:

Having Same issue. I need an example or some help making a vi for storing data. I wish to have several drop down boxes with selections of data pulled from a file and a blank text box that manual text can be entered and on a click of a button I would like the information saved into some sort of file somewhere, this data needs to be added to when new data is added at future dates. I would also require some information on how to have another vi for viewing the information thats in the spreadsheet from labview itself. Is this possible? I may be asking alot here, but I am learning and I wish to make a labview project based on the current database I am collecting information in. thank you,

TellTims Survey

---

First of all I would recommend starting another thread instead of hijacking someone else's thread.

Next, have you reviewed any of the LabVIEW training material? Also, have you searched through the examples that are supplied with LabVIEW? There are a number of examples that include writing to files. 

Finally, the description of what you are trying to do is extremely shallow. If I understand correctly you want to populate ring controls with text from files, have a text entry box, and then write data to a separate file upon a push of a button. Are you writing data from all controls or just individual controls? What is the format of the files that you're reading from? You mentioned a database - what type of database are you using?