02-10-2017 01:07 PM
What is the best practice for writing to TDMS file. I want to sample data at high frequency and continously write to TDMS.
I found this. however it only saves to TDMS when stopped. Is this best pratice? How do I get it to continously save/append?
02-10-2017 01:13 PM - edited 02-10-2017 01:16 PM
The example you attached uses the (relatively) new DAQmx Logging feature and does write the TDMS file as the data is read from the Task. That would be the way to go!
The Example code link (using a PC Loop) was authored before the DAQmx logging feature was available. It would be a suitable method to do continuous logging for anything other than a DAQmx device (or if you are stuck using an older version of DAQmx)
02-10-2017 01:31 PM
It does record data from the task but I don't see the file updating. How well does this method perform for long duration tests?
02-10-2017 02:17 PM
TDMS streams to memory and then to file so you may not see the file size updating unless you repeatedly hit "f5" while file is being writtein.
If you are concerned about data being lost due to a power outage etc, you can use a property node...
DAQmx Read >>> Logging.FilePath
and periodically change the file name to close the previoust and start a new TDMS file
OR
DAQmx Read >>> Logging. SampsPerFile
And let the new file creation happen when the file number of samples you specify have been written to disk.
Do help on those properties to get the details.
Ben
02-10-2017 04:20 PM - edited 02-10-2017 04:20 PM
Yeah TDMS has an API that can buffer data for efficiency. When using the DAQmx streaming, you won't see the file have any real size until it is flushed to disk, which will happen when the file is closed, or when the API feels like it. That last one has several variables associated with it but NI wants you to not worry about it. Logging to a TDMS file will not eat up all of your RAM until it crashes, it will flush to disk if you have really long running tasks. But if you are conserned you can force a new file as Ben mentioned.
Unofficial Forum Rules and Guidelines
Get going with G! - LabVIEW Wiki.
17 Part Blog on Automotive CAN bus. - Hooovahh - LabVIEW Overlord
09-06-2017 04:07 PM
Came across this post looking for best practices for TDMS files. This is not really that discussion but was wondering if there is such a page or doc online?
09-06-2017 04:43 PM
I presented on TDMS at NI Week 2016, here is the content, sorry no video. I think what you will find is that TDMS can be a large topic. There's plenty of things to avoid (like unnecessary writes), but doing these things won't generally break things, it just might result in a poor experience, or you might not even notice. The design and structure is up to you and that might be one place where people make mistakes since they don't really know what works and what doesn't when they first try it out. If you have N devices should you have N TDMS files? or N groups in a single TDMS file? Or just all the data in one group? What properties should I write to the channel? Group? File? These all depend on the application and I can make suggestions based on a design but until you use it for a little while generating data, and seeing how the other users consume that data, you won't know if the design was a good one. Sorry for such a long answer when I could have just said "It's up to you".
Unofficial Forum Rules and Guidelines
Get going with G! - LabVIEW Wiki.
17 Part Blog on Automotive CAN bus. - Hooovahh - LabVIEW Overlord
10-10-2017 08:53 AM
The DAQmx TDMS logging feature is excellent but what if I have 2 separate sync tasks?What would be best practice then? Also I will have large TDMS files which will need to be split into smaller managable sizes. In ni examples synchronization vi, I have used this template to syn two different chassis, so I guess question would what would be best pratice in this case, I am asuming i cant use 1 DAQmx TDMS logging vi.
10-10-2017 09:28 AM
@isitthereur wrote:
The DAQmx TDMS logging feature is excellent but what if I have 2 separate sync tasks?
I'd suggest logging to two separate files, and then merge the two TDMS files at the end. You could also not use the built in logging and read the DAQ then write them separately but I imagine it would be better to merge them. Merging is a pretty fast process and is mentioned in the linked presentation.
Splitting isn't as straight forward. You can split a TDMS file at every segment but if you defrag your file there will only be one segment. If your files are getting to large, just close the file and start a new one mid test. Also if you defrag at the end of the test your file size is generally going to be reduced. There are other methods of splitting files that are less straight forward, and require more resources but can be split in very custom ways like ever N samples.
Unofficial Forum Rules and Guidelines
Get going with G! - LabVIEW Wiki.
17 Part Blog on Automotive CAN bus. - Hooovahh - LabVIEW Overlord
10-10-2017 10:17 AM
@isitthereur wrote:
Also I will have large TDMS files which will need to be split into smaller managable sizes.
The DAQmx TDMS Streaming allows for this. Just use a DAQmx Read property node and write to the Logging->Samples Per File property. And new TDMS file will be created ever X samples automagically for you.