LabVIEW
TDMS file in Circular Buffer
One reason I do like my TDMS file concatenating method (even if it is clunky and a pain at times), is that with it if there is an abrupt stopping of the application, software crash, or power outage, the last N files will still be on disk in the temporary folder. Putting all your data in a queue that flushes at end is much cleaner but doesn't handle this. Also at the start of the project I didn't know how much data I would need to collect or for how long, and in practice can be quite a bit. So using the disk for caching made sense in my situation.
@AAI88 wrote:
Hi Mcduff,
Thanks for the suggestion. I had read about your idea before, but until yesterday I didn't think to implement it as I think the method I am trying might be more appropriate. I did, however, try your method, but it seems that with a lossy queue you can miss out on some of the information you might want to archive. I have not implemented it to the full extent, but I am attaching it here for your viewing in case you are interested in taking a look and giving me an idea of how to fix the bug of it missing out on some of the data.
Some Suggestions:
Change the boolean mechanical action on the front to "Latch When Released"
You need to fix your TDMS save. You have a 5 element queue, each queue element has a 6 element array. I changed the DDT wire to a waveform wire, see attached. Since you are auto-indexing the two inner for loops in your file saving section, you need to make sure all of the number of elements match otherwise your for loop will not run correctly. To me it appears you have too many for loops, so I removed 1. See Attached.
This is what I get when I make these mods to your VI.
mcduff
PS The VI can be cleaned up significantly and improved, I assume this is just a test case you sent.
@AAI88 wrote:
Hi Hooovahh,
I have seen this idea mentioned before and I am planning on taking a crack at your idea here pretty soon if I don't have a viable solution to the archive file problem on my current code. If you have the time to or feel comfortable sharing an example code of your idea I would really appreciate it. Thanks again for the share and for your advice.
Okay attached is a very rough first draft of what I am describing (saved in 2018). Unzip the code and run Demo TDMS Wrapper. I believe this will need the OpenG File I/O package. Set the path to the file you want to create and hit run. By default when you hit stop (the button not the abort button in the VI), it will create the TDMS file you selected and it will contain between 1 minute, and 1 minute 6 seconds of data. It does this by first creating a TDMS file in your temp directory making a new file every at buffer size divided by 10 (which in this demo is 6 seconds).
Also notice that the write function is a VIM which accepts different data types as the input. It should work just like the normal TDMS write and you can write any data type it accepts.
I think this could be a much larger conversation about TDMS wrapping functionality and this class can be expanded to handle things like making a new file after so much time, or making a new file after the size reaches a certain size, or handling buffering data before invoking writes, or allowing to read and write clusters. I just don't have the time at the moment but hope I get time to revisit this.
After a few months I decided to look into doing this better and in doing so realized I wanted to create a TDMS toolkit with a few other feature. I just release version 1.0 of my Tremendous TDMS Toolkit over on LAVA. It has lots of stuff but one of them is an improved Circular TDMS Example which works by using the In Memory TDMS tools instead of writing to disk and then deleting old ones. I believe this to be a better design and should have better performance.
I also added a few other features like the option to zip the TDMS before writing it to disk, which uses the built in LabVIEW zip utilities. I'm unsure how this effects performance but suspect it is worth it to not have to write as large amounts of data at once. There is also an option for writing data to the final file, no matter how long the write session has been open.
