Getting correct Date/Time data in DIAdem 11 from LabView tdms data

[eberg]

I've got a handful of large tdms  files (9-18 million data points) collected with LabView that I need to extract elements of, and export as a text file, with the time data as a string.

I'm using DataFileLoadSelect and the Register option and then DataBlCopy to bring in a chunk of data at a time, but I'm having to explicitly convert each time value from a LabView timebase (1/1/1904) to the DIAdem timebase (1/1/0000), not to mention the offset,  before writing the date-time as a text string - this really slows the process down. 

I't strikes me that DIAdem(11.1) has got to have a way to read LabView timestamps correctly, but I'm a DIAdem novice, so any suggestions would be helpful. And no, I don't have LabView installed locally, so using a LabView vi is not an option for me.

Thanks,

Eric

[eberg]

Well, apparently DIAdem can't figure out the correct waveform start for tdms files generated by LabView, but I did find that the channel calculator speed things up rather than converting each row with a loop, still wish I could write the displayed time value directly, but for now I use the calculator to create the text-based time values:

'Correct the time column

 Call ChnCalculate("Ch(""[2]/Timestamp"")= Ch(""[2]/Timestamp"")+60084295200")

'Create a text-based time column

Call ChnCalculate("Ch(""[2]/DateTime"")= Str(Ch(""[2]/Timestamp""),""#MM/DD/YYYY hh:nn:ss.fff"")")

The correction factor is 1904 years plus the offset between Austin, Tx and Warzaw, Poland; in seconds.  Go figure!

Eric

[Brad_Turpin]

Hi Eric,

Did you mean 20 million points per channel or total in the file?  20 million points in a TDMS file does not constitute a large TDMS file for DIAdem.  Assuming that each point is an 8 byte DBL data type, that only adds up to 180 MB, and you really should have enough available RAM to load that data file outright without registering.

Instead of using both DataFileLoadSel() and DataBlCopy(), why don't you just use DataFileLoadRed(), which will enable you to restrict the channels as well as the rows (by index)?

Now regarding the date/time channels, if LabVIEW created the TDMS file and actually wrote a date/time wire to the TDMS Write.vi, then DIAdem WILL automatically read the date/time values correctly.  The TDMS VIs convert from LabVIEW date/time stamps to TDMS date/time channels while writing the data to disk, and both LabVIEW and DIAdem correctly read TDMS date/time channels.  Note that LabVIEW stores UTC date/time (including geographical shift from Greenwich, England) while TDMS and DIAdem do not, so your time zone information will disappear when you write the date/time from LabVIEW to TDMS.

So, what do the date/time values look like in DIAdem, that you conclude that they were read incorrectly?  Could you send me one of these TDMS files?  (brad.turpin@ni.com)

Brad Turpin
DIAdem Product Support Engineer
National Instruments

[Brad_Turpin · Accepted solution]

Hi Eric,

I saw your second post after I has posted my first answer.  Your second post indicated that the main issue was the start date/time value in DIAdem.  All DIAdem versions that support waveforms (10.0 - 11.1 currently) by default hide the absolute start date/time of each waveform in an invisible property.  It is loaded, but you'd never know it by scanning the Data Portal property table.  R&D's idea with this implementation was that most people will want to look at the relative time of the waveforms, and they further hid this "wf_start_time" property so that users would not be confused by the inconsistency between what is displayed graphically and what they see in the Data Portal properties.  They certainly achieved their goal, but only for the customers they had in mind.  For your use case this only exacerbates an already frustrating problem.

But there is a way to read out this invisible absolute start date/time property and create a date/time channel like you need for ASCII output-- I'm attaching two examples of this below.  If you need to do further geography-based shifting, then I'd recommend the ChnLinScale() function instead of the Channel Calculator-- it should be even faster.

Hope this helps,

Brad Turpin
DIAdem Product Support Engineer
National Instruments 

[eberg]

Brad,

Thanks.  The hidden wfm data did the trick.  And unlike with my brute force conversion, the regional offset is now 6 hours which is more in line with my expectations.

Eric

[eberg]

WfmToDateWfm works great, but there is a bug in the code if you want to use DataFileLoadRed and your starting point is not at the beginning of the file.  There is a provision in WfmToDateWfm to handle the offset, but it's not hooked up:

OffStartTime = ChnPropValGet(Channel, "wf_start_offset")

ChnDeltaTime = ChnPropValGet(Channel, "wf_increment")

RelStartTime = 24*3600*(Trunc(ChnStartTime) + 693958 + Abs(Frac(ChnStartTime)))

To make WfmToDateWfm work with an offset, simly add the variable OffStartTime to the RHS of RelStartTime

Eric 

[Brad_Turpin]

Hi everyone,

I'm re-posting the examples which I've now updated with Eric's bug fix.

Brad Turpin

DIAdem Product Support Engineer
National Instruments