TDMS file with AutoSave on corrupts data.

[MiF]

Whenever I try to stream data to a TDMS file using AutoSave = true, data gets corrupted in "packet" #3 and onwards.

I am writing paired double/DateTime data to a TDMS file configured as TdmsWaveformLayout.PairedTimeAndSampleChannels.

When I try to read the TDMS file, e.g. using the Reader example app, I get all kinds of errors. Notably, the number of timestamps is not equal to the number of samples (y-values). In addition data is corrupt. That is, timestamps are not valid and the sample data is basically randomized.

This happens every time I write the third packet/batch and onwards. If I write one single sample, followed by another single sample, the third one will be corrupt. If I write 1 kilosample, followed by 1 kilosample, sample no# 2001 will be corrupt.

I assume something is very wrong in the AutoSave routine, as data is always written correctly when I explicitly save the TDMS file.

This is using MS 8.9.0.246 in Visual Studio 2008 SP1 using .NET framework 3.5 SP1 on Windows 7.

The TDMS file is created in the following manner. Note that AutoSave is set to false to avoid the bug.

TdmsFile f = new TdmsFile(tdmsFilename, new TdmsFileOptions(TdmsFileFormat.Version20, TdmsFileAccess.ReadWrite, true, TdmsByteOrder.Native));
f.AutoSave = false; // Must not be set to =TRUE due to a bug in Measurement Studio!

TdmsChannelGroupCollection channelGroups = f.GetChannelGroups();
TdmsChannelGroup channelGroup = new TdmsChannelGroup(... unique ID ...);
channelGroups.Add(channelGroup);

f.Save(); // Needed when AutoSave is false.

TdmsChannelCollection tdmsChannels = channelGroup.GetChannels();
foreach (IDicomInstance instance in series.Instances)
{
	tdmsChannels.Add(new TdmsChannel(string.Format("{0} (time)", instance.Signal.Name), TdmsDataType.DateTime));
	tdmsChannels.Add(new TdmsChannel(string.Format("{0}", instance.Signal.Name), TdmsDataType.Double));
}
channelGroup.WaveformLayout = TdmsWaveformLayout.PairedTimeAndSampleChannels;

Every 10 ms, a double[100] array arrives (with timing info in a DateTime[100] array). The data is stored to the TDMS file in a straightforward manner:

AnalogWaveform<double> wf = AnalogWaveform<double>.FromArray1D(sample.Data.Samples);
wf.Timing = WaveformTiming.CreateWithIrregularInterval(sample.Data.Timing);
wf.ChannelName = sample.Source.Name;
wf.UnitDescription = sample.Source.UnitOfMeasurement;

TdmsChannelGroup channelGroup = _tdmsFile.GetChannelGroup(... unique ID ...);
TdmsChannel channel = channelGroup.GetChannel(... channel name ...);
channelGroup.AppendAnalogWaveform<double>(channel, wf);
channelGroup.Parent.Save();  // Needed when AutoSave is false.

 This is what the TDMS file looks like in the Reader example application:

[nickb]

Hello MiF - 

Thanks for the very detailed description.  I have been able to reproduce the behavior you've described, and have filed a bug report with ID 224471.  I will post back to this thread as I have more information.

NickB

National Instruments 

[nickb]

Hello MiF - 

We are planning on creating a fix for this issue.  If it would be ok with you, I would like to get your contact information from the forum administrator to contact you offline about some of these details.  Could you please respond indicating whether or not it is ok for me to get your contact information?

thanks,

NickB

National Instruments 

[MiF]

No problem. Please do ...

Mikael

[nickb · Accepted solution]

For anyone else who may run into this before we are able to get a fix out, a viable workaround is to disable AutoSave and manually save the TDMS file any time a change to the file is made, such as after adding channels or groups, or writing data.  This will not adversely affect performance, as it is exactly what the AutoSave property does. 

It should also be noted this issue only arises when using the TdmsWaveformLayout.PairedTimeAndSampleChannels Waveform Layout.

I will post back later when we have a fix for this issue. 

Sorry for any inconvenience this may cause,

NickB

National Instruments