Corrupted TDMS File

[JW-JnJ]

Ok, this file transfer just isn't happening.

I zipped the file into 3 seperate 650MB files, which finally got me past all the problems on our side. Now the FTP is timing out every 14 minutes.

[deppSu]

As you already know, a TDMS file is composed of segments, if the "corrupted" segment is located, then you can cut off the file right before that segment and the TDMS file with the segments before should be opened successfully.

Since you are trying splitting your TDMS file, I suggest (maybe you are already doing in that way) you can use methods similar to bisection search to split the file.

[JW-JnJ]

---

@deppSu wrote:

As you already know, a TDMS file is composed of segments, if the "corrupted" segment is located, then you can cut off the file right before that segment and the TDMS file with the segments before should be opened successfully.

Since you are trying splitting your TDMS file, I suggest (maybe you are already doing in that way) you can use methods similar to bisection search to split the file.

---

I attached my splitting VI (pardon the hasty/messy code). I grabbed the first segment and stored it. Then each X segments, write the first segment then the X segments. This puts all of the same metadata up front. Is this proper, or should I just put the first metadata segment on the first file?

Anyway, they all opened in the TDMS viewer but not diadem.

Now I'm really confused.

[YongqingYe]

Hi JW-L3CE,

I'm surprised and appreciate that you have analyzed the TDMS files by yourself, but I would reccommend you sending the TDMS file to us (NI R&D) for investigation if the data is not very confidential. If the FTP cannot work for you, we can think about other ways, maybe some network space can also work for us to transfer files.

[JW-JnJ]

Sure, I would like to get them to you. We just may hit some more obstacles with my IT policies.

Let me know.

[deppSu]

Use an exmaple to illustrate my idea,

If we have a "corrupted" TDMS file with 100 segments, we assume the offsets of the 100 segments are correct. We can locate the "corrupted" segment by the following steps:

Set variable "left" to 0 and "right" to 100

1) Truncate the TDMS file at the "middle", middle=(left+right)/2

2) Try use "TDMS Open" to open the truncated TDMS file. 

3) If open succeeds, set left=middle+1; If any error thrown out, set right= middle-1

4) Repeat steps 1 to 3,  until left>= right, where left should be the corrupted segment!

If we found left is 70 for the example given with the above procedure. Then that means the TDMS file with the first 69 segments is sane and you can truncate at the 70 segment to keep the sane segments. You can also post the corrupted 70th segment. Then, NI developer can check it byte by byte and try to see if it possible to make it work.

I can make out such a VI but I'm afraid it might not can be directly adapted in your case and you don't know how to modify it. What do you think of this method?

[deppSu]

I have written an program in LV2011 which try to locate the corrupted segment. You can have a try.

[JW-JnJ]

---

@deppSu wrote:

I have written an program in LV2011 which try to locate the corrupted segment. You can have a try.

---

Great idea. I altered the VI so that it quits on error and doesn't delete the temporary TDMS file. It got to segment 663441 (of 884587). Attached is the last segment of that temporary file (corrupted) and the first if you need it.

Thanks for the help.

(by the way, not letting people attach tdms files in the forum is a little odd)

[JW-JnJ]

Oh, and for future prosperity, here are the altered VIs.

For large files they take a LOOOOONG time to run. However, it will leave a temporary tdms file where the last segment is the one that corrupted the tdms file.

[YongqingYe]

Hi JW-L3CE,

deppSu will return to office in a couple of days, before that, could you please let us know which version of TDMS component you are using?

Please go to c:\program files\national instruments\shared\TDMS and find out tdms.dll, right click the file and go to the tab "details", for the "product version" field, if the versoin is earlier than 2.4.0f0, I would high recommend you go the below page to download and install the latest TDMS component to avoid any more corruption in the future.

http://zone.ni.com/devzone/cda/epd/p/id/3008