Recovering TDMS files with DIAdem Connectivity Library

[chasadams]

I've found myself in the situation where my application dies and I have unsaved TDMS files. I need to recover the data in them.

The file logic is arcane. For one file I have three: *.TDMS, *.TDMS_Index and *.TDMS~. The last one is always the largest, so I suspect this is where my data is, along with some scaffolding perhaps. If I were to save and close my file, I would end up with one, well-formed TDMS file. Unfortunately, my customer is killing my application so I never have a chance to properly save and close the file. Yet, I need to recover the data. Is there some magic I can perform to get my data back?

[Josh_L1]

Hi chasadams,

How large is the *.TDMS~ file? Why is your customer killing the application before there is a chance to save the file? Your question makes it sound like this is a reoccurring issue, please confirm that. Have you thought about programmatically saving your data via VB script.

Regards,

Josh L.

[chasadams]

My use of TDMS is a small part of a large application, which is experiencing unrelated problems. At the time of the crash, the most important data is trapped in the .TDMS~ file.

This set of three files remain closed and intact. I need to reopen them, save the data and close them again, but there is some special relationship between .TDMS and .TDMS~. This seemingly cannot be recreated or reestablished with existing files.

Meanwhile, my data is lost -- maddeningly unrecoverable.

[Josh_L1]

Hi chasadams,

I did some research on your issue. Unfortunately, I did not find any direction answers. The file with the ~ symbol after it is probably a backup file that was automatically generated. You may need a third party file recovery software to open these files. I'm going to escalate this issue to our R&D engineers and get back to you.

On a side note, could you elaborate on what you mean by "special relationship between .TDMS and .TDMS~"? Could you also give me the size of the .TDMS~ file? Are the files grayed out? Or can you not open the file? Have you tried using another program to force open the files? If yes, what happens when you do?

Regards,

Josh L. 

[AndreasK]

Hi chasadams,

how big are the tdms~ files?

how do you save the tdms file. (DataFileSave?)

Is it really a crash or when the application is terminated or does it just take a while?

The tdms~ is just a temporary file containing channel binary data.

It is channel orientated but no information about the data itself.

The order of the data in the tdms~ is radom. So its almost impossible

to reconstruct the data from it.

Greetings

Andreas

[chasadams]

When I start a trace, I create and open the file, in this case 'Ruido', with DDC_CreateFile. After a few minutes, the trio of files look like this:

Ruido.tdms [0] July 18, 2011 8:00:25.000
Ruido.tdms_index [0] July 18, 2011 8:00:25.000
Ruido.tdms~ [192460] July 18, 2011 8:04:45.000

In this case, I'm writing 100 floating point numbers to 10 channels every 10 seconds. As far as I know, the file remains open while my trace is active and closes only when I tell it to close, DDC_CloseFile. What I hope is obvious is that the file size continues to increase as long as my trace is active. So, I'm not sure what difference the file size makes except that .TDMS~ is always non-empty and .TDMS and .TDMS_INDEX are always empty.

I am running under Windows 7. While my application has the file open, no other application can open it. When my application is rudely terminated -- SIGKILL, end process, taskkill, terminate with extreme prejudice -- I never have a chance to close the file properly. Although, I am releasing the file handles. Windows shows all three files remain in the file system and I am free to open all of them. I can strip off the ~ on the .TDMS~ filename and try to open it in the TDMS Viewerprovided by NI, but I get TdsErrNotTdsFile(-2503).

From these observations, I submit that .TDMS~ contains data and some scaffolding. The size relationship between .TDMS~ and .TDMS at the time of closure is about 5:4. Hence, .TDMS~ contains more than just data -- something gets stripped out when the data is committed to .TDMS. It might well be that the contents of .TDMS~ gets split between .TDMS and .TDMS_INDEX. Regardless, I seek some way of reinterpreting these files and rescuing my data.

And, Josh, thank you for your help.

Charles Adams

Lam Research

[chasadams]

Andreas, thanks for responding. Josh asked many of the same questions so see my answers in the previous post.

I'm disappointed that there may be no solution but I recognize that the DDC Connectivity Library is a free service and comes with some requirements.

[Josh_L1]

Hi chasadams,

One of our senior Applications Engineers suggested to move the .TDMS~ file to a different folder, rename the file, and try to open it. I've also found a KB based on your error code that includes a TDMS file repair toolkit. The link is listed below. The issue is not the same, but hopefully the repair toolkit can fix your file. Please try my first recommendation first and making a backup file before using the repair toolkit.

http://digital.ni.com/public.nsf/allkb/DD94C6858DE6FBA4862577FB0065B231?OpenDocument

Regards,

Josh L.

[chasadams]

Josh,

I've tested moving the file before opening but still no go.

Your file repair toolkit sounds promising, but I have no LabView runtime environment of any sort. So, I guess I'm out of luck.

Charles Adams

Lam Research

[YongqingYe]

Hi chasadams,

Are you using DIAdem? Do you have any other NI software installed, like LabVIEW? After you removing the .tdms~ file, can you load the .tdms file in DIAdem? If not, what error code you got? It sounds like it's possible that the last segment of your TDMS file is ruined when the application is killed, I have a recovery VI (just remove the last segment in a TDMS file), but it needs to be opened in NI LabVIEW, please let me know whether you have LV or not, maybe we can have a try.

Yongqing Ye

NI R&D