Extracting data from logs that do not complete?

[JasonKoravos]

I'm using SE 2.0.0, I have several log files that were interrupted by errors before they completed.  The logs folders appear in the "Signal Express Data" folder and have the TDMS and TDMS_Index files in them, but the incomplete log folders are missing the "meta" .txt file.  Any logs that completed normally have the "XXXX_meta.txt" file in them.

If I try to import the incomplete log from the TDMS files, it shows up in the project as "New Group" and cannot be accessed.

Is there a way to gain access to the data in these logs? 

[Wallace_F]

Hello Jason,

Can you explain the procedure that you are using when you try to import the incomplete logs?  Are you going to File -> Open Project, and then browsing to the log file, or are you using File -> Import -> Logged Signals from SignalExpress TDMS files...?  If you are using the former, I recommend that you try the latter method.

Regards,

Wallace F.

[JasonKoravos]

Wallace,

I have project open and I am trying to import logs into the project using the latter method (File -> Import -> Logged Signals from SignalExpress TDMS files).

-Jason 

[Wallace_F]

Hello Jason,

I have been trying, but have not succeeded in duplicating the conditions you are experiencing.  Have you tried copying the XXX_meta.txt file from one of the completed logs and pasting a copy to the incomplete log folder? Try this, and if it doesn't work, you will likely have to address the cause of the errors that are interrupting the incomplete log files. What are the errors, and when do they occur?

Best wishes,

Wallace F

[JasonKoravos]

I'm not sure that the existence of the meta file is the culprit.  I've tried moving existing meta files to the "damaged" data log folders, as well as manually altering the meta files to match the parameters of the "damaged" data log.  It has no effect.

Something I have noticed in the TDMS files is that there is a footer at the bottom of working log files that doesn't exist in the "damaged" log files.  The footer looks like this (in Notepad):

TDSm   h             
  
   /ÿÿÿÿ#  
   format string           iteration based timing!       timing mode       HWTimed_Continuous   unit string   
   V       unit GUID    &   {91FEC1EF-A358-44c4-BC5E-C13C17674E12}
   X unit string   
   s   X unit GUID    &   {0748B676-82D4-4318-A843-708972E53760}   unit version          title       09/02/2008 05:36:29 PM   author       Bil-LabBldg9   description       tld301 10micron cont'd   log duration
       @î@   number of channel in log         log channel init time0D   @ì#'›Ó[‚bãÄ       log channel name0    )   09/02/2008 05:36:29 PM - Voltage - Vacuum
   maxYvalue0
   O:¨üG@       maxYtime0
   szK6ŠœèA
   minYvalue0
   ¬o‚z@       minYtime0
   sz˶}œèA   log channel init time1D   @ì#'›Ó[‚bãÄ       log channel name1    0   09/02/2008 05:36:29 PM - Voltage - Boil-off Flow
   maxYvalue1
   ‹€.
Äô?       maxYtime1
   szk¤pœèA
   minYvalue1
   רÄ
©‚¿       minYtime1
   szkLƒœèA   log channel init time2D   @ì#'›Ó[‚bãÄ       log channel name2    -   09/02/2008 05:36:29 PM - Voltage - Boil-off P
   maxYvalue2
   ÿ½á,Oò?       maxYtime2
   szKclœèA
   minYvalue2
   aÚ¢’[Oï?       minYtime2
   szk“‡œèA   log version       SignalExpress 2.5   log datatype   
   Time-WDT-Real   log dt
         ð?   log iteration length
         ð?

I'm not sure if this is relevant or not.  I tried to copy this footer to one of the "damaged" log files but it didn't help.

Several of the errors that abrupty stopped the logging were "Not Enough Memory To Complete This Operation", which I think had to do with updating views while running.  I haven't recieved that error since I changed the setting to "Update None" while running the logs.  I've also had a few unexplained Windows errors, such as "error trying to access location (some hex number)", or something along those lines.  I don't remember the exact error message.  Most of these errors completely froze the workstation and I had to do a hard reboot.

[Wallace_F]

Hi Jason,

What data acquisition hardware are you using in your application?  Also, what sampling rate, and how many channels are you sampling with? I'm still investigating whether it is actually possible to import "broken" log files in SignalExpress.

Regards,

Wallace F.

[DFGray]

The TDMS File Format Internal Structure may help you out a bit.  Be aware that the log files have a specific format using this structure.  It is not documented well, so you are probably better off using the French Cafe method of determining what it is (look at good files).  You seem to already be doing this.

Note that TDMS typically buffers data and writes it in chunks to avoid low write speeds.  In the event of a power failure or other catastrophic failure, that section of the file will not be written.  This appears to be what occurred.