Format difficulty with text read from file

[PolymerTim3]

I am trying to process some data that is output to a text file by another application I use.  The text file seems like a perfectly normal, tab delimited file when I open it in either notepad or MS Excel, but the formatting is strange when I import it into LabVIEW.  I also noticed in another program I have that is supposed to be able to import text data, it cannot recognize the formatting in these files.  If I simply open them with Excel and copy/paste the columns into a new notepad file, then LabView can import it just fine, so Im' pretty sure this is just something strange about the formatting of the original text file.  Basically, every line is double spaced and there is an extra space between every character.

I originally started with the "read from spreadsheet file" but have even tried the more basic "read from text file" and the strange formatting is already there.  Is there a simple way to import this file correctly?  I am including a sample data file as an attachment.  I deleted most of the data because the original is over 1 MB, but the formatting is the same.

Thanks,

-Tim

[RavensFan]

Are you sure those "extra spaces" aren't already in your data file?  What created this text file?

Viewing it has hex code format, the extra spaces show up as 00.  A non-printable character in places like notepad often does not show up, (but perhaps the Labview string indicator allocates a space for it.)

There were also some non-printable characters at the beginning (FF FE).  I calculated the string length and it came out to be 5822 characters, which is exactly the size of the data file in bytes.  So that indicates to me that the "extra" stuff is in the file.  I think the real issue is in whatever is creating the file is adding the unnecessary characters.

Message Edited by Ravens Fan on 10-16-2007 07:32 PM

[Matthew_Kelton]

Your text file is in Unicode.  LabVIEW doesn't handle Unicode directly.

It also appears it may have been created on a Unix or MAC machine.

Use the attached llb file to decode properly (use Convert UTF16 String to ASCII).  I didn't write it, but it works quite well.  This is assuming you are using a Windows PC, as it makes WinAPI calls.  If you're using Mac or Linux, try the VI in this message.

To use the attached llb, write a VI as follows:

Message Edited by Matthew Kelton on 10-16-2007 08:30 PM

[PolymerTim3]

That works perfectly!  Thanks for all your help.  I figured it was something to do with the format.  I had forgotten all about different encodings (I'm only vaguely familiar with them).  Once you specified unicode though, it sounded familiar.  The software I use is called Universal Analysis.  It runs on a Windows machine, as does my LabVIEW.  Once I started digging through the export options, I noticed a box for encoding and Unicode is the default, but there are also options for Windows (ANSI) and OEM (PC-8).  Now I know both how to export in ANSI and to read in the Unicode if I need to.

Since I may share this program with others in my group, I'm wondering if there is a simple way to programmatically identify what the encoding is so the program could intelligently load either type of text file.  At the very least, I can probably have it check for that first non-printing character, but that's probably not a very robust solution and probably specific to this programs output.

Thanks again!

-Tim

[smercurio_fc]

The first few characters do indicate the type of Unicode: http://msdn2.microsoft.com/en-us/library/ms776429.aspx.

[PolymerTim3]

Thanks very much for that info.  That makes things much easier.  I have now set my program to read the first byte in binary form and check to see if value equals FEFF.  Then  I just use a case structure to handle each situation.  Thanks again for all the help guys!

-Tim