View multiple text files as single channel

[dave_helmut]

Hi,

I have a camera that FTP's its data to me.  It is measuring Height, Width, and Length (It's a 3D scanner).

Files arrive as <measurement><##>.txt where <measurement> is one of: a) length, b) width, c) height, or d) volume and ## is the file number starting from 1 and going to however many we collect.  every N samples are bundled into a file and ftp'd as a chunk.

Basically, I need Diadem to analyse each channel as data is coming in.  Problem is, each channel comes in as many files.

Can Diadem do this?

Thanks!

Dave Sell

[Brad_Turpin]

Hi Dave,

The short answer is Yes, DIAdem can do this.  But now we need to more closely define what you mean by this.  What sort of analysis do you want DIAdem to do with the data once it's concatenated into the 4 conglomerate channels?  What do you mean by "as it is coming in"?  Do you want DIAdem react to new data files automatically, or do you want DIAdem to simply find all the relevant files each time you ask it to load the data set?

I think the simplest way to pull together all these files into a single virtual data set file is to create a header file and a DataPlugin.  The DataPlugin will be able to find all the appropriate segmented files of the different quantities and expose those as 4 conglomerate channels as if they were in the header file, and you will only see the header file in the NAVIGATOR for simplicity.

Would you willing/able to send me a representative data set of all these data files?  I should be able to suggest a header file format and create a quick DataPlugin for you.

Brad Turpin
DIAdem Product Support Engineer
National Instruments

[dave_helmut]

---

Brad Turpin wrote:

Hi Dave,

The short answer is Yes, DIAdem can do this.  But now we need to more closely define what you mean by this.  What sort of analysis do you want DIAdem to do with the data once it's concatenated into the 4 conglomerate channels?  What do you mean by "as it is coming in"?  Do you want DIAdem react to new data files automatically, or do you want DIAdem to simply find all the relevant files each time you ask it to load the data set?

I think the simplest way to pull together all these files into a single virtual data set file is to create a header file and a DataPlugin.  The DataPlugin will be able to find all the appropriate segmented files of the different quantities and expose those as 4 conglomerate channels as if they were in the header file, and you will only see the header file in the NAVIGATOR for simplicity.

Would you willing/able to send me a representative data set of all these data files?  I should be able to suggest a header file format and create a quick DataPlugin for you.

Brad Turpin
DIAdem Product Support Engineer
National Instruments

---

Hi,

Sorry for the delay.  The camera programmer has changed what he is sending me now a little.

Now, 5 data will appear in a single file, every few minutes, of possibly 1000 data points, in the following format:

length1;width1;height1;volume1;timestamp1,length2;width2;height2;volume2;timestamp2,...length999;width999;height999;volume999;timestamp999

I guess this will appear as a single line (not delimited by CR/LF).

I don't have any sample files yet, but I will post something when I get it.

I am a VB programmer, so I could probably walk through some VBA code to do as you suggest, but I a so new to DIADem, not much of the examples I found on the forum make sense to me yet.

I would be interested in learning to "refresh" a channel.

Thanks!

Dave Sell, CLAD

[Brad_Turpin]

Hi Dave,

The first step is to build a DataPlugin to read the raw files you're getting ftp'd to you.  I will be happy to do this as soon as you send me several example files.  It is no problem for the file to have a semicolon instead of a CRLF.  If you would prefer, you can email them to brad.turpin@ni.com.

What file extension will these files have (*.txt, *.ftp, *.abc) ?

Brad Turpin

DIAdem Product Support Engineer
National Instruments

[Brad_Turpin]

Hi Dave,

Here's a prototype DataPlugin that works with a file I made up based on what you posted.  We'll need to tweak it to cover the differences between my made up file and the ones you're getting ftp'd to you.

Brad Turpin

DIAdem Product Support Engineer
National Instruments