Plugin Script for DAT File with Multiple Channel Rows

[bbueter343]

So, I'll attempt to explain this the best that I can.  I have a *.dat file that we have written a data plugin to handle importing our particular data file type and identify header properties, channel descriptions, channel units, channel names, and then associated data.  However, certain files that I would like to read in will have a block of data with a set of channel names/units and then if our data user added a channel later we get another block of data with a new set of channel names/units.  I've attached a sample of the data which I've removed most of the columns and actual data but still provided the format that it's in.  I've changed the data file format from *.dat to *.txt to upload.

What I want to do is have DIAdem create a single channel name/unit/description row that concatenates all of the channels from multiple rows and delete the rows with extra channel names in them prior to loading in the data portal.

I currently have an Excel Macro that will match the channel names and create this format for me, but I would really like to get away from using excel as an interface for my team. If DIAdem has to call Excel in the background to export/import data, I suppose that would be okay but would prefer not if at all possible.

Any thoughts on how to do this effectively?

Thanks in advance!

Bueter

[ML927]

Hi Bueter,

Have you tried creating a DataPlugin with the DataPlugin Wizard for Text Files ?

When doing so, you can tell DIAdem to ignore certain lines of your input file which, as far as I understand your problem, is what you want to do.

More info here : https://www.ni.com/en/support/downloads/dataplugins/using-the-text-and-excel-dataplugin-wizards.html

[AndreasK]

I am not sure if I got you right but I will try to draw a picture of a solution.

I am assuming your files are of medium size because you are able to read them with Excel.

So I am just parsing the file inside of the vbs code instead of using the DataPlugin string blocks which would potentially give a better performance but a more complex code.

I also assume that you are using vb code in Excel too to manipulate the content, which means the attached code will not scare you.

It is meant as a statng point for you.

Your format looks like this. So we can parse it using vbTab and vbCrLf

The attached DataPlugin code will result in DIAdem Content which I hope is somehow what you are expecting

Find zip attached including the dataplugin itself.

Hope this helps starting with your DataPlugin.

Dataplugin Code:

Option Explicit

Sub ReadStore(File)
  File.Formatter.Delimiters = vbTab
  File.Formatter.LineFeeds = vbCrLF

  dim dummy : dummy = file.GetNextStringValue(eString)
  if(1 <> instr(dummy, "vsSysTestStartDate=")) then call RaiseError() ' Not my file
  
  Dim grp : Set grp = Root.ChannelGroups.Add(file.Info.FileName)

  dim chOArray(1001) ' array to know the order of the channels
  
  ' add fix blocks with dynamic dta types
  set chOArray(1) = grp.Channels.Add("Date",eTime)
  set chOArray(2) = grp.Channels.Add("Time",eTime)
  set chOArray(3) = grp.Channels.Add("numTest1Run",eR64)
  set chOArray(4) = grp.Channels.Add("vsSysOperatorPreLogComment",eString)
  set chOArray(5) = grp.Channels.Add("vsSysOperatorPostLogComment",eString)

  call file.SkipLines(4) ' skip header, can be parsed and added as properties

  file.Formatter.TimeFormat = "MM/DD/YYY hh:mm:ss:.fff" ' set formatter for data
  dim currNrOfChannels : currNrOfChannels = 0
  dim pos, i : pos = 0
  dim leader : leader = 0
  dim currLeader : currLeader = 0
  do
    leader = file.GetNextStringValue(eI32)
    if VBNULL = VarType(leader) then
      ' empty or invalid line
    elseif leader <> currLeader then
      ' new block
      currLeader = leader
      currNrOfChannels = 5
      call file.SkipValues(9) ' Alias
      dim chName : chName = file.GetNextStringValue(eString)
      while "" <> chName and grp.Channels.count < 1000
        dim chO : set chO = nothing
        if not grp.Channels.Exists(chName) then
          set chO = grp.Channels.Add(chName,eR64)
        else
          set chO = grp.Channels(chName)
        end if
        currNrOfChannels = currNrOfChannels + 1
        set chOArray(currNrOfChannels) = chO
        chName = file.GetNextStringValue(eString) ' Next element
      wend
      file.SkipLine() : file.SkipValues(5) ' PARAMS
      for i = 1 to currNrOfChannels 
        chOArray(i).properties("description").value = file.GetNextStringValue(eString)
        Next
      file.SkipLine() : file.SkipValues(5) ' UNITS
      for i = 1 to currNrOfChannels
        chOArray(i).properties("unit_string").value = file.GetNextStringValue(eString) 
      Next
    else
      ' data line
      pos = pos+1
      call file.SkipValues(4)
      ' fill the leading once
      chOArray(1).Values(pos) = file.GetNextStringValue(eTime) 
      chOArray(2).Values(pos) = file.GetNextStringValue(eTime) 
      chOArray(3).Values(pos) = file.GetNextStringValue(eR64) 
      chOArray(4).Values(pos) = file.GetNextStringValue(eString) 
      chOArray(5).Values(pos) = file.GetNextStringValue(eString) 
      for i = 6 to currNrOfChannels ' add the dynamic numeric ones
        chOArray(i).Values(pos) = file.GetNextStringValue(eR64) 
      Next
    end if
  Loop until Not file.SkipLine()

End Sub