Create array of different property clusters

[DrNO23]

Hello All,

    I'm creating a VI which will read Excel sheets.  Each Excel sheet has multiple columns of data corresponding to measurements taken on various tools.  For example in the same sheet I'll have a Voltage vs. Current data, Current vs. Temperature Data, and Capacitance vs. Temperature data.   I would like to create a cluster array where each spot corresponds to a column of data regardless of the type of data.  So the first index may have information about a single Current vs. Voltage measurement, and the second will have information about a Temperature vs. Capacitance measurement.

The problem is that each of the measurements require different information so that the cluster at each spot in the array will have different properties (i.e. some may have 3 doubles and a double array while some will have 1 int and 2 double arrays).  Is there a way to accomplish this?

[DrNO23]

Ok so the method I'm using now is to convert the different clusters into a variant with an attribute and then putting the variants into an array.  Is there a way to do it without this abstraction?

[mikeporter]

Actually unless you are wanting to jump into OOP, that looks like a perfectly fine way of handling the problem.

Mike...

[Bob_Schor]

I'm a little confused.  Am I correct that if you look at a particular Excel WorkSheet, you will see a rectangular array of data, each row representing a single measurement (except the first rows might have Header information) and each column representing one type of measurement (Current, Voltage, temperature, etc.)?

I'm assuming that different Sheets might contain differing "mixes" of columns (but if not, the problem simplifies a lot!).

If the data on a single WorkSheet is rectangular, what prevents you from reading it into an array of "Measurement" (where "Measurement" is a Cluster designed to hold a single Measurement from various instruments, all taken at the same time, and possibly also including the time of the measurement)?  Note that if you group multiple Measurement types into the Measurement cluster, you could also have a Boolean "Enabled" for each Measurement type that you could turn off if there is no appropriate data on the Worksheet.

If the problem is that the data structure on the various Worksheets is different (but "human-recognizable", perhaps by looking that the first Header Row that says "Temperature vs Voltage", and the second Header Row that says "Temperature", then "Voltage", with blank spaces before the "Capacitance vs Phase of the Moon" columns), then you only need to read in the Headers, have a LabVIEW parser, and "do the same thing".

I've done exactly this.  We use Excel to program Behavioral Studies, where each Trial can have up to 80 different parameters specified.  Some Experiments use only 5-10 parameters, of which, say, 4 are fixed for the entire Experiment.  We have an Experiment Worksheet where each Row consists of a Parameter ("Laser Delay") and a value ("1000" (msec)), the Rule being that Parameters that are not specified in the Trials Worksheet will take their (Experiment-constant) values from the Experiment Worksheet.  The Trials Worksheet is arranged in rows, each Row being a different Trial, with the Experiment Parameters arranged in columns (with Header information).

We have a Master Excel Worksheet that has all of the Trial Columns, with most of the Trial variables duplicated in Column 1 of the Experiment Worksheet.  Our Users do the following:

1. Decide which Variables are not being used in the Experiment, and delete those columns from the Trials Worksheet and those rows from the Experiment Worksheet.  LabVIEW will set "suitable default values" for these variables (e.g. time duration, amplitude, etc. = 0).
2. Decide which Variables need to be set at a constant value during the Experiment.  Either put this constant in all rows of that Variable on the Trials Worksheet, or eliminate the column on the Trials Worksheet and set the constant into the Experiment Worksheet.
3. Put the "variable" variables into the Trials Worksheet.

I create a LabVIEW cluster containing an entry for every possible Trial value (many of which might not be specified in a particular Experiment).  I then proceed as follows:

1. Set all of the values to a "safe default value".
2. Read the Experiment Worksheet.  From parsing Column 1, figure out which Variables are present on the Experiment worksheet and set the appropriate value into the cluster.
3. I now build an array of Trial Values corresponding to the entries on the Trial Worksheet.  I start with the "Default cluster" constructed before, with the "not-specified" default values and the "Experiment-constant" Experiment values already in place.  I read a row from the Trials Worksheet, use the header information to put the values into the correct Cluster entries (over-riding any otherwise-default value already there), then save this in my array.

This sounds complicated, but it works very smoothly, and surprisingly quickly.

Bob Schor

[DrNO23]

Bob_Schor,

    If I am reading your reply correctly it seems like you have a cluster with all of your data from an Excel file which is exactly what I would like to do.   However, it seems as through you have the same experiment with different measuring parameters which would allow you to have a single type of cluster but with different data inside to represent the measurement.  I would like to mesh different types of data with totally different parameters types. For example:

Measurement Type 1:

X aray of doubles

Y array of doubles

Temperature double

Time Double

Measurement 2:

X array of doubles

Y array of doubles

Z array of doubles

Averaged samples Uint64

Temperature double

I would like to have an array of clusters but each spot in the array could vary depending on which measurement was located next in the excel sheet.  The whole reason for doing this is to have a very robust plotting method which would allow me to search through my huge data sets and just plot measurements of a certain type taken from some date and with some parameters.

[Bob_Schor]

Look into using TDMS and Diadem.  Huge data sets with multiple different kinds of data, where you want to make various plots segregated by the types of data is (I believe) the design goal of this product.  I participated in a Hands On session about three years ago, but decided that it wasn't appropriate for my Experiment data.

In the studies I described, the varied Excel Worksheet describes the varying Stimulus parameters.  There are actually 4 data files generated by these studies.  One is the Excel Worksheet itself -- for historic reasons (which I argued for changing, but was over-ruled), we write a few "end-of-trial" values as part of the Trial row, including the (LabVIEW) run number of the Trial.  [The Experimenter has the ability to start and stop the Excel Worksheet at any row, repeating rows as he/she sees fit, thereby over-writing the Excel data, but keeping the old + new LabVIEW data].  Our data files are a Header File that contains both global Experiment information and a Trial-by-Trial set of all of the Excel variables used in that Trial, an Event file that records any and all "point-in-time" Events, including timing of the Trials and any Digital I/O transitions, and a Samples file that contains N channels of A/D and sampled Digital (port) data, recorded at 1KHz.

BS