Dr. Damien's Development - Object Based File I/O I - Representative File Types

[Ben]

---

@DFGray wrote:

From my OO programming expert (AristosQueue) - my questions in black, his answers in blue.

 Is it valid to have overrideable functions in a parent class which 

> are not required because they make no sense in some of 

> the possible children types?

Is it valid to have overrideable functions in a parent class which are not required because they make no sense in some of the possible children types? 
That would be bad design.

Or should I create further children which are still templates containing the extra functionality?

This is strongly preferred. As far as names, it's just further specialization of the parent class, no different than when the first child inherited from the parent. 

Now, having said the above, let me point out that sometimes it is hard to split your hierarchy -- you have two functions, and three child classes, and you want function A on two of them and function B on a different pair of them. There's no way to create intermediate children with the right set of functions. This triggers the classic desire for multiple inheritance or interfaces, and LabVIEW doesn't have these features, so you may decide the poorer design (putting everything on the base class) is the best strategy. 

Another option is a variant of the Delegate pattern. You add to the parent a "Get Extension" VI that returns an internal object that has more functionality than the parent itself. That generally ends up with some type testing though.

Felix, thanks for pushing me back down the multiple object path again.  I had considered it, but not really thought about it much.  Thanks for the feedback.

---

Well that explains why I felt dirty when I did that.

let me also join in thanking Felix. I have now read that post multiple times and it is starting to make sense. I'll sleep on and try to get the model into my head.

Ben

[Ben]

---

F. Schubert wrote:

Part 1: uml soapbox (in reply to Ben)

The diagram is called Class Diagram.

In uml there is the ValueSpecification class with method booleanValue. Only the (grand)child LiteralBoolean is overriding this to return it's boolean value. As you might guess, stringValue, integerValue also exist. So this seems to be a valid design. I was already thinking about this 'pattern' before Damien published his method list.

Part 2: my model

I wouldn't make it all a single class. Having more than one base class will resolve some of the conflicts.

File Class

Basically all File Functions such as Open, Read, Write, Close, Delete

Attributes such as isReadOnly, FileExtension, ...

ReaderAndWriter Class (not the best name)

In the childs of this class is the format specific code.

Methods: Load, Save, GetRawData, GetMetaData, GetHeader, GetFooter, GetData

Data Class

The ReaderAndWriter objects will return Data Objects, this allows to already define the methods in the parent class. AudioReader will return an AudioData Object, MeasurementReader will return a MeasurementData Object, ...

Note that ReaderAndWriter class could have some protected(?) methods for the format conversion which are called by the public GetData method, instead of overriding the complete GetData method.

Felix

---

I slept on it.

So you will have an instance of the Class "ReaderWriter" in the private data of the "File" class and each child of the class File?

To help make this htread easier to follow here are some images of what I have been working on, what I like and what I don't and hopefully where this thread will help me improve what I have.

First the File I/O stuff.

THe above image shows what I have built so far. I did not have to resort to "Place-Holder" for the file stuff. Most of these classes seem to fall together naturally and I have been able to re-use them in multiple projcts.

What I don't like is the size of the TDMS Class.

For comparison this is what the File Class looks like.

Now contrast that with the TDMS functions...

That class seems to be "over-loaded" if that is an appropriate term for classes.

I am thinking that this thread will help me with the TDMS.

So that is the libaray I have been working with and have posted about.

I have also started an ambitious attempt to encapsulate DAQmx functions as well as third party drivers since this is the bread and butter of what I do. TO that end I have a Hardware Class.

This has presented a number of challenges but has started to pay-off in the second project it was used in (the first project started this hierachy) and has been expanded to handle Digital outputs not shown above.

Aside:

The big challenge in getting this together was avoiding recursive calls in my agregating classes.

e.g. The Class MUX uses multiple instances of the Class Cheetah (one for each SPI port on the machine). When the "Init" was executing and calling the parents method "init" in the Class "Hardware" it in turn was calling the "init" method of the Class Cheetah which in turn was calling the method "init" also in Hardware.

Back to the topic!

It was while developing this that I ran into the issue I mentioned above where the "place-holder" was introduced so that I could morph from any class to any class. What I mean by morph to any class I wanted to be able to wire any of the class constants in this diagram and still have it work.

And it was only by using the "Place-holder" (aka Bad Design) could I get the code to update without errors.

So that is the challenge I hope to learn a little more about and hope this thread will help out.

Ben

[F._Schubert]

My ideas are based on Ben's post:

---

* High Cohesion is another Gang of Four pattern that ask the questions "Can someone familiar with the real world system capable of understanding the software model used to implement same?" Or a real-world example "Can someone that understands using the third party driver understand the LVOOP implementation?"

---

So I was just asking what a user of such a class lib would expect if I present him a 'file class'. This led me to the separation into different classes, because if you double click on an xls file, the Excel.exe is called by the Explorer and this in turn opens the file and displays it to the user. So already in this process I have three different 'object's: MyExelFile, TheExcelApp, TheExplorer.

I don't know if it is a good design how MS organized the WIN OS, but this is what 99% of users expect (see all the posts that find the multi-Windows of open VI's unpractical, or the strange behaviour when using gimp). Ok, we can say the users are used to LV in this case, so they would not request an MDI but prefer a new window popping up.

I see that it's really easy to make this really big with a huge number of classes. So if you follow the same path, you'll end up with just some scatches before the budget is spent.

I think that an plugin mechanism would be very useful, so we can add new file formats without refactoring the base package.

Here is my new set of classe (in addition to the one of the previous post):

Filesystem Class

This is the analog to the explorer. For the moment we only need two features (I don't think about public methods yet):

* Browse (this method opens a file dialog)

* CreateFileObj (So the design pattern is a factory that returns the FileObj)

Registry Class

If I double click on a file in the explorer, what happens? Well, the explorer looks in the registry if the extension is registered and searches the corresponding keyword like 'open' and then calls the exe and passes the filename to the exe.

So this class would check the file extension, check which ReaderWriter class is associated with the extension and create (return) the appripiate ReaderWriter (again a factory design pattern, here we have the plugin possibility).

Types package

This would be so big, that we need a complete package just to cover the basic types. Just consider a video: a large number of images, an audio-track and information to syncronize video and audio. This would be our internal representation of the data, indepenend of the format the codec uses. The plugin classes (ReaderWriter) would transform the file data to a collection of types (our dataType).

In the basic implementation, we would only have scalars and arrays (according to Damien's method list).

Here the biggest design challange would be interlinking the types, so how can we model a syncronization element that tells us how the audio track runs with the pictures or if the voltage array and the current array form XY pairs.

Felix

[Ben]

---

@f. Schubert wrote:

My ideas are based on Ben's post:

---

* High Cohesion is another Gang of Four pattern that ask the questions "Can someone familiar with the real world system capable of understanding the software model used to implement same?" Or a real-world example "Can someone that understands using the third party driver understand the LVOOP implementation?"

---

So I was just asking what a user of such a class lib would expect if I present him a 'file class'. This led me to the separation into different classes, because if you double click on an xls file, the Excel.exe is called by the Explorer and this in turn opens the file and displays it to the user. So already in this process I have three different 'object's: MyExelFile, TheExcelApp, TheExplorer.

I don't know if it is a good design how MS organized the WIN OS, but this is what 99% of users expect (see all the posts that find the multi-Windows of open VI's unpractical, or the strange behaviour when using gimp). Ok, we can say the users are used to LV in this case, so they would not request an MDI but prefer a new window popping up.

I see that it's really easy to make this really big with a huge number of classes. So if you follow the same path, you'll end up with just some scatches before the budget is spent.

I think that an plugin mechanism would be very useful, so we can add new file formats without refactoring the base package.

Here is my new set of classe (in addition to the one of the previous post):

Filesystem Class

This is the analog to the explorer. For the moment we only need two features (I don't think about public methods yet):

* Browse (this method opens a file dialog)

* CreateFileObj (So the design pattern is a factory that returns the FileObj)

Registry Class

If I double click on a file in the explorer, what happens? Well, the explorer looks in the registry if the extension is registered and searches the corresponding keyword like 'open' and then calls the exe and passes the filename to the exe.

So this class would check the file extension, check which ReaderWriter class is associated with the extension and create (return) the appripiate ReaderWriter (again a factory design pattern, here we have the plugin possibility).

Types package

This would be so big, that we need a complete package just to cover the basic types. Just consider a video: a large number of images, an audio-track and information to syncronize video and audio. This would be our internal representation of the data, indepenend of the format the codec uses. The plugin classes (ReaderWriter) would transform the file data to a collection of types (our dataType).

In the basic implementation, we would only have scalars and arrays (according to Damien's method list).

Here the biggest design challange would be interlinking the types, so how can we model a syncronization element that tells us how the audio track runs with the pictures or if the voltage array and the current array form XY pairs.

Felix

---

Thank you Felix for the reply and yes I finally figured out how to get to page two to see your Blog.

1st

The blog is over my head but not so far that I can't get there. I am going to have to study it before it will sink in. But lacking an full understanding of your blog is not going to stop me from trying to understand what you have posted here.

I like your approach of looking to the Excel model and the Registry class. I was trying to apply the Expert pattern saying that the Object using the File functions would know the file type and make the selection rather than (is this correct?) delagating that decision to the Registry Class.

So if I followed that much OK then we get down to the details of the "Types" where aside from the common scalars and arrays, I suspect we will have to implement unique methods.

Again thanks for slowing down and spoon feeding. I may be a slow learner but I can commit that once I learn it, I will do my best to share it with others.

Ben

[F._Schubert]

Hi Ben,

I didn't see your long post before my last reply. Your issues with the TDMS method list is exactly what I try to target with the DataType's. The TDMS would only have the attributes (and accessor methods for) Channel, Global and Group. They in turn would be objects with a common parent class that implements the Read_Property(dbl), ...

Concerning my blog, these are actually my own notes when I study the uml metamodel. I'm far away from trying to teach uml. Until now I was just dealing with the abstract meta-meta-classes that will be used to model the meta-languages such as uml and mof. A really academic project at the moment and without a relevance when you just want to use uml to model your code. But my self-imposed challenge is to be able to adjust uml syntax to fit LabVIEW.

If something is unclear to you, please use the comment function. Thinking that a question is stupid is a bad excuse.

Felix

[Ben]

No not a stupid question. I feel there are some data points staring me in the face and I want to process those before I go off and start building bad ideas in my head. So I am still processing.

Ben

[DFGray]

I was also targeting complexity like your TDMS object by generalizing to a data path rather than file specific terminology.  File formats like LVM, TDMS, TDM, INI, TIFF, JPEG, and WAV have fairly rigid formats, but can have huge numbers of and variations in how you get to the various parts.  Other file formats, such as XML and HDF5, have flexible formats and take take virtually any form.  Using an XPath like or HDF5 like data path format solves these issues and is certainly usable by rigid format files, provided the path is one acceptable to the format.  For TDMS, which has a strict root/group/channel hierarchy, only three elements could be specified, and they would correspond to the three levels.  INI files have only two levels, but can still be accessed in the same manner.

I am still thinking over the consequences of using a generic data object.  On the one hand, it permits simply asking for a piece of data without specifying the actual data type if the file is either rigidly specified (e.g. WAV) or is self-documenting (e.g. TDMS).  It also greatly reduces the number of read/write VIs needed on the palette (but not actually written).  On the other hand, once you have the data object, you still need to find out what it is and get the actual data, leading to more complexity for the end user.

But, the data object also allows you to add metadata to the raw data.  For example, it is easy to add scaling information to an integer waveform.  You can also do things like compression (on disk or in memory) and decimation for display.

But I am getting ahead of myself.  Requirements first...

[Ben]

---

@DFGray wrote:

I was also targeting complexity like your TDMS object by generalizing to a data path rather than file specific terminology.  File formats like LVM, TDMS, TDM, INI, TIFF, JPEG, and WAV have fairly rigid formats, but can have huge numbers of and variations in how you get to the various parts.  Other file formats, such as XML and HDF5, have flexible formats and take take virtually any form.  Using an XPath like or HDF5 like data path format solves these issues and is certainly usable by rigid format files, provided the path is one acceptable to the format.  For TDMS, which has a strict root/group/channel hierarchy, only three elements could be specified, and they would correspond to the three levels.  INI files have only two levels, but can still be accessed in the same manner.

I am still thinking over the consequences of using a generic data object.  On the one hand, it permits simply asking for a piece of data without specifying the actual data type if the file is either rigidly specified (e.g. WAV) or is self-documenting (e.g. TDMS).  It also greatly reduces the number of read/write VIs needed on the palette (but not actually written).  On the other hand, once you have the data object, you still need to find out what it is and get the actual data, leading to more complexity for the end user.

But, the data object also allows you to add metadata to the raw data.  For example, it is easy to add scaling information to an integer waveform.  You can also do things like compression (on disk or in memory) and decimation for display.

But I am getting ahead of myself.  Requirements first...

---

Are we doing Waterfall?

Agile seems more appropriate since we will never know all of the requirements and besides, it will highlight the flexibility of LVOOP.

Re: On the other hand

The techniques used would be applicable to other design challenges where we will have to provide interactions between objects using data types that will change in many cases.

It seems that if we focus on

text

Config

TDMS

that other flavors should be able to build on the foundations used for the above.

Ben

[DFGray]

One further question we may want to resolve, are we writing for flexibility or ease-of-use?  Flexibility would imply a lot of low-level objects which can be combined in any way the user could possible imagine to do almost anything.  Ease-of-use would imply hiding as much complexity as possible, but have lots of permutations at the user level.  Ease-of-use could be accomplished using an underlying flexible architecture, so I am inclined to go for that.  I have always been burned when trying to make things too easy; assumptions have a way of getting in the way.

[Ben]

---

@DFGray wrote:

One further question we may want to resolve, are we writing for flexibility or ease-of-use?  Flexibility would imply a lot of low-level objects which can be combined in any way the user could possible imagine to do almost anything.  Ease-of-use would imply hiding as much complexity as possible, but have lots of permutations at the user level.  Ease-of-use could be accomplished using an underlying flexible architecture, so I am inclined to go for that.  I have always been burned when trying to make things too easy; assumptions have a way of getting in the way.

---

Speaking for myself, I still have to "sell LVOOP" into my shop since we have been getting along quite well without LVOOP for more than a decade.

So my customers are already CLA's that can get by without LVOOP. For me to be able to get them to start using LVOOP I can not complicate their lives becuase they will simply say

"That looks too complicated and I will be forced to work outside my comfort zone and concidering I (my CLA customers) am responsible for delivering in-budget and under time, I will not take the chance!"

So I would lean toward a solution that "looks easy to use" by non-LVOOP devlopers but offers a good foundation for growth and flexibilty.

Note:

For my part I can rule out "Cheep". I'll do the work on my own time.

Ben