LabVIEW
Thoughts/Feedback on Channel Based App Hierarchy
Background: All the channels in this are the same template. This is my Flex Channels template that's published to VIPM: https://www.vipm.io/package/illuminatedg_lib_flexchannels/
Unlike the shipped channel templates where the mechanism (lossy/lossless/etc) is determined by the channel template itself, flex channels provide lossy, lossless, and event-based reads all in the same wire depending on which reader endpoints are active. I'm starting to work on some module templates (akin to a starter DQMH template or CMH/QMH project template) and am curious what people think of it. With Flex, the main idea is to further abstract the mechanism away in an aim to have more flexible and reusable code. Want to tack a chart onto a data channel and get a lossy view of the data? Easy. Want to tack on a lossless logging module to an existing data path for record/playback debugging? Easy.
This application will be a reproduction of the Continuous Measurement & Logging template that ships with LabVIEW:
Then the control, acquisition, and logging modules can be used without the GUI for programmatic timed data logging:
This does present a difficulty in channels that all wires must be terminated at an endpoint, and the lossy Control Status ready is there only to accomplish this. It's tempting me to create a "null" terminator that allows terminating unused channel wires without causing any data transfer to be done. This also allows commonly combined modules to be combined into SubVIs:
Currently, the logging module is the only one completed and is implemented as shown in the following screenshot. This is a pattern I've landed at though I'm still fiddling with details about helper loops. It does demonstrate the flexibility of the wire template in that they're forkable and can support different reader types. The helper loop leverages the module's own "external" command channel to close the module if the data channel gets closed.
What do you think? This Shirley falls under "solves a problem I don't have" for most people and those responses are just as welcome. Something I'm also working on figuring out is what error handling and propagation would look with a module pattern like this. I'm tempted to create an error channel that all lowest level modules could write to so they could be read by a higher level module and propagated further as necessary. Does anyone see value in this at the inter-module level? Anyone besides Bob Schor using channel wires for their architectures?
I'm thinking of providing a way to "merge" channels so that modules have a singular channel in/out and other modules can pull out merged channels they know about and need which will eliminate the explosion of connector pane connections for more complex modules. A feature I've already played around with in unpublished versions is to "convert" channel wires into classes so that they can be passed around more freely; a lot of the work for internals of merging/unmerging is already prototyped.
One of my favorite features of Channel Wires is the "obvious metaphor" they represent. They are "pipes" that go "over the edge" of structures, a metaphor for "Structure Edges don't deter the normal flow of data out of one structure and into another.
By at least 30 years of experience, the "output" edge of a structure is the right edge, the input edge is the left edge, and when you abstract the code as a sub-VI, with an Icon and Connectors, all the connectors connected to the right edge are outputs, the left edge are inputs, and for top and bottom edges, the outer-most connectors follow the same pattern.
So I look at your first illustration, showing "Acq Status" connecting two outputs (right edges) together (which makes no sense to me), and a little lower, Acq Data apparently connected to three inputs.
I'm currently working on a LabVIEW Real-Time application that has a half-dozen (or so) parallel loops running, two of which are the usual CMH Event and Message Handler loops, others handle communications to and from the Real-Time side (linked by 5 Network Streams, two carrying Messages from Host-to-Remote and Remote-to-Host and the others carrying various data streams to the Host (for logging and data storage). Here's a picture (just to show the "idea" of the intuitive "direction of Data Flow" in Channel Wires) from the Host code (still under development ...):
The VIs whose outer borders look a little like a While Loop (three are at the bottom) are "Parallel Loops" (hmm, I seem to have a few more than I mentioned, above). Look at the first one at the bottom, "Get Message: UI from RT" -- it has an Input coming from the Blue (Tag) Channel Wire which also "feeds" the inputs of two more Loops, as well as going up to the Event Loop (located above the CMH Loop), and has a Messenger Output (the Pink Pipe) that goes up and "in" to the CMH Message Reader (LOOP Receive Message, UI to UI). All done by "positioning the Pipes".
I'll be interested to see where you go with this ...
Bob Schor
@DerrickB wrote:
With Flex, the main idea is to further abstract the mechanism away in an aim to have more flexible and reusable code. Want to tack a chart onto a data channel and get a lossy view of the data? Easy. Want to tack on a lossless logging module to an existing data path for record/playback debugging? Easy.
You should have a look at my Messenger Library sometime. No Channels, but the "Messengers" in it were motivated by the same kind of easy flexibility of hooking things together (even extended to TCP and UDP).
Then the control, acquisition, and logging modules can be used without the GUI for programmatic timed data logging:
I've not used Channels much myself, but I like the block-diagram clarity it can bring (though we could really do with those directional arrows being actually automatically present, rather than added manually). But I worry such clarity gets lost as one scales up to a more complex real-world app (rather than simple examples).
Just in case you haven't already seen it, here is an interesting talk about Channel architecture: A Visible Actor Framework - Jeff Kodosky.
I have installed your package and had a very quick look. It looks interesting and I hope I get the time to test it to give you some real feedback.
In 2019, I started using channel wires for my applications. I had been using all kinds of frameworks and the frameworks themselves seemed to get in my way too often.
For my small to medium sized applications, channel wires work very well. For them, I don't need a framework, just a channel messenger class and some design principles. It was a struggle until LV2020 which seemed to have solved most issues. But since then, I can just code and don't have to battle any golden-hammer framework.
Some thoughts on JÞB's points:
""Channels" are a hardware term."
- agree. I always call them "channel wires".
"They don't share well."
- I don't see that anymore. It took 1 second to open the OPs example on a 4-year old laptop. I don't think this is an issue at all anymore. I had issues with LV2019 and before but not since then. Opening other code displays the loading dialog and that one is often hard to stop too (although sometimes possible).
"I really like the VI Hierarchy view!"
- Why should that limit how they are used? It's not uncommon for code-libraries to have layers of code that are never used and cannot be filtered out in the hierarchy view. It would be nice though to filter them out from the hierarchy view, even though I rarely use it myself, but then I would like to be able to filter out my own reuse-code too. I got curious though, to how you use and make use of the VI hierarchy, as I rarely find it useful.
(for reference: another thread about general advantages/disadvantages of channel wires)
Some of this feedback request is based around the fact that Channel Wires have some lack of clarity inherently baked into their implementation. With this Flex Channel Wire implementation I'm trying to feel out usefulness of but I'm also curious of channel usage at larger scales. (Thanks for that screenshot Bob, it's exactly what I was looking for in previous messages). I've got another framework I'm writing that's targeted specifically for parallelizing DSP / SDR / etc. processing pipeline algorithms and am starting to think what a more generalize framework/module pattern might look like. I have seen Jeff's visible actor presentation and the thread about advantages/disadvantages but everything about channels I've seen is very high level and pretty simple cases, loops with single reader and/or single writer, so I'm interested in some more real world considerations of their usage.
Bob, I started that way but didn't like the amount of overlapping and "loopiness". For some of the channel wires, there are writers on both sides so it's not always purely directional. It feels like having a convention like that, particularly with such a flexible channel template, could lead to invalid assumptions when other constraints mean violating those conventions. How about a convention based on the sink/source direction of terminals and adding glyphs? Here's a very quick mockup that shows limitations of it as just subVI glyphs but thoughts on if some kind of overlay was implemented?
These are the kinds of details that are pushing me towards something like a channel template specifically for defining in/out relationships for a module instead of having different channels for different directions or different data types. I know the messenger channel wire template can transfer different overrides with class-based data and I think that could be combined with how my flex channels allow choosing and mixing different underlying mechanisms based on the endpoints used.
I've also mentioned converting channel wires to classes and am thinking of a channel wire template where the endpoints are used to create/obtain messaging mechanisms which would provide better scoping than globally named queues. For instance, inside a child module an endpoint could be called to register an event for a named event to use with an event loop and another endpoint added to get a queue for high speed data transfer from another module, but the channel wire itself doesn't provide data transfer endpoints. (One of my complaints about channel wires is them being focused solely on data transfer read/write schemes because the channel wire mechanism can be used for things that aren't quite read/write exchanges).
Another common issue I hear about something like AF is the effort in getting things like data queues bubbled up to a higher level actor and pushed back down to another branch of the tree for efficient data transfer without violating hierarchy constraints. If going with this "module relationship channel wire" a mechanism could be implemented to bridge levels for specific identifiers/data such that one side of a tree could expect that a leaf will be providing data via a queue and the other side could push that down so that another leaf module could get access to that queue but all intermediate modules must enforce exposing it. I'm kind of word vomiting and brainstorm out-loud here but does this sound useful to anyone? Doing this would definitely push a lot of potential edit time error recognition to run-time errors but that'd generally be the case with any abstracted messaging mechanism.
Slightly related, regarding any Channel Wire implementation, something I've run into with some customers is their incompatibility with the run-time engine which can wreak havoc with improperly implemented plugin architectures or trying to use the run-time adapter in TestStand. I've contemplated creating some tooling based around pulling specialized channel implementations from ExtraVILib into projects and therefore into SCC. This would also eliminate the chances of name collisions if different projects have different typedefs with the same name that aren't namespaced into differently named libraries/classes.
I've seen the intro to messengers video before but I'll have to take a closer look and see what larger examples would look like.
