Is 764 diagrams too many?

[jcrooke]

My LV6.1 main vi hangs upon saving.  Reviewing the statistics, it has 4568 nodes, 255 structures (interesting number), 764 diagrams, 5387 wire sources, 77 controls, and 243 indicators.  The total project has roughly 3.5 times these numbers in all categories.

It contains bunches of state machines (32) on the block diagram.  The states are set and read from functional globals so that each state machine can affect the state of the other state machine.  The code ran just fine up until it refused to be saved.  I tried  the procedure recommended in the Knowledgebase

http://digital.ni.com/public.nsf/allkb/E43B32C14030C3B686256C1E0044D9B6

which worked to allow me to save the project with the new edits.  The article didn't mention, however, that once the behemoth is saved, it cannot be run.  The new problem is that it hangs when I click the run button.

"Must" I, not "should" I reduce the size of the block diagram?  Why?

One reason so many state machines are on the block diagram is to optimize LV multithreading.  I was advised by NI engineers that literally wiring parallel tasks in parallel on the block diagram is the best way to ensure optimal multithreading in LV.  And really, until LV choked on it, the code performed very well, with no noticeable performance issues.  I have 20 serial com sessions, 2 UDP sessions, and 5 TCP/IP sockets running unsynchronized in parallel, managed by these state machines, all connected to a rather massive User Control Panel, configured as a tab control with various control and display options available on the various tabs.  The state machines are designed such that shared information is passed through non-reentrant functional globals which have very low latency functions in them so that each state machine gets in and out of the functional globals in a hurry, ensuring that the machines are as pseudo parallel as possible.  One machine can hang in its communication with it's instrument (a too-common event due to fiber network switch instability) without holding up the other machines.  Since each state machine is on my main vi's diagram, I have full access to state information for each state machine, which makes for powerful diagnostics through indicators on my "Diagnostics" tab.  Aside from the multithreading issue, I am hesitant to move the functionality into a subvi without a plan to access the internal workings of the state machine on my "Diagnostics" tab on the main GUI.

I am interested in discussing better ways of designing this program, but I am mostly interested in finding out what limit I am against so that I may most quickly modify my program to work.

I humbly request all thoughtful advice and suggestions, as I work to quickly subdue this monster.

Thanks all,

Jayme

[mikeporter]

Yes, you "must".

I can't imagine what a single diagram with 32(!) state machines on it would look like--not to mention the race conditions resulting from all the globals...

You need to put some effort into decomposing your application's functionality into reasonable sized chunks that you can implement independently. Those functional units can then communicate with each other as required using queues, notifiers, events--you name it, just not globals. I have been doing this work a long time and have never seen an application that could not be adequately implemented using proper design techniques.

Start by finding a good book on structured programming techniques. You may not appear to be "productive" for a few days, but the time will pay for itself many times over.

Mike...

[jcrooke]

Please, have some charity here.  I don't use any globals, just functional globals, the LV recommended way of storing data to be accessible by various code structures.  I have taken great care to avoid race conditions.  The code works very well in fact.  We must work hard to come up with a solution other than, "You must learn to code correctly."  The code works very well for what it was intended to do.  Why the abrubt halt in editability?  That is the question, not, am I handling race conditions and so forth correctly.  If we know that adding one more indicator is not allowed, then let's agree to that.  Generalized comments about code design do not directly address the question.  What are the limits?  What limit am I up against?

Jayme

[jcrooke]

Mike,

Let me describe the system to you.  Maybe you can recommend a better way to achieve my automation goals.  I have 5 dataloggers that communicate over RS232 from 5 different remote sites.  I have NI ENET 232/4 devices at the remote sites to handle the RS232 communication.  I have other serial sensors at the sites as well.  These other sensors may be moved from site to site depending on the desired configuration for the whole system.  Potentially 4 RS232 devices can be connected to each ENET box for a total of 16 RS232 devices.  Each device has its own protocol.  I have four state machines for handling each of the four types of sensors.  Since each site may have four sensors, there are 4x5=20 state machines required to handle the RS232 communication.  Five of these state machines are exactly the same with the exception of port identification. 

If it is indeed true that I have surpassed some memory limit on the block diagram, then I must try to reduce the number of items on the block diagram.  In this case, how can I package these state machines such that I can constantly monitor the state of the machines from my main panel without using indicators inside the state machines published on my main GUI?  Should I make my state machines into subvis and send indicator references to the subvis to give the state machine access to main GUI indicators?

 The other state machines in my system handle other tasks, but this description will provide some insight into my motivation.

I don't want to seem defensive, but, I really fear that we are tempted to use this thread as a forum for coding critique rather than as a forum for sleuthing the precise problem that I have encountered and perhaps providing hard evidence of LV6.1 limitations, or uncovering a system bomb, file corruption idiosyncrasy, or any number of other problems that could explain the hang-up other than, "You need to code better."

The essential question remains the same, "What limit is preventing me from editing my vi?"

Jayme

[jcrooke]

note: I mistakenly said I have potentially 16 RS232 devices.  That number should have been 4X5 =20, as in my second total, not 16 as in my first total.  I have 4 devices (4 types of state machines) x 5 locations.  They operate continuously, 24 hrs per day, 7 days per week.  The state machines write data to functional globals that are used to store data.  They write one component of data.  When I read the data from the functional globals for logging, I get the latest value written to the functional global by the device.  I don't have to worry about arrays growing too large because I only hold one sample in the functional global for each data field supplied by the instruments.  There is typically more than one data field per device.  I read the functrional globals on even intervals equal to the fastest update and let the device state machines write to them as the communication allows.  They vary in update cycle.

[tbob]

In Mike Porter's defense, most of the problems we see here are a result of bad coding from newbies.  Mike does not know you nor does he know your abilities.  When you stated that "it has 4568 nodes, 255 structures (interesting number), 764 diagrams, 5387 wire sources, 77 controls, and 243 indicators", it sure seems to me that your main vi is way too large to handle.  It may work well, but it seems as if a vi this large would be terribly difficult to troubleshoot in case of problems.  Apparently this is true or you would not be asking for help here.  What Mike was suggesting was that you find a way to make your vi more manageable, like making parts of the main into subvi's that can be easier to debug.  Remeber, we are all volunteers here.  We try to help in the best way we can, without knowing anything about the person asking the questions.  In Mike's opinion (and in mine too), your problem is that your vi is too large to manage.  I don't know the limitations of LV 6.1, how many nodes it can have, etc., and probably nobody here knows what those limits are. Perhaps this is why Mike was suggesting better programming practices rather than answering your question on limitations.

[jcrooke]

tbob,

Thanks for taking the time to answer the question.  It appears that this is a rather rare failure of LabVIEW, one that is not easily predicted nor diagnosed.  I searched for quite a while before posting a new question.  I knew that it wasn't a common problem, but I thought it would be worth asking if anyone else had insight into the cause of this failure.  My experience with the forum is that it is great for polling mental resources far deeper and broader than my own in pursuit of answers.  I have been helped time and again by the forum and its many brilliant members.  I suppose I should have mentioned up front that I have a lot of experience in LabVIEW.  I know more than one way to skin a cat and I could just set off recoding the project with a different approach.  But what if the problem is not really tied to the number of diagrams, indicators, wires, or the like?  This nagging uncertainty is the reason I insisted on addressing the source of the problem, not the plethora of possible redesigns that might avoid it.   

The new development in this thread is that I loaded the code into LV7.0 and it opened up, it accepted edits, it saved and it ran.  It performs at the same blazing speed with all of the robustness that it experienced before, in response to shin kicks and bullet wounds.

I just heard from Mark, the LV engineer servicing this problem.  He seems to think that the problem is the total project memory, not the size of a single vi.  In that case, it doesn't matter how big the main GUI diagram is.  In the solution to my problem, I would be better served trying to make my subvis leaner and meaner, eliminating vestigal data components, functions, or diagrams, since my main vi is more or less optimized, doing no more than I need it to do.  Instead, I think I'll opt to give NI another upgrade sale.

On a lighter note, if you've never seen a large color-coded block diagram spread out over an E-sized plot, you should try it.  It's quite breathtaking. 

Jayme

[tbob]

The closest I ever came to displaying a large block diagram on one screen was when I had a dual monitor system.  It involved a special video card that would split the video screen in half (left and right).  Left half went to one monitor, and the right half went to the 2nd monitor.  If using flat screen monitors and setting them side by side, I could get a 1280 x 2048 screen.  It helped for those looooonnnnggg block diagrams.  Now that I'm back to a single monitor, I cut those long diagrams, by using subvi's, to what I can view.  I sure do miss that dual monitor systems.  I'm pushing to get one here at my latest job.

I can understand your reluctance to recode into a smaller main, especially if your vi works.  Why fix something that is not broken.  If you are going to upgrade, go all the way to LV7.1.1. 

Message Edited by Support on 08-17-2005 03:49 PM

[jcrooke]

tbob,

I am taking your advice, or,  rather, I have convinced the customer to upgrade to 7.1 with the SSP.  It seems I am always behind the curve on LV upgrades.  I find that I can't contribute to forum questions because I can't read the submitted vi's unless I happen to be on my Mac when I am in the forum.  Most of the time I work for somebody who owns the development system.  They pay for the source code and they want it developed in their environment.  They don't like paying for upgrades when their development environment works fine for their needs.  I find that, if you have a 6.1 project, you should develop in 6.1.  I have also found that porting to higher versions is uneventful, but downgrading is complicated to impossible.

I bought myself the Mac version of LV 7, but I rarely use it.  Still, I just can't bring myself to purchase a windows machine with my precious computer budget, even though 100% of my coding projects have been windows projects. 

Jayme

Message Edited by Support on 08-17-2005 04:04 PM

[mikeporter]

Ok, a few more comments:

1) I understood the point you made about the functional globals the first time. The point you need to understand is that while functional globals are better in some ways than the built-in one, they will not protect you from race conditions. You should never have main dataflows being carried in globals.

2) You say that there several of you state machines that are identical--good they should at least be subVIs, but more likely they should be independently operating processes that pass data to a central destination using event, queues or the like.

3) You keep making the statement that your current version works. Well apparently it doesn't otherwise we wouldn't be having this conversation. "Working" includes whether or not the code is maintainable--which your code clearly isn't. In addition, even in terms of functionality, the best you can say is that it "appears" to be functional.

4) You state that: "The essential question remains the same, 'What limit is preventing me from editing my vi?'. Well, the essential answer is still the same: "The code is very poorly written."

As someone else stated, many of the issues we see on this forum are "newbie" errors--and this is another one. I can't teach you good software enginnering in a few posts, and there are many fundamental problems in the code you described. That is why I originally suggested finding a good book on the topic of software design.

Mike...