Parallel while loops for failsafe system and control system

[andrewjoe777]

Hello all,

So, I'm needing some direction for the best way to go about making a control system and failsafe system run in parallel. I've attached the VIs for both systems. I'm wanting it where if the failsafe VI detects any problems in the outputs or inputs, then it tells the control VI to send 0 volts to the proportional valve (input to the plant in the simulation).

In the failsafe VI, it checks all the outputs and inputs (simulated signals at the moment) and if any thresholds are suprassed then a false value is assigned to the particular global variable. Then in the next sequence it checks if any of these variables are false. If so, then it assigns the "normal state" to false. In the final sequence it reads the value of "normal state" and if true, nothing happens and it goes to the first sequence and checks the I/O values again. If false, it sends a 0 voltage to the proportional valve, sends an email and text to someone telling them a failure happened, and then gives a siren.

In the control system VI, it continually sends a voltage to the valve based off of the error in the set point. 

Now how would I run these in parallel and have it where the failsafe VI while loop would stop the control VI (which is actually in a case statement inside the while loop) while loop if any failures are detected? My only guess is that I could put them in the same VI, one below the other, and then before the control while loop iterates and it goes into the case statement, it checks the normal state. If the normal state is false, it sends 0 volts to the valve, if true is continues as normal. I'm not sure if this is the most efficient way to do things though so I'd appreciate any advice anyone could give. Thanks!

[andrewjoe777]

Here's the three subVIs for the plant that I forgot to add.

[johnsold]

First, you should consider what happens if the computer fails, rather than equipment in the plant or the plant control VI. Running a separate VI as a failsafe only captures certain kinds of failures.  Depending on the consequences of a failure, you may want to consider a hardware failsafe, a PLC for failsafe, or a standalone real-time system dedicated to failsafe.

Next, you should learn how to use the dataflow paradigm of LabVIEW. The use of sequence structures and global and local variables violates dataflow, is prone to race conditions, and makes troubleshooting more difficult.  The Normal State global has a race condition. It is probably read before it is written. Since the default for a boolean is False, the loop might stop after one iteration. A True is written in frame 1 but most likely the instance wired to the continuation terminal will have been read before that write occurs. 

The two Stop buttons on the Failsafe VI may confuse the user.  At least one of them should be renamed. The way the one on the lower right is wired in an event structure inside a while loop inside a sequence structure inside a case strucure inside a sequence structure inside a while loop inside a sequence structure could lock up the VI so that the only way out would be to abort the VI. That of course would defeat the failsafe function.

I recommend that you learn about state machines and the Producer/Consumer Design Pattern.  A combination of those designs will allow you to eliminate the sequence structures and global variables, while providing a more robust, expandable, and maintainable software architecture.

The failsafe VI would run as an independent subVI of the main program. Independent means that no output of the main program (or other subVIs) is wired to inputs of the failsafe VI. The data it needs could be passed by a queue or via a functional global variable.  The failsafe signal would be returned via a notifier.

Lynn

[andrewjoe777]

Sorry for the late reply but I really appreciate your advice, Lynn! This is just what I needed to know! I had no idea that my programming structure was so flawed. I am going to restructure my program and then post the results here. It'd be awesome if you could look it over after I'm done to make sure I made the correct improvements.

Thanks for the advice on using an external failsafe system. I might think about using a PLC for failsafe but the consequences of failure is pretty minimal. Nothing catastropic or anything. The failsafe system is really more like a safety system I guess rather than failsafe. We are just lifting 20 lb loads with a cylinder rod to different set points. The biggest dangers are only if the cylinder rod hits the maximum displacement too fast or if a pipe busts.

[johnsold]

Please do post your updated VIs.

Only you or your customer can decide on the consequences of failures. I often mention that when someone starts asking about software failsafes just to make sure that you think about those things.

Lynn

[andrewjoe777]

OK. Here's the new VIs. I eliminated all the global variables I had in the safety system by using the state machine. I also make sure I adhered to the data flow structure but I couldn't figure out how to relay a message to the control system when a failure occured by not using a global variable. It seems to work fine while using the global variable. Is this okay?

I know you said to use a notifier but as far as I read when you use that in the master/slave configuration it doesn't allow the slave loop to execute until it receives a notification. That's not what I want it to do though. I want it to keep running the control loop and then just change the output going to the valve when a failure occurs. 

I also didn't know how to run the safety system as a subVI so I just put the code in the same program. Is this okay? well besides it creating a large block diagram?

Oh and the other 3 subVIs that I posted before for the plant are still the same.

Thanks!

[johnsold]

The way to transfer a message via a notifier without blocking is to use the timeout feature.  The Wait on Notifier function has a timeout input which defaults to -1, meaning wait forever. It also has a timed out? boolean output. If that is True, no notification was received before the timeout. So rather than using the Wait function, use Wait on Notifier with 1000* Sampling Time as the timeout. Probably move Wait on Notifier outside the case structure where the Failure global is. If timed out? is True, this is the same as Failure = False. If timed out is False, use the value of the notifier output (which is probably the failure boolean, but not the global.)

To keep the diagram size down, make the failsafe state machine a subVI. Then it only takes the space of a subVI icon.

Lynn

[djs_at_eaton]

I don't think there's any way to "failsafe" a system using only software.  We program the software to run a watchdog timer.  If the timer doesn't see a pulse (from the program) every 100mS (you decide the time), then the relay drops out and the e-stop system activates.  The e-stop is part of a safety relay system.

Recovering from the e-stop requires at least the following steps.

1) The safety relay has to detect that the appropriate equipment contacts are open.  In other words, check for welded contacts or any other parts of the safety circuits that may have failed.

2) The watchdog timer must resume.

3) The software must acknowledge that it has "seen" the e-stop condition and placed itself in a "safe to resume operating" mode.  This means the software is in a state where no motion or forces will be present when operation resumes.

4) The operator has to press a button to restart the equipment.

If all those conditions are met, the machine resumes operation.

This method ensures that the software can't get into strange loops where the machine is essentially out of control.  One more requirement is that the watchdog timer software is called from only one place in the main software loop.  The software architecture must be such that any "bad" loop will cause the watchdog to time out.