Erratic timing in RT Timed Loop

[SammyDee]

Hi,

I am trying to use our PXI-8186 to perform tasks at precisely 500ms intervals. However, despite using the 1 MHz clock, the timing of my loop was all over the place (varying by 100s of ms). So I wrote the simplest possible VI to test this and the same thing kept happening. Occasionally it works as expected (hitting 500ms within only a couple of microseconds), but mostly the timing is out by bucket-loads (as you should see by the numbers I've saved as default in the array). The VI is attached but you will notice there it is not really doing anything, so I can't imagine there's anything to slow it down. There are no other VI's running at the same time either. Any ideas? Would this be a hardware problem with PXI or perhaps a problem with the real-time OS?

Many thanks!

Sam

[SammyDee]

Sorry, the attachments drops off when you preview it seems... try again...

[Doug M]

SammyDee,
That is very strange behavior.  I tested your VI on an 8176 RT controller and it ran fine and gave either 499999 or 500000 on every iteration.  The only things I can think of are 1) you are using default communication to the host and network traffic is somehow so heavy that the background communication task is taking over the CPU, or 2) you have something else running on the target.

What version of LabVIEW RT are you running on the controller? Have you tried running this not in default communication mode? Have you tried using the millisecond timing source instead of the microsecond source?

[SammyDee]

Thanks Doug,
Yes, I see almost perfect timing on occcassion too... i.e. if I run the VI over and over sometimes it seems to "get it right" and keep time to ±1µs. It's like it either works or it doesn't.

My answers to your questions:
1) I haven't played with the communication so I gather I'm on the default setting. The host and PXI are connected to the same 10/100 Fast Ethernet Switch. Perhaps I could use a direct connection from the PC to the PXI to test this idea....? Otherwise, I'll see if I can work out how to use a non-default setting.

2) I don't have anything else running on the target. (Unless something's running unauthorized somehow!)

3) I'm using Labview Real-Time Update 7.1.1

4) No I haven't tried non-default communication - I'll give it a go.

5) I tried on the millisecond timer and it seems well behaved. The microsecond tick count reports numbers like 500340±30, meaning it is actually accurate to ~0.3ms but stable to about 0.03ms. That's good in other words!

Thanks again,

Sam

[aasenjo]

Sam,

There are two very likely causes for the timing variation (jitter):

1) The Stop control terminal inside your loop will cause your Timed-Loop thread to interact with LabVIEW UI thread. One should avoid controls and indicators in time-critical tasks. See Chapter 3 of the LabVIEW Real-Time User Manual for more information on this topic.

2) The array allocation thought auto-indexing in the output tunnel of the loop will cause memory allocations. Memory allocations should also be avoided because they require access to the OS Memory Manager, which is a shared system resource and can therefore be held for an indefinite amount of time by lower priority tasks, inducing jitter on your time-critical task. The jitter will only be evident every so often, only when a memory-manager contention occurs. For more information on this topic see Chapter 5 of the LabVIEW Real-Time User Manual.

I attach a screenshot that highlights both points.

The LabVIEW Execution Trace-Tool is a great tool for finding the source of these jitter sources, allowing you to optimize the timing of your application

I hope this helps,

Alejandro

[Doug M]

Sammy,
What you may want to do is use more of a real-time architecture to test this.  When you use default communication (default communication is when you have the front panel of the Real-Time VI open on the host machine) there is actually a communication thread running on the background of the real-time target transferring the front panel information back to the host.  This can sometimes degrade performance.  In a normal real-time application you want to split your time-critical operations into a separate VI and make the priority of the VI time-critical, then have a communications VI that is normal priority that communicates information back to a host VI, usually using TCP/IP VIs or Datasocket.

The Real-Time Communications Wizard can take a single VI and split it into these three VIs for you.  However, the Timed Loop structure should not be used in a VI that is set to Time-Critical Priority.

Even with the overhead of communication, it doesn't seem like you should see the disparity in numbers that you are seeing, however, so I'm a bit baffled as that goes, but I would still try splitting your application up to see if that improves the reliability.

[SammyDee]

Alejandro,

Thanks for your input. I am aware that those tasks are potentially disruptive to the determinism of the loop but (like Doug) I would have thought the overheads, in this case, to be almost insignificant... at least not enough to cause delays of the order of hundreds of ms. I tested the impact of the stop button by removing it and stopping at i=10 instead. The same systems remain (see numbers in attached VI). So this would presumably rule out point 1 for my case.

That leaves your second point which I don't know how to deal with just yet. I'll have to read up on how to report those numbers in a deterministic way.... although I still find it hard to believe reporting such a small amount of info could cause such major problems, but I guess it's a bit complicated.

Thanks also for alerting me to the jitter tool, although I'm not willing to invest in it yet because I reckon the task I am trying to achieve should be well within the ability of our system as provided.

Cheers,

Sam

[SammyDee]

systems=symptoms in the above.. whoops 🙂

[Doug M]

sammydee,
See the following for info on preallocating resources to avoid the jitter alejandro talked about: http://zone.ni.com/reference/en-XX/help/370622C-01/lvrtconcepts/avoiding_shared_res/

[SammyDee]

Thanks again Doug!

I tried to implement the structure on that page (see attached) but the same problem remains (see the numbers again). I'm not sure what to try next. The only other thing is to implement the communication protocols you mentioned but now we have no communication (do we?) occurring in the VI... So I don't think that's going to help. Must be something low-level don't you think?

Cheers,

Sam

P.S. Sorry about the delay - I got busy! 🙂