ProcessSystemEvents() slowing down

[crazyfwed]

Hello,

I'm facing a strange behavior of my CVI project.

The project is quite big: several timers, probably more than 50 panels, and more than 500 controls (including graphs, trees and tables).

I'm facing to a slow down of my project over time. Usually, it is noticeable after several hours. At some point the slow down renders Ring manipulation extremely slow: the mouse cursor highlights items after 1 or 2 seconds (!).

In order to measure the slow down, I changed the call to RunUserInterface() to a loop including a call to ProcessSystemEvents().

This way, I've been able to measure execution time of the ProcessSystemEvents() call using clock().

At program start, the execution is below 1 ms.

However, I can make it rise (I will explain later how I can do this) up to 3 ms. At that point, the execution time rises slowly over time. I haven't made precise measurement, but I would say it increases of 1 ms every 10 minutes.

I tried to put breakpoints at many callback functions, but I don't get any callback calls (when not doing anything), excepted in timer callbacks.

So I tried to break on ProcessSystemEvents() and then "Step Into".

I was able to confirm that I get only timer callbacks (apparently).

One strange thing is that I get up to 3 calls to each of my callbacks. But OK, that's CVI's internal way of doing, it's not important.

Anyway, it's looks like I don't get my own code executed, only timer callbacks (which I emptied) and CVI black magic.

So, I can only conclude that the ProcessSystemEvents() call is spending time at code I don't know.

Does anybody know what could happen ? Most likely I'm doing something wrong at my program start which triggers a bad behavior. If, ever, somebody has suggestions, I would be really interested.

Now, I'll try to explain what my program is doing when starting, and what I'm doing to "trigger" the slow down, or maybe only what i'm doing to "accelerate" the beginning of the slow down.

My program is quite big and consists of many panels which I have organized more or less in tabs.

In fact, it looks like the popular Firefox extension "Tree Style Tab".

The main panel has a tree control which lists panels. So, there is a graphic representation of the child/sibling relationship of the panels.

At program start, all the panels are loaded and "inserted" in this tree. Some of them are loaded several times.

I also start some timers: one is running at 1s, one is running at 1 ms (I removed almost all code inside the timer callbacks).

When I want to trigger the slow down, I just click on one item of this tree which then shows the corresponding panel (and hides the previous one). I then use the keyboard to navigate in the tree (and thus showing different panels).

If I navigate quickly (using keypress repeats) over all the panels I have (let's say, 20 panels in 1 second) I usually trigger the slow down as soon as I stop changing panels (sometimes I need to repeat the "navigation").

If I don't navigate quickly (let's say 1 panel every 2 seconds), I don't trigger the slow down (maybe it would appear hours after, I haven't tried).

If I disable the timers, I don't trigger the slow down (maybe it would appear hours after, I haven't tried).

Up to now, I haven't been able to remove enough code to be able to publish a simplified version of my program which triggers the slow down fast enough, but I'm still trying...

So, now that you have read up to this point (I would be truly impressed if anybody have read this far...), here are some questions which could be answered to many people (from newbies to NI experts and NI officials):

- has anybody ever seen such a behavior ?

- how could I replace the ProcessSystemEvents call with something I can trace ?

- has anybody have suggestions on how to investigate ?

- why is there up to 3 timer callback calls in 1 single ProcessSystemEvents call ?

- can my problem be related to EVENT stack explosion or something related ?

- is there any way to know what's inside the event stack ?

For information:

- I'm using CVI 2017 (but I have the problem since CVI 2012 at least, I just never had dedicated time to investigate)

- the slow down appears at different rates (unless I trigger it) on different computers

- launching the program from CVI (shift-F5) helps triggering the slow down much faster

- the slow down also appears on computer which only have CVIRTE

- I'm running windows 10, but I had the problem when I was running windows 7

Thanks for reading such a long and mysterious message 🙂

[Jun']

   Hi Crazyfwed,

I believe it would help us to have a little more information to work with. In the meantime, I've found a KnowledgeBase Article (that I also mention in my questions below) that goes over improving performance or LabWindows¿/CVI¿ applications. I'm copying a link to this article below.
Improving Performance of LabWindows™/CVI Applications

1. Have you gotten the status return on the ProcessSystemEvents function? If so, were there any errors?

I just want to make sure that the function was being called and run correctly.

2. What behavior do you expect when calling ProcessSystemEvents?
The function is meant to process all system events that it recognizes and are stored in the event queue. There is a known issue with processing tracking loops, and I want to make sure that the function is behaving as expected.

3. What behavior are you seeing?
When you say that your code is solwing down, does that mean that there is a delay between user input and the callback being processed? Is this delay increasing continuously overtime? What happens if your program runs continuously?

4. Have you looked into adjusting the set sleep policy?
ProcessSystemEvents can sometimes sleep too long between calling events. You can set the sleep policy such that it sleeps less between calling events either programmatically or through the environment. I'm linking a KnowledgeBase Article below that goes into a little more detail on this topic.
http://digital.ni.com/public.nsf/allkb/E9584F559AF4512D86256AB5006D7C00

As an aside, the sleep function may not have been performing the way you wanted since it is a blocking function. If you were trying to perform your ProcessSystemEvents call in the same thread as your sleep call, it would effectively block that function from performing.

Hope it helps!

Junior

[crazyfwed]

Hello,

1. I just checked the status return of the ProcessSystemEvents call. I have neved seen something else than 0.

2. I changed from RunUserInterface() to a loop containing a ProcessSystemEvents(), because I believe that's the only way to observe the evolution of my program reactivity time.

Up to now, there is a good correlation between execution time for ProcessSystemEvents() call and bad reactivity of the interface.

Of course, execution time depends on many things like mouse movement and displayed panel. But for a given displayed panel and no mouse movement, it's OK.

3. What I am seeing is: at some point (after several hours) my program is so slow that when I click on a ring, CVI displays the ring item list after 1 second. This delay is always increasing over time.

4. I tried to change the sleep policy, but I haven't noticed any real difference. But it's hard to say because the slowing down can take quite some time to occur.

crazyfwed

[Wolfgang]

hm, what does Windows task manager say? Does memory consumption stay constant? And what happens if you use 'empty' timer callback routines (or longer timer intervals)?

[crazyfwed]

oops, that's something I forgot to mention.

Memory consumption is not constant, it keeps increasing over time. But.. I don't think it's my fault, because I use CVIDynamicMemoryInfo to monitor.

I haven't tried to "plot" memory consumption over time, so I can't tell whether it's increasing by big steps or regularly.

In the past I had troubles with CVI's malloc (which doesn't truly free memory), so I try not to do much malloc/free and I prefer using realloc when needed. All in all, I'm quite convinced I don't have memory leaks in my C code.

When I empty timer callbacks, it's a little bit more difficult to trigger the slow down (which does not help much for debugging...).

When I use longer timer intervals, it's also a little bit more difficult to trigger the slow down.

More generally, the more controls I update in timers, the more frequently I plot graphs and the more timers I have, the faster the slow down appears.

So, maybe I'm doing something wrong with the timers or the controls, but it's easier for me to blame CVI 😉

More seriously, I checked several things:

- almost all my callbacks needs EVENT_COMMIT (or clicks sometimes) to spend CPU time

- few others callbacks don't need such events (e.g. timers callbacks). For these ones, I'm measuring CPU time (using clock() calls)

Up to now, I'm quite convinced I don't spend excess time inside my callbacks (but I keep in mind this, because maybe I have forgotten something).

So, to me, if I don't do anything, program should not slow down.

Anyway, as I already told, I'm still trying to reduce my program in order to provide reasonnably small program, but... if I reduce it too much, the slow down takes longer to occur... quite frustating.

[Wolfgang]

something else I did not see mentioned in your post: are you talking about the executable or the debug version of your code? If you are running the debug version: what does resource tracking report?

[crazyfwed]

Both release and debug version have the problem.

I haven't seen any clear difference both.

I don't have access to resource tracking, as I only have access to "base" license.

One more thing, the ProcessSytemEvents call also spends more CPU time over time (in my big program, at least) when I keep moving the move over a panel which has a callback or over a control which has a callback.

Maybe that's because of some CVI optimization, but I don't understand why it would change over running time.

Anyway, I managed to build a simple program which has some strange behavior, which, I suppose, could be related to the problem I have.

In this program, I have timers (with associated callbacks) and buttons (with no associated callbacks).

When the program starts, the main loop takes about 30 ms (on my computer).

When I click on a button, the main loop now takes 60 ms.

If I press space bar (and keep it pressed), the loop goes up to 3 s.

If, while keeping pressed the space bar, I move the mouse over the different OK buttons, you can see there is some delay before the buttons get hilighted.

Release the space bar, the loop takes 60 ms.

Now, remove the timers in the UIR and rebuild the program.

The loop now always takes 30 ms (there are some variations, but usually it's around 30 ms).

I haven't find any way to make it take more CPU time.

I haven't tried to simplify even more this program, but I'll probably try next week.

Can anybody confirm my observations ?

Can anybody explain me what's happening ?

Thanks.

crazyfwed

[layosh.b]

What about running ProcessSystemEvents from a new thread or use assynchron timer that automatically starts in an independent NEW thread?

[crazyfwed]

Hello,

I'm not against suggestions, however as I don't know yet where is my problem, I'm rather interested in knowing what I'm doing wrong or why something is behaving in an un-natural way.

Anyway, as you suggested, I will probably try to change my timers, but I would like to avoid having several threads (if possible).

[RobertoBozzolo]

It seems to me that you are fighting against intrinsic limitations of UI timers which you cannot avoid unless you switch to a multithreading scenario.

UI timers are subject to the limitations of user interface handling mechanism the same as every other control; tight loops like those you have in your timer callbacks collide with that mechanism: while inside the loop the system is not able to process UI events (timer scheduling the same as button clicks and so on). Additionally, some events completely block the user interface, including timers: as an example, dragging a window or even simply clicking on the window title halts UI events processing as long as the mouse button is pressed!

You can escape all these limitations by using threads: the simplest way it to substitute UI timers with asynchronous timers, a special timing object installed in a separate library that ships with CVI. The relevant example program can be found by searching for "Asyncronous" in the example finder (Help >> Find examples... menu item).