Saving AVI with Overlay.. slow.. any workaround?

[Corny]

Hello there

I couldnt find any clear answers so I'll just ask here 🙂

My application needs to do the following:

1 - grab an image from camera

2 - add overlay (overlay is consisting of dynamic items)

3 - view image w/overlay in a external window

4 - save AVI with overlay

The most importent thing is that the external window showing the overlay and picture has a high refresh rate. Below is the code for this: (wow it got quite huge.. Im sorry about that.,.)

First, it grabs an image from the camera. This grab function removes any overlay that was there before (blæh.). Second a clear error is used (Im lazy and havent included errorhandling.. yet), third, each 200ms (the loop runs at 20 ms) the dynamic overlay items are updated on my own overlay picture. The reason I use an own picture with the overlay is that I want to minimize redrawing of static elements on the overlay. Fourth - copy the overlay from my static overlay picture to the grabbed image. Fifth - show the picture in an external window. Sixth - Set the IMAQ reference to a func. global.

This loop runs perfectly. The reason for not using "merge overlay" here is that is goes to slow and messes with the update frequency..

Second, the loop that saves the AVI file:

First - the func global is red with the IMAQ ref that has both the grabbed image and the overlay.

Second - This IMAQ pic is duplicated. Why? well, if the merge overlay is going to merge the original IMAQ picture, it would somehow slow up the external display (and that gets choppy) (althrough, beats me why this happends)

Third - The overlay is merged

fourth - The merged overlay is saved.

Now what happends is that this saving is just going way to slow, (but by doing it the way I did, it doesnt messes with the update frequency of the external display) And it all has something to do with the merge overlay. If I disable the merging everything is fine - but the video does not show any overlay.

Below is an example of how the overlay may look like:

Does anyone have any clues how I can further improve this to be faster? my AVI file gets like mega choppy when merging overlays like the picture above!

[Corny]

An update:

I changed the code that writes the AVI to rather use a wait until multiplum timer in that case, calculated using 1000ms/(frames per second). This gives the AVI correct sample rate, since I saw the overlay flickers if you push it to far. Also I played with it by putting it on an own core.

Using MPEG-4 codec I managed to save it stable with 3 frames per second. Going higher resulted in flickering of the overlay in the recorded video, why I dont know (it doesnt give out any error), but it seems pushing the limits affects something, even the CPU isnt max?

Still its slow though. The most importent thing is that the external display performs excelent including overlay with no latency (here there is no overlay flicker like in the video recordings)

Still the problem isnt solved.

[BlueCheese · Accepted solution]

Hi Corny,

It seems your problem might be (can't tell from the screenshots) that you are just copying a reference to your same image you are using to Grab into that your other loop is pulling from. The problem there would be that you would have a synchronization issue where as you are using the image in your AVI loop you will be modifying that same image when it is acquired into. A simple modification would be to have your Acquire loop Grab into a local image and only copy into the global one. Then you could have your AVI loop copy from that image before doing any processing. This would ensure the image is write-only in one loop and read-only in the other, as well as ensuring those accesses are atomic with respect to each other (since Copy will lock the images during the operation).

Eric

[Corny]

BlueCheese, thanks a lot.

I understand it is difficult to read. The camera application stuff is only a smaller part of a larger software

FYI: The reason why I used one loop for aquisition & showing, and one for saving (and merging) was simply that if I put the merging in the aquisation loop, it slows it down significantly and the poor operator who looks at the camera picture w/overlay would grow tired of that laggy stuff. So I needed to handled aquisation w/overlay in one loop, and saving/merging in another.

It perfectly makes sense what you said, and yes, this solves the flickering in the AVI. However logically, the performance dropped a bit to the ext. display due to that the copy function was introduced in the aquire loop, and created a bit longer lag there. Without overlay you wont notice, but when the dynamic overlay gets updated its noticed - I then introduced wait on multiplum in the update case, due to that previous using the Queue timeout (20ms) the total time would be a lot higher since the copy function is used in addition the total aq time would be around 30-40 ms I guess (or more) each time the update dynamic overlay function kicks in. A possible solution would be to put the overlay updating in an own loop,and introducing another paralell case (and two more IMAQ picture refs) but I hope to avoid this due to that the compexity increases. I will try to see if I can tune in the timers, however they are depended on the amount of overlay that gets drawn. Since the user can draw and create its own overlay (the software will be used on many sites) I have no clue what they will configure individually.

The AVI save loop now does not have any flickering, and Im able to save to file with approx 5 frames per second. Going higher results in that the copy function & merge overlay function, in addition save to AVI would consume more time and it not able to save to AVI with the faster frame rate. I will try and see how much time that case uses, althrough again - its depended on how large the overlay is.

I'll leave this open for some days in case someone sees something else (or that I used the wrong strategy) that could be done to increase performance, if not I'll mark your comment as a solution and try to live with the performance issue.

[Corny]

Copying the reference to a global solved it and am using that today, however having a dynamic overlay with a lot of items still has poor performance. Today I'm using the solution above, however have limited the items on the overlay to avoid any flicking of the overlay.

I'm changing over to IP cameras and will use an own OCX for that, that has overlay functions included, support for different devices, and a lot more. This works perfectly - smooth - and after some long term test it is stable. I even can get out movement information etc. for image processing afterwards, recognition of movement and simply build functions upon that.