ARGB to RGB, array processing performance

[RavensFan]

I either forgot about that statement or just missed it.

Assuming that this can't be sped up (it's just a matter of time before Altenbach steps in and shows us how), supposed there is a different way to do this in parallel threads. 
Create 4 different instances of this subVI and make it reentrant.  Send the data of each frame to a different subVI instance in a round-robin format.

[Jester_v01]

Just some context:

This is for an application that is basically a game, so I want a tight loop with my input processing (easy) game logic (which is simple), my output creation (pretty graphics, some animation, simple and quick with Cairo) and then display (inexplicably hard!).  Currently the display bit is consuming about 80% of my loop time.

The problem with doing the processing in parallel is that the target frame buffer array (U8 R and G then B) has to be split up (easy -  can be done by preallocating R, G and B arrays, or splitting it into thirds) but then stuck back together again to be converted into a string to be sent to the picture control.  Every time this happens Labview makes copies of it (used Profile - Show Buffer Allocations) which, for a full size frame of 720 * 1366 pixels  = array size of 983,520 this is pretty slow.  I think this is what slows down the decimate --> interleave technique also - the buffer allocation for the intermediate arrays - I'm not aware of a way to put the output of the decimate block into preallocated arrays.  I've also tried just having a massive preallocated string rather than a U8 array and using 'Replace string subsection' to update the data.  This was also slower.

I don't understand why missing this step out altogether and passing the all the data (converted to string) to the picture control after telling it to expect 32 bits per pixel is also slower.  Presumably it does it's internal processing in an even less efficient manner...  It also screws the colours up.

What I *really* want is a pointer to the back buffer of the 2D picture control so I can just MoveBlock the whole ARGB array into it's buffer and then tell it to update.  Or have some Directx guru tell me a really simple way to get the window handle and create an overlay I can dump the frame buffer to.

Or NI to create a 2D picture control that doesn't suck..

[Andy.G]

Hey Jester,

I would strongly reccomend that you do not use loops when dealing with the image data, if you can find a function that will do the process for you all the better. What I would suggest is rather than using Cairo and Moveblock DLL's that you use the IMAQdx functions rather, as they work directly with Active X within windows and therefore will use the graphics card to do the processing rather than your CPU. Also you will find that the data will be outputted in the corrct format for you, you may find this would greatly reduce development time. Failing that you could always try using the "Draw Unflattened Pixmap" function?

P.S. IMAQdx is phenominal, if you want advanced picture control functions and image manipulation i suggest you try it.

[Jester_v01]

Not sure how I can avoid using loops - the application responds to inputs and needs to update it's output regularly - not sure how to do this without a loop.  As for processing the image data - the draw unflattened pixmap uses exactly the same loop as I am to process the ARGB array into an RGB one, with lots of extra stuff for range checking and so on.  By moving only the code I needed out of the unflatten pixmap vi and onto the main diagram I managed to speed it up usefully (less buffer allocations?)

I'm sure that IMAQdx is great but I can't justify the cost for this developement, and in any case, it seems a little poor to need to get a whole toolkit just to be able to display an image quickly, especially since the Cairo/moveblock bit to actually create the image works so well!

[Jester_v01]

Interesting discovery:  The 'no array processing' solution!

Tried this:

No array processing, preallocated array of U32 (ARGB) converted via cast to a string and then thrown straight to the picture control (note change to 32bit color depth).  Very neat code... but slower.  Probably because of the buffer allocation after the cast block.  Any ideas how I can prevent that?

Luke

[Ben]

Hi Luke,

As you know these investigations take time (something I don't have a lot of) so the best Ican do to possibly help is offer this link to a Post where Norm (the Captain, who was one of the first to unlock LV scripting) posted some info on how he managed to get a 40 FPS in an app.

Now if you can get what you want from his public postings then I suggest you pick-up the phone and call your favorite NI rep and ask them if they can get Norm to look at your thread and advise (Norm now works for NI Austin).

If you do get this going fast enough for your needs, I for one would be very interested in your final solution or an example that would help others.

Take care,

Ben

[Jester_v01]

Fascinating - thank you very much for that information.  It explains what I've managed to do by with a bit of trial and error and looking at the various pixmap .vis.  I'm already using his technique - constructing the picture from opcodes sent to the picture control in a flattened string.  Unfortunately I can't take full advantage of this (by directly editing the instructions that are changing from frame to frame rather than redrawing everything (building the string from scratch) every frame) since the only op-code I'm sending the control is 29, which I guess is the 'display array' command.

I've also discovered that the directx 'wait for screen blank' bit isn't doing much - with the image I'm animating the difference is negligable.  By removing that and reducing the picture control to 750*500 I'm managing to get a fairly stable 30fps (up from 20 with the wait for blank).  This falls to pretty lamentable 14fps when I try to have a 720*1366 size control. 

By changing the size of the Cairo surface I'm using (from 750*500 to 1366*720), but copying across the same amount of data (750*500) I've established that about 1/3 of the time increase is due to Cairo having to blank the larger surface every frame (I could avoid this) and the rest is due to the moveblock and string formating / copy to the picture control.

The only way I can see to increase the speed of this operation is to use moveblock to put the Cairo surface directly into the string going to the picture control.  Everything crashes when I try to do this....(Ideas welcome!)

In conclusion - I don't think I'm going to get it to go any faster, it would be nice if the Labview draw .vis supported antialiasing and gradient fills (the lack of this was the reason I had to use Cairo in the first place), and it would be nice if the 2D picture control op codes / data structures were documented somewhere.

Luke

(I'll knock together a demo in a bit)

[Jester_v01]

Ok - here is cut version of the application, with dlls and wrapper vis.  I'm getting about 35fps.  You can disable / enable sections of the diagram to see what bits are taking time.  Redrawing the background (Draw Axis sub vi) each time is fairly time consuming, but could be speeded up. 

Comments welcome! - dramatic speed ups even more so!

http://dl.dropbox.com/u/710852/Demo.zip

Luke

[andre2]

Hi,

Could you please tell me how you managed to use Cairo Graphics within LabVIEW ?

I tried to download your .zip file but the link is broken.

[ToeCutter]

You could try '2D in 3D' rather than a chuggy old picture control. Take a look at LabVIEW's 'solar system' example, maybe there's a quick way to pipe your ARGB texture onto an object. If you can do that, it's just a case of exchanging a QUAD for the target object and scaling it to the correct size for pixel perfect reproduction.