How to average a 3D array into a 2D array in the fastest way?

[ccyang]

Dear folks: I need to calculate the average value through Z axis for each (X,Y) for several 3D matrix. The speed of calculation is very essential since there are many data coming in and each one must be calculated, Thus, I can't use 3 do loops to do this work. Can anyone please help me? Thank you!

[Kevin_Price]

Not sure I'm entirely clear on what you're dealing with.

I'm going to suppose that your X,Y, and Z are just integer indices into a 3D matrix, and the value found at (X,Y,Z) is your measurement.  For conversation's sake, I'll imagine this as a cubic grid of temperature probes in a fluid of interest.

If I'm on the right track, then each value of index Z represents a different X-Y plane.  It sounds like you want to average over Z, i.e., find an average X-Y plane.

If so, then at least half the battle is making sure you build your 3D matrix properly, so a single auto-indexed For loop will index on the Z dimension, extracting each of the XY planes.  You can then easily use a Shift Register to sum up your X-Y planes and then divide by count after the loop for an average.

Then again, maybe that's not even close to what you're after.  If not, can you post again with more detail?

-Kevin P.

[Wiebe@CARYA]

"Kevin Price" <x@no.email> wrote in message news:1161738607098-431978@exchange.ni.com...
Not sure I'm entirely clear on what you're dealing with.
I'm going to suppose that your X,Y, and Z&nbsp;are just integer indices into a 3D matrix, and the value found at (X,Y,Z) is your measurement.&nbsp; For conversation's sake, I'll imagine this as a cubic grid of temperature probes in a fluid of interest.
If I'm on the right track, then each value of index Z represents a different X-Y plane.&nbsp;&nbsp;It sounds like you want to average over Z, i.e., find an average X-Y plane.
If so, then at least half the battle is making sure you build your 3D matrix properly, so a single auto-indexed For loop will index on the Z dimension, extracting each of the XY planes.&nbsp; You can then easily use a Shift Register to sum up your X-Y planes and then divide by count after the loop for an average.
Then again, maybe that's not even close to what you're after.&nbsp; If not, can you post again with more detail?
-Kevin P.



Another tactic might be to flatten the 3d array to one 1d array. If you do this correct (depending on how the 3d array is build up), you should be able to get one subset from the array, average it, and store the result in a new 1d array. Afterwards you can convert the 1d array to a 2d array.


It might be faster, but the only way to find out is to try it out.


Have you thought about avoiding the 3d array?


Or averaging the data each time you add a sample (by keeping the last average and the number of samples, you can add a new sample to the average without averaging all the data)?


Regards,


Wiebe.

[Kevin_Price]

Wiebe@CARYA wrote:
Another tactic might be to flatten the 3d array to one 1d array. If you do this correct (depending on how the 3d array is build up)...
It might be faster, but the only way to find out is to try it out.

I *think* that the 3d build-up that makes Wiebe's idea work would be different than the one that makes auto-indexing work.  A speed trial (including any gyrations required to perform each type of build-up) would be a good idea.   That said, I expect his other ideas might well prove to be faster yet.  Here they are again for emphasis:

Have you thought about avoiding the 3d array?

Or averaging the data each time you add a sample (by keeping the last average and the number of samples, you can add a new sample to the average without averaging all the data)?

-Kevin P.

[ccyang]

Thank you, Kevin and Weibe and friends. I am new to LabView, so your suggestions are very helpful. My problem is that I am taking a lot of pictures in a short time, each shot contains X by Y pixels. In the end, all of the pictures will be stored into a 3D array by X by Y by Z, where Z means Z pictures. Then I need to discard the first one and combine Z-1 pictures into one by averaging the intensities of Z-1 pictures at each coordinate (x,y) into one value and assigned to this coordinate. It sounds simple, but my problem is that the camera needs to be ready for next group of pictures for less than 1 second, and this averaging work must be done as soon as possible. I can not avoid 3D array because the data come from the built-in driver which I can not change. Can anyone help me figure out how to solve this problem, and if possible, send me a simple example? I would reall appreciate for your important help and any of your suggestions. Thank you, folks!

[Kevin_Price]

Are you sure that there isn't a different driver call that would just give you each 2D snapshot one at a time rather than all at once as a 3D matrix?  This would let you build up a cumulative average with each successive frame.  Dunno your device, but it "feels like" an option that ought to be available.  Perhaps you can't get the frame rate you need that way, but don't discount it too fast.  You don't have to snap--calc--snap--calc in series.  You can have parallel loops with a queue for the 2D pic.  One loop does snap--enqueue--snap--enqueue... while the other loop does dequeue--calc--dequeue--calc.

But if you're really really stuck with 3D data, you're at the mercy of the arrangement used by the driver.  It certainly seems logical that it'd be arranged such that Wiebe's idea to reshape into 1D and operate on subsets could work.  One frame probably occupies one contiguous chunk of memory, and each successive frame probably appends additional contiguous chunks. 

(Note to purists: Yeah, I know, if the imaging code is C-like, individual frames wouldn't need to be laid out in consecutive memory.  But the part of the driver that's going to return a by-value LV-compatible 3D array still would.)

-Kevin P.

[ccyang]

Sorry that I did not explain my problem well. the camera is shooting one target by one target. For every target, the camera continues taking the image until the stop signal is sent, then the camera needs to be ready to shoot for next target. Each target comes into the front of the camera very quickly and no target can be missed, which is why the running speed is essential to me. For each target, the camera shoots Z ofl X-by-Y images and simply send this X-by-Y-by-Z images to me. I can not change this part. I need to calculate one X-by-Y image by averaging intensities at each (x,y) through Z axis, and send the result to the user's model for judgement. I can not simply convert 3D array into 1D and average the whole values because each pixel of an image is independent. I tried to wire and index the 3D array into a for loop and sum up itself one Z frame by one Z frame, and divid it by the number of Z. However, the result is incorrect. I wish I explain my problem clearly and someone can give me an example for how to do it correctly and quickly. Thank you a lot!

[Kevin_Price]

Ok, I hope to show an example later but first need to know how the data is arranged in the 3D matrix.  If you were to index a value out of the 3D matrix using "Index Array", you'll have 3 index terminals to wire.  Going from top to bottom, which is X, which is Y, and which is Z?

Also, since speed is a concern, what are your standard sizes for the X and Y dimensions?  What is a reasonable range to expect for Z, the # of frames to average?  From the time you receive the 3D matrix, how soon must you have an averaged result?

-Kevin P.

[ccyang]

Thanks Kevin, your help is great. For my image, X means the number of rows, which is usually 512 or 256. Y means the number of columns, which is usually 5 to 10. Z means the number of images or frames per target, which may really varied from target to target. The numbers of X and Y will be determined before running the camera, and the number of Z will be determined by how big the target is. when the target comes, the camera will start to shoot the images automatically, and it will end automatically when the target leaves. Do you think the information is sufficient? Thanks a lot for your effort.

[ccyang]

Sorry that I forgot to mention the speed need. The time between 2 targets varies between 0.1 to 0.3 seconds, much less than 1 second I mention before.