video editing in labview 3D Array

[younes_hajjar]

I need the value of gray for each pixel of 300 frames

thanks

[altenbach]

Ok, so you have a 400x500x300 3D U8 array instead, that's already 60MB of contiguous memory. A convolution typically requires DBL, so this would be 8x more, so make sure you have sufficient RAM and LabVIEW 64bit. I would recommend to leave it at U8 and only convert frames and lines to DBL, re-using the same memory for each iteration.

As I said, all you probably need is a 3D comvolution with a simple 3D gaussian kernel, for example. You can iterate over each frame and then over each line in time. DBL 2D convolution for each frame is already built into LabVIEW (use output option: size=x) and a simple gaussian kernel (e.g. 3x3 would be:

1 2 1

2 4 2

1 2 1

Or any of the higher kernels found on the internet (5x5, 7x7, etc.) while the convolution for each pixel line in time is a 1D convolution with a simple 1D kernel..

With a bit of skill, you can probably implement an efficient U8 version (direct instead of FFT). You can just iterate over all elements, get the surrounding 3D subset, process and generate the output element.

Once you attach a VI with typical data, I can probably cook something up. Attach it with fewer frames (e.g. 10) to keep the file size reasonable.

Converting the 3D U8 array back to a video is up to you. You already know how do do it in the other (video  >>> 3D U8 array) direction, so I am sure you have the tools.

[Bob_Schor]

What is your Frame Rate?  If you have 300 frames and your Frame Rate is 30 fps, then your Video will be 10 seconds long.

Think about a 1D signal of 10 seconds that you sample at 30 Hz -- you'll have 300 samples.  If you want to apply a filter to the signal, you probably want to use a "moving window" that replaces the "point in the center" by the average of points on both sides of a central point (for example, replacing point 5 by the average of points 1..9, point 6 with avg (2..10), and so on).  [Such a "box-car" averager is a kind of low-pass filter].  Is this what you want to do with your Images?  Do you want to "smooth out" (low-pass filter) any motion artifacts in your images?

I also forgot to ask -- how many dimensions do your pixels have?  If you are using Grey Scale images, the answer is simple, they are "shades of Grey", or 1D.  But if you are working with color, how do you plan on averaging?  

Bob Schor

[altenbach]

---

@Bob_Schor wrote:

I also forgot to ask -- how many dimensions do your pixels have? 

---

If you look at the VI attached earlier, the 3D array is U8.

[younes_hajjar]

hello everyone,

I have a question regarding the video processing in Labview, is there a way that I can edit the video as colored, until now I only work with white and balck . although the orgial video is colored I can unfortunately only edit as grayscale (U8). and the output is only as grayscale (U8).

thanks in advance

[Bob_Schor]

You want to edit a Video (what format?  what codex?) using LabVIEW (what Version (year and bitness), what Toolkits and Drivers, what licenses)?  What do you mean by "edit" (recolor?  do fancy "transitions" similar to what professional Video Editing software can do)?  Why LabVIEW, rather than a commercial Video Editing routine?

Have you experience and/or training in IMAQdx?  Have you a "reasonable" amount of experience with LabVIEW? (I had been doing LabVIEW programming for several years before a colleague came to me to work with him on a Video capture routine, and it was a challenge to learn).

One way to answer the above questions would be to attach some of your code showing how you work with grayscale videos.  If it is only a question of video format, that should be relatively simple (Famous Last Words) to answer.  But you failed to attach anything to help us to help you.

Bob Schor

[altenbach]

I agree with Bob that you don't provide sufficient information. There are millions of way to "process" a video, some are easy and some are harder. Do you want to simply process each frame separately or do you also need to e.g. process in time.

Color (picture, video) is just three combined "gray" levels, representing red, green, and blue. If red=green=blue, you get gray. Typical is 24bit color and you get any possible of 2^24 shades, eight bits for each color. If you know how to process 8bit greyscale, you'll just do it 3 times, one for each color channel.

I assume this related to your earlier discussion. Maybe you should have kept it all in one place.

[younes_hajjar]

I have used the following vI to edit the video.
problem is when I use the VI ffmpeg_Video_read_no_IMAQ, I get colored 2D array as output, with the ffmpeg_Video_read I get 3D array but as grayscale as output.

yes those belong to the previous questions, you can put them all together.

i am currently working on a project where i have to edit a video (3D array) in labview, the editing consists of two parts (spatial and temporal). the spatial filtering was successful as i had only worked with 2D array.
for the temporal filtering I have to apply a temporal filtering for each pixel where it is just hard because I work with 3D array.
does anyone know a literature or course for video processing in labview ?
for further recommendation how the video processing works  i would be grateful

Thanks
I used Labview 2022 Q3

[altenbach]

You have the three U8 RGB components and merge them into color inside your Rube Goldberg FOR loop (see picture. Yes there are much simpler ways than doing the silly "array-to-cluster-unbundle" dance.. Just process each of the three elements according to your needs before combining them into the U32 color array..

For example, the following with also turn a 1D U8 array of R,G,B,R,G,B,R,G,B... into a 1D U32 array of 0RGB, 0RGB, 0RGB, etc.

(Arguably simpler. You don't even need to measure the size!)

[altenbach]

Here is a quick example to do a 3D convolution of three 3D U8 arrays (R, G, B) using an approximate 3x3x3 Gaussian Kernel. You can easily substitute your own processing of each color plane, whatever, your definition of "processing" is. 😄

Note that this is just a quick draft and probably needs some tweaks. Many things could be optimized, especially if the dataset is much larger. Still, it should give you some ideas. Just run it and use the slide to look at the various frames. I am sure you can figure out where to substitute your own 3D U8 arrays, e.g. as obtained from your interlaced data as explained earlier.