LabVIEW take pictures, put into array

[mb056145]

Hi,

I've written a program that works extremely well for my Pulnix TM-6740cl camera. However, it does have one small problem: I run one while loop to take the pictures, then another to decode the raw bayer and save them to the computer in order to improve the speed at which the pictures are taken (I'm looking for approximately 200 FPS, and if I put everything into 1 while loop, then the camera only takes pictures at about 50 FPS maximum.) I'm new at LabVIEW, and I was wondering if anyone had a solution to my problem. I thought the best solution was to put the pictures that I'm taking into an array in the first while loop, and then read from that array in the second while loop. However, there are two problems with this: 1, I have no idea how to make an array since I'm new to LabVIEW, and 2, I still need the program to run quickly (although I'm pretty sure the array will not impact the speed much.) Thanks!

[mb056145]

Here's a copy of my LabVIEW program, if it helps...

[Matt_W1]

How many frames are you acquiring. I'm guessing you have 640*480@8 bpp@200Hz so ~60 megs per second. A typical LabVIEW setup can only allocated about 1.5 gigs of memory, so you can't record more than about 24 seconds of video before you run out of memory. You can potentially get 3GB of memory (4 on vista64), with some tweaking of the system, but that would only get you another ~24 seconds.

You can use "find examples" under help to get an idea on how arrays work, or better yet read the "LV_Fundamentals.pdf" in the LabVIEW manuals directory. One important thing to keep in mind is that unlike most LabVIEW data the IMAQ images are reference based instead of value based. I think the fundamentals pdf explains the difference.

Warning: I've only used the IMAQdx commands, so my interpretation of the IMAQ setup may be a bit off.

If you only need a few seconds of video.
The simplest method would be to grab a sequence instead of continuously. Basically just pass an array of IMAQ images in (make sure each has a unique name), then take the array of images out and process over that. I'm unsure whether it's possible to see what the camera sees while doing a sequence grab though (grab acquire or copy buffer in a seperate loop may work for that).

If you only need a few seconds of video and want to see what the camera sees.
You could do what you originally said. You'll need to make a new image for each frame, you can set the output of the while loop to index mode to get an array.

If you want to capture a stream of any length (drive space permitting).
This is a bit more complicated but can probably be done (it depends on the speed of you computer and hard drive, relative to the amount of data your capturing). I doubt you can save pngs faster than you can capture frames so those are out. I use avi files with the huffyuv codec in ffdshow-tryouts (a very fast loss less codec), but getting that to work with the IMAQ avi functions is tricky. I could try to explain if you want to go this route, but be aware if others need to read the files, then they'll need compatible codecs (ffdshow on windows, perian on mac) or player (vlc on almost anything). You'll also need to record data in a separate loop from the one acquiring frames (since saving the first few frames can go too slow) so you would probably use a queue for that. Hard drive speed could be an issue, I get ~15 mb/s for 1024*768@30Hz with huffyuv, assuming a direct scaling yours should be ~40 mb/s which should be doable on a decent modern hard drive that's dedicated to the task. Without compression your video in color is about 180 mb/s, which would require a raid array to stream data fast enough. I usually recompress my videos in XviD when I'm done since I don't need them to be loss less, it shinks the file size to less than a tenth, and is far more common a codec for other people to have.

I modified your code to capture with a sequence and an array. But since I only have a IMAQdx setup and can't test them so there's a very good chance I messed something up. If you try to capture too much video, you'll run out of memory.


Message Edited by Matt W on 06-24-2008 07:02 PM

Message Edited by Matt W on 06-24-2008 07:04 PM

[mb056145]

Thanks Matt,

Like you said, the examples didn't work. However, your explanation of the memory issues was extremely useful. I've been looking for a good explanation for that for a while. I've actually put together a couple of programs that work, now; however, I need to tweak them a little before they're fully operational. For the most part, they run the camera at roughly 130FPS when I scale the image size down to half its original size (320X240). The images of something moving in front of the camera are also slightly blurry.

The last things I need to tweak on my program is to make the images clearer, and collect them at 200FPS and at full resolution. I'm 99% sure that my computer is capable of running at this speed since I have 1 gigabyte of RAM, >30 gigs of hard drive space, a hard drive that writes at 78 megabytes/second, and an AMD Athlon dual core 3800+ processor. If this is true, then I only have to change the program. I have two examples that I've listed below, if you want to take a look at them. Once again, thanks for your help!

[Matt_W1]

In the "Sequence To Save AVIz" your measuring the frame rate of saving the files, not of capturing. The "IMAQ Sequence.vi" is where all the capturing is done. So the video should be at full speed (I'm only familiar with IMAQdx so there maybe a fps setting that needs to be set to get 200 fps). The camera captures everything to ram then you save it to PNGs. So unless you need to capture longer video (which as I said is a lot more tricky), or need a faster saving speed (to capture another event that's about to occur), you should have the basics of what you need already. The other one looks like it should be doing about the same thing. Assuming the sequence capture is good enough then you don't need to scale the images down to capture at 200FPS, since it's already doing that. As for the blurryness it could be a lot of things, the scaling down (which shouldn't be needed), the lens is out of focus, motion blur (which means you need to shorten the shutter time, you might need to increase lighting to compensent), or maybe you have a smudge on the lens.

As for hardware.

With Sequence capture
Everything is done in ram so harddrive speed isn't really that important. Ram size can be important if you need longer captures (To take advantage of more than 2 gigs of pyhsical ram requires some changes to windows and the sequence capture setup).

With Continuous capture.
You'll probably want a second harddrive. My C: drive does 100 MB/s at the begining to 50 MB/s at the end (the average is 84). The random access time is on average 11.4 ms, so each time the drive has to seek (once due to fragmentation or twice due to windows using the drive and going back to recording video), I loss about (11.4 ms * 84 MB/s) 1 MB/s of bandwidth. For instance after a quick check of my C: drive for data access while doing nothing in windows I had 23 events during one busy second. So I'd loss 23 * 2 * 11.4ms * 84 MB/s = 44 mb/s of bandwidth just due to drive seeking during that second. There's more bandwith lost to to time to reading the bytes as well. It isn't really this bad since a lot of stuff is cached, and the accesses may go directly to the next which only cost one seek time instead of two. But it works out for me that I can't stream my 15 MB/s video to my C drive due to fragmentation and all the other work it's doing, but my 😧 drive will easily handle it.

The requirements for CPU speed depend on how your compressing the video. With no compression (which I'm not sure how to setup with the avi functions), you'd get by with almost no cpu power (you'll need a lot of harddrive bandwidth). The huffyuv codec is extremely fast and uses very little cpu power (and compresses by a third). Most other codecs were too slow to stream video for my setup (which is faster than yours), the only one that almost worked was xvid in real-time mode (the files were a order of magnitude smaller, but a lossy format wasn't acceptable for some of my uses). With the expection of xvid most of codecs produced poor quality videos.

With ram, size is most important, since it determines how much you can buffer incase something has a hiccup. I'm sure speed can be a factor with very high video bandwidth requirements.

Matt

[mb056145]

Hi Matt,

Thanks again for your help. Thanks to your last post, I've been able to get closer to getting this camera to work properly. I've determined that the RAM is, in fact, not the problem. I've also determined that my program is also definitely not the problem, and that my cpu is not the problem. However, there are a few questionmarks that I still have.

For one, I can't figure out if the program is compressing the pictures or not. It is likely compressing the pictures because they are all under 300 kb in file size, as opposed to being megabytes in size, but I don't know how to check that out for sure.

Also, when I configure the camera in MAX, and I play live video from the camera through MAX in full resolution, I see at the bottom-right corner of MAX that the frames per second is only about 77FPS. The camera is rated at 200FPS at full resolution. I called the camera company, and they said that the camera was fine, and that the problem was elsewhere. To clarify, the MAX frames per second that I'm reading in MAX state: "77FPS displayed/77 FPS recieved." This means that somewhere in between the camera cord (I checked that, too, and it is the proper type of cord for the camera) and MAX, there is something that is slowing the frames per second. It could still possibly be the cpu; however, I tested the CPU using task manager, and while the camera capture or my program ran the graph of % cup usage didn't even budge except for saving the files to my computer, which as you could see in my above program (the AVI one,) is in a separate loop than the image capture itself. This is a very perplexing problem, but any help/ideas will be greatly appreciated! Thanks!

[Matt_W1]

If all of the pictures are exactly the same size independent of what the picture was of then they are not compressed. It looks like you're using PNG compression which is loss less (like bmp but with compression unlike bmp). PNG doesn't compress normal pictures very well (but jpeg does) but it does compress simple stuff (like the image of a LabVIEW block diagram) very well. You could try jpeg if you want something smaller and a lossy format is good enough (it is in most cases). You could also save to an avi file if you want a video you can play back in a media player.

Max may be slowing down the frame rate to display every image. You could try timing the amount of time the sequence call takes. Then calculating a rough fps from that. Video capture is mostly just memory copying which is handled by the DMA controller and not the CPU. So if your only doing capturing then the cpu should be doing very little work. The video compression doesn't happen until all of the capturing has been finished, so capturing and saving wont run in parallel (changing that is part of the complication with saving a continuous video). This is due to a data flow dependency of the saving loop waiting on the images out of the IMAQ sequence node (node and boxes don't start until all of their inputs have data). You can use the highlight execution button then running a test to see how that works.

Matt