Access multiple USB Webcams simulaneously

[armmic]

Hi

I have an application whereby I am using 4 cheap (Lattepanda) webcams to simultaneously read product QR codes on a tester in a factory environment.

The flow is as follows:

Operator loads 4 products onto tester

Operator operates physical buttons to initiate test

QR codes on products are scanned in order to determine that they are correct product and to store test results against the unique serial number in the QR code

Test adaptors close (so long as the product fitted is correct)

Test executes

Test Results are stored alongside Serial numbers from QR code.

I am using cheap cameras as I need them to be physically small (due to where they are situated) and cheap - I need approximately 100 of them so I cannot easily budget for £1k+ Keyence or similar. 

I am using the IMAQ pallettes to provide the functionality for opening the cameras, snapping images, interpreting QR codes etc.

My problem is that the images take a long time to collect and it seems that they cannot be taken simultaneously (i.e. a re-entrant vi crashes - guess down to a shared resource somewhere?). Any ideas on how I can access the cameras at the same time?

The user experience for the operator suffers as they press the buttons to initiate a test but the test adaptor takes a couple of seconds to close (usually it is instant). The whole process is time critical - in that the whole cycle time is less than 20 seconds - I don't want 10% of the time to be taken up just performing QR scans as opposed to actual testing.

Any ideas/solutions greatly appreciated!
Thanks

Mike

[LVNinja]

Have you tested create dedicated VIs for image acquisition for each camera, instead of using re-entrant execution and optimize execution when  running in parallel ? 

[armmic]

I haven't - but I'll give it a go. I guess you mean just taking a copy of the vi (with a new name) and attempt to run in parallel?

Thanks

[armmic]

@LVNinja

I've tried your suggestion - didn't work. I looked inside the IMAQ vis but they are just DLL calls so can these be saved off and duplicated too?

[Eric1977]

How about uploading your VI so others can see what is going on. If you can't do that, then an example that exhibits(?) that behavior would be the next best thing.

[Bob_Schor]

About a decade ago, I worked with a colleague who was monitoring animal behavior using IMAQdx.  We had 24 cameras (either by Axis or Basler, I forget which) that were connected via TCP/IP, and were taking 320 x 240 color images at 10 fps (again, I'm not entirely sure of the details).  At one point, we did try to substitute USB cameras, but found it much more difficult to handle multiple USB lines than TCP/IP (as I recall, we had two "concentrators" that gave us 12 "ports" on two IP addresses).

We wrote the code for handling a single camera (there were other sensors, including serial lines from each "station"), then basically "spawned" as many "clones" of this basic code as we had running Stations (we rarely ran all 24 at the same time).  Note that we were not recording continuously from all 24 channels -- we monitored each "station" and only acquired videos from a camera when the station signaled us to capture the video.

We basically stuck with the TCP/IP cameras, as (at the time) handling multiple USB video feeds seemed much more problematical (and we couldn't imagine handling 24 USB cameras simultaneously) ...

Bob Schor

[armmic]

Hi

I've attached a zip file (hope this is acceptable). It contains:

'Take Multiple Images.vi' - this runs 3 vis in parallel and takes benchmark timings

'Take an image.vi' - there are 3 versions of this in an attempt to demonstrate LVNinja's suggestion.

These examples take a little longer to run than my production code as they include opening/closing the session which I wouldn't do every time, and I haven't set the camera mode (I can get away with 640x480 which is a bit faster).

I'm not having any problems with reliability - just with the speed.

I'm considering having a service in the background which periodically takes pictures (and maybe analyses them) and keeps them in a globally accessible place. This way the main code which needs them can just take the most recent picture without having to wait for the acquisition. This may be acceptable as the operator that is placing and removing the product takes a few seconds to do so (so hopefully the images have updated by the time they ready to press the buttons to start the test).

Thanks for all the suggestions...

Mike

[Bob_Schor]

Close, but no cigar (as they used to say).  Seeing this code reminds me of what we probably did when we tried to use 4 USB Webcams instead of the 24 Ethernet cameras 15 years ago.

First question -- are you doing "Snaps" (single images) or "Grabs" (video)?  [I clearly assumed it was "Grabs", much more demanding].

Second question -- are you acquiring multiple images, or just one (from multiple cameras)?

If the answer the Q2 is "multiple", then your "flow" model is wrong.  You want to put the "Snap" in a While Loop, and only worry about timing that loop!  You should see that (all of) the Loops run at the frame rate of the cameras (I'm assuming all the frame rates are identical, and "reasonably slow" (like 30 fps).  I would write a sub-VI that implements a State Machine with the following States:  Open Camera, Snap, Close Camera, Error, and Exit.  You haven't said what you want to do with the Images (from multiple Cameras, and potentially from multiple Snaps), but you might want a parallel Loop as the Consumer of the "Snap" Producer, particularly if you are doing disk I/O.  I would run this sub-VI as a pre-allocated Clone, so that you only have one VI to modify.

Bob Schor

[armmic]

Hi Bob

Thanks for your response.

Q1

I am performing 'Snaps' - this is in order to read a QR code from a product. As my tester can test 4 unit simultaneously I would generally have 4 cameras - 1 pointing at each product.

Before starting a test I want to take an image of each QR code and interpret it - the taking of the images is what is time consuming as it seems I can only do one at a time.

Q2

One image from each camera. The code I posted is just a mockup of my real code as I have stripped out things like camera configuration settings etc. In reality I'm opening once, snapping multiple times, closing once.

Regarding the 'pre allocated clone' idea. As you can see from my code - I even created separate vis (identical) in order to get around the crash I get when using a re-entrant vi - to no avail. I'm guessing that maybe it's the underlying DLLs which cannot be called multiple times simultaneously...

I'm going to try an idea similar to yours with the state machine - leave it looping in the background taking snaps. I'll then just analyse the latest snap. I may even be able to access the images from elsewhere purely by name as I believe that the image wire only carries a reference?

Thanks again!

[Bob_Schor · Accepted solution]

---

@armmic wrote:

Hi Bob

Regarding the 'pre allocated clone' idea. As you can see from my code - I even created separate vis (identical) in order to get around the crash I get when using a re-entrant vi - to no avail. I'm guessing that maybe it's the underlying DLLs which cannot be called multiple times simultaneously...

---

A "pre-allocated clone" should get around the problem.  You allocate, say, 4 clones, one for each Camera/Station, and save the references to the clones in an Array.  You then launch the 4 clones, passing to each the reference (or however you handle the multiple cameras -- in my case, I passed the IP address of the camera assigned to that clone) and away you go!  You do want to use Producer/Consumer patterns to let image processing not materially interfere with image acquisition, but this wasn't a problem for us (fortunately, the behaviors we were trying to "capture" were about 10 seconds, occurring minutes apart, so even with multiple stations, we were only doing "video processing" on (a) the one station we might be "viewing" and (b) the 1-2 stations that might be saving a video.

Bob Schor

Bob Schor