Help on Perspective correction, Image stitching and then pattern matching.

[moderator1983]

Hello All,

Being a newbie in LabVIEW, I don't know much about machine vision.

For my automation project, i need to stitch pictures (from 4 cameras, 90 degrees apart). Do the perspective correction, and then form a label (which is actually sticked on the jar).

Please share your experience on similar kind of project. And help me in proceeding in the project development.

Also find the attached pictures taken from the camera.

P.S: Because of budget constraint, I'm using very basic USB cameras available, and attached picture is the actual picture.

[ChristopheC · Accepted solution]

To be able to correct the image for distortion, you first need to calibrate it. Assuming your camera setting is constant (i.e. the bottle always at the same place with respect to the cameras, I would first stick a calibration grid around the bottle, and then take some images of this grid with all four cameras.

We install a calibration grid you can print: C:\Program Files (x86)\National Instruments\Vision\Documentation\CalibrationGrid.pdf

This example will then show how you can learn the calibration using the grid images, and then correct images you would have taken without the grid.

<National Instruments>\LabVIEW 2010\examples\Vision\2. Functions\Calibration\Nonlinear Calibration Example.llb\Nonlinear Calibration Example.vi

Note that you need to recalibrate the system if anything changes (lens focus, camera or bottle position.

After that step, you should have 4 "straight" images that have been corrected for distortion and that can be used for stitching.

For the stitching, unfortunately, Vision Development Module does not include an out of the box stiching function.

If this is acceptable, you can use some third party software that provide stitching functionality, such as Autostitch, Hugin, Ptgui, Panorama Tools, Microsoft Research Image Composite Editor, CleVR Stitcher or Adobe Systems's Photoshop.

If you need to do it all in LabVIEW, I can point you to some of the tools provided with the library that you can use to create your own stitching algorithm. This part is definitely the most challenging one of your project.

The image stitching process can be divided into three main components - image calibration, registration and blending. We already talked about calibration.

- Image registration involves matching features in your images to be able to align them. You can use tools like pattern matching and edge detection for that purpose. Look at the LabVIEW vision examples for how to code them in LabVIEW.

- Image blending involves processing the images to adjust for the difference in brightness of the images acquired from multiple cameras.

Histogram analysis of the different images, and applying a lookup table are simple tools that you could use for that purpose.

Images can then be blended together with functions like IMAQ ImageToImage.

This thread also talks about stitching:

http://forums.ni.com/t5/Machine-Vision/Image-stitching/m-p/860414

Hope this helps getting you started. Good luck with your project.

-Christophe

[moderator1983]

Dear Christophe,

Thank you sooooo much for taking out your valuable time and for the reply.

Thanx a lot again.