Depth map from stereo pair

[JasonC1017]

I have a stereo pair image (taken with a stereopair camera - right and left images next to each other that, when used with a stereo viewer makes them look 3D - think of those old wooden viewers from long ago or viewmaster from your childhood).  Has anyone generated code or knows how to make a depth map from the stereo pair?  I can put in objects of known depths, to calibrate (or am open to any other suggestions on how/what is needed to calibrate).  Basically, I want to be able to take an image with the stereo pair camera and get measurement of depth.  Would be OK if the depth map output is something like pixel intensity (the brighter the pixel, the closer to the camera, etc).  Any suggestions would be greatly appreciate.

Thanks!

Jason

[BruceAmmons]

The general idea is to take small pieces from one image and use correlation to find a match in the other image.  You only need to search in the local neighborhood, and if the cameras are side by side you only need to do a horizontal correlation.  I haven't seen any code to do it, but it wouldn't be too difficult.  There are probably some tricks to make it fast, but if you don't mind slow it would be fairly easy.

I would start by using an edge finding filter to locate the details in each image, then shift the image horizontally one pixel at a time, multiplying it by the other image and storing the result.  Use a local summing filter on each image to get the correlation value, then for each pixel you find the image where the correlation is the strongest.  The index of the image is the intensity.

The problem is areas with very little detail.  You would have to flag the areas with no correlation values and fill them in using surrounding values.

Okay, maybe it isn't that easy.  I would be interested in seeing the details of a working algorithm.

Bruce

[Hornless.Rhino]

The hardest bit will be linking the points in one image with those in the other image (for example the middle of the left eye of a teddy bear would need to be found in both images).  (See the above post for more on this)

Assuming you know some basic camera information there are simple formulas which can be used to get the depth of each real object point.

Then you can just build your own depth map using an iterative process.

[JasonC1017]

Thank you both for your responses.  The challenge definitely is about finding the same point in both images to know the pixel delta to calculate depth. 

[BruceAmmons]

If you post some image pairs, I would be interested in trying some ideas on them.  I wouldn't be able to do a complete algorithm, but I could get you close.

Bruce

[JasonC1017]

Bruce,

Thank you for your willingness to help.  Greatly appreciated.  While I'm still working on generating mine, I pulled some free ones from the web.  Let me know your thoughts on these.  THANKS!!!

[BruceAmmons]

Jason,

This is the best version I could come up with, and it isn't that great.

Here's the algorithm I used:

Shift the right image by different amounts to be on top of the left image.  Zero is directly on top, negative is left, positive is right.  Loop through all the different shift values.

At each position, subtract the two images using absolute subtraction.  Parts that match exactly will be zero, and non-matching parts will be non-zero.  Use a smoothing filter to average pixels with the surrounding pixels.

For each pixel, extract the value from each shifted image.  Output the index of the smallest value, which is the best match.

It works okay, but areas with low contrast don't work at all.  Tweaking the size of the filter can make a difference, also.  You can at least see the shape of the flowers separated from the background.

One possibility to improve it would be to iteratively reduce the size of the smoothing filter.  Start with seven, then five, etc.  Use some sort of logic to fill in the background spaces using the larger filter, and get the detail from the smaller filters.

Good luck.

Bruce

[JasonC1017]

Bruce,

THANK YOU!  This gets me a lot closer.  I really appreciate your time and effort to help and welcome any additional thoughts you may have.  You  definitely deserve "KUDOS" from me for all of your hard work.

Jason

[Mahmoud82]

Hello,

I have NI 1762 Smart Camera and I can connect this camera to vision builder software then I can take pictures. I am wondering if I can take picture using Matlab.

 My case, I have two cameras (right and left camera) and want to take images in real time using Matlab. Is that possible?

Hello,I have NI 1762 Smart Camera and I can connect this camera to vision builder software then I can take pictures. I am wondering if I can take picture using Matlab.
 My case, I have two cameras (right and left camera) and want to take images in real time using Matlab. Is that possible?

[Justin__D]

Hi Mahmoud82,

Since your question is about a different topic, it would be best to either start a new discussion thread or find a thread related to your question.  We like to keep each discussion limited to one specific topic.

As for your question, you will still need to have some sort of LabVIEW interface to acquire the images using the IMAQ driver.  You could then import the images into your script after they have been acquired.