Seeing a Soccer ball

[LockheedJoe]

I would like to be able to lock on the Soccer Ball with the camera, I've tried using the vision assistant to build a script to find it but it didn't work. Does anyone have any other idea's, thoughts, Vi's, that might help me to learn how to track a soccer ball?

[OliviaC]

Hi Joe,

What do you mean when you say your vision script doesn't work? Cannot it not find the ball? Cannot it not follow the ball? Which steps are using to find the ball?

If you are using LabVIEW, to process the images you are acquiring, you need to use the Vision Assistant (or the Vision VI's which are the same as the steps in the Vision Assistant).

If you want an idea of what steps are needed to track an object take a look at the Vision Processing code in the Robot Main VI. In there, open the VI for Find Circular Target (it's the VI with the text Find and a circle) and take a look at the code in there. As an overview, what this VI does is takes an image from the camera and examines the luminance (or brightness) features of the image by essentially changing it to a grayscale image. This is useful when looking for the target because it has areas of high contrast. From this grayscale image, we search for anything in the image that fits a template of an ellipse. From all (if any) ellipses found in the image, the VI tries to determine which ellipse corresponds to the target. This done in the Normalized Score VI and if you open that up, you'll see that each ellipse found in the image is given a score based on how well it fits the desired object (in this case the target) we're looking for.

To track the soccer ball, you'll need to do something similar. The added challenge will be that the soccer ball will look slightly different based on which angle/side you're looking at and you'll need to think of a way for your code to identify a characteristic of the soccer ball that allows the robot to know it's the soccer ball and not some other round object.

~Olivia

NI FIRST Support

[LockheedJoe]

I apologize for not being clear about "didn't work". The vision assistant did build a script, ouside of the vision assistant I couldn't track the ball.

On the robot, we started using the Find Circular target.vi (in the Vision.vi) which would find the target on the wall and will occasionally (inconsistanly) track on the ball. It would track on a white round circle on a medium brown piece of wood (our ball simulation) pretty well.

We then changed the drop down under one of the Vi's that that said "ellipese" to "Circle" and re-did the broken wiring. It tracked a spot on the wall better than the ball or our ball simulation. It would not track the ball. We tried breaking out color planes (Red, Green, Blue) as well as hue, lumineces, etc.(each of the drop downs). We tried comparing various outputs. (IMAQ Compare.vi)

Anything else we might try tonight when I go back to the Robot?

Thanks

Joe V

[OliviaC]

Hi Joe,

Did you change anything else to the Find Circular Target VI to make it track the soccer ball?

Keep in mind that this code was optimized to find the target, so it's looking for an ellipse of a certain size (min/max radius). Have you change any of these parameters to fit an image of the soccer ball?

Also, when the IMAQ Detect VI finds an object, it gives the found object a 'score'. This score relates to how well the found object(s) fits the template it's using. The reason your robot might be following a spot on the wall instead of the ball is because the spot on the wall 'scored' higher. Take a look at the output of the IMAQ Detect VI (you can do this in Vision Assistant as well) to see what the scores are for the objects it finds. From this, you might be able to figure out why it thinks that it should track a spot on the wall.

~Olivia

NI FIRST Support

[LockheedJoe]

We have been trying different input parameters for the various vi's. I think that is what we're going to keep trying to do. I beleive the key is there, we just have to find the right combination of parameters.

I was also thinking of trying the "IMAQ find circle.vi" or some vi like that to try to find a small white spot of ball. For teleop mode this might not be good with a bunch of robots running around, but for autonomous it might work.

Thanks, I appreciate your help.

[Greg_McKaskle2]

Tracking a ball using an edge detection algorithm is not likely to be very successful. The ball is 3D, and while human vision can easily identify a circle around the ball, that is in part because you know it has a circular cross section and because your are being very adaptive as you follow the edge. If you measure the line profile at different points around the ball, the biggest issue you'll probably run into is beneath the ball where any shadow caused by the ball darkens both the ball and the floor. This makes the edge transition much weaker, and will likely not be identified as an edge, thus preventing the circle fitting from giving a high score to the shape.

I'd recommend building a script that thresholds based on either the color of the carpet or on the luminance plane. Think of it this way. The light colored ball sitting on top of the darker carpet can be thresholded to produce a hole in the carpet. The hole is pretty circular, depending on the shadow cast by the ball. Next you can invert the binary image, remove objects that touch the edge of the screen, probably remove small particles, and you'll have something that you can do a particle analysis on.

One of the more interesting values in the particle analysis report is the Heywood circularity factor. You may want to read about it and the other particle measurements in the Vision Concepts manual located in the Start>>National Instruments>>Vision>>Documentation>>NI Vision folder. You can likely use this parameter perhaps along with some others to robustly identify circular holes in the carpet.

I hope this helps a bit.

Greg McKaskle

[LockheedJoe]

Thank you for your suggestion and information. Thresholds/luminace plane/image inversion a shot. I am a compter vision noob so a lot of the stuff you said was over my head. When you say to remove the small particles, I'm assuming there's something in the vision assistant that says "remove particals - small"? And once I have something to do partical analysis on, what do I do to it?

I will read the information in the start>>na...>>NI Vision folder. One thing I wonder is if there is enough processing time in the robot to do all of that. And we only really have 2 weeks to do it.

[OliviaC]

Hi Joe,

When you threshold an image, it creates a binary image. All pixels that are within the threshold (pixel values) that you have specified turn red and all the that are outside of the limits turn black. You are now left with a black image with red blobs or particles. By removing the small particles, you focus your attention to larger objects/blobs/particles in your image that have the potential to fit the shape of the ball. To remove the small particles you'll want to use in the Vision Assistant is Advanced Morphology and in that step you can select Remove small objects.

After using a filter and removing the small particles, you should be left with some larger looking blobs. To determine if any of them have the potential to be the ball, you can use a Circle Detection or Shape Matching step to see if any of the blobs fit the shape of a circle.

As for your last question about whether there is enough processing time to do all this, the answer depends on what else you have going on in your code and what image size you choose to use. The larger the image, the longer it takes to process it. After running a few tests, if I used the smallest image size and modified the default code to do run the image processing steps discribed above. These additional steps added only a few milliseconds.

~Olivia

[Drucifur]

Hi Olivia,

Where can I find the documentation on all this?

Thanks,

Drew

[OliviaC]

Hi Drew,

If you want some documentation on what each of the steps available in Vision Assistant do, you can open up the context help (press Ctrl+h). When you click on a palette it gives a brief discription of what each of the steps do.

If you want some more detailed documentation on vision concepts in general (such as, what is particle analysis?), you can find a NI Vision Concept Help file at Program Files/National Instruments/Vision/Documentation.

Hope this helps!

~Olivia

NI FIRST Support