motion recognition with labview

[pallen]

Hello KupiKupi,

Are these the actual images you're using in your application?

If so, you might want to make some camera and lighting adjustments to get a better image.  I think you might have a difficult time tracking motion with these particular images.

[oliver_sidla]

Hello 'kupikupi',

first of all let me tell you that your problem is really a hard one and not easy to solve. It is to my knowledge still being researched.

Some ideas how you could start:

From your images I see that you have marked the hand with color blobs. Try to segment and recognize those blobs
in the image sequence, get their image coordinates. When you optimize your illumination and camera setup that should be possible.
What you should get from this is a more or less well defined motion path for each of the blobs (maybe you try first with one blob only to make things easier).
Since the motion paths for one and the same gesture are always slightly different (positon in video, speed,  etc.) you need to
match the paths in a flexible way. One idea that I have is that you code the motion path as delta increments in  image coordinates - this
is very similar to the chain codes used for contour coding (look for 'Freeman code'). The idea is that you
convert your absolute image path  positions to relative coordinates (blobs moves 3 pix down and 2 pix left, then moves 4 pix down and 2 pix left, etc..)
and thus make it tranlsation independent.
What I suggest now is that you try some kind of inexact string matching on the motion paths: Code each relative move of the blob as 'character' (a = 1pix left/1 pix down,  b = 1 pix right/1 pix down), etc.) and so represent the motion path as a string. Each gesture should be
transformed to a string which represents it. No two gestures (of the same word) will be expressed as the same string but the strings will be
similar. You can try to match them with an inexact string matching function (look for 'Levenstein algorithmus). 
This is relatively easy to implement and fast.

I don't know how well above procedure will work but it is at least an approach worth trying, in my opintion...

Good luck

Oliver

[kupikupi]

hi  oliver,

    thanks for the suggestion.. i'm not sure if i understood it correctly but i wonder whether it would work on mine as my relative position would not be exactly the same everytime. im acquiring image at 30 frames per second and sampling them at 4 frames per second. it might not be affected by the speed and poisition but the relative coordinates would not always be the same for the same motion, it might deviate a little. please correct me if i'm wrong.

just for a clearer pic of my work

i've managed to recognize still signs by segmenting different colors and getting the position and matching it with the database which works for me.
for the motion part, i separate different signs into a separate folder to be recognized separately. my plan is to plot a graph from the motion trajectory and compare the graph's statistic with a database. that way, the shape of the same signs done by different users will be almost similar and won't affect the position speed or the relative distance. but i'm not too sure that thyis way will be efficient or whether i have enough data to be manipulated.. thanks very much for the input ..


 

[o_sidla]

Hi,

I think you are on the right track.
Certainly the motion paths will always be more or less different (speed of hand, etc.).
You could resample the motion paths into equally spaced small linear pieces, which is simple.
What I meant with the string matching procedure is that you code each linear motion path segment (with a length of 1 - 5 pixels) with a character (vertical = a, 45 deg left = b, etc..) and so get a string which represents each path.
Since paths will never be identical (and so their strings) you need a matching function that
tolerates differences between strings but still gives you the most similar pairs
like (aaaabaaa is very similar to aaaabaaaa etc.). This is exactly what Levenstein does.
You can even use wildcards for matching. In this way I believe it could be possible to match
motion paths based on their structural similarity.

regards

Oliver

[kupikupi]

hi oliver...

     i get wat u mean ... but is it possible that u give me a simple example of how to do it in LabVIEW ? i will try work on it myself ... but now i dont really know how to start.. thanks very much

i've got another question , lets say i plot the motions in a graph... for similar graphs .. what characteristics of the graph do i compare??   

[kupikupi]

hi .. im working on plotting a graph based on the coordinates of the motion .. but i've got a problem, when i plot the graph.. the lines connecting the points does not follow the sequence of the motion, and how can i compare 2 similar graphs?? wat statistics can i use..
and also .. is there anyway to analyse a motion  path based on the coordinates and the graph ?? i used the XY graph to plot the coodinates.

im also wondering if vision builder or motion assistant could provide more help for me ??

oliver,

 about the levenshtein algorithm, do i sample images in such a way that each subsequent images has a fixed distance between them ?? im not so sure how to implement taht algorithm .. i hope u can provide more detail .. thanks very much

[Torba]

Hello kupikupi,

Probably the best function to use for this situation is geometric matching.  This function is similar to pattern matching with the exception that it focuses more on the distinct geometrical characteristics of the image and the template as opposed to direct pixel comparison.  You can teach templates for this function to make comparisons with, and specify what types of geometrical shapes you intend to detect.  This function will then try and locate all of the instances in the new image that contains the shapes that fall within your specifications.  You can then compare these geometrical shapes to the ones that are defined in the templates.

Detecting the shapes as opposed to pattern matching gives you the flexibility to accept different relative locations of the moves that are being captured in the image.  The training interface for geometric matching in VBAI also offers a variety of methods to help filter out the information that you need to extract from the move.

Regards,

Mike T

National Instruments

[kupikupi]

hi ... im workig  on an algorithm to recognize motion .. could anyone please help me have a look at it ?? there's something wrong with it .. by right similar motions should have similar graph shape .. but it doesnt work in this case ... please help me .. thanks. .
i've attached myprogram ... and also the sub vi used.... thanks

Message Edited by kupikupi on 09-13-2006 09:46 AM

[Torba]

Hello kupi kupi,

I'm afraid that I don't understand what you are asking.  Can you provide some more detail on the problem, and where it is occurring?  Do you have any screenshots that can demonstrate the problem?  I can't run your code because I am missing some files.

Thanks,

Mike T

National Instruments

[kupikupi]

hi mike ...

im not sure if i could explain properly ...

this is wat im trying to do ...

im getting a set of images which represents one motion..

i get the centroids and if i plot it on a graph .. i will get the path of the motion ..

so for similar motion .. i will get similar motion path which also means that the shape of the graph will will similar...

im trying to find how much the path differs...

so this is what i did ...

i get the centroid of the motions...
every motion will have a different number of points and different position and length ..... the only thing similar will be the shape of the path ...

that's the only thing i can compare...

so i use spline to get the rest of the points on the graph .. 

i use the same method for the template to be matched... compare the deviation of each point... and set a threshold for mtaching

i think the file u are missing is the sub vi .. which i have attached.. but that's only for reference... u can use the array cluster and connect them to the spline loop... those are the centroid i got from the motion ..

i could u some opinion on how i can compare shape of graphs.. i hope u can help me .. thanks