Real-time face and eye detection in LabView using OpenCV Harr feature classifier

[Klemen]

(check out this link: https://decibel.ni.com/content/blogs/kl3m3n/2014/09/05/point-cloud-library-pcl-and-open-computer-vis...)

Hello,

I have been playing around with the face and eye detection algorithms in OpenCV and have again made a dll library, which can be called in Labview to perform face and eye tracking in real time (on my computer I achieve an average detection time of ~50 ms per loop, which equals ~20 fps using a webcamera with VGA resolution).

The algorithm first detects the faces in a live camera stream and then detects the eyes inside the "detected faces" regions of interest. The algorithm is based on Haar cascade classifier with OpenCV's default training samples.

I am attaching the dll and the source code along with the LabView sample code saved for LV2010. The dll was built using VS2010x86 and OpenCV 2.4.6.

I hope this helps someone in their project.

Be creative.

Best regards,

K

P.S.: Be careful to synchronize the path to the classifier files (can be seen in the source code). The LV algorithm tells you, if the classifiers are not loaded correctly.

P.P.S.: The .dll included detects only one face that corresponds to the biggest object in the scene. If you want multiple face detection, you need remove "CV_HAAR_FIND_BIGGEST_OBJECT" in the source code and rebuild the .dll. Or if you have any problems doing this, leave a comment and I will post the multi-face detection .dll.

[TaxusInSE]

Dear K

I'm tested the programe and then having show error on openCV_core246.dll, i don't corrected that's, Would do you like to help me?

Thank You So much.

Taxus

[Klemen]

Do you have OpenCV installed? You need to install OpenCV 2.4.6:

http://opencv.org/

Then add:

C:\Program Files (x86)\OpenCV 2.4.6\opencv\build\x86\vc10\bin

to  environment variables ("path" under "system variables").

Tell me how it goes.

[TaxusInSE]

Dear K

Yeah,Your recommended That's so work, but program is not detected my eye or face,

How should i do?

Thank you.

[Klemen]

Hello,

your classifiers are not loaded (as the indicator shows). Check the .cpp file and put the classifiers in the folder specified there. You can alternatively rebuild the dll with custom (your default) path.

Best regards,

K

[TaxusInSE]

Hello K,

Now i'm attempting to correctively your code,In fact it's so hard for me but, thank for all

Best regards and Good night (in Thailand)

Taxus

[Klemen]

To be even more clear:

create a folder "C:\Program Files (x86)\OpenCV 2.4.6\opencv\data\haarcascades"

and copy  "haarcascade_frontalface_alt.xml" from your OpenCV data folder to the newly created folder.

and copy  "haarcascade_eye_tree_eyeglasses.xml" from your OpenCV data folder to the newly created folder.

This is if you want to use the .dll. Alternatively you can change the path in the .cpp to match your own OpenCV installation folder. In this case, you have to rebuild the .dll from the source.

Either should work. Tell me how it goes.

Best regards,

K

[TaxusInSE]

Dear K

Finished, i'm tested and extremely work, i follow step by you told and not edit your code,just created a folder for appropriately your directory.

Best regards

Taxus

[Klemen]

Nice to see you've got it working.

Have fun

Best regards,

K

[TaxusInSE]

Thank for all K,and nice to met you.

Best regards

Taxus

[uncleu]

Hello Kelmen.

Your jobs are so impressive.

I have some questions on using labview with opencv. Could you give me some advice?

I'm trying to calculate 'the Center of mass' on each frame by using the dlls.

So I'm using the ImaqGetImagePixelPtr to get image's pointer. However, it returns repeatedly the first frame's pointer out.

How can I get the image's pointer on each frame separately?

and how did you deal with the real-time processing?

Thanks in advance

[Klemen]

Hello,

thank you.

If you have any questions that do not apply to the subject of this post, it is best to post them to the Machine Vision forum here:

http://forums.ni.com/t5/Machine-Vision/bd-p/200

Best regards,

K

[JLHaworth]

Is it possible to use this on a 64-bit machine Windows 7? When I associate the Labbview call .dll function to the location of the face_detection_haar_DLL.dll in the downloaded zipfile, I get an error saying that the dll will not work for 64-bit, because it is built for 32-bit...

[Klemen]

Hello,

yes of course it its possible. I am actually using x64 Windows 7.

Do you have "...\OpenCV 2.4.6\opencv\build\x86\vc10\bin" under path variables? Is your Labview x86?

Best regards,

K

[JLHaworth]

Hello,

I was getting this message when trying to rebuild the dll on my machine:

LINK : fatal error LNK1104: cannot open file 'kernel32.lib

I ended up hard coding the path to this library and all has worked great since!

Thank you for this contribution!

Best,

Josh

[Oskr1709]

Hello! I think this is a great explanation of detecting faces with Labview. There are no info about this. I did face and any object detection previously with openCV and simpleCV, the file that I needed to do this is the one that is with the extension ".xml". The files that you mentioned: "haarcascade_frontalface_alt.xml" and "haarcascade_eye_tree_eyeglasses.xml" I think are the one that I need to do the recognition, right? So, for example, if I do my own ".xml" file for detecting another thing just I need to put this file in the folder you mentioned? I´m new using Labview with image processing so I have to install all the packages before I can try your code. Hope you can help me. Thanks.

[Klemen]

Hello,

yes, you would need to train new classifiers for your object and store them in the appropriate .xml format. Then, you would just need to load the file.

Best regards,

K

[Oskr1709]

Hey! I have the same error of Taxusln: The program can´t start because opencv_core246.dll is missing from your computer... I installed opencv 2.4.6 but It´s the same. I don´t understand when you said:

"Then add:

C:\Program Files (x86)\OpenCV 2.4.6\opencv\build\x86\vc10\bin

to  environment variables ("path" under "system variables")."

Thank you.

[Klemen]

Hello,

assuming you've got Windows 7:

Control panel -> System -> Advanced System Settings -> Environment Variables -> select Path (under System Variables sub-window -> Edit -> add "C:\Program Files (x86)\OpenCV 2.4.6\opencv\build\x86\vc10\bin" (the paths must be seperated by semicolons) -> Log off -> Log on (to save settings).

Best regards,

K

[Oskr1709]

Thanks K! I did it! Excelent program!!! Now I´m going to try to modify it to find objects!!

[Klemen]

Hello,

glad you've got it working.

When you modify the example to detect other objects, you are welcome to share your results/procedure! I was planning to do this also, but currently have no time... And tips ad help will be much appreciated.

Best regards,

K

[Shenglan]

Dear K!

Your programme works perfectly on my laptop. My groupmate and I would like to do some modification so that we could detect people's mouth and output the width of it as well. However we have difficulties in compiling the dll from the source code. When we try to get your source code compiled, we get a .dll file that is not working as yours. How did you manage to compile this .dll file from the source code? Btw we are using visual studio 2012x86.

[Klemen]

Hello,

the easiest way (for me) is to use cmake:

http://www.cmake.org/

In the past I have been manually adding the libraries and dependencies, but I switched to cmake, which saved me a lot of time. I use only cmake now.

I plan to dedicate a post on how to build the pcl and opencv project using cmake. Maybe in a couple of days. It's really simple...

You can search the web also - you will find a lot of examples.

Best regards,

K

[Klemen]

Hello,

check out this post on how to create the Visual Studio project from the source file:

https://decibel.ni.com/content/blogs/kl3m3n/2014/04/22/create-a-visual-studio-project-from-source-fi...

Best regards,

K

[candy0821]

Hello,
I have some questions about this project.
When I run this program,it will be delayed.The delay time around one second.The condition is normal?
Do you see a way of getting round the problem?

thank you

[Klemen]

Hello,

of course, the initialization part introduces some delay. In my case, the delay to initialize the camera is ~300 ms and the delay to load the classifier files ~150 ms.

I don't see a way to reduce the delay. Why do you need to reduce it anyway? Your initialization should occur only once in your application...

Best regards,

K

[candy0821]

Ok,I got it.
When the program been runned,eye or face are detected all the time??Is it need to check the STOP button and then can detecte the eye and face??

And I want to ask another qusetion about detection.
Some time the program can't detecte my eye or face,it have to wait a period of time.

What cause this condition?

thank you.

[Klemen]

Hello,

yes this is a real-time application. Use STOP only when quitting the program.

I have no problem with the face and eye detection when looking at the camera. This example uses the default classifiers from OpenCV. If you want to improve the accuracy for your setup, you can train new classifiers. Also be careful of the illumination - for example, direct sunlight from one side can cause problems, especially of the eye detection algorithm...

Best regards,

K

[nilohdez1967]

Hi GURU Klemen:

Thanks very much for your contribution. I tested it in windowsXP and work perfect. Now it's the time to send the face coordinates to my microcontroller and control the camera with a pan&tilt head and surprise my work colleges

[testerpce]

Hello,

I am using labview 2013 and it says that I need to load a cpl file and not not a cpp file. If I use the cpp file then it is showing me a whole lot of errors. Can you please help me out on this one?

Sayambhu

[Klemen]

Hello,

what exactly is a .cpl file? Can you post the screenshot of your error?

Best regards,

K

[testerpce]

No it works now! Yay! Turns out that I had to download all of the driver files(All 6 GBs of them). And then it works perfectly after adjusting the path of the xml files and all.
But now I have one question:
Suppose I wanted to make a standalone embedded system using the National Instruments myRIO platform to detect the face using the camera input of the myRIO and then control a servo accordingly using that. How would I do that?

I mean how would I make a standalone face detection vi which can be implemented on myRIO without having to use something to  load opencv dependencies from other locations on my computer?

[ayansengupta17]

As this requires a dll file, can we implement it on any NI hardware like NI myRIO?? If Yes, how?
Thanks in advance

[ayansengupta17]

That issue has been solved. #Driver issues.

Thanks

[s10059040]

Hello,

I have some questions about this project,

i want to modify your source code,

but i don't know how to rebuild .dll,

can you help me?

thank you!

[Klemen]

Where exactly do you encounter problems?

You need to rebuild the .dll from the source code after setting up the additional dependencies and libraries. Use cmake (there is a step by step tutorial on this blog also)...

Best regards,

K

[candy0821]

Hello,

I  implement it on myroi ,

but some error happened 

can you help me?

Thank you!

[Klemen]

Hello,

I am not familiar with RIO. Seems the .dll  is missing on your RT target.

Sorry I cannot help you more. Maybe you could try the forum, asking how to add custom .dll's to your target.

Best regards,

K

[s10059040]

Hello,

I thought that you have implement it on the myRIO,

can you help me?

If can,how I can do it ?

Thanks in advance

[muks]

Thanks for this. I am going to check this out and let you know

[candy0821]

Excuse me,

Becsuse we need to modify some code and then debug,but the cpp file you put can't compile.

Can you send me your "face_detection_haar_DLL" project.
thanks you very much.

[Klemen]

Please follow the instructions here to succesfully compile the source code (for OpenCV):

https://decibel.ni.com/content/blogs/kl3m3n/2014/04/22/create-a-visual-studio-project-from-source-fi...

Best regards,

K

[slads]

hi i, m using labview 2013 and i m running it on win 7 vm ware ....

i added the path of opencv core246.dll but it is still giving error  the program cant run opencv core246 dll missing and i m using open cv 2.4.10 ....

plz help me regarding this

[sk_a]

Hi,

I have installed the opencv2.4.8 and added the environment veriable also, still the program says can´t start because opencv_core246.dll is missing.

do i need to instell opencv2.4.6 for program to run or can program run with opencv2.4.8?

regards

Sk

[Klemen]

Hello,

if you are using a different version of OpenCV, you need to rebuild the code (from source) for your version.

Best regards,

K

[slads]

thanks for reply ,,

[slads]

hi I need opencv 2.4.6 bcause now this version is out ...or

opencv_core246.dll and opencv_imgproc246.dll I need them if u have these

things I need them pls help me regarding this

[Klemen]

Hello,

like I've said, the source code is also included, which you can rebuild with your version of OpenCV.

Or if you don't know how or you don't want to do this for any other reason, see:

https://decibel.ni.com/content/blogs/kl3m3n/2014/09/05/point-cloud-library-pcl-and-open-computer-vis...

You can find the example of face detection there (built with OpenCV 2.4.9).

Hope this helps.

Best regards,

K

[slads]

hello

Thanks now the code is working[?] . you r doing great work .

[Woody247]

Hi Klemen,

I went through all your help comments and found it very useful, I have rebuilt the .dll, as per your instruction, with my location for the .xml files.

When I run in LV2014, openCV 2.4.10, windows 8.1 x64 OS, but x86 LV and compiled dll I get the error code shown in the screen shot. (it is from the first call library)

The cascade classifiers are loading so not sure what is going wrong. I know the error code is pretty generic, is it anything you have seen before? Sorry this is the first time I've used call dll.

Any help appreciated, I think this is exactly what I need for a university project.

Many thanks

edit: also getting this message when restarting labview after trying to run the code. "The last time you ran LabVIEW, internal warning 0x50CBD7C1 occurred in ExtFuncRunTime.cpp."

[Klemen]

Hello,

no, I cannot say what the error is. Which version of Visual Studio are you using?

Best regards,

K