Student Projects
Contact Information
University: Fudan University (China)
Team Member(s): Huang Xinyuan, Zhu Dan, Liu Zhidong, Liu Yang
Faculty Advisors: Li Dan
Email Address: 08300720023@fudan.edu.cn 08300720020@fudan.edu.cn 08300720040@fudan.edu.cn 08300720033@fudan.edu.cn lidan@fudan.edu.cn
Project Information
Title:
Gesture Controller
Description:
The “Gesture Controller” is an all-in-one PC device which makes it possible for you to control mouse, scrollwheel and joystick simply with your gestures. It also has an interface for external programs which enables you to create your own gesture-controlled-system and take control of any custom devices with Gesture Controller.
Products:
NI myDAQ
LabVIEW 2010
Fig.1 The logo and the appearance of Gesture Controller
The Challenge:
Our original aim is to improve the user experience of PC devices such as mouse and joystick, because people sometimes feel it uncomfortable using these traditional devices since they do not keep a proper posture. We hope to make a new device with a more interactive user experience that frees people from uncomfortable postures. For example, people will control the cursor on the screen by simply waving hand in the air, instead of moving a mouse on the desk. The new controller should also integrate multiple functions thus to be more useful in people’s daily life.
Our further aim is to set up an open-source platform which enables users to customize their own applications with our controller.
The Solution:
1, Hardware
The Gesture Controller mainly consists of two parts: an accelerometer and a button group, which are both worn on the hand. The accelerometer’s output voltage (in 2 axes) changes with the movement and rotation of the hand, which provides gesture information. The button group can be used to switch among different working modes (eg. mouse, joystick, etc.). It also gives you some more controls such as mouse button press.
In addition, we use NI myDAQ to acquire both analog data from the accelerometer and digital data from the button group.
2, System architecture
Fig.2 System architecture
First, the analog and digital data are acquired by NI myDAQ, and input into the signal processing module, which converts the accelerometer’s voltage signal into the obliquity and rotating speed of the hand, and the button group’s on/off signals are converted into button-press events. Then, according to the button-press events, a working-mode-switch is called to switch among different working modes dynamically by calling corresponding control programs. Finally, the control programs recognize your gestures and call Windows API functions to realize the function of corresponding PC devices. In addition, the control program for “custom mode” can also output data through NI myDAQ’s digital I/O port in order to control custom hardware devices.
3, User interface
There are three panels on the user interface, which are called “state panel”,“options panel” and “calibration panel”, correspondingly.
Fig.3 State panel (made with LabVIEW)
On the state panel, the physical states of the sensor are displayed, as well as the obliquity and speed of your hand, which indicate your gestures. You can also see current working mode on the right hand.
Fig.4 Options panel (made with LabVIEW)
On the options panel, you can configure the physical channels and the performance of the sensor in different working modes. You can also set the directory of you customized control function so it can be linked to the main program and used to control your custom devices.
Fig.5 Calibration panel (made with LabVIEW)
On the calibration panel, you can easily calibrate the sensor by simply following the guide of the program. Generally you can finish calibration within 30 seconds.
Benefits using NI products
We have used NI myDAQ in our project, which is a small but very useful instrument. It is easy to acquire and output multiple kinds of signals with myDAQ and that helped us to achieve our goals smoothly.
In addition, LabVIEW is also a very nice tool. Thanks to its graphical programming environment, LabVIEW is easy to learn and comfortable to work with, which largely accelerated the process of our project.
Acknowledgement
We would like to thank our advisor, Mr. Li Dan (lidan@fudan.edu.cn) for his valuable advices during the whole process of the project. Without his help, we would not have solved so many troublesome problems successfully.
We also appreciate Ms. Wu Hongyan (hywu@fudan.edu.cn) for her generous support of laboratory apparatus for our experiments. With rich experience in virtual instrument, she also helped us a lot to improve our product.
Watch the video to see how it works!
(If you can not see the video, please visit this URL: http://v.youku.com/v_show/id_XMjc0OTE0MTMy.html)
