Professional Documents
Culture Documents
Introduction
Functionalities developed :
Move mouse pointer by hand gesture . Right Click mouse event by hand gesture Left Click mouse event by hand gesture . Drag/Drop an item by hand gesture .
Specifications
Platform : Windows7 Processor : Core2Duo RAM : 2GB
The application has not been tested on lower configurations , however it should work on system with lesser RAM and slower processor as well .
Tools Used
MSDN library : The functions defined in <windows.h> have been used to move
perform various events on mouse .
OpenCV library : The funtions defined in OpenCV libraries have been used for
quick reference .
Implementation of Algorithm
Algorithm Implemented
Initially start the camera using cvCameracapture( ) . Bring both caps in front of the camera and click an image using cvQueryFrame( ) . In the image , click at any point on the image of caps one by one . Determine the position of the point clicked on using GetCursorPos( ) . Find the hue component of the corresponding pixel in the image using setLimit( ).
Detecting Colors
Algorithm Implemented
Step 2 : Determine the presence of each colored cap and segment the colored caps in each frame
Capture the image continuosly using while( ) loop . Change the image in HSV plane using cvConvertScale( ) Find pixels which lies in the range of hue of a colored cap . Find the contour of the caps in the image using cvFindContour( ) .
Problems Faced
Step 1 : Learn colors of caps to be used for segmentation . Step 2 : Determine the presence of each colored cap and segment the colored caps in each frame
Segmentation using RGB space results in poor segmentation . Segmentation using connected components algorithm results in slower segmentation . Determining Hue of the colored caps using getPixel( ) of <windows.h> give different range of Hue .
Algorithm Implemented
Step 3 : Determine the mouse event to be performed Assumption : First color is green color and second color is red color .
Case 1 : Only green color present in image taken : Move mouse according to the displacement of green cap in present and previous image . Case 2 : Only red cap is present : Use left movement of red cap for Left Click , and right movement of red cap for Right Click Case 3.a : Both green and red cap present in image : No movement of mouse even if green cap is moved Case 3.b: Both green and red cap present in image and are nearby : Drag the mouse according to the displacement of caps .
Problems Faced
Step3 : Determine the mouse event to be performed
A lot of combinations were tried before deciding on final combination . A nice observation is that finger in the image moves in the opposite direction
Algorithm Implemented
Step 4 : Find the displacement of colored cap in consecutive frames and move the cursor on screen .
Stepwise velocity taken to make cursor movement look realistic . For very slow motion of finger , cursor doesnt move . For very fast motion of finger , cursor doesnt move .
Problems Faced
Step4 : Find the displacement of colored cap in consecutive frames and move the cursor on screen .
A lot of combinations were tried to make mouse pointer movement look realistic . Since finger doesnt have any support , so it keeps on displacing , and hence valid assumptions need to be tried to ignore small motion . If finger moves out of range of the camera, then bringing back finger cause motion of cursor in opposite direction , which might not be desirable .
Other Problems
Improper Illumination Problem Illumination Changes Color of background shouldnt contain any portion having same color as that of caps . The finger movement needs to be constrained in the range of camera which sometimes is impalpable experience for first time user .
Future Aspects
For our project we intend to use mobile camera for capturing images and using bluetooth to communicate with computer . Otherwise , Hand Gesture recognition to control events has a lot of scope e.g. streching an image by hand , playing games .
Not yet !!!! Since the most important component of a software cycle ,i.e, testing has not been done rigorously . However , we believe with little more hit and trial and testing , it can be made good enough for commercial purposes .
ThanK YoU