Gesture Controlled Robot Using Arduino 2018 - 2019

INTRODUCTION

The history of hand gesture recognition for computer control started with the invention
of glove-based control interfaces. Researchers realized that gestures inspired by sign language
can be used to offer simple commands for a computer interface. This gradually evolved with the
development of much accurate accelerometers, infrared cameras and even fiber optic bend-
sensors (optical goniometry).
The robot is usually an electro-mechanical machine that can perform tasks automatically.
Some robots require some degree of guidance, which may be done using a remote control or with
a computer interface. Robots can be autonomous, semi-autonomous or remotely controlled.
Robots have evolved so much and are capable of mimicking humans that they seem to have a
mind of their own.
An important aspect of a successful robotic system is the Human-Machine interaction. In
the early years the only way to communicate with a robot was to program which required
extensive hard work. With the development in science and robotics, gesture based recognition
came into life. Gestures originate from any bodily motion or state but commonly originate from
the face or hand.
Gesture recognition can be considered as a way for computer to understand human body
language. This has minimized the need for text interfaces and GUIs (Graphical User Interface).
Gesture recognition technologies are much younger in the world of today. At this time there is
much active research in the field and little in the way of publicly available implementations.
Several approaches have been developed for sensing gestures and controlling robots.
Glove based technique is a well-known means of recognizing hand gestures. It utilizes a sensor
attached to a glove that directly measures hand movements.

OBJECTIVES

The System objectives are created throughout the span of the project. They resemble the vital
outputs which were needed to successfully create the system.

 The Project is based on wireless communication, where data is transmitted to the robot
over RF link (RF Transmitter-Receiver pair).
 The main objective here in construction of this robot is to use as a hand gesture controlled
moving device.
 It enables or leads to the creation of Graphical User Interface (GUI).
 This eliminates the usage of joystick, buttons or any remote controlled operations for the
movement of the device.
 As the robot works on hand gesture, the commands are received based on the movement
of the hand in different orientations.
 Usage of accelerometer enables the movement of the robot in all three axes.

BLOCK DIAGRAM

Dept. of EIE, Dr.AIT Page 1

Gesture Controlled Robot Using Arduino 2018 - 2019

The following images show the simple block diagram of Hand Gesture Controlled Robot
for both Transmitter and Receiver Parts.

TRANSMITTER BLOCK DIAGRAM

 Accelerometer is a three-axis acceleration measuring device.

 The Arduino board is a open source microcontroller board that is equipped with sets of
digital input/output pins that may be interfaced to various expansion boards on other
circuits.
 Encoder IC is mainly used in interfacing RF and infrared circuits.
 RF Transmitter (Radio Frequency Transmitter) is a small electronic device in an
embedded system used for wireless communication between devices.

RECEIVER BLOCK DIAGRAM

 RF Receiver is used to receive the signal from the RF Transmitter.

 Decoder IC is used to decode the encoded data at the receiver end.
 Motor Driver IC (L293D) is an Integrated Circuit chip which is usually used to control
motors in autonomous robots.
 Motors are used to drive the wheels of the robot.
WORKING
Dept. of EIE, Dr.AIT Page 2
Gesture Controlled Robot Using Arduino 2018 - 2019

Gesture controlled robot works on the principle of accelerometer which records hand
movements and sends that data to the comparator which assigns proper voltage levels to the
recorded movements. That information is then transferred to an encoder which makes it ready for
RF transmission. On the receiving end, the information is received wirelessly via RF, decoded
and then passed onto the microcontroller which takes various decisions based on the received
information. These decisions are passed to the motor driver IC which triggers the motors in
different configurations to make the robot move in a specific direction.
The brain of the robot is an Arduino Uno(Atmega32). It is fed with a set of code. The
gestures/motion made by hand is recognized by the acceleration measuring device called
accelerometer (ADXL335).

Accelerometer based gesture controlled robot moves according to the movement of hand
as we place the accelerometer on your hand. When we tilt hand with an accelerometer in front of
the robot, then the robot starts moving forward until the next movement is given. When we tilt
hand in backward direction, then the robot changes its direction and state. Then it starts moving
in backward direction until the next signal is given. When we tilt hand on left side, then the robot
moves into left side until the next signal is given. In the same way, when we tilt hand in right
side, then the robot moves right side.

SCHEMATIC DIAGRAM OF ACCELEROMETER ADXL335

The Accelerometer shown above reads the X Y Z coordinates when we make hand
gestures. It then sends the X Y Z coordinates to the Arduino. We don’t need the Z axis. We need
only X and Y. The Arduino checks the values of coordinates and sends a 4bit data to the Encoder
IC in accordance with the data received from the accelerometer. The Encoder passes the data to
RF Transmitter. And the transmitted data is received by the RF Receiver. The receiver sends the
4bit data to Decoder IC which decodes it and passes to Motor Driver IC. Later the motor driver
makes decision to turn the two motor in required direction

Project is divided into two parts to make the task easy and simple and to avoid
complexity and make it error free.
Dept. of EIE, Dr.AIT Page 3
Gesture Controlled Robot Using Arduino 2018 - 2019

The first is the Transmitting section which includes the following components: Accelerometer,
encoder IC, RF Transmitter Module.
The transmitter section consists of an accelerometer which detects the hand gesture
and sends the data to the Arduino. Later Arduino sends data to the Encoder IC in
accordance to the data received from accelerometer and the data is transmitted to the
receiver.
The second is the Receiving end which comprises of following main components: RF
Receiver Module, Decoder IC, Microcontroller, Motor Driver IC and DC Motors. The receiver
circuit consists of 2 IC (HT12D decoder, L293D motor driver), RF receiver module.

SCHEMATIC DIAGRAM OF TRANSMITTING AND RECEIVING

SECTION

This robot is designed for recognizing five sets of gestures: forward, backward, left, right
and stop.

LITERATURE SURVEY

Dept. of EIE, Dr.AIT Page 4

Gesture Controlled Robot Using Arduino 2018 - 2019

Different technologies have been implemented for hand based recognition system and
few of them have shown good results. The ultimate goal is to control certain systems like smart
phones, air conditioners using these techniques. One of the most common approaches is data
glove based approach. In this approach there are sensors attached to glove which acquire the
gestures and the signals generated by sensors are processed and corresponding instructions are
performed.

Previously, the robots were controlled using joysticks, buttons or any kind of RF remote
controlled devices. So this hand gesture controlled robotic technique has eased the operations
and has made convenient for the user, as it can be operated based on the hand gesture commands.

Accelerometers are attached to the gloves as sensors to convert positional changes of

hand into signals which are then interpreted by the processor to perform the instructions. The
industrial robot is programmed by gestures and speech by natural means. ANN (Artificial
Neutral Networks) was applied for gesture and posture recognition.

The main benefit of using Arduino is faster code. Depending on the compiler we use, our
code can be a lot more efficient. Sure there are times writing our own USB driver is impossible,
but there are times we just want something simple and fast.

The expected upgradation in our project is that we are adding speed controller,
rechargeable battery and proximity sensors for the Gesture controlled Robot. Hence, we can
adjust the speed according to our wish and requirement. There will be no need to change
batteries again and again instead we can recharge it whenever we want. When an obstacle is
faced by the robot, even if we give commands to go forward the robot will not accept these
commands unless the obstacle is removed or a command to move away from the obstacle is
given.

HARDWARE AND SOFTWARE COMPONENTS

Dept. of EIE, Dr.AIT Page 5

Gesture Controlled Robot Using Arduino 2018 - 2019

 Arduino Board
 RF Transmitter
 Encoder IC
 Accelerometer/Gyroscope Sensor
 Resistors
 Motor Driver IC
 Decoder IC
 RF Receiver
 LED
 Four Geared Motors With Wheels
 Arduino IDE Software

APPLICATIONS

 The applications are huge, they can be used by physically challenged people to navigate
their robot around.

 The accelerometer is also used in IMU (Inertial Measurement Unit) for stabilizing drones
and ROV’s in 3D space, along with a Gyroscope. Ex- Balancing robot, robot arms etc.

 This sensor can also be used to calculate the distance traveled over time on a particular
axis. Suppose you measure the acceleration on X-axis over 10 seconds, double
integrating this expression will give you distance traveled, the mathematical model is bit
more complex. This feature is used in mapping and localization.

 This mechanism is used in military applications to operate robots.

 These robots are used in medical applications for the purpose of surgery.

 These robots are used in the construction fields.

 These robots are used in industries to control trolley and lift.

 Wireless controlled robots are very useful in many applications like remote surveillance,
military etc.

 Hand gesture controlled robot can be used by physically challenged in wheelchairs.

 Hand gesture controlled industrial grade robotic arms can be developed.

Dept. of EIE, Dr.AIT Page 6

Gesture Controlled Robot Using Arduino 2018 - 2019

REFERENCES

[1] Human Computer Interaction Using Hand Gestures

Chapter 2: “Historical Development of Hand Gesture Recognition”
Author: Prashan Premaratne, Originally published: 20 March 2014

[2] Beginning Arduino

Author: Michael McRoberts
Originally published: 22 December 2010

[3] http://2embeddedrobotics.blogspot.in/2012/05/gesture-controlled-robot-is-kind-of.html

[4] https://www.electronicshub.org/hand-gesture-controlled-robot/?amp

Dept. of EIE, Dr.AIT Page 7