A Novel Approach for Communication among
Blind, Deaf and Dumb People
Rohit Rastogi Shashank Mittal
Sr. Asst Professor, CSE-Dept CSE-Dept
ABES Engg. College ABES Engg. College
Ghaziabad (U.P.), INDIA Ghaziabad (U.P.), INDIA
Email Id:rohit.rastogi@abes.ac.in Email Id:shashank.12csl07@abes.ac.in
Abstract - Science and Technology have made Human life
addictive to comfort but still there exists an underprivileged
group of people who are fighting for finding an innovative
way that can make the process of communication easier for
them. According to the World Health Organization, about 285
million people in the world are blind, 300 million are deaf
and 1 million are dumb. In this project, we are going to
propose a new system-prototype called the SHARO]AN
BRIDGE in an effort to bridge the gap in the process of
communication between the Blind, Deaf and Dumb people.
The SHARO]AN BRIDGE will make use of the Wearable
Technology, Texas Instrumentation Circuitry and Arduino
Circuit Boards to provide a means of communication to
differently-abled people having one or all of the above
mention disabilities. It is assumed that a person who is deaf is
also dumb but vice versa is not true.
Keywords - Accelerometer, Arduino Circuit Boards, Blind,
Deaf, Dumb, Flex Sensors, Gesture, Microcontroller, Tactile
Sensors, Wearable Technology.
I. INTRODUCTION
According to the statistics given by the World Health
Organization, about 285 million people in the world are blind,
300 million are deaf, 1 million are dumb and many more
suffering from one or more of the above mentioned physical
disabilities. The developments in Science and Technology have
reached to great heights in making the Human Life easier and
comfortable within a short span of time. During the last few
decades, we have come across various technologies that have
made our life so easier and comfortable that we even do not
have to move our body to do a task. But always running in the
race to be ahead of everyone we have forgotten that we still
have a section of our population called the physically disabled
people who are deprived of the advancements of Science and
Technology because it has not given them that comfort that is
required by them to feel that they too are the part of the society
and they too can walk hand in hand with others.
Communication being a fundamental aspect of human life is
very much difficult for the people who are Blind, Deaf or
978-9-3805-4416-8/15/$31.00 2015 IEEE 605
Sajan Agarwal
CSE-Dept
ABES Engg. College
Ghaziabad , INDIA
Email Id:sajan.12cs1l6@abes.ac.in
Dumb. There are a little means of communication between
there people like the Braille Language [10] for communication
between Blind people and the Sign Language for Dumb and
Deaf people.
This paper is going to concentrate on the above mentioned fact
and tries to develop a new instrument which can help
differently abled people (Blind, Deaf and Dumb) to
communicate easily in the living world with other normal
persons or the persons of their own kind.
The main aim of this paper is to bridge the gap in
communication and bring forward some technology that can
help out the people who are suffering from Blindness or
Deafness or Dumbness or any combination of these three.
Here we have put forwarded the proposal for a new instrument
called the SHAROJAN BRIDGE which can help in solving
the above mentioned problem.
The SHAROJAN BRIDGE is based on the concept of
Wearable Technology that the gadget is wearable by the
person and thus making the device easily portable. Here we
are using Texas Instrumentation Circuitry and Arduino Circuit
Boards to make the transfer of message between differently
abled people possible. The message transfer include the
transfer of message in the form of Text or Audio or Braille
Language [10] as per the level of disability of the person.
II. CURRENT TECHNOLOGY IN LITERATURE
This research investigates a new way that can be productize so
that a new gadget can be developed that can bridge the gap in
communication among differently abled people who suffer
from any of the possible combinations of the disability of
Blindness, Deafness and Dumbness. For this we are looking
for some sort of wearable technology that can satisfy our
purpose. Some of the present technologies in concern with our
idea are:
A. The Data Entry Glove [9]
The Data Entry Glove was presented by Gary Grimes from
Bell Telephone Laboratories in 1983, and was the first widely
published sensor glove [6, 7, and 8]. The Data Entry Glove TABLE I. ALL CASES OF DISABILITIES IN A PERSON AND FORM OF
MESSAGE TRANSMITTED.
was originally devised as an alternative to the keyboard, and
made it possible to generate 96 printable ASCII characters
Sr.N Sending Message by Reply to be send by
from 80 different finger positions. The glove was made out
o. Communication Case A B
of cloth and had flex sensors along the fingers, tactile sensors /Vormal perso n A to Blind Audio message willi Audio message will
on the fingertips, and inertial sensors positioned on the knuckle 1 person S be sent by A. be sent by S.
side of the hands. The distribution of the sensors was specified Text message by B to,
with the aim of recognizing the Single Hand Manual Alphabet A converted to
for the American Deaf [7]. /Vormal perso n A to Dumb Audio message willi Audio/text message
2 person S be sent by A. only.
B. Multi-Modal Interfaces (Tyflos-Koufos) [3]
Message sent by B
This project was proposed in order to achieve the need to will be un the form of
convert different modalities into common medium shared and Braille and will be
understandable by deaf and blind individuals, for instance, Normal person A to "Blind Audio message will converted to
3 and Dumb" person B be sent by A. text/Audio message.
converting images into Natural Language (NL) text. [5].
Audio message will Text message by B tt)
This project gave a prototype that consisted of cameras
be sent by A A converted to
attached to dark glasses along with speaker and microphone Normal person A to "Deaf converted to text Audio/text message
and a portable Pc. 4 and Dumb" person B Ion IV text message. only.
Motivation: We came across various technologies that can help Blind person A to Blind Audio message will Audio message will
differently abled people to communicate among themselves 5 person S be sent by A. be sent by B.
and with the normal world easily but all of the technologies
studied so far were focusing only a certain parameter or extent Blind person A to Dumb Audio message will Text message by B tt)
of disability among the three of Blindness, Deafness and 6 person S be sent by A. A converted to Audio.
Message sent bV B
Dumbness. None of the technology was so developed that it
Audio message will will be in the form of
can be used as a general approach that can tackle any
be sent by A will be Text and will be
combination of these three disabilities. Blind person A to "Oea/and converted to text converted to Audio
So to solve this purpose, we proposed an approach that can be 7 Dumb" person B message. message.
used as a general way in which people suffering from any type
of combination of these three disabilities can think themselves Text message sent by
as a part of this beautiful world. Dumb person A to Blind A will be wrwerted to Audio message will
8 person S audio message. be sent by B.
III. PROPOSED METHOD
Text message sent by Text message sent by
The main motive of our paper is to introduce an idea that can Dumb person A to Dumb A will be (olwerted to B wi II be converted to
help to ease the way the disabled people use to communicate 9 person S audio message. Audio message.
with each other or with the world. The SHAROJAN BRIDGE Dumb person A to "Deaf Text message will be Text message will be

will make the communication easy between the disabled people 10 and Dumb" person B sent bVA. sent by B.
based on the extent of their abilities. Audio message will
In our approach we are considering all the possible Text message sent by be sent lly B will be
combinations of the disabilities of Blindness, Deafness and "Deaf and Dumb "personA A will be converted to converted to text
Dumbness by which a person can suffer. Our device called the 11 10' Blind person 8 audiD message. message.
SHAROJAN BRIDGE will take the input message form the "Deaf and Dumb" person A
differently abled sender as per his abilities and facility and It) -Deaf and Dumb" person Text message will be Text message will be
convert that message to be transferred to long or short distances 12 B sent bVA. sent by B.
as per the requirements. Once the message is transmitted to the "DeDf Dnd Dumb" person A Text message will be Text message will be
receiver then it is again converted as per the facility and 13 to' Dumb person B sent bV A. sent by B.
abilities of the receiver. Message will be sent
We have come across several combinations that are possible in "Slind and Dumb" person A in the form of Braille Audio message will
case of the three above mentioned disabilities and the solutions 14 to, Blind person B byA. be sentlly B.
to these possibilities. The following Table 1 describes every Message will be sent
possible combination and the forms of message that will be in the form 01 Braille
converted in every possible combination of disability between by A and then Text message will be
the differently abled people. converted to text sent by B will be
"S/lnd and Dumb" person A message/Audio converted 10
15 to, Dumb person B message. Braille/Audio.

606 2015 2nd International Conference on Computing for Sustainable Global Development (INDIA Com)
 Message will be sent
i n he foomof Sraille Ted message will be
"mind and Dumb" person A be

)if
byA and then sent by B will
-
to "Dealand .Dumb person converted to text converted to
16 S message. Braill e/Audio.

Message wi'll be sent
in the fonmof Scaille
by A and then
convert.ed to text DUMB PERSON BLIND PERSON
"mind and Dumb" person A message/Audio Audio message will
17 to Normal persa" B message. be sent by B.
Audio message will ., .,

"Dealr>nd flumb" person A
1.8 to Normal Ipe"a" B
"mind and De.'" and Dumb"
19 lpersan A to Blind person B
W W
be sent by B wiU be
Text message will be converted to text
sent by A. message.
Message will be sent Message wm b e sent
in he fonmof Sraille in th e fonmof Sraille
byA. by S.
BLIND PERSON DEAF ANO OUMB PERSON

Message win be sent

in the fonmDf Sraille Fig. I. Communication between a Dumb and a Blind Person and a Blind and a
"mind and De'" and Dumb" byA and then Ted message will be Deaf and Dumb Person
lpersa" A tD "CJeafand converted tD teu sent by B will be
20 Dumb " person B message. oonverted to Braille.
V. STRUCTURE AND COMPONENTS OF THE
Message will be sent
SHAROJAN BRIDGE
in the fonmDf Sraille
bVA and then
The SHAROJAN BRIDGE will consists of many small
convert.ed tD teu Ted message will be
"mind and De'" and Dumb" message/AudiD sent by B will be structural units that are assembled together to form a complete
21 Dumb persD" B
lpersa" A tD message. converted to Braille. circuity that will be capable of transmitting any type of
Message will b" sent messages between differently-abled people. The structure of
in the foomDf S.raille
the SHAROJAN BRIDGE consists of the following
by A and then
converted tD text Ted message will be
components:
"mind and De'" and Dumb" message/AudiD sent bV B will be
21 lpersa" A tD Normal person B message. converted to BraiUe. A. Sensor Glove. [1 and 2J
"mind and De'" and Message will b" se"t Message will b" sent
Dumb "person A to "mind in the fDrmDf Sraille in the fDnm .of Sraille The Sensor Glove is a latest proposed technology that is used
23 and Dumb Ipersa" B byA. by s. to convert the American Sign Language (ASN) [11] into audio
"mind and De."'and Dumb " Message will b" se"t Message will b" senl
which can further be converted into text to be displayed on the
rsa" A tD "Blind and l>ea/ in the fonmDf S.raille in the fDnm.of Sraille
LCD screen as per the requirements.
24 IandDumb Ipersa" B byA. Iby S. The Sensor Glove consists of 4 major components namely
Arduino Circuit board [12], Flex Sensors [1], Accelerometer
IV. HOW OUR METHOD IS USEFUL [1] and Tactile Sensors [1]. The block diagram of the Sensor
Glove is shown below.
The SHAROJAN BRIDGE we have proposed will act as a
connection link between people suffering with any combination
of disabilities we are tackling in this approach. Flex

The SHAROJAN BRIDGE will convert the message that is Sensor
sent by any of the disabled person in any form like text, audio,

Gesmre Texllo
gestures or Braille Language [10] in the form that is

Recognition Speech Speaker
understandable by the other disabled person easily and the Accelerometer
r. (ARDUINO) -. Conversion

transmission of the data is also valid up to large distances
1odle

t
because we are using the mobile network technology to
Hand
transmit the information over the GSM network. The following
chart will describe all the possibilities of combinations of the
gesture Tactile t
Sensor
extent of disability of the disabled people and will also describe
that how message is converted from one form to another so that
Fig.2. Block Diagram of the Sensor Glove
it is easy for both the communicators to understand and
communicate easily.
The Gestures made by the hand are feed as an input to the Flex
Let us illustrate the working of the SHAROJAN BRIDGE by
Sensors, Accelerometers and Tactile Sensors whose output is
showing the communication between a Blind person and
given to the Arduino Circuit Board for gesture recognition.
persons suffering from some other combinations of disabilities
Once the gesture is recognized, it is converted into speech
and vice versa.
which is taken as an audio output with the help of the speaker.

2015 2nd International Conference on Computing for Sustainable Global Development (INDIA Com) 607
The components of the Sensor glove are described in detail as and works on the technique of accepting the serial data ASCII
follows: characters and translating them to syllabic sounds.

A.I Arduino Circuit Board with built in ATMEGA 328 {12J B. Arduino GSM Shield. {J3J
Arduino is a simple microcontroller board bases open source
We are using the Arduino GSM Shield for making our device
platform. Here in our Sensor Glove we are using Arduino
capable of communicating over large distances. Arduino GSM
duemilanove with inbuilt atmega328. Atmega328 has 32KB
Shield enables the user to connect his Arduino to the Internet
on-chip flash memory for storing codes of which 2KB used for
using a GPRS wireless network. We only need to connect this
boot loader. It also includes a 2KB of SRAM and lKB of
should to our Arduino Board along with a GSM SIM card and
EEPROM.
some simple instructions only. This also allows the user to
save numbers, make phone calls and send text messages.
A.2 Flex Sensors {lJ
Here we are using the Flex Sensors in accordance with the
voltage divider circuit. The Flex Sensors are resistive carbon
elements that produces a variation in the resistance when bent
correlated to the bent radius. The variation in resistance varies
from lOKohms when unflexed to 30Kohms when bent at an
angle of 90 degrees.

FigA. Arduino GSM Shield

A.3 Tactile Sensors {lJ
A tactile sensor is used for controller resets and input
C. Texas Instrumenting's BeagleBone (Black) & IC's. {l4, ISJ
operation. This is a type of switch which creates a temporary
electrical connection when pressed. The tactile sensor is The Texas Instrumentation's BeagleBone (Black) will be used
connected to the digital pin of Arduino and one pin is grounded to implement the whole circuitry that will consists of Analog
and other is supplied with +5 volts. to Digital Converters and Digital to Analog Converters i.e.
ADS1605 and ADS1606 and . DAC8534 and DAC858.
A.4 Accelerometer {IJ
Accelerometers are used for tilt sensing. They measure both
Static and dynamic acceleration. The sensor has a g-select
input which switches the accelerometer between 1.5g and
6g measurement ranges. It has a signal conditioning unit with
a I-pole low pass filter, temperature compensation, Self-test,
and Og-detect which detects linear free fall.
A.S Text to Speech Conversion {lJ
Here we are using a mechanism that will convert the gesture Fig.5. Texas Inslrumenlalion's BeagleBone (Black).
once recorded into the speech. The block diagram for this
mechanism is shown below.
Data i.n D. Arduino Uno Circuit Board and Vibrating Motor {l6, 17J
ARDUINO
The Arduino Uno is a microcontroller board based on
Gesture input
Allophone fr0 111 glove the ATmega328 (datasheet). It has 14 digital input/output pins
codes (of which 6 can be used as PWM outputs), 6 analog inputs, a
16 MHz ceramic resonator, a USB connection, a power jack,
an ICSP header, and a reset button. Vibrating Motor is to
produce a vibrating sensation to the user to make him aware of
the message.
Speaker

Voice output
Amplified speech
signal

Fig.3.Block Diagram for Texl lo Speech conversion

The output in the form of text from the Arduino is converted

into sound using the TTS. SpeakJet is used here for facilitating
the conversion from text to speech and LM386 is an amplifier
used to amplify the sound. TTS 256 is a 28 pin DIP, 8 bit
Fig.6. Arduino UNO Circuil Board and Vibraling Molor
microprocessor programmed with 600 letters to sound rules

608 2015 2nd International Conference on Computing for Sustainable Global Development (INDIA Com)
 VI. METHODOLOGY AND WORKING
The SHAROJAN BRIDGE is proposed by taking care of all the
possibilities and combinations of the three disabilities namely
Blindness, Deafness and Dumbness from which a person can
suffer. Moreover it also considers distance as a major barrier in
communication for which it uses a technique that will remove
the barrier of distance between communications of such people.
The working of the SHAROJAN BRIDGE for transmission of
a message from one disabled person to another is described in
the following steps:
The SHAROJAN BRIDGE we are using is a type of
wearable technology and thus it is wearable on the wrist of
the user. First of all, the output and input of the
SHAROJAN BRIDGE is set to the desired form of the
user.
For example: For a blind user, the input can be text using
the Sensor Glove or the Audio as per his wish and
requirements and the output must only be Audio. Thus, the
device is set so that it can take input and give output as per
the requirements of the user.
The message to be sent by the user is taken as an input to
the gadget. The input can be text, audio or Braille
Language [10]. The gadget has a Microphone to take the
audio input, the Sensor Glove for text input and a Braille
Language [10] Converter for taking the Braille Language
[10] input and converting it into text.
If the message to be delivered by the sender is in the form
that is acceptable and understandable by the receiver and
the communication is a direct type of Communication,
then the message is transferred directly to the receiver.
For long distance communication, the input message is
converted into audio message independent of the initial
form and then it is transmitted through Wireless GSM
Network to the receiver. For long distance communication,
the sender must have the phone number of the receiver.
The block diagram of the SHAROJAN BRIDGE is shown
in the below figure.
Fig.7. SHAROJAN BRIDGE Block Diagram
VII. AN EXAMPLE OF A SPECIFIC CASE
2015 2nd International Conference on Computing for Sustainable Global Development (INDIA Com)
Let us take the example in which a Deaf and Dumb person
wants to communicate to a Blind Deaf and Dumb. Following
are the steps that must be followed to make the
communication possible.
The Deaf and Dumb person will send a text message
to the SHAROJAN BRIDGE through the Sensor
Glove by the use of hand gestures as described in the
American Sign Language.
The Sensor Glove will convert the Gestures into an
audio message.
The audio message is then converted into text
message by the use of Analog to Digital Converters
i.e. ADS1606 and ADS1605 embedded on the
BeagleBone Board.
The output from the BeagleBone board will be
converted from text to Braille Language [10] using
the Braille Lite circuitry so that the Blind can easily
understand it. The vibrating motor also vibrates when
the receiver receives the message.
As in case the message is to be transferred to a long
distance, the audio message obtained at the Step 2 is
first transmitted to the long distant receiver by the use
of Arduino GSM Shield and then Step 3 and Step 4
are followed to obtain the final Braille Language [10]
Message output at the receiver's end.
BRAIUf OUTPUT'"HEUO"
BliND, DEAf AND DUMB
Fig.S. Communication between Deaf and Dumb person with a Blind, Deaf and
Dumb person.
VIII. CONCLUSION FUTURE SCOPE
In this paper we have proposed the basic approach of the
system we named as SHAROJAN BRIDGE which can be a
useful tool in banishing the barrier of disabilities in
communication of the people suffering from any of the
possible combination of Blindness, Deafness and Dumbness
among themselves as well as normal people.
We have taken into consideration that can arrive in case of the
three type of disabilities and facilitate every disabled person
and the normal person to communicate with the disabled ones.
The person can communicate and transfer the message as per
his ability and desire. The dumb can use their Sign language to
transmit the message while those who are unable to
understand the Sign Language can make use of the device to
609
get the output in the audio form for normal or blind people and
in the form of Braille Language [10] for Blind and Deaf person.
Moreover the message can also be displayed in the form of text
on the LCD screen for Deaf people and even the transmission
of the message can be made over large distances by the use
GSM Wireless Networks. Thus this approach can tackle to any
type of difficulty that can come across the process of
communication among differently abled people and the normal
world.
There can be numerous future advancements that can be
associated with this research work some of which are described
below as:
We are using the Arduino GSM Shield to make
connectivity over long distances. But we are using
only the call and text message feature of this
technology. In future some new way can be developed
that can use the Internet connectivity feature of GSM
Shield to make the connectivity of the device better
and for longer distances.
Arduino Technology also provides Ethernet, WI-PI
and Bluetooth support. Thun incorporate these
features to our device to make it capable of connecting
to any other device with WI-PI and Bluetooth support.
There can be more perfection in sensing the
movements and gestures so that the message
transmission can be made smoother.
Since this is a type of wearable technology, we can
think of new advancements that can be implanted so
that the device can be made more compact, faster and
reliable.
IX. NOVELTY IN OUR PAPER THROUGH THIS
APPROACH
We are using the latest and trending wearable technology
which makes it possible to carry the device easily anywhere
and everywhere by the disabled person which makes our device
portable. Moreover we have not came across any such
technology so far that can help to all the people who are
suffering from any combination of disabilities of Blindness,
Deafness and Dumbness. Thus we have followed a general
approach due to which it is possible to productize a single
gadget which can solve the problems of all the people with
different type of disabilities. At present there is no such
technology that can provide long distance communication
facility to differently abled people but we are giving the way by
which it is possible for these people to communicate over long
distances with many future scopes also.
X. LIMITATIONS
Although SHAROJAN BRIDGE provides a novel approach for
providing a link between differently abled people but even then
it has several limitations.
We are using the wearable technology along with several types
of Arduino Boards and circuity which makes the device a little
bulky and massive. Moreover, long distance communication is
610 2015 2nd International Conference on Computing for Sustainable Global Development (INDIA Com)
completely based on mobile calling feature due to which the
sender must have the contact number of the receiver to whom
he wants to communicate. The disabled person must aware of
the American Sign Language and the Braille Language [10] in
order to send and receive the message.
ACKNOWLEDGEMENT
With due regards, we would like to thank the actual people
behind this paper. We would like to thank our Head of
Department, Dr. Prof. RRadhaKrishnan, HOD-CSE of ABES
Engineering College, Ghaziabad for his guidance at every step.
We would also like to thank Mr. Rajat Sharma, B.Tech CSE
Dept. Final Year for his kind support at every step of this
paper. We would like to thank our friends, family members for
their extreme support and blessings and to the ALMIGHTY for
HIS blessings.
REFERENCES
[1] Anbarasi Rajamohan, Hemavathy R., Dhanalakshmi M.
"Deaf-Mute Communication Interpreter", International
Journal of Scientific Engineering and Technology Volume
2 Issue 5, pp: 336-341 (ISSN: 2277-1581)
[2] Kuldeep Singh Rajput, Shashank Deshpande, Uma
Mudenagudi, "INTERACTIVE ACCELEROMETRIC
GLOVE FOR HEARING IMPAIRED".
[3] Nikolaos Bourbakis, Anna Esposito, D. Kabraki, "Multi
modal Interfaces for Interaction-Communication between
Hearing and Visually Impaired Individuals: Problems &
Issues", 19th IEEE International Conference on Tools with
Artificial Intelligence.
[4] NetchanokTanyawiwat and SurapaThiemjarus, Design
of an Assistive Communication Glove using Combined
Sensory Channels, 2012, Ninth International Conference
on Wearable and Implantable Body Sensor Networks.
[5] N.Bourbakis, An SPNG based method for image to NL
text conversion, PR Journal.
[6] G. Grimes, Digital Data Entry Glove Interface Device,
AT & T Bell Labs, 1983.
[7] D. Sturman and D. ZeIter, -A survey of glove-based
input, II IEEE Computer Graphics and Applications, vol.
14, no. l, pp. 30-39, 1994.
[8] M. Mohandes and S. Buraiky, -Automation of the Arabic
sign language recognition using the powerglove, II AIML
Journal, vol. 7, no. 1, pp. 41-46, 2007.
[9] S. Sidney and E. Geoffrey, -Glove talk-a
neuralnetwork interface between a data-glove and a
speech synthesizer, II IEEE Transactions on Neural
Networks, vol. 4, no. 1, pp. 2-8, 1993.
[10] en.wikipedia.org/wikilBraille
[11] en.wikipedia.orglwikilAmerican_Sign_Language
[12] arduino.cc/en/MainlProducts
[13] arduino.cc/en/MainiArduinoGSMShield
[14] www.ti.com/product
[15] www.ti.com/toollbeaglebk?DCMP=PPC_Google_TI&k_c
lickid= 1ccc422b-a6a2-03a8-420c-
0000782fc37a#descriptionArea
[16] arduino.cc/en/MainlArduinoBoardUno
[17] www.sparkfun.com/products/8449