Voice Activated Presentation Assistant

V O I C E A C T I VA T E D P R E S E N T A T I O N A S S I S T A N T
Brian Chen Eunsang Cho Vahe Jabagchourian Vatche Jabagchourian
EECS 129B Senior Design Project Professor Raymond Klefstad Winter 2007
.
UNIVERSITY OF CALIFORNIA IRVINE
Voice Activated Presentation Assistant (VAPA)
Executive Summary The Voice Activated Presentation Assistant (or VAPA) allows the presenter to use his/her voice commands to control PowerPoint slides and activate a guiding countdown timer. Key Benefits Plug and Play Hands Free Easy to Use Wireless Modular
EECS 129B
Voice Activated Presentation Assistant
Page 2
Imagination is everything. It is the preview of lifes coming attractions. - Albert Einstein
EECS 129B
Page 3
Table of Contents
Page
1. OBJECTIVE ................................................................................................................................................. 6 2. OVERVIEW .................................................................................................................................................. 7 3. SYSTEM ASSEMBLY.................................................................................................................................. 8 4. OPERATION INSTRUCTIONS .................................................................................................................... 9 5. SETUP........................................................................................................................................................ 10 6. TRAINING .................................................................................................................................................. 11 7. SYSTEM LEVEL BLOCK DIAGRAM AND COMPONENT DESCRIPTIONS .......................................... 12 8. CIRCUIT DIAGRAM................................................................................................................................... 13 9. SR-06 (SPEECH RECOGNITION) DIAGRAM ..........................................................................................14 10. COUNT DOWN TIMER DIAGRAM.......................................................................................................... 15 11. ARDUINO/DECODER CIRCUITRY......................................................................................................... 16 12. ARDUINO/KEYPAD INTERFACE........................................................................................................... 17 13. VOLTAGE THRESHOLD LEVELS FOR KEYPAD................................................................................. 18 14. USER LEVEL FLOW CHART.................................................................................................................. 19 15. SYSTEM LEVEL FLOW CHART............................................................................................................. 20 16. SOFTWARE SYSTEM OVERVIEW ........................................................................................................ 21 17. MASTER CONTROLLER SOFTWARE CODE ....................................................................................... 25 18. SLAVE CONTROLLER SOFTWARE CODE .......................................................................................... 37 19. COMPONENT DESCRIPTIONS.............................................................................................................. 45 20. ENCLOSED ASSEMBLY ........................................................................................................................ 47 21. COST SHEET .......................................................................................................................................... 48 22. TEST PLAN.............................................................................................................................................. 49 23. CONCLUSION ......................................................................................................................................... 52 APPENDIX ..................................................................................................................................................... 53
EECS 129B
Page 4
Table of Figures
Page
FIGURE 1 AVAILABLE VOICE COMMANDS ............................................................................................. 6 FIGURE 2 FUNCTIONAL OVERVIEW......................................................................................................... 7 FIGURE 3 PROJECT IMAGE (TOP VIEW).................................................................................................. 8 FIGURE 4 HIGH LEVEL BLOCK DIAGRAM ............................................................................................... 9 FIGURE 5 TRAINED WORD INDICES......................................................................................................... 9 FIGURE 6 SETUP OPTIONS...................................................................................................................... 10 FIGURE 7 KEYPAD DIAGRAM.................................................................................................................. 10 FIGURE 8 HOW TO TRAIN WORDS ......................................................................................................... 11 FIGURE 9 KEYPAD DIAGRAM.................................................................................................................. 11 FIGURE 10 SYSTEM LEVEL BLOCK DIAGRAM ..................................................................................... 12 FIGURE 11 CIRCUIT DIAGRAM ................................................................................................................ 13 FIGURE 12 SPEECH RECOGNITION CIRCUIT DIAGRAM ..................................................................... 14 FIGURE 13 COUNTDOWN TIMER DIAGRAM .......................................................................................... 15 FIGURE 14 ARDUINO/DECODER INTERFACE ....................................................................................... 16 FIGURE 15 ARDUINO/KEYPAD INTERFACE .......................................................................................... 17 FIGURE 16 VOLTAGE THRESHOLD LEVELS......................................................................................... 18 FIGURE 17 USER LEVEL FLOW CHART................................................................................................. 19 FIGURE 18 SYSTEM LEVEL FLOW CHART ............................................................................................ 20 FIGURE 19 TOP LEVEL COMPONENT DESCRIPTIONS ........................................................................ 45 FIGURE 20 ENCLOSED ASSEMBLY........................................................................................................ 47 FIGURE 21 LED TRANSISTOR CIRCUIT ................................................................................................. 49 FIGURE 22 DECODER PIN LAYOUT DIAGRAM...................................................................................... 50 FIGURE 23 SPEECH RECOGNITION PIN LAYOUT................................................................................. 53 FIGURE 24 SPEECH RECOGNITION PIN DESCRIPTIONS .................................................................... 54 FIGURE 25 KEYPAD PIN DESCRIPTIONS............................................................................................... 55 FIGURE 26 ARDUINO / SPEECH RECOGNITION INTERFACE.............................................................. 56 FIGURE 27 -- TRANSMITTER CIRCUIT BOARD......................................................................................... 57 FIGURE 28 MICROPROCESSOR PIN LAYOUT (ON TRANSMITTER) ................................................... 58 FIGURE 29 FUNCTIONAL BLOCK DIAGRAM ......................................................................................... 59 FIGURE 30 TRANSMITTER PIN LAYOUT ................................................................................................ 60 FIGURE 31 FUNCTIONAL BLOCK DIAGRAM ......................................................................................... 61
EECS 129B
Page 5
1. Objective
Purpose of Project To design a voice activated presentation assistant with a count down timer. This will allow the user to control PowerPoint without using his/her hands and also be able to note the amount of time left on his/her presentation by using the count down timer. Main Features Voice Activated PowerPoint Commands Keypad Interface Large Countdown Timer Keypad Input Five Unique PowerPoint Commands
Available Voice Commands

Command NEXT PREVIOUS EXIT BEGIN CLEAR TIME Action Advances slide Recalls previous slide Exits presentation mode Begins presentation Blanks screen Activates Time
Figure 1 Available Voice Commands
EECS 129B
Page 6
2. Overview
Summary of Functionality Speaker says command into microphone Timer begins countdown sequence if Begin command received Binary code is relayed from voice recognition board to remote control Remote control sends signal to laptop with receiver PowerPoint responds by performing action
Laptop with PowerPoint Microprocessor Countdown Timer Transmitter
Speaker
Speech Recognition Board
Figure 2 Functional Overview
EECS 129B
Page 7
3. System Assembly
Figure 3 Project Image (Top View)
EECS 129B
Page 8
4. Operation Instructions 1. 2. 3. 4. 5. 6. Power-Up Unit Press Reset button on Master and Slave Arduino Verify that LEDs on Timer blink Once and then Twice Set mode of operation using Keypad Train Commands or Set Time Begin Presentation
VersaPoint Tranmitter
Arduino MicroProcessor
Timer & 7-Segment Displays
Figure 4 High Level Block Diagram
Word Index 01 02 03 04 08 09
Command Begin Next Previous Blank Exit Time
Figure 5 Trained Word Indices
EECS 129B
Page 9
5. Setup
Key 1 2 3 4 5
Option Speech and Timer Speech Only Timer Only (Speech Activated) Timer Only (Keypad Activated) Speech and Timer (Skip Training)
Figure 6 Setup Options
1 4 7
2 5 8
3 6 9
CLR
TRN
Figure 7 Keypad Diagram
EECS 129B
Page 10
6. Training
Keypad Command 0 1 TRN 2 1 TRN 9 9 CLR 0 1 CLR
Action Trains First Pattern Trains 21st Pattern Clears memory Clears 1st pattern
Figure 8 How to Train Words
1 4 7
2 5 8
3 6 9
CLR
TRN
Figure 9 Keypad Diagram
EECS 129B
Page 11
7. System Level Block Diagram and Component Descriptions
Countdown Timer
Arduino Controller
Keypad
Transmitter
Decoder and PNP Transistors
PNP Switch Array
Arduino Controller
Figure 10 System Level Block Diagram
Component Keypad Main Arduino Microcontroller Secondary Arduino Controller Digital Displays Decoder and PNP Transistors PNP Switch Array Speech Recognition Board Transmitter
Functional Description Allows user to train words into memory and set timer Controls timer and accepts input from keypad Inputs commands to Speech Recognition circuit and transmitter Displays remaining time to presenter Deocdes binary code from Arduino to PNP transistors which control transmitter Activates buttons on speech recognition ports from decoder output Processes speech recognition functions and outputs status codes to Second Arduino Tranmits RF signlal to USB reciever module to control PowerPoint
EECS 129B
Page 12
8. Circuit Diagram
Figure 11 Circuit Diagram
EECS 129B
Page 13
9. SR-06 (Speech Recognition) Diagram
Figure 12 Speech Recognition Circuit Diagram

Quantity 1 1 1 2 16 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 Item HM2007 CY6264 74LS373 4511 220 Ohm Resistor XTAL 3.57 MHz Main Circuit Board Keypad Power Switch Seven Segment Displays Headset and Microphone 9V Battery Snap Coin Battery Holder 100K ohm Resistor 6.8K ohm Resistor 22K ohm resistor .1FCapacitor 0.0047F Capacitor 100F Capacitor 10-22 pF Capacitor 7805 Linear Voltage Regulator Resistor (100K Ohm 1/4 Watt) Resistor (6.8K 1.4 Watt) Resistor (22K 1/4 Watt) Description Speech Recognition IC 8K X 8 SRAM 8 Bit Register BCD to 7-Segment Decoder/Driver Resistor (200 Ohm 1/8 Watt) Crystal Oscillator Printed Circuit Board 12 Switches On / Off
EECS 129B
Page 14
10. Count Down Timer Diagram
Figure 13 Countdown Timer Diagram
Quantity 2 4 1 28 1
Item LED (Green) 74HCT4543E (Texas Instruments) 74LS373 110 Ohm Resistor 520 Ohm Resistor
Description Bright Green LED 7-Segment Decoder Driver with Latch 8 Bit Register 110 Ohm Resistor 520 Ohm Resistor
EECS 129B
Page 15
11. Arduino/Decoder Circuitry
Figure 14 Arduino/Decoder Interface Digital Pin 8, 9, 10, 11 on Arduino connect to Pins on 1, 2, 3, 4 on Decoder (74HC154)
Quantity 5 1 1 5 1
Item 1K Resistor Arduino 74HC154 2N3906 VersaPoint Communicator (Transmitter)
Description Resistor Microcontroller 4-16 Decoder PNP Transistor RF Transmitter for PowerPoint
EECS 129B
Page 16
12. Arduino/Keypad Interface

Most keypads use row plus column number of pins connected to a keypad decoder which connects to a microcontroller. If the microcontroller has a binary keypad decoder then there is no need for a keypad decoder. In our project we have used the principle of voltage dividers by using a resistor matrix. Each key produces an analog voltage ranging from 0 to 5 volts. The analog voltage is decoupled with a tiny capacitor to filter out high frequencies. The press of a key produces a voltage on the input of the ADC pin of the Arduino. The Arduino consists of a 10 bit ADC (0 to 1023 quantized values can be represented by the ADC). Using a driver code we are able to represent the button that has been pressed by its corresponding binary value. The benefits of this resistor matrix are massive as this architecture uses only one pin when interfaced to the host controller. We can use the unoccupied pins for handshaking between the two microcontrollers and controlling the timer circuitry.
Arduino
ADC0 1nF 1K 820 1K 15K 3.3K 680 180 +5 VDC
1 4 7 *
2 5 8 0
3 6 9 #
Keypad
Figure 15 Arduino/Keypad Interface
EECS 129B
Page 17
13. Voltage Threshold Levels for Keypad
Key 1 2 3 4 5 6 7 8 9 * 0 #
Voltage at ADC0 Min Avg Max 0.225 0.248 0.272 0.396 0.434 0.474 0.588 0.641 0.698 0.930 0.969 1.048 1.468 1.522 1.627 1.959 2.020 2.139 2.563 2.688 2.809 3.285 3.396 3.500 3.740 3.832 3.917 4.170 4.237 4.298 4.507 4.550 4.588 4.671 4.700 4.726
Figure 16 Voltage Threshold Levels Read By ADC on Arduino
EECS 129B
Page 18
14. User Level Flow Chart
Start
Begin Presentation
Train Mode?
Continue Presentation
No Yes No
Train Words Is Presentation Over?
Yes
Set Time End
Figure 17 User Level Flow Chart
EECS 129B
Page 19
15. System Level Flow Chart
Master
Start
Handshake
Slave
Start
Select Mode
NO
Mode 1, 2, 3, 5
YES Keypad buttons To SR
Train Words
Press 5,6
Train Words
Press 5,6
Mode 2, 3, 4, 5
YES NO
Recognize Words
From SR
Set Time
CLR/Begin Press CLR
Valid word?
YES To To Transmitte Transmitter r
Count Down
TRN/ Time TRN/ Time To Timer NO
Transmit
Paused
Time/Begin
YES
Figure 18 System Level Flow Chart
EECS 129B
Page 20
16. Software System Overview

The Voice Activated Presentation Assistant (VAPA) uses two Arduino microcontrollers: a master and a slave. The master Arduino takes in keypad inputs and controls the timer while the slave Arduino interfaces with the speech recognition circuit as well as the transmitter. Both Arduinos communicate together through their serial transmit and receive lines.
Operation of the VAPA is conducted as shown below:

1. 2. 3. 4. 5. 6. 7. 8. Plug power into an Arduino. Wait for the : LEDs on the timer to flash once, then twice Turn on the speech recognition circuit Select an operation mode by pressing a key between 1 and 5 on the keypad Train the commands by pressing the appropriate number and then TRN Press 5 and then 6 to finish training Enter time on timer Begin speaking commands
Main Components:
Keypad The keypad, which is connected to the master Arduino through one analog pin, transmits the different button values using a resistor network. The Arduino will read the analog pin and depending on the voltage value it gets, will assign a value that corresponds to the pressed button to a global variable and will return 0 if there was a valid key pressed, and -1 otherwise. To distinguish between a key being pressed and one that is held down, the analog values is compared to the previous value and only if the two are not equal will it be possible for a valid press to register. This is because when no buttons are pressed, the analog pin would read a different value from the keypad. Therefore, buttons must be released in between presses.
Speech Recognition Circuit The speech recognition (SR) circuit is connected to the slave Arduino for both input as well as output. Originally, the keypad was connected directly to the speech recognition circuit. A button press would short two of the pins on the input bus of the SR, resulting in an input. Since the keypad is used in all aspects of VAPA, it cannot be connected directly to the SR. Instead, transistors that connect the pins of the SR input bus allow the Arduino to simulate the button presses by sending a signal, which activates the transistor and shorts the two pins together.
EECS 129B
Page 21
The output of the SR is also connected to the slave Arduino. It reads the data through its connection to six of the eight data pins of the output bus, which are in binary coded decimal. This allows the Arduino to read numbers up to 39.
Timer The timer consists of four seven-segment displays, each with a display driver, which also serves as a latch. The master Arduino interfaces with the timer using 10 digital pins. Four pins are used as a data line to be able to send the numbers 0 through 9 in BCD to the timers universal data bus that connects to all four display drivers. Another four pins are used to connect to a latch disable pin on each of the 4 display drivers. Sending a HIGH signal to the latch disable pin will all the input of the driver to pass to the output. A LOW on the latch disable will capture what is on the input during the high to low transition, hold it on the output, and not let any inputs pass through. One pin is used to connect to the blank pin on all of the latches, which turns off the displays. Finally, one pin is used to control the : between the minutes and seconds. Displaying a number on one of the displays of the timer is done as followed: Put the number of the data bus Set the latch disable pin of the appropriate driver HIGH, and then LOW This is repeated for the remaining three displays.
Decoder To save pins, a decoder is used to connect between the slave Arduino and the transistors for simulating the button presses on the SR and the transmitter. The decoder is a 4-to-16 decoder and has 13 of its outputs used by the VAPA. Five of the outputs are used for the five buttons on the transmitter. Eight outputs are used for the input bus of the SR. These are 1, 2, 3, 4, 8, 9, TRN, and CLR. 5, 6, and 7 are unnecessary and therefore not used.
VAPA Software System Operation:

Startup When the system is powered up, the master Arduino will flash the timers : indicating that it is ready and wait for the slave to start up. When it does, it will send the character R to the master through the serial lines and the master will flash the timers : twice.
EECS 129B
Page 22
Mode Selection Once the system is ready the user will first press a key on the keypad to select a mode of operation. The entered key will set a global variable on the master to determine which of the SR and timer will be used. In each function to train words and to set the time on the timer, this global variable will be checked before proceeding. Train Words In this mode, keypad buttons are checked to ensure they are valid values for the SR, and will then be sent to the slave Arduino. The slave Arduino will then activate the corresponding buttons on the SR. When the user presses the keys 5 and 6 in succession, training will be complete and the master will no longer send button values to the slave. Set Time After commands are trained, the user will enter the time of the presentation length on the timer. Times are entered in a similar way as on a microwave. Numbers are shifted in from right to left with each entry, until four numbers have been entered. Each time a number is entered, variables for minutes and seconds are updated. When the CLR buttons is pressed during set mode, numbers are all shifted right by one, discarding the rightmost number. Command Recognition Mode Once a time has been entered, the user can immediately begin speaking voice commands. The slave Arduino handles the outputs of the SR. If the slave reads a valid command it will take one of the following actions:
Command Begin Next Previous Blank Exit Time Transmit YES YES YES YES YES NO Timer Start time
Pause/Restart
The first 5 commands will all cause their corresponding commands to be sent by the transmitter. If begin is spoken during set time mode, the timer will also start counting down. If the Time command is spoken while the timer is counting down, it will pause the timer, while speaking Time when it is paused will start the timer counting down again. The SR works by placing the value it gets on the output bus and keeping it there until a different word is recognized or an error code is output. This presents the problem of not being able tell when a certain word is spoken twice in a row because there would be no EECS 129B Voice Activated Presentation Assistant Page 23
change in the output when the word is recognized the second time. To alleviate this problem, upon recognizing a valid word, the slave Arduino will send the pattern 1, 8, CLR to the SR, which serves to separate back-to-back recognitions. Timer Countdown Mode Once a time has been entered, countdown can begin by either speaking the word Begin, or by pressing the TRN button on the keypad. In either case, the time will be checked to ensure it is valid by using the variables for minutes and seconds. Minutes must be between 0 and 99 while seconds must be between 0 and 59, providing a maximum time duration of 99:59. A time of 00:00 will also not be a valid time. Once the time runs out, the timer will blink 00:00. The software for the timer works by using a pair of nested while loops. The outer while loop is for minutes which checks minutes > 0 while the inner loop checks seconds > 0. For both loops, their respective time variable, minutes or seconds, is decremented after one iteration. Each time the while loop for seconds exits (seconds = 0), seconds is set to 59. When the time variables are updated, they are printed to the displays and master delays for 1 second. During a standard delay, the Arduino cannot do anything. However, the timer needs to be able to accept input from the keypad and slave to start, pause, or clear the time. Therefore, VAPA uses a special function, which breaks up the 1-second delay into 40 equal time chunks. During each time chunk, the Arduino will delay for 25ms and then perform a non-blocking read of the keypad and the serial port. If it gets input, it will take the appropriate action. Otherwise, it will continue delaying and checking for inputs until the second is over. By checking for inputs 40 times per second, the timer is very responsive to inputs from either the keypad or the serial port. The timer can be cleared during countdown or while it is paused by pressing CLR on the keypad. This puts the timer back into set time mode. When the time finishes and the displays are blinking 00:00, pressing CLR will put the timer into set time mode as well.
EECS 129B
Page 24
17. Master Controller Software Code

//-------CONSTANTS---------// //Keypad Buttons const int KEY_0 = 0; const int KEY_1 = 1; const int KEY_2 = 2; const int KEY_3 = 3; const int KEY_4 = 4; const int KEY_5 = 5; const int KEY_6 = 6; const int KEY_7 = 7; const int KEY_8 = 8; const int KEY_9 = 9; const int KEY_CLR = 10; const int KEY_TRN = 11; const char keyChar[12] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'C', 'T'}; //Operation modes const int MODE_SPEECH_TIMER const int MODE_SPEECH const int MODE_TIMER_SC const int MODE_TIMER_KC const int MODE_SPEECH_TIMER_DT //System states const int STATE_MODE = 0; const int STATE_TRAIN = 1; const int STATE_SETTIME = 2; const int STATE_COUNTDOWN = 3; const int STATE_PAUSED = 4; //LED const int ledPin = 2; //Latch disable const int LD3 = const int LD2 = const int LD1 = const int LD0 = //BCD const const const pins 11; 10; 9; 8;
= = = = =
1; 2; 3; 4; 5;
decoder inputs int D3 = 7; int D2 = 6; int D1 = 5; Voice Activated Presentation Assistant Page 25
EECS 129B
Master Controller Software Code (Continued)

const int D0 = 4; //Blank const int BL = 3; const int keypadPin = 5; /////////////////////////////// //------VARIABLES------------// int opMode = -1; int button = -1; int lastButton = -2; char srCommand = -1; int mins = 0; int secs = 0; int inByte = -1; int keysToGet = 4; int int int int digit3 digit2 digit1 digit0 = = = = 0; 0; 0; 0;
int inChar = -1; int sysState = STATE_MODE; boolean hasRun = false; /////////////////////////////// //--------PROTOTYPES----------// void modeSelect(); void trainWords(); void setTime(); void getKey(); int getKeypadButton(); boolean voiceButton(char); //Timer Functions void ready(); EECS 129B Voice Activated Presentation Assistant Page 26

void latch(int); void dBus(int); void flashLED(int); void countDown(); void getKeyDW(); void kpNumber(); void kpClear(); void kpEnter(); void polledDelay(int); void printTime(int, int); //////////////////////////////// void setup() { pinMode(ledPin, OUTPUT); pinMode(LD3, OUTPUT); pinMode(LD2, OUTPUT); pinMode(LD1, OUTPUT); pinMode(LD0, OUTPUT); pinMode(D3, OUTPUT); pinMode(D2, OUTPUT); pinMode(D1, OUTPUT); pinMode(D0, OUTPUT); pinMode(BL, OUTPUT); Serial.begin(9600); flashLED(500); } void loop() { if( hasRun == false ) { hasRun = true; ready(); modeSelect(); trainWords(); setTime(); } }
EECS 129B
Page 27

void ready() { while( true ) { if( Serial.available() > 0 ) { inChar = Serial.read(); if( inChar == 'R' ) { flashLED(500); delay(250); flashLED(500); break; } } } } void modeSelect() { sysState = STATE_MODE; getKey(); opMode = button; } void trainWords() { if( opMode == MODE_SPEECH_TIMER || opMode == MODE_SPEECH || opMode == MODE_TIMER_SC ) { sysState = STATE_TRAIN; int leftDigit = 0; int rightDigit = 0; while( !(leftDigit == KEY_5 && rightDigit == KEY_6) ) { getKey(); if( voiceButton(keyChar[button]) ) Serial.print(keyChar[button]);
EECS 129B
Page 28

leftDigit = rightDigit; rightDigit = button; } Serial.print('R'); } } void setTime() { if( opMode == MODE_SPEECH_TIMER || opMode == MODE_TIMER_SC || opMode == MODE_TIMER_KC || opMode == MODE_SPEECH_TIMER_DT ) { sysState = STATE_SETTIME; while( true ) { getKeyDW(); } } } void kpNumber() { if( (keysToGet > 0) && !((keysToGet == 4) && (inByte == 0)) && (sysState == STATE_SETTIME) ) { mins = mins*10 + (secs/10); secs = (secs%10)*10 + inByte; keysToGet--; printTime(mins, secs); } } void kpClear() { if( sysState == STATE_SETTIME ) { secs = (mins%10)*10 + secs/10; mins = mins/10; if( keysToGet < 4 ) keysToGet++; EECS 129B Voice Activated Presentation Assistant Page 29 //Tell slave Arduino to Run

} else if( sysState == STATE_PAUSED || sysState == STATE_COUNTDOWN ) { sysState = STATE_SETTIME; mins = 0; secs = 0; keysToGet = 4; } printTime(mins, secs); } void kpEnter() { if( (validTime(mins, secs) && sysState == STATE_SETTIME) || (sysState == STATE_PAUSED) && (srCommand != 'B') ) { sysState = STATE_COUNTDOWN; countDown(); } //"Begin" cannot pause or restart time else if( (sysState == STATE_COUNTDOWN) && (srCommand != 'B') ) { sysState = STATE_PAUSED; } } void getKey() { boolean hasKey = false; while( !hasKey ) { if( getKeypadButton() == 0 ) { srCommand = -1; hasKey = true; } } EECS 129B Voice Activated Presentation Assistant Page 30

} void getKeyDW() { srCommand = -1; //Speech input if( Serial.available() > 0 ) { srCommand = Serial.read(); if( srCommand == 'B' || srCommand == 'P') kpEnter(); } //Keypad input else if( getKeypadButton() == 0 ) { inByte = button; if( inByte != -1 ) { if( (inByte >= KEY_0) && (inByte <= KEY_9) ) kpNumber(); else if( inByte == KEY_CLR ) kpClear(); else if( inByte == KEY_TRN ) kpEnter(); } } } boolean validTime(int minutes, int seconds) { if( (minutes >= 0 && minutes <= 99) && (seconds >= 0 && seconds <= 59) ) { if( !(minutes == 0 && seconds == 0) ) return true; } return false; EECS 129B Voice Activated Presentation Assistant Page 31 //Begin or Pause

} boolean voiceButton(char keyPressed) { if( keyPressed == '1' || keyPressed == '2' || keyPressed == '3' || keyPressed == '4' || keyPressed == '8' || keyPressed == '9' || keyPressed == 'T' || keyPressed == 'C') return true; return false; } //Returns 0 if valid key pressed, else -1 int getKeypadButton() { int keyVal = analogRead(keypadPin); int keyButton = -1; lastButton = button; if( keyVal >= 40 && keyVal <= 55 ) keyButton = KEY_1; else if( keyVal >= 80 && keyVal <= 90 ) keyButton = KEY_2; else if( keyVal >= 120 && keyVal <= 135 keyButton = KEY_3; else if( keyVal >= 190 && keyVal <= 200 keyButton = KEY_4; else if( keyVal >= 300 && keyVal <= 315 keyButton = KEY_5; else if( keyVal >= 400 && keyVal <= 415 keyButton = KEY_6; else if( keyVal >= 540 && keyVal <= 550 keyButton = KEY_7; else if( keyVal >= 685 && keyVal <= 700 keyButton = KEY_8; else if( keyVal >= 775 && keyVal <= 790 keyButton = KEY_9; else if( keyVal >= 860 && keyVal <= 870 keyButton = KEY_CLR; else if( keyVal >= 920 && keyVal <= 935 keyButton = KEY_0; else if( keyVal >= 955 && keyVal <= 970 keyButton = KEY_TRN; EECS 129B
) ) ) ) ) ) ) ) ) )
Page 32

else keyButton = -1; button = keyButton; if( button != lastButton && button != -1 ) return 0; else return -1; } void latch(int displayNum) { digitalWrite(displayNum, HIGH); digitalWrite(displayNum, LOW); } void dBus(int number) { int d3 = LOW; int d2 = LOW; int d1 = LOW; int d0 = LOW; if (number % 2 == 1) d0 = HIGH; number = number >> 1; if ( number % 2 == 1) d1 = HIGH; number = number >> 1; if (number % 2 == 1) d2 = HIGH; number = number >> 1; if (number % 2 == 1) d3 = HIGH; digitalWrite(D3, d3); EECS 129B Voice Activated Presentation Assistant Page 33

digitalWrite(D2, d2); digitalWrite(D1, d1); digitalWrite(D0, d0); } void flashLED(int time) { digitalWrite(ledPin, HIGH); delay(time); digitalWrite(ledPin, LOW); } void countDown() { boolean stop = false; digitalWrite(ledPin, HIGH); while( (mins>=0) && (sysState == STATE_COUNTDOWN) ) { while( (secs >= 0) && (sysState == STATE_COUNTDOWN) ) { printTime(mins, secs); if( sysState == STATE_COUNTDOWN ) flashLED(250); else if ( sysState == STATE_PAUSED ) digitalWrite(ledPin, HIGH); if( sysState != STATE_PAUSED ) secs--; polledDelay(1000); } if( sysState == STATE_COUNTDOWN ) { mins--; secs = 59; } } while (true) EECS 129B Voice Activated Presentation Assistant Page 34

{ if( sysState == STATE_SETTIME || sysState == STATE_PAUSED ) { break; } printTime(0, 0); flashLED(500); polledDelay(500); digitalWrite(BL, HIGH); polledDelay(500); digitalWrite(BL, LOW); } } void printTime(int minutes, int seconds) { int M1 = 0; int M0 = 0; int S1 = 0; int S0 = 0; M1 M0 S1 S0 = = = = minutes/10; minutes%10; seconds/10; seconds%10;
dBus(M1); latch(LD3); dBus(M0); latch(LD2); dBus(S1); latch(LD1); dBus(S0); latch(LD0); } void polledDelay(int length) { int period = 25; int times = length/period; while( times > 0 ) EECS 129B Voice Activated Presentation Assistant Page 35

{ getKeyDW(); delay(period); times--; } }
EECS 129B
Page 36
18. Slave Controller Software Code

//-------CONSTANTS----------// //Keypad Buttons const char KEY_1 = '1'; const char KEY_2 = '2'; const char KEY_3 = '3'; const char KEY_4 = '4'; const char KEY_8 = '8'; const char KEY_9 = '9'; const char KEY_TRN = 'T'; const char KEY_CLR = 'C'; //SR values const int TR_BEGIN const int TR_NEXT const int TR_PREVIOUS const int TR_BLANK const int TR_EXIT const int TR_TIME //SR output pins const int SR_OUT_0 const int SR_OUT_1 const int SR_OUT_2 const int SR_OUT_3 const int SR_OUT_4 const int SR_OUT_5
= = = = = =
1; 2; 3; 4; 8; 9;
= = = = = =
2; 3; 4; 5; 6; 7;
//Transmitter decoder inputs const int DEC_BEGIN = 3; const int DEC_NEXT = 4; const int DEC_PREVIOUS = 2; const int DEC_BLANK = 1; const int DEC_EXIT = 5; //Decoder const int const int const int const int const int const int const int const int Inputs SR_1 SR_2 SR_3 SR_4 SR_8 SR_9 SR_TRN SR_CLR
= = = = = = = =
6; 7; 8; 9; 10; 11; 12; 13;
EECS 129B
Page 37
Slave Controller Software Code (Continued)

const const const const int int int int DEC_0 DEC_1 DEC_2 DEC_3 = = = = 8; 9; 10; 11;
const int STATE_INIT = 0; const int STATE_TRAIN = 1; const int STATE_RUN = 2; ////////////////////////////// //--------VARIABLES---------// char inChar = -1; boolean hasRun = false; int sysState = STATE_INIT; int srVal = 0; int srPrevVal = -1; ////////////////////////////// //---------PROTOTYPES-------// void trainWords(); void slideNav(); void srPress(char); void getInput(); void readSR(); void interruptSRCom(); int binToInt(int, int, int, int); ///////////////////////////// void setup() { pinMode(DEC_0, pinMode(DEC_1, pinMode(DEC_2, pinMode(DEC_3,
OUTPUT); OUTPUT); OUTPUT); OUTPUT); INPUT); INPUT); INPUT); INPUT); INPUT); INPUT);
pinMode(SR_OUT_0, pinMode(SR_OUT_1, pinMode(SR_OUT_2, pinMode(SR_OUT_3, pinMode(SR_OUT_4, pinMode(SR_OUT_5,
digitalWrite(DEC_0, LOW); digitalWrite(DEC_1, LOW); digitalWrite(DEC_2, LOW); EECS 129B Voice Activated Presentation Assistant Page 38

digitalWrite(DEC_3, LOW); Serial.begin(9600); } void loop() { if( !hasRun ) { hasRun = true; delay(5000); Serial.print('R'); trainWords(); slideNav(); } } void trainWords() { sysState = STATE_TRAIN; getInput(); while( inChar != 'R' ) { srPress(inChar); getInput(); } } void slideNav() { sysState = STATE_RUN; interruptSRCom(); while( true ) { readSR(); } } EECS 129B Voice Activated Presentation Assistant Page 39 //Tell master I am ready

void srPress(char srButton) { int decIn = -1; if( srButton == KEY_1 ) decIn = SR_1; else if( srButton == KEY_2 ) decIn = SR_2; else if( srButton == KEY_3 ) decIn = SR_3; else if( srButton == KEY_4 ) decIn = SR_4; else if( srButton == KEY_8 ) decIn = SR_8; else if( srButton == KEY_9 ) decIn = SR_9; else if( srButton == KEY_TRN ) decIn = SR_TRN; else if( srButton == KEY_CLR ) decIn = SR_CLR; if( decIn != -1 ) { decoder(decIn, false); } } void decoder(int number, boolean transmit) { int i = 0; int origNumber = number; boolean digit[4] = {false, false, false, false}; int decPin[4] = {DEC_0, DEC_1, DEC_2, DEC_3}; for(i=0; i <= 3; i++) { if( number%2 == 1 ) digit[i] = true; number = number/2; } for(i=0; i <= 3; i++) EECS 129B Voice Activated Presentation Assistant Page 40

{ digitalWrite(decPin[i], digit[i]); } if( transmit ) { if( (origNumber == DEC_EXIT) && (sysState == STATE_RUN) ) delay(3000); else delay(500); } else delay(100); for(i=0; i<= 3; i++) { digitalWrite(decPin[i], LOW); } } void getInput() { boolean haveInput = false; while( !haveInput ) { if( Serial.available() > 0 ) { inChar = Serial.read(); haveInput = true; } } } void readSR() { int sr0 = 0; int sr1 = 0; int sr2 = 0; int sr3 = 0; int sr4 = 0; int sr5 = 0; EECS 129B Voice Activated Presentation Assistant Page 41

int command[2][5] = {TR_BEGIN, TR_NEXT, TR_PREVIOUS, TR_BLANK, TR_EXIT, DEC_BEGIN, DEC_NEXT, DEC_PREVIOUS, DEC_BLANK, DEC_EXIT}; int val0 = 0; int val1 = 0; int val = 0; sr0 sr1 sr2 sr3 sr4 sr5 = = = = = = digitalRead(SR_OUT_0); digitalRead(SR_OUT_1); digitalRead(SR_OUT_2); digitalRead(SR_OUT_3); digitalRead(SR_OUT_4); digitalRead(SR_OUT_5);
val0 = binToInt(sr3, sr2, sr1, sr0); val1 = binToInt(sr5, sr4); val = (val1 * 10) + val0; srPrevVal = srVal; srVal = val; int i = 0; for(i = 0; i <= 5; i++) { if( val == command[0][i] ) { delay(75); sr0 sr1 sr2 sr3 sr4 sr5 = = = = = = digitalRead(SR_OUT_0); digitalRead(SR_OUT_1); digitalRead(SR_OUT_2); digitalRead(SR_OUT_3); digitalRead(SR_OUT_4); digitalRead(SR_OUT_5);
val0 = binToInt(sr3, sr2, sr1, sr0); val1 = binToInt(sr5, sr4); val = (val1 * 10) + val0;
EECS 129B
Page 42

break; } } for(i = 0; i <= 5; i++) { if( val == command[0][i] ) { decoder(command[1][i], true); interruptSRCom(); if( val == TR_BEGIN ) Serial.print('B'); break; } } if( val == TR_TIME ) { Serial.print('P'); interruptSRCom(); } } int binToInt(int d1, int d0) { int val = 0; if( d0 == 1 ) val += 1; if( d1 == 1 ) val += 2; return val; } void interruptSRCom() { delay(50); srPress(KEY_1); delay(50); EECS 129B Voice Activated Presentation Assistant Page 43
//Pause for time

srPress(KEY_8); delay(50); srPress(KEY_CLR); delay(75); } int binToInt(int d3, int d2, int d1, int d0) { int val = 0; if( d0 == 1 ) val += 1; if( d1 == 1 ) val += 2; if( d2 == 1 ) val += 4; if( d3 == 1 ) val += 8; return val; }
EECS 129B
Page 44
19. Component Descriptions
4 5 7 8 6
Figure 19 Top Level Component Descriptions
EECS 129B
Page 45
Component Descriptions (Continued)
1. Timer 2. Master Controller 3. Keypad Interface 4. Transmitter Switches 5. Transmitter 6. Speech Recognition Switches 7. Speech Recognition Board 8. Slave Controller
EECS 129B
Page 46
20. Enclosed Assembly
Figure 20 Enclosed Assembly
EECS 129B
Page 47
21. Cost Sheet
Arduino Microcontrollers VersaPoint Transmitter Speech Recognition Board 74HC154 Decoder PNP Transistors 74HC4543 Drivers 7-Segment Displays Breadboards Power Supply Resistors and Wires Cover and Wooden Support Total Cost
$60.00 $40.00 $100.00 $2.00 $5.00 $8.00 $10.00 $30.00 $20.00 $15.00 $60.00 $350.00
EECS 129B
Page 48
22. Test Plan Check for Damaged Transistors Verify that both PNP and MOSFET devices are working. Construct simple LED Transistor circuit as shown below to determine if LED turns on. Replace defective transistors.
Figure 21 LED Transistor Circuit
EECS 129B
Page 49
Check Decoder Voltages on Input / Output Lines Apply Voltmeter on pins 1 to 13 of Decoder and verify that the voltages are 5V for an input of 0000 on the input pins. Note that the decoder outputs active low output. Verify that E1 and E0 are both grounded.
Figure 22 Decoder Pin Layout Diagram
Test Speech Recognition Circuit Connect keypad and status display board to SR-06 and train words. Power on circuit using external power source or 9V battery. Train words by pressing pattern number followed by TRN button. Speak command into microphone. Make sure command is no more than 1.92 seconds. If word is successfully processed the red LED will blink once. Test that the words will be recognized by saying the word clearly. The pattern number of the word should appear on the status display board. If the word is not recognized, too long, or too short the following error codes should display on the board. SR-06 Error Codes: 55 = word too long 66 = word too short 77 = word no match
EECS 129B Voice Activated Presentation Assistant Page 50
IMPORTANT Never short power and ground on SR-06 as this will rapidly heat up voltage regulator and may cause permanent damage to the circuit.
Verify that VersaPoint Communicator can transmit Turn on PowerPoint and open a presentation. Connect 3V source to Transmitter and short each pair of terminals (red and black) Check that LED on the other side will blink to indicate successful transmission of signal. Test Countdown Timer Select Timer Only Mode and Set time with Keypad. Press TRN to start countdown. Verify that Keypad inputs work Use a test program that will read inputs from keypad and output both to status board and timer displays Check power and ground connections Verify that power source is being properly distributed to all major components. Measure input voltage on transmitter from 3V regulator. Measure input voltage on SR-06 from Arduino. (Verify that it is 9V) Measure output voltage across 5V regulator on SR-06. Measure input voltages on Timer and Keypad.
EECS 129B
Page 51
23. Conclusion Overall we have concluded that our project works successfully. Before reaching this milestone we encountered several learning experiences that will be discussed. We realized that the Microcontroller Interface between the Arduino and Speech Recognition circuit could have been easily done after trying to directly interface the Speech Kit to the Microcontroller in CPU mode. We debugged the system for about 3 weeks after we abandoned the decision to use CPU mode. We used a much easier interface which connects the Speech Kit to the Microcontroller using switches. The mode of operation for the Speech Kit was Manual mode which was much easier to work with. In addition to this problem we had to debug the noise on the board. We determined that there was noise causing the timer to flicker values. However, once the Arduinos reset we noticed that the timers blanked out which was good. We learned about Common Annode and Common Cathode configurations for Display Components and learned how to use them with our LCD drivers. Eventually with some ingenious programming we successfully programmed a countdown timer with reset capabilities. We learned that we could use a keypad configured with a resistor matrix to reduce the number of ports used for the keypad and programmed a driver program to accept the commands from the keypad. After noticing that the Speech Kits onboard regulator heated up considerably we purchased a clip on heat sink and mounted it on the regulator after relocating the regulator to the breadboard.
EECS 129B
Page 52
Appendix
HM2007 Pin Descriptions
Figure 23 Speech Recognition Pin Layout
Pin Names SA0-12 D0-7 MR/MW and ME X1 and X2 S1-3 K1-4 TEST LINE MICin WLEN CPUM
Pin Number(s) 17-24, 27-31 36-43 35, 34 2, 3 4-6 8-11 12 45 46 13 14
Description Memory address bus connections are between the HM2007 and the static RAM IC Data input bus three connections for each line: the HM2007, the static RAM, and the microcontrollers inputs Additional control lines for writing to and reading from external RAM connection is between HM2007 and RAM Connection pins for 3.58 MHz crystal Read/write control pins connection to microcontroller I/O Bi-directional data bus (K-bus) connection to microcontroller I/O Not used pull low Not used leave disconnected Microphone connection Word-length select pin. Pull high to use 1.92-second words, pull low to use 0.9-second words CPU/manual-mode select pin. Pull low for manual mode, pull high for CPU mode. Pulled high to select CPU mode in this report
EECS 129B
Page 53
Speech Recognition Circuit

Voltage Regulator
Status LED
Figure 24 Speech Recognition Pin Descriptions
EECS 129B
S1 S2 S3 K1 K2 K3 K4
GND POWER D7 D6 D5 D4 D3 D2 D1 D0
Page 54
Keypad Diagram
Pin # 4 5 6 8 9 10 11 Corresponding Pin Name S1 S2 S3 K1 K2 K3 K4
Figure 25 Keypad Pin Descriptions
EECS 129B
Page 55
SR-06/Arduino Interface
Figure 26 Arduino / Speech Recognition Interface Pin Mapping Data Bus On SR-06 D0 D1 D2 D3 D4 D5 Arduino Pins 2 3 4 5 6 7
EECS 129B
Page 56
The VersaPoint Commuinicator
Figure 27 -- Transmitter Circuit Board
EECS 129B
Page 57
Main Components On The VersaPoint Communicator

The Microprocessor The EM78P156EL is an 8-bit microprocessor designed by Elan Microelectronics Corporation that is designed with CMOS technology. A unique feature of the chip is its OTP-ROM. OTP-ROM stands for One Time Programmable Read Only Memory. Major features include. 1K X 13 bits on chip ROM 48 X 8 bits on chip registers (SRAM, and General Purpose Registers) 2 bi-directional I/O ports Low power consumption Power down mode
Figure 28 Microprocessor Pin Layout (On Transmitter)
EECS 129B
Page 58
Figure 29 Functional Block Diagram The instruction set in the microprocessor consists of 13-bit words that are composed of an OP code and several operands. Usually, the instructions are executed with one instruction cycle. However, if the PC becomes changed by a MOV, ADD, or by instructions of arithmetic or logic operation then the execution takes two cycles.
EECS 129B
Page 59
The Transmitter The TDA5102 is designed by Infineon Technologies and is an ASK, FSK transmitter whose transmission frequency band is between 905 and 925 MHz. Major features include High level of integration Need for only a few external components Integration of a PLL (frequency) synthesizer High efficiency power amplifier to drive a loop antenna Power down mode Selectable XTAL oscillator at 7.15MHz/14.3MHz
Figure 30 Transmitter Pin Layout
EECS 129B
Page 60
Figure 31 Functional Block Diagram
EECS 129B
Page 61

Voice Activated Presentation Assistant

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Voice Activated Presentation Assistant

Uploaded by

Copyright:

Available Formats

V O I C E A C T I VA T E D P R E S E N T A T I O N A S S I S T A N T

Brian Chen Eunsang Cho Vahe Jabagchourian Vatche Jabagchourian

UNIVERSITY OF CALIFORNIA IRVINE