Human Controlled Remote Robotic Arm (HCRRA)

Team Members: Amithash E. Prasad (995-22-0945) Murtuza Tambawala (630-04-6490)

Index
Introduction Design Overview Detailed Design System Testing Users Guide Conclusion References Appendix 03-03 04-05 06-09 09-11 12-13 13-13 14-14 15-19

Introduction
The Human Controlled Remote Robotic Arm (HCRRA) is a system created to maneuver a robotic arm remotely, to perform simple tasks i.e. lifting and moving simple light weighted object. We started this project with the idea that a system like this is presently used in automated industries and tele - robotics. The input to control the robotic arm comes from a sensor glove. The sensor glove we have used is an off the shelf data glove, which uses fiber optic flexure sensors and accelerometers to detect motion. We have made use of the 5DT Data Glove 5. The user wearing the sensor glove can be at a reasonable distance from the robotic arm. Our design uses RS232 interfaces and thus has a maximum allowable distance between user and arm of 50 feet. This can easily be increased to up to 4050 feet by making use of alternate interface cables. HCRRA Project Minimum Objectives: 1. Achieving clamping mechanism (Index finger and thumb movement) 2. Achieving elbow movement (1 DOF) HCRRA Project Target Objectives: 1. Detection and achieving elbow, rotational movement (2 DOF) 2. Achieving clamping mechanism (Index finger and thumb movement) 3. Limiting control to project over-shoot of a DOF which will damage the gear or robotic arm HCRRA Project Goal Objectives: 1. Detection and achieving elbow, shoulder and rotational movement (3 DOF) 2. Achieving clamping mechanism (index finger and thumb movement) 3. Non noticeable delay between sensing to actuation (less than 500ms) So far in the project we have successfully reached the project goal objectives. With additional timing management and synchronization we will be able to exceed our goals.

Design Overview
Hardware
The HCRRA consists of three main modules that is the 5DT Glove with flexor sensors and accelerometer, the robotic arm with a minimum of 4 Degrees of Freedom (DOF) of movement and finally a multi-tasking computer system which will monitor sensor signals and hence control the actuators in real time on the robotic arm.

Figure 1: High Level Hardware Design

Software
The software is divided into 3 main modules namely: The glove interface, Data Manipulation and finally the Arm interface. The glove interface module interfaces to the Com port 1 serial interface driver and polls for data every 500ms. Once this glove data is in, the Data manipulation task collects the data, manipulates it and creates commands to drive the relay interface. The Robotic Arm interface module collects these commands and drives the relays via a serial interface through com port 0. All these modules must work in real-time else a noticeable lag between the glove control and the actual actuation of the robotic arm will be unavoidable.

Glove Interface Module (Serial Port)

Data Manipulation Module

Robotic Arm Interface Module (Serial Port)

Serial Com Port 1

Serial Com Port 0 Figure 2: Software Design Overview

Hardware and software data flow diagram:

Figure 3 below shows the general data flow diagram of the HCRRA. This gives an overview of the functionality of the software and hardware working in synergy to achieve the final goal of controlling the robotic arm with a user wearing the glove to cause actuation.

HARDWARE
Data Glove with flexor sensors and accelerometers to detect hand motion

Serial RS-232 Interface to the computer.

Glove interface module which polls for data over the com port every 500ms

RS-232 serial interface to the computer to turn on relays. Robotic Arm

Sends commands over an RS-232 serial interface to the relay module to switch on the DC motors on the robotic arm in either direction. Figure 3: Data Flow Diagram

Data manipulation module takes the data and thresholds them to digitize the data from the glove

Detailed Design
Hardware
The robotic arm OWI-007 used is capable of 5 degrees of motion and hence equipped with 5 DC motors. These are interfaced to the relay board. 2 relays are used per DC Motor one for either direction. Robotic Arm Robotic Arm Base Shoulder Elbow Relay Board Wrist Gripper RS-232 Serial Interface Limiting Switches Board to switch off relays based on limits 16 Relay Control Board with RS-232 Interface

Target: Com port 0

Target Board

Target Com port 1

5DT Data Glove 5

Figure 4: Detailed Hardware Design

The robotic arm used in the project is an OWI-007 five-degree of movement (5-DOF) arm, namely: Gripper Wrist Elbow Shoulder Base The 5 DOF covers the major and most common arm movements to cover a large area, and it also makes the arm easy to maneuver to lift and move objects in any direction. This is connected to a relay expansion card. A small interface board, which is used to connect the limiting switches, which turns the relay off when a limit is reached and finally to the relay control board which has 16 relays of which 10 are used. This connects to the target over a RS232 serial interface at a baud rate of 19200. The sensor glove has 7 degrees of movement namely: 5 Fingers Pitch Roll In the HCRRA project we make use of 4 DOF of the arm, which is mapped to 6 DOF of the data glove, to give both up-down and left-right movement of the 4 DOF in the robotic arm. The glove data is communicated to the target at a baud rate of 19200 through an RS-232 interface.

Software
Serial Interface: Data Glove The glove operates in two modes (Glove mode and mouse emulation mode) and sends out two kinds of frames based on the mode. The data from now on is termed as gdata (For glove mode) and mdata (for mouse emulation mode). The project is executed only in the glove mode, thus the description of the mouse emulation mode is out of scope of this text. Glove Frame: H F1 F2 F3 F4 F5 P R C

Figure 5: Frame format coming from the Serial Port H (Header) - Is always a byte with a value of 0x80 (128 decimal). And denotes the start of the frame. F1 (Finger 1) Thumb F2 (Finger 2) - Index F3 (Finger 3) - Middle 7

F4 (Finger 4) - Ring F5 (Finger 5) Little F1 F5 indicates the flexures of the corresponding finger. Each flexure value has a decimal range of 0 to 255, with a low value indicating an un-flexed finger, and a high value indicating a flexed finger. P (Pitch) and R (Roll) are byte values ranging from 0 to 255. A value of 128 for either sensor indicates that the hand is un-tilted in that axis (center value). Finally C (Checksum) can be used to check if the frame has no bit-errors. Serial Interface: Relay Board 1. The relay ON addresses are calculated as follows: (Relay number * 2) 1 2. The relay OFF addresses are calculated as follows: (Relay number * 2) 2 Relay numbers start from 17 32. We use an EX-16 relay expansion card connected to the relay control board having 16 relays. The exact mapping can be found in arm.h in the Appendix.

START

Include header file Arm.h this includes definitions for arm commands

Open and initialize serial driver for COM-2

Set baud rate 19200 for Com-2 and set data receive and send to type raw
YES

Send Glove initializations commands A to reset to command mode

Give Sem and delay for 500ms

Threshold the data and save to array Movement

Send command C and read 9 bytes from the glove into an array

If abortall No End and Exit Task

TASK GLOVE

Figure 6: Task Glove software flow diagram

End and Exit Task

START

NO YES If abortall=0 Take Sem

Initialize and open COM-1 (Arm) and set baud rate to 19200

TASK ARM

Send commands to relay based on data in Movement array

Figure 7: Task arm software flow diagram Start task spawns task glove (priority 70) and task arm (priority 80), and also initializes the semaphore shared by these two tasks. Shut task sets abortall to 1 so both the tasks ends and also closes the opened serial ports.

System Testing
We divided our testing into module based testing of separate functional units. In the end we intensively tested the combined final unit. Robotic Arm Test Plan and Results: 1. After assembling the robotic arm we soldered the control board to test if all the 5 DOF in both directions were functioning properly. 2. By alternating the DC motor inputs between +6VDC and 6VDC we were able to test each DOF in both directions. 3. All movements in either direction were functioning properly. Relay Board Test Plan and Results: 1. We wrote a RS-232 serial protocol to connect to the relay board, to study the addresses to switch the relay ON or OFF based on the address. 2. The relay board test program consisted of a simple for loop to run over the addresses to switch each relay ON alternatively and then one for loop to switch them OFF. 9

3. The test program ran at a baud rate of 19200 and the results were as follows: Relay 17 not functioning Relay 22 not functioning Relay 32 not functioning Relay 33 not functioning Data Glove Test Plan and Results: 1. The data glove test plan was to test the basic functioning of the glove and test the input data stream. We also had to test if each degree of motion was functioning and producing expected results. 2. We wrote a simple serial interface protocol to connect to the glove at 19200-baud rate. 3. Character A was sent to the glove to enable it in command mode, we waited for a character U to make sure the glove went into the command mode successfully. 4. Character C was sent to the glove to enable the user to receive streaming data i.e. gdata, continuously from the glove. 5. We saved the input array from the glove continuously in a global array in 300 ms loops. This 300ms delay in loops was set based on a trial and error to match the arm movement speed in real time. 6. We then split the 9 data bytes received and printed them out separately to view the changes in received data per movement and to calibrate the glove. 7. Based on these observations we came up with the threshold limits for the glove data. 8. The result of this test was the output data stream and mapping the output vales from each DOF of the glove to a particular gesture of the arm. It helped us calibrate the glove and gestures to best fit our robotic arm movement. It also served as a basic test to make sure all DOF of the glove functioned and gave expected outputs. Limiting Sensors Test Plan and Results: 1. Force every DOF direction, to go to its extreme position and observe if the relay is turned off. Final System Test Plan and Result: 1. Keep the arm with the glove straight ahead with only the thumb flexed and palm facing the floor. This is the default position. 2. Flex the index finger and the thumb. This should cause the gripper to close 3. Un-Flex the index by keeping the thumb flexed. This should cause the gripper to stop. 4. Un-Flex both the index finger and thumb. This should cause the gripper to open. 5. Repeat step 3 to stop the gripper. 6. Keep Thumb flexed and now flex the middle finger. This should cause only the elbow to move down. 7. Just un-flex the middle finger. Thus should stop the elbow from moving. 8. Flex the ring finger. This should cause the elbow joint to move up.

10

9. Repeat step 7 to stop the elbow from moving. 10. Now raise the arm above the head. This should cause the shoulder to move up. 11. Return to the initial poison. Thus should stop the arm from moving. 12. Now release the arm and lower it to the waist. This should cause the arm to move down. 13. Return to the initial position. 14. Now rotate the arm clock wise such that the palm is facing the left. This should cause the base of the arm to rotate to the right. 15. Return to the default position. This should cause the base to stop rotating. 16. Rotate the arm counter clockwise such that the palm is facing the right. This should cause the base to rotate to the left. 17. Return to the default position to stop the base from rotating. 18. At any of the above steps, make a fist by flexing all the fingers. This should stop all movement. All the above tests passed on test.

11

Users Guide
Setup: 1. Connect power to the glove, the relay board and the Robotic Arm. 2. Connect the Glove to Com 2 and the relay board to Com 1 port of the target. 3. Wear the Glove and execute the Start function in the target. 4. The below pictures give details of the gestures to control the robotic arm.

Default Position: No movement

Emergency Arm shut off

Close Gripper

Open Gripper

Elbow Down

Elbow Up.

12

Shoulder Up

Shoulder Down

Base Right

Base Left

Conclusion
This project was a great learning experience for us. We started a project with the idea of a project, which would have the plasticity of future expansion, and also keeping in mind its practical uses. We learnt new concepts on how to interface separate units of a system keeping in mind all timing issues and making it real time. We also learnt how to use and configure off the shelf units, to match our requirement. We believe our most enjoyable part of the project was testing each section independently, and then testing the entire unit. This led us into many issues, which we over came with minor, and sometimes major changes. This project taught us to configure serial interfaces and manage 2 serial interfaces at the same time. Finally we like to conclude by saying that this project can lead to many more practical implementation and expansion into bigger and useful systems. One thing we would like to have done if time permitted was to implement soft limits using the parallel interface of the target.

13

References
1. 2. 3. 4. 5. OWI-07 Robotic arm user manual EX-16 Relay Expansion Card technical reference 5DT Data Glove 5-user manual. Robotic arm chapter in Real-time Embedded systems and components by Sam Siewert Sam Siewert for providing the data glove and useful comments, which started us off in the right direction. 6. The Serial interface example code from the examples section of the 5623 course home page.

14

Appendix
Source Code:
Arm.h
/*Wrist*/ #define WRIST_CW_OFF #define WRIST_CW_ON #define WRIST_CC_OFF #define WRIST_CC_ON /*Shoulder*/ #define SHOULDER_DOWN_OFF #define SHOULDER_DOWN_ON #define SHOULDER_UP_OFF #define SHOULDER_UP_ON /*Gripper*/ #define GRIPPER_CLOSE_OFF #define GRIPPER_CLOSE_ON #define GRIPPER_OPEN_OFF #define GRIPPER_OPEN_ON /*Elbow*/ #define ELBOW_DOWN_OFF #define ELBOW_DOWN_ON #define ELBOW_UP_OFF #define ELBOW_UP_ON /*Base*/ #define BASE_RIGHT_OFF #define BASE_RIGHT_ON #define BASE_LEFT_OFF #define BASE_LEFT_ON #define #define #define #define #define GRIPPER WRIST ELBOW SHOULDER BASE 0 1 2 4 44 45 46 47 40 41 58 59 52 53 34 35 36 37 38 39 56 57 50 51

/*global variables*/ /*global variables*/ static int arm1; unsigned char outbyte; static int sd;/*the serial device id returned by open*/ unsigned char output;/*variable which outputs to serial device*/ unsigned char input;/*variable which takes only 1 char from serial device*/ unsigned char inputarr[9]={0};/*9 byte array to recieve the frame from the glove*/ int bytes;/*no of bytes*/ int abortall = 0; SEM_ID Glove_Sem;

15

const int Threshold[7][2] = {{0x45, 0X45}, {0x55, 0X55}, {0x55, 0X55}, {0x55, 0X55}, {0x55, 0X55}, {0xE9, 0X0A}, {0xC9, 0X35}}; int Movement[7] = {0}; int count = 0; void Arm_Init(void) { int status; printf("Serial Driver Test\n"); if((arm1 = open("/tyCo/0", O_RDWR, 0)) == ERROR) { printf ("Error opening serial interface\r\n"); fflush(stdout); } /* put the serial interface into "raw" mode */ status = ioctl(arm1, SIO_BAUD_SET, 19200); status = ioctl(arm1, FIOSETOPTIONS, OPT_RAW); printf("driver intialized and opened with status=%d\n", status); fflush(stdout); } void Arm(unsigned int Command) { int j=0; outbyte = (unsigned char) Command; j=write(arm1, &outbyte,1); }

Glove.c
#include <stdio.h> #include <stdlib.h> #include <errnoLib.h> #include <ioLib.h> #include <sioLib.h> #include <semLib.h> #include "Arm.h" #define GLOVE_TASK_PERIOD 500 void glove(void) { int status, i; int j = 0; printf("Serial Driver Test\n"); if((sd = open("/tyCo/1", O_RDWR, 0)) == ERROR) { printf ("Error opening serial interface\r\n"); } /* put the serial interface into "raw" mode */ status = ioctl(sd, SIO_BAUD_SET, 19200); status = ioctl(sd, FIOSETOPTIONS, OPT_RAW);

16

printf("driver intialized and opened with status=%d\n", status); fflush(stdout); printf("reset glove to command mode\n"); fflush(stdout); output = 0x41; /*send 'A' to set the glove in test IO mode*/ bytes += write(sd, &output, 1); fflush(sd); bytes += read(sd, &input, 1); /*the glove must send 'U' back*/ while (!abortall) { output = 0x43;/*send 'C' to put the glove in report mode to send the gdata*/ bytes = write(sd, &output, 1); fflush(sd); bytes = read(sd, inputarr, 9);/*read the 9 bytes of data*/ fflush(sd); j=0; for(i = 1; i<8; i++) { /*Threshold the data and store in Movement[]*/ if(inputarr[i] > Threshold[i-1][0]) Movement[j] = 1; else if(inputarr[i] < Threshold[i-1][1]) Movement[j] = -1; else Movement[j] = 0; j++; } /*ignore the first 5 input data as these are usually erroneous then semGive so the arm task can take over*/ if(count > 4); semGive(Glove_Sem); count++; /*delay for the time in ms mentioned in GLOVE_TASK_PERIOD*/ taskDelay(GLOVE_TASK_PERIOD);/*delay for 100ms and then repeat 150 times*/ } } void Arm_task(void) { Arm_Init(); /*initialize the arm*/ while(!abortall) { /*take sem if available, else block forever*/ semTake(Glove_Sem, WAIT_FOREVER); /*if all the fingers are flexed, shut off all motors*/ if(Movement[0] != -1 && Movement[1] != -1 && Movement[2] != -1 && Movement[3] != -1 && Movement[4] != -1) { Arm(GRIPPER_OPEN_OFF); Arm(GRIPPER_CLOSE_OFF); Arm(WRIST_CC_OFF);

17

} /*else, read and do the following*/ else { /*Gripper, index finger and thumb*/ if(Movement[1] == -1 && Movement[0] == -1) { Arm(GRIPPER_CLOSE_OFF); Arm(GRIPPER_OPEN_ON); } else if(Movement[1] != -1 && Movement[0] != -1) { Arm(GRIPPER_OPEN_OFF); Arm(GRIPPER_CLOSE_ON); } else { Arm(GRIPPER_OPEN_OFF); Arm(GRIPPER_CLOSE_OFF); } /*Shoulder, pitch data*/ if(Movement[5] == -1) { Arm(SHOULDER_DOWN_OFF); Arm(SHOULDER_UP_ON); } else if(Movement[5] == 1) { Arm(SHOULDER_UP_OFF); Arm(SHOULDER_DOWN_ON); } else { Arm(SHOULDER_UP_OFF); Arm(SHOULDER_DOWN_OFF); } /*Base, roll data*/ if(Movement[6] == -1) { Arm(BASE_RIGHT_OFF); Arm(BASE_LEFT_ON); } else if(Movement[6] == 1) { Arm(BASE_LEFT_OFF); Arm(BASE_RIGHT_ON); } else

Arm(WRIST_CW_OFF); Arm(ELBOW_UP_OFF); Arm(ELBOW_DOWN_OFF); Arm(SHOULDER_UP_OFF); Arm(SHOULDER_DOWN_OFF); Arm(BASE_LEFT_OFF); Arm(BASE_RIGHT_OFF);

18

{ }

Arm(BASE_LEFT_OFF); Arm(BASE_RIGHT_OFF);

} }

/*Elbow, middle and ring finger*/ if(Movement[2] == 1) { Arm(ELBOW_UP_OFF); Arm(ELBOW_DOWN_ON); } else if (Movement[3] == 1) { Arm(ELBOW_DOWN_OFF); Arm(ELBOW_UP_ON); } else { Arm(ELBOW_DOWN_OFF); Arm(ELBOW_UP_OFF); }

} /*call this to shut the program*/ void shut(void) { abortall=1;/*so all the loops end*/ close(sd);/*close both the file descriptors*/ close(arm1); } /*Call this to start the program, it spawns both the tasks*/ void Start(void) { /*set system clock rate to 1000*/ sysClkRateSet(1000); /*create and initialize the semaphore, initially empty*/ Glove_Sem = semBCreate(SEM_Q_FIFO, SEM_EMPTY); count = 0; /*arm task spawned with priority 80*/ if(taskSpawn("Arm_task", 80, 0, 4000, Arm_task, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) == ERROR) { printf("Task Arm_task task spawn failed\n"); } else printf("Task Arm_task task spawned\n"); /*glove task spawned with priority 70(higher priority)*/ if(taskSpawn("glove", 70, 0, 4000, glove, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) == ERROR) { printf("Task glove task spawn failed\n"); } else printf("Task glove task spawned\n"); }

19