Automation Framework 1.

Introduction This Automation Framework Document outlines the automation architecture, approach, execution and result flow in SELENIUM RC. 2. Approach The and The and test automation framework is developed using open source tool SELENIUM (IDE & RC) JAVA. The proposed Framework provides Data-driven and the Keyword-driven support. User Actions, Input values and Test Data are recorded and read from XLS & other files write results in to the XLS files.

3. Framework Features In addition to standard features such as performing operations and verifications on the objects the framework includes other sophisticated features. The key features of the automation framework are listed below: Supports-Multiple Browsers and Operating Systems (Higher Automation ROI). Windows Internet Explorer and Google Chrome. Mac Mozilla Firefox and Safari. Test scripts are generated using MS-Excel (Keyword Action Approach). Handles multiple datasets captured in MS-Excel. Use of variables: Variables can be defined and used across the generated test script. Reports: Customized reporter messages can also be used to perform effective analysis on execution reports. These reports can be customized to display the pass or fail condition of any functionality. Generates HTML Summary Report on completion of test execution. Exception handling: Runtime errors can be effectively handled and reported using this framework. Highly Modularized and Reusable Functions: This framework supports a functional decomposition approach. This increases the reusability of functions, which in turn reduces the unnecessary repetition of steps. Easy to update and maintain (Scripts & MS-Excel). Developed using Selenium with Java interface. Keyword-Driven Automation Framework: Keyword-based test design and test automation is formed on the idea that the discrete functional business events that makeup any application can be described using short text description (keywords). Data-driven testing: This framework supports data-driven testing by importing data from an external data sheet. Maintenance: Simple modifications to the application can be easily handled in the code. The changes will be done only in the external file containing the code and the scripts need not be changed. Test Execution Report will be sent automatically through the Email. Selenium was first written in Java but it also supports .Net, Ruby, Perl, PHP and Python. This is a big plus when you want to build your framework in a language that has the highest adoption in the organization it is being built within.

You can instantiate several concurrent tests with Selenium. Test Automation Framework is application-independent: This Automation Framework is application-independent because the Open Source Test Automation Framework is designed to perform operations for all standard object types (WebEdit, Weblist, Webcheckbox, etc.) of technology commonly used to build a Web application. This Test Automation Framework reduces maintenance and increases productivity.

Effectiveness in handling unexpected application behavior. Keywords are application-independent. Benefits: Reusability, greater test productivity, optimum utilization of the tool through keyword support, and minimum effort needed to build scripts, Conditional checking, Data-driven testing, Reports, Exception handling, Handling Web objects. Reduced dependency on technically skilled resources. Reliable: The framework provides standardized, tested code. Flexible in test suite and test case configuration. Test Script development is possible without a functioning application. Test cases designed in this framework are less prone to human errors. If functionality changes, only the specific test scenario needs to be changed. Actual and expected results are stored in Result Folder. You can Run the AUT in different Browsers in single Testscript. It minimizes the requirement of manual software tester.

4. Folder Structure

Figure: Folder Structure. 4.1. IATF: This is the main folder with subfolders as shown in the above diagram. Here the IATF is the Project Name. The Project IATF contains subfolders or files that should not be renamed or moved. The project Folder (IATF) contains the sub folders like libraries, properties, Results, Testscripts, Testsuites and JAR file (iatf). As shown in the above figure. 4.2. Libraries: The libraries folder contains the Selenium related JAR files 4.3. Properties: The properties folder contains the two type of property files. 1. Application Property. 2. UI Element Property.

Figure: Application Property File and UI Element Property File. 1. Application Property. This Property file contains some one time Configuration details such as serverHost, serverPortNo, browser, application URL and some others details which are changes based on Application build such as buildNo ,buildDate, MODE_OF_RUN. This file contains data related to the particular Application under Automation Test. Example: # Application properties file serverHost=localhost

serverPortNo=4444 browser=*firefox,*iexplore MODE_OF_RUN=N buildNo=32 buildDate=3rd Feb, 2011

2.

UI Element Property File.

This Property file contains the actual Application related UI element objects which are map with some logical name. This logical name must match with the data under column named as UI Element of testscript in order to fetch element object for process. This file contains data related to the particular Application under Automation Test. Example: #UI Element Property file for Login and Logout Flow. frmLogin.lblLoginTitle=x-auto-1-label frmLogin.txtUID=username-input frmLogin.txtPWD=password-input frmLogin.chkBoxRememberMe=x-auto-12 frmLogin.btnLogin=//table[@id='aff']/tbody/tr[2]/td[2]/em/button frmLogin.lnkLogout=link=Log out

Before running the Driver script the Property Files should be updated appropriately and the scripts should available under TestSuite folder. 4.4. Results:

The Results folder contains the status of each testscript(Including testscript status along with individual steps whether Pass, Fail etc). We can see in the above figure the Results folder contains the Timestamp and each timestamp will consists of below sub folders as follows:

Abort: Once the testscript under execution mark as aborted then that will move to Abort Folder Fail: Once the testscript under execution mark as fail then that will move to Fail Folder Pass: Once the testscript under execution mark as pass then that will move to Pass Folder The testScript based on the status (pass/Fail/Abort) move to the respective folder and saved with the name that included suffix at the end of the file name that specify for which browser the particular script run for. The suffix could be: FF- shortcut for Firefox IE- shortcut for Inter SF- shortcut for Internet Explorer CH- shortcut for Chrome

ScreenShots: while executing the testsuite, if any steps mark as fail ,then the ScreenShots will be captured and moved to ScreenShots Folder TestSummary_timestamp: After the execution of all testScript inside TestSuite this file will be updated with several information.

4.5. Testscripts: This folder contains all Application related testscript. Only those scripts would execute which are available under Testsuites folder. 4.6. TestSuites: All the testscripts(copy from Testscripts folder) available under this folder will be executed as part of automation.

4.7. iatf.jar This file act as a driver Script. Once everything done as mentioned above, user can run this file from command prompt. 4.6. Test Suite: Test Suite folder contains the list of test scripts (N number of test scripts) for execution. The scripts that need to be executed are copied under Test Suite folder before execution. Remember that the scripts name should be unique in the Test Suite. In test Suite we can run N number of Test Scripts.

In TestSuite folder contains the TestScript or N number of TestScripts as shown below:

In TestSuite you just copy and paste the TestScripts / TestScript (xls).

5. Test Script Structure (xls) Test Script (xls) is the sequence of keywords/actions designed to test one or more aspects of the application (AUT) being tested. Test Scripts (.xls) written by the tester have 4 worksheets as follows MainTestScript, Datapool, Iteration and Log These above mentioned sheet name must be appear as written above. 5.1. MainTestScript / Test Script Worksheet: The MainTestScript should be in following Format. This worksheet has 9 columns: Step, Browser, Page, UI Element, Action, Test Data, Table Identifier, Comments and Severity. The Column name and their positioned should not be changed.

5.1.1. Step:

Driver Script reads the MainTestScript sequentially based on the Step number and the step will be executed. The step number should starts with number 1.When the system update the Log, these step number make it easy for user to track which part of the application got failed/pass/abort etc.

5.1.2. Browser: Browser column we have to specify that which browser we are using for Test or Execution. Example: S H A R P. 5.1.3. Page: Page column will show that the Page of the Browser window. Some times Browser window and Page will be same or it may differ. Note: Browser and Page are different. Example: Page column will show that the Web, my browser window (Ex: S H A R P) opens up on the Web page (Ex: S H A R P). 5.1.4. UI Element: The data value for this field follow standard naming convention as follow: Object Name Link Image Button List Box Text Field Table Check Box Radio Button Label Prefix lnk img btn lst txt tbl chk rbt lbl

The data value for this field should be prefix with the one specified above. The naming convention listed above should be used consistently in test scripts, any deviation will result in script failure. Below example shows how to make UI Element properties. Note: The data value for this field must be available in UIElement.properties (under Properties Folder) 5.1.5. User Action / Action: Action is nothing but a USER ACTION indicates operation to be performed on AUT (Application Under Test). The action name is a short description of the actions that keyword of a keyboard performs. Examples: Click: Used to click any Button in the AUT. Type: Used to enter the value in the Edit field in the AUT. Check: Check is used for Check the Check Boxes in the AUT. AssertTitle: Check is used to check the Title or Labels in the AUT.

5.1.6. Test Data / Data: The Input values are recorded in this column. In addition to the input values other keywords are implemented to provide more flexibility in the framework. We can give test data according to our need. The Test Data is very important in testing an application. Test data is the Input feed for Testing the Application. To feed the test data to check that the outputs are derived correct. Test Data varied for different scenarios in testing (valid/invalid). While defining the Test data might be useful for other users/developers that what system gave for the given inputs. Test Data help the developers to find the problem during fixes. 5.1.7. Table Identifier: Main purpose of this field is to inform the system about important of a particular step as validation failed at this step, then the present testscript will be aborted and further process would be terminated. The input value for this field could be either Blank or required. Default value is Blank. When value specified as required means it Mandatory i.e. validation should be pass at this step. 5.1.8. Comments: Comments are general comments which are used by the QA person about a particular step. 5.1.9. Severity: When any fail occur in application this field value specify the type of severity. It is of three type as follow: Critical Major Minor Which are refer as S1, S2 and S3 respectively.

the

5.2.

Datapool:

In the program we will fetch the dynamic value from the iteration no 1 without user involvement. A datapool is a test dataset, a collection of related data records which supplies data values to the variables in a test script during test script playback.

5.3.

Iteration:

Iteration refer repeating the same set of step again & again to fix number of times. When we will run the TestScript the sheet named MainTestScript will start executing stepby-step and when it will reach to the user action IterationStart that time the control goes to the sheet named Iteration, and execute some set of steps and again come back to main sheet The number of time the set of steps will execute will depend on the column value label as Data user MainTestScript

5.4.

Log Worksheet:

The sheet name Log initially blank and filled at runtime step-by-step as the execution progress. The main purpose of having Log sheet is to easily track the status of each track of the testScript. In below Figure we can see the it appear blank.

Below Log (xls) filled step-by-step when testScript execution progress

6.

Test Execution Flow:

Following are steps to run the framework:

1) Install java (jdk1.6.x) 2) ANT(apache-ant-1.7.x) 3) Set environment variable

4) Start selenium server 5) Run application Set environment variable 1) Click on start button 2) Right click on label My Computer 3) Click last option i.e properties 4) Click on tab label as Advanced 5) Click on the button labeled as Environment Variables 6) Click on the button labeled as Edit under system variables box and provide the following value one by one. variable name ANT_HOME IATF_HOME JAVA_HOME JAVA_LIB Start selenium server: variable value C:\apache-ant-1.7.0 C:\IATF C:\Program Files\Java\jdk1.6.0_14 C:\Program Files\Java\jre6\lib

1.

Start Selenium RC Server using command prompt.

a)
c)

Open command prompt.

b) Go to the directory where selenium-server. Jar is available in system

Execute the following command java -jar selenium-server. Jar

After that below window will display.

Run application: 1) open command prompt

2) Go to the directory where driver script available in your system

3) Execute the following command: Java jar iatf.jar

Application Run Mode

The application under automation can be run in two different mode:

1) Run/Unattended mode
2) Debug/Attended mode.

Launching a application in debug mode: Launching a program in debug mode allows you to suspend and resume the program, inspect variables, and evaluate expressions using the debugger.

To launch a Test Script/Java program in debug mode, first go to the Application Property ( eg C:\IATF\Properties\ application.properties) Set the following variable value as follows: MODE_OF_RUN=A. Here A means Attended mode Example: When we will execute the Test Script in Debug/Attended mode, while running the Test Script if object is not found then system will ask the user for their input as shown below: DO YOU WANT TO CONTINUE (Y/N) option if you press Y Then it will ask from which step you want to continue the execution i.e. Enter StepNo :: if you say 2 then system continue start executing as per user input

Launching a application in run mode: To launch a Test Script/Java program in debug mode, first go to the Application Property ( eg C:\IATF\Properties\ application.properties) Set the following variable value as follows: MODE_OF_RUN=R. Here R means Run mode The Run/Unattended mode is normal mode.

Test Report Analysis After the completion of the execution of the all scripts, system will generate one report which are placed under result folder that consist of two type of details. Which are as follows:

1) 2)

Summary Report. Detail Report.

Summary Report: This part of the report consist of the following information: 1) 2) 3) 4) 5) 6) 7) Build Information of the application Date of execution Total number of scripts run Application run on which all browser Scripts status like Pass/Fail/Abort based on browser Total scripts Pass/Fail/Abort on each browser Total severity like critical, major and minor Script status along with total number of fail if occur.

8)

7.1.

Test Report/Detail Report:

Sheet Name: The value for this field display the testScript name along with the suffix which indicate the browser type(like FF,IE,CH,SF)

Step: It display the step number of the scripts for which validation have been passed /Failed/Abort. Field : Display the element object of the application.

Action: It will show which action Performed on the AUT. Expected Result: This value for this field hold the user expected value Actual Result: Actual value that system captured while validating the element Object during execution time

Result: It will show whether particular validation of test script is Passed, Failed, etc. a) Failed : If actual result deviates with the expected b) Passed : If the actual result is same as expected result c) Aborted : If actual result deviates with the expected but in this case it is mandatory for these two value to be same Severity: It is user defined value, based on the severity of a particular step. Example: S1 Critical. S2 Major. S3 Minor.