CSE 403

Lecture 15
UI Automation / Functional Testing
Reading:
How to Break Software, Ch. 2, Whittaker
slides created b !art "te##
htt#://$$$.cs.$ashington.edu/%&'/
2
Recall: Kinds of testing
(
unit testing: looks )or errors in ob*ects or subsstems
(
integration testing: )ind errors $hen connecting subsstems
(
system testing: test entire sstem beha+ior as a $hole, $ith
res#ect to scenarios and re,uirements
-
)unctional testing: test $hether sstem meets re,uirements
-
#er)ormance testing: non)unctional re,uirements, design goals
-
acce#tance / installation testing: done b client
3
Functional testing
(
ad-oc: .ust run the #roduct and click things.
(
!" automation: "imulate usage o) a #roduct/s UI in code.
- 0record0 usage and #la back later
-
or $rite code to simulate mouse clicks
(
!an de+elo#ers rel too much on ad1hoc testing.
-
#ro: "im#le2 )ast2 does not re,uire s#eciali3ed kno$ledge
-
con: Inaccurate2 must be re#eated man times2 #oor at catching
regressions2 costs more and more time later in the #ro*ect
-
The ideal is a mi4 o) both kinds o) UI testing.
4
Flus out error messages
-
em#t strings 5or strings made entirel o) s#aces, etc.6
-
in+alid strings 5too short, too long2 s#ecial characters6
-
& or negati+e numbers
- settings that do not make sense in combination
5
"n#ut $uffer o%erflo&s
(
When #rom#ted )or in#ut, tr to #ut in a +er long string
-
Will it be acce#ted, leading to a strange a##earance on the UI7
6
'%erflo& data structures
(
Whene+er a UI sho$s a #age or list, tr to add to that list until it
o+er)lo$s, causing crashes, errors, or a$k$ard a##earance
7
(iolate a##)s assum#tions
(
What does the a##/s 8UI do i):
-
The )ile it is using gets e4ternall modi)ied or deleted7
-
The net$ork goes do$n 5or *ust slo$s do$n6 une4#ectedl7
-
The 9" amount o) memor a+ailable dro#s7
- The #rocessor becomes bus and the a## slo$s do$n7
8
Re#eat*du#licate in#uts
(
Tr the same in#ut multi#le times to e4#ose bugs:
- re1add an e4isting user
-
create a )ile that alread e4ists
-
delete a )ile that is alread deleted
or that does not e4ist
-
click the button to #er)orm an
action multi#le times
( 0:u0, 09rder0, 0Check 9ut0
( Will the customer be charged t$ice7
-
$eb a##s: click 0:ack0 and then
tr an action again
( Was the de+elo#er e4#ecting this7
9
Cause in%alid out#uts
(
!ost 8UIs sto# ou )rom su##ling bad in#ut.
-
:ut mabe ou can still cause bad out#ut.
(
;4am#le: "et calendar to an in+alid date:
-
The UI #ro#erl restricts ou to Feb <12=.
-
Choose a lea# ear, then select Feb 2>.
-
Change ear back to a non1lea# ear.
-
Feb 2> $ill still be sho$n as a +alid choice.
(
;4am#le: Te4t?ad 0:lock "elect0 )eature
-
toggle on, co# te4t, toggle o)), #aste
10
+est mo%ing * si,ing
(
!an UI designers don/t consider $hat their screen or #age $ill
look like $hen resi3ed to e4tremes
-
tr resi3ing the $indo$ or adding in#ut to gro$ a $indo$/s si3e
(
does the $indo$ add scrollbars7
(
do some controls disa##ear or o+erla#7
(
does te4t begin to $ra# in odd $as7
11
+est ena$ling * disa$ling
(
;nable/disable elements to indicate $hether the can be used.
(
Test the enabling/disabling o) all UI elements.
-
@o elements disable/re1enable $hen the are su##osed to7
-
Is it e+er #ossible to click an element that shouldn/t be clickable,
or im#ossible to click an element that should be clickable7
12
-ndroid testing
(
8oogle recommends creating an entire se#arate test ;cli#se
#ro*ect to store our unit tests )or an Android a##
-
htt#://de+elo#er.android.com/tools/testing/
(
#ut in tests/ subdir o) main a##
-
!?ro*ect/
( Android!ani)est.4ml
( res/ ... 5resources )or main a##6
( src/ ... 5source code )or main a##6 ...
( tests*
- -ndroid.anifest/0ml
- res* ... 5resources )or tests6
- src* ... 5source code )or tests6
13
-ndroid !" testing
(
uiautomator+ie$er
-
allo$s ou to ins#ect current
state o) an on1screen UI
(
UiAutomatorTestCase
-
a s#eciali3ed .Unit test that can
construct and interact $ith UI controls
(
UI Automater !onke
-
simulates #seudo1random UI interaction
to test UI robustness and stress testing
14
-ndroid !" test e0am#le
import com.android.uiautomator.core.*;
import com.android.uiautomator.testrunner.*;
public class LaunchSettings extends UiAutomatorTestCase {
public void testDemo() throws UiObjectNotFoundxception {
!etUi"evice().press#ome();

$$ simulate a user brin!in! up the %ll %pps screen
UiObject all%pps&utton ' new UiObject(new Ui(elector().description()%pps)));
all%pps&utton.clic*%nd+aitForNew+indow();

$$ simulate the user brin!in! up the %pps tab
UiObject apps,ab ' new UiObject(new Ui(elector().text()%pps)));
apps,ab.clic*();
$$ simulate a user swipin! until the- come to the (ettin!s app icon
Ui(crollable app.iews ' new Ui(crollable(new Ui(elector().scrollable(true));
app.iews.set%s#ori/ontal0ist();

$$ simulate a user clic* to launch the app
UiObject settin!s%pp ' app.iews.!et1hild&-,ext(new Ui(elector()
.className(android.wid!et.,ext.iew.class.!etName())2 )(ettin!s));
settin!s%pp.clic*%nd+aitForNew+indow();

$$ validate that the pac*a!e name is the expected one
UiObject settin!s.alidation ' new UiObject(new Ui(elector()
.pac*a!eName()com.android.settin!s)));
assert,rue()Unable to detect (ettin!s)2 settin!s.alidation.exists());
3
3
15
-ndroid !" test code 1
$$ (tart main activit- o4 the application under test
m%ctivit- ' !et%ctivit-();
$$ 5et a handle to %ctivit- object6s main U7 wid!et2 a (pinner
m(pinner ' ((pinner) m%ctivit-.4ind.iew&-7d(
com.android.example.spinner.8.id.(pinner9:);
$$ (et (pinner to a *nown position
m%ctivit-.set(pinner;osition(,(,<(,%,<"(,8O=<;O(7,7ON);
$$ (top activit- > on"estro-() should save state o4 (pinner
m%ctivit-.4inish();
$$ 8e>start %ctivit- > on8esume() should restore (pinner state
m%ctivit- ' !et%ctivit-();
$$ 5et (pinner6s current position
int current;osition ' m%ctivit-.!et(pinner;osition();
$$ %ssert that current position is same as the startin! position
assertEquals(,(,<(,%,<"(,8O=<;O(7,7ON2 current;osition);
16
Ro$otium
(
Robotium
-
UI test automation tool
)or Android a##s
-
based on +er #o#ular "elenium $eb a## UI test tool
- htt#://code.google.com/#/robotium/
-
tutorials:
( htt#://$$$.outube.com/$atch7+ABCk<Dk#"3Eg
( htt#s://code.google.com/#/robotium/$iki/RobotiumTutorials
17
Ro$otium test code
import com.ja-wa-.android.robotium.solo.*;
public class ditor,est extends
%ctivit-7nstrumentation,est1ase?@ditor%ctivit-A {
private (olo solo;
public ditor,est() {
super(ditor%ctivit-.class);
3
public void setUp() throws xception {
solo ' new Solo(!et7nstrumentation()2 !et%ctivit-());
3

public void test;re4erence7s(aved() throws xception {
solo.sendKey((olo.BNU);
solo.clickOnText()Bore));
solo.clic*On,ext();re4erences));
solo.clic*On,ext()dit File xtensions));
assert,rue(solo.search,ext()rt4)));
solo.clic*On,ext()txt));
solo.cleardit,ext(?);
solo.enterText(?2 )robotium));
solo.clickOnButton()(ave));
solo.goBack();
solo.clic*On,ext()dit File xtensions));
assertTrue(solo.search,ext()application$robotium)));
3
public void tear"own() throws xception {
solo.4inishOpened%ctivities();
3
3
18
Selenium
(
Records and #las back automated 0test cases0
o) $alking through a $eb a##/s UI
(
can assert +arious as#ects o) the $eb #age state
to make sure the #age looks right
(
tests can be sa+ed as FT!G
-
or can be $ritten in:
( .a+a
( Rub
( ?thon
( ...
19
Com#onents of Selenium
(
"elenium I@; 1 record/#laback tool as Fire)o4 add1on
-
#roduces "elenium Core test cases
(
"elenium Core 1 FT!G/." )rame$ork that runs in an bro$ser
-
)or testing bro$ser com#atibilit
(
"elenium Remote Control 5RC6 1
automation )rame$ork
-
)or running tests on a schedule
-
used $ith ;cli#se or a dedicated ser+er
20
E0am#le Selenium test
import com.thou!htwor*s.selenium.*;
public class New,est extends SeleneseTestCase {
public void setUp() throws xception {
setU()httpC$$www.!oo!le.com$)2 )*4ire4ox));
3

public void testNew() throws xception {
selenium!oen()$));
selenium!tye()D)2 )mart- stepp));
selenium!click()btn5));
selenium!"ait#or$ageToLoad()E9999));
assert,rue(selenium!isText$resent(
)Universit- o4 +ashin!ton)));
3
3
21
Selenium e0am#le 1
import java.util.*;
import or!.openDa.selenium.*;
import or!.openDa.selenium.4ire4ox.*;
public class 5oo!le(u!!est {
public static void main((trin!FG ar!s) throws xception {
%e&Dri'er driver ' new Fire4ox"river();
driver.!et()httpC$$www.!oo!le.com$webhpHcomplete':Ihl'en));
$$ nter the Duer- strin! )1heese)
+eblement Duer- ' dri'er!(indElement(By!name()D)));
Duer-.sendJe-s()1heese));
lon! end ' (-stem.current,imeBillis() K L999; $$ (leep L sec
while ((-stem.current,imeBillis() @ end) {
%e&Element results"iv ' driver.4indlement(
By!class)ame()!ac<m)));
i4 (results"iv.is"ispla-ed()) brea*;
3
$$ %nd now list the su!!estions
0ist@+eblementA all(u!!estions ' dri'er!(indElements(
By!xath()$$tdFMclass'6!ac<c6G)));
4or (+eblement su!!estion C all(u!!estions) {
(-stem.out.println(su!!estion.!et,ext());
3 3 3
22
2a%a S&ing !" testing
(
Abbot 1 Functional UI testing )or .a+a deskto# a## 8UIs
-
5not )or Android a##s6
-
$orks $ith Costello com#anion a##
-
htt#://abbot.source)orge.net/