Mobile Application Testing : Challenges & Solutions

Ashwini Phalle

1. Mobile Testing The Need 2. Mobile Application overview 3. Mobile Application Testing Requirements 4. Testing Challenges 5. Possible Solutions

The Mobile Testing The Need

Brand new devices smart phones, tablets being released every week

Analysts at Forrester Research predicted within a few years, the mobile channel will displace the online channel for day-to-day banking tasks

Mobile apps have limited computing resources still need to be agile and reliable

Complex mobile environment: Devices, OS, Carries

Mobile Application Test Requirements

Mobile Application Test Requirements

Functional & UI related testing
UI functional Testing System Testing Regression Testing

Usability Testing
Usability of applications functionality Usability with various devices

External factor (Device related) testing

Device Related Testing: Battery, SD card interactions, Voice Call Handling, SMS Handling, Notifications External incoming communications Infrared etc

Performance Testing
Application response time Use of resources Application loading speed

Security Testing
Security Testing for financial apps Authentication checks

Mobile Testing Challenges

Mobile Test Environment Elements

Mobile Testing Matrix

1. Increased Testing Cost

2.

Increased Testing Efforts

Devices at multiple locations Not Feasible

Features

3.

Locations

Device Models OS & Browsers

8

Possible Solutions

Test Effort Optimization Through Prioritization

Risk Based Testing approach for optimized Testing Effort : With weighted Device Platform Matrix Testing Requirement and application types matrix Test Cases Business priority matrix

Device OS
Relative Importance Device1 Device2 Device3 Device4

Relative Importanc e

OS1

OS2

OS3

OS4

OS5

App Testing Type

Relative Importance Functional
1 2 3

Relative Importanc Financial e

Native

Webapp

App4

App5

2
3

Usability External Factors Performan ce Security

10

Mobile Emulators
Mobile devices with different resolutions, screen sizes and browser implementations Good for finding browser\platform specific bugs Test the apps on device emulators, interact with OS, access in browser simulators

Note : Not all activities can be realistically emulated Mobile apps can exhibit different behavior on device due to
Due to lower device power and memory, Different DPI resolution

Not for Usability Testing

11

Outsourcing

Crowd-sourced Mobile Testing

Community that serves as a virtual testing team or as an on-demand extension for in-house QA departments

Mobile Could

Remote device access service, gives online access to numerous real devices on various networks Perfectomobile.com DeviceAnywhere.com

12

Best Practices

13

Best Practices
Browser metrics: Subject to change - depends on changes in technology, popularity of the browser, performance issues Device popularity: Analysis on the popular devices in the market, metrics are going to change frequently

Metrics for regions, carriers: Analysis for device/carrier popularity in terms of web traffic from devices in particular countries and regions
Test on desktop browser first for early bug detection Testing at an early stage for the priority devices from the metrics Test on emulator then on device for device specific bugs Early Usability testing on multiple devices Performance Testing on different device and carrier mix based on country and region

14

Q&A

15