White Paper accepted for plenary presentation at STeP-IN SUMMIT 2009

A Foray Into Cloud-based Software Testing

Author(s): Sunil Joglekar

Designation: Technical Architect

Email: sunil.joglekar@patni.com

Company Address : Unit 17, SDF-VII, SEEPZ, Andheri (East), Mumbai – 400 096

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 1 of 10

 Abstract:

Introduction :

Modern Software Development entails many challenges e.g. rapidly changing business
scenario, rising complexity, shorter time to market, ensuring agility without
compromising on the quality.

Agile methodology for Modern Software Development includes many practices such as
automated source code analysis for reviewing the architecture / design / code review,
frequent build & integration and automated execution of tests.

Taken together, these activities ensure rapid, immediate, useful feedback throughout
the span of software development thereby transforming a vicious circle into a virtuous
cycle.

Even when we succeed in implementing the mechanisms and inculcating the best
practices, there’s a price to pay. Doing these things takes Time, Money & Resources.
Besides, we need to ensure that we create value rather than incurring costs.

Cloud Computing is an innovation we can apply to bring economy to our solution in

terms of time, resources and of course, the money !

This paper describes by means of an example our foray into cloud-based software
testing using Amazon Web Services like EC-2, S3 et al.

Audience: Test Architects, CTO, CIO

Area of Application: Testing Modern Software Systems

Benefits:
Making modern software testing economical in terms on resources, time and money

Issues and Challenges:

How to make agile practices economical.

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 2 of 10

 Table of Contents
ABSTRACT: ................................................................................................... 2
1.0 CHALLENGES IN MODERN SOFTWARE DEVELOPMENT :.......................... 4
2.0 TESTING MODERN SOFTWARE SYSTEMS : .............................................. 4
3.0 PROBLEMS WITH THE CONVENTIONAL SOLUTION :............................... 6
4.0 AMAZON CLOUD COMPUTING PLATFORM :............................................. 6
5.0 CLOUD-BASED MODERN SOFTWARE TESTING :...................................... 8
6.0 BENEFITS & LIMITATIONS..................................................................... 9
7.0 DEFINITIONS, ABBREVIATION AND ACRONYMS.................................... 9
8.0 REFERENCES ........................................................................................ 10
9.0 ACKNOWLEDGEMENTS ......................................................................... 10
BIOGRAPHY OF THE AUTHORS ................................................................... 10

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 3 of 10

1.0 Challenges in Modern Software Development :

The Modern Software Systems are growing beyond the boundaries of silo-ed enterprise
applications to meet the requirements of rapidly changing markets. They are getting integrated
within and across the enterprise thereby metamorphosing into ecosystems.

This has resulted in complexity which seems to be ever increasing and also in a high number of
changes these systems have to go through. Naturally this has lead to increased risks and has
also increased cost of evolving these systems to meet the requirements of rapidly changing
business landscape.

There are pressures on time to market and there is a need to ensure the agility of the
enterprise at several levels : Technical, Organizational and Business. The cost of finding &
fixing defects is becoming exponential. There’s a high premium for Enduring Quality in Modern
Software Development.

2.0 Testing Modern Software Systems :

The above mentioned challenges can be tackled by working in an iterative / incremental

manner. We need to embellish the Continued Integration discipline by embedding & integrating
the following activities :

• Source Code Analysis

o Common Coding Mistakes
o Compliance to Coding Standards
o Common Design Mistakes
o Compliance to Design / Architectural Policies
• Unit & Integration Testing
• Acceptance Testing
• Load Testing

These activities ensure rapid, immediate, useful feedback throughout the span of software
development thereby transforming a vicious circle into a virtuous cycle.

Let us consider the case of modern software development using Java / JEE.

ANT is a build tool for Java. It provides a facility to carry out tasks such as check out from the
source code control, compiling java files, running unit tests et al. Using the build file
mechanism, we can compose these tasks in a pipe-line manner.

We have developed an ANT pipeline using a bunch of open source tools to provide solutions to
some of the commonly recurring problems during software development as mentioned above.

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 4 of 10

 Many of the available open source tools do a pretty decent job of analyzing source code as well
the byte code, jar files at al.

• Compliance to Coding Standards : CheckStyle

• Common Coding Mistakes : PMD, CPD
• Analyzing byte-code : JLint, FindBugs
• Common Design Mistakes : JDepend
• Compliance to Design / Architectural Policies : Utilities developed in-house
• Unit & integration testing : JUnit Family
• Acceptance testing : Fit/Fitnesse
• Database testing : dbFit
• Load / Performance Testing : JUnitPerf, Meter

A sample ANT pipeline would looks like the figure shown below :

However, this is easier said than done and even when we succeed in implementing the
mechanisms and inculcating the best practices, there’s a price to pay.

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 5 of 10

 3.0 Problems with the Conventional Solution :

We need to invest in infrastructure to implement the mechanisms necessary for continued

integration. Also, we need to set up machines for carrying out the tasks like analyzing the
source code, running the tests et al. Naturally, this leads to up-front capital expenditure as
well as operational expenditure.

As the modern software systems are getting sizable, tasks like analyzing the source code and
the large test base is implying lengthy test execution cycles !

Under such circumstances, it would become difficult to ensure that the feedback is rapid.
This would get in the way of ensuring the required agility and would also hamper the time to
market.

The problem can be summed up in just a line : given that doing this stuff takes Time, Money &
Resources, how do we ensure that we create value instead of incurring costs ?

Cloud Computing is an innovation we can apply to bring economy to our solution in terms of
time, resources and of course, the money !

4.0 Amazon Cloud Computing Platform :

Cloud computing can be described as the scalable computing resources provided as a service
from outside our environment on a pay-per-use basis.

Amazon Cloud Computing Platform provides the following services in a quite cost effective
manner.
• Simple Storage Solution (S3)
• Elastic Computing Cloud (EC-2)
• Simple DB
• Simple Queue Service (SQS)

Simple Storage Service (S3) provides a Web services interface for the storage and retrieval of
data. The data can be of any kind, and can be stored and accessed from anywhere across the
Internet.

Amazon Elastic Compute Cloud (EC2) makes it possible to run multiple virtual Linux servers on
demand, providing as many computers as we need to process data or run our web application
without having to purchase or rent physical machines.

Once we have set up an EC2 server the way you like it, we can save it permanently as a server
image, also called as Amazon Machine Image (AMI). We can then launch new servers from this
image to create virtual machines that are preconfigured and ready to carry out our tasks.

EC-2 lets us easily scale our capacity based on the demand by quickly creating virtual instances
and terminating them once the demand is satisfied.

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 6 of 10

 SimpleDB is a web service for storing, processing, and querying structured datasets. It is not a
relational database in the traditional sense, but it is a highly available schema, with a less
structured data store in the cloud, and which you can use to store and retrieve keyed values.

Simple Queue Service (SQS) provides access to the reliable messaging infrastructure used by
Amazon. You can send and retrieve messages from anywhere using simple REST-based HTTP
requests. There’s nothing to install and nothing to be configured. We can create an unlimited
number of queues, and send an unlimited number of messages.

As this is a brief overview, please refer to [1], [2] and [3] for more details. The following figure
shows how a user interacts with AWS :

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 7 of 10

 5.0 Cloud-based Modern Software Testing :

Let us consider the sample ANT pipeline shown earlier as an example to understand our
solution. We created an AMI as a foundation for the executing the following activities on the
various instances --
• Source Code Analysis
• Unit / Integration Testing
• Acceptance Testing
• Load Testing

We mapped the ANT tasks pertaining to these activities onto the Amazon EC-2 instances as
shown below :

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 8 of 10

 6.0 Benefits & Limitations

Applying Amazon cloud computing facilities helped us accrue many benefits.

• Reduction in Cost – Capital Expenditure (owing to zero infrastructure)
• Reduction in Cost – Operational Expenditure (owing to a small “rent”)
• Reduction in execution time => Rapid Feedback cycle
• Quick Turn-around => Increase in Productivity
• Reduction in Risk => Increase in Quality

Our current PoC is just a baby step in the right direction. We would like to grow our prototype
in the following directions --
• Augment the PoC by using a blog server to blog the build process
• Make the current prototype cloud-aware
• Create a Cloud-based Test Lab

The following are the limitations of our PoC :

• At present Amazon does not support other platforms such as Windows, Macintosh, but
only Linux and Open Solaris.
• Currently, we are yet to hear from commercial test tool vendors whether their tools
can be deployed in the cloud

We believe that these issues would get resolved in the future. So, we are optimistic about
applying Cloud Computing.

7.0 Definitions, Abbreviation and Acronyms

Acronym Description
AWS Amazon Web Service
AMI Amazon Machine Image
EC-2 Elastic Computing Cloud
S3 Simple Storage Solution

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 9 of 10

 8.0 References

[1] Programming Amazon Web Services by James Murty. O’Reilly Media 2008

[2] Amazon Web Services : Articles and Tutorials

[3] Amazon Web Services : Developer Tools – Useful AMI tools

[4] Other Resources:

• Wikipedia - Cloud Computing
• NY Times - Cloud Computing: So You Don’t Have to Stand Still
• Infoworld - What cloud computing really means
• Brief Introduction to Cloud Computing

9.0 Acknowledgements

I would like to express my sincere thanks to Dr. Prasad Ramanathan for the encouragement,
support and feedback.

Biography of the authors

Author 1: (minimum 200 words)

Sunil Joglekar is a Technical Architect with the Product & Technology Initiative Group
at Patni Computer Systems.

He has 18 years experience as a Software Professional working in Global Software

Teams. He has spent about 5 years in United States mostly as a consultant at Sun
Microsystems.

Sunil has extensive experience in the development of products / product suites / product
lines. Also, he is well-versed in applying the Agile Methods / Practices.

Sunil specializes in architecting testable solutions for complex enterprise software

systems and their integrations such as EAI / B2Bi / SOA.

His primary areas of interest are : strategizing & architecting innovative solutions with
Web 2.0, Cloud Computing, Autonomic Computing et al.

 Patni Computer Systems A Foray Into Cloud-based Software Testing Page 10 of 10