Linking Agile software development, Cloud Computing,

and DevOps
Michael Ali
July 2016

A. Motivation
The strategic goals and operational objectives of the business are captured
in the Core Diagram (left side of Figure 1 below) [MITa]. The business
processes and structures in the Core Diagram are enabled and supported by
an underlying application architecture as shown on the right side of Figure 1
below. The Customer Channels in the Core Diagram are enabled and
supported by the corresponding eCommerce, Internet of Things (IoT), Pointof-Sale (POS), and Customer Relationship Management (CRM) systems.
Analytics requires Business Intelligence (BI) tools. Backend Operations are
usually handled by an ERP platform. Shared Information Services are
provided by legacy Enterprise Data Warehouse (EDW) as well as newer
NOSQL databases. The Semantic Layer infers relationships between data
elements using machine intelligence.
The application architecture is a multi-tiered combination of legacy, new,
purchased, purpose-built, on-premise, and cloud applications that interface
with customers, employees, suppliers, and other partners. Data is both
structured and unstructured.
Successfully developing, deploying, and
maintaining this complex landscape requires the latest tools and techniques.
Three of these tools and techniques: Agile, DevOps, and Cloud, are discussed
below.

Analy cs

Customer Channels

Shared Informa on
Services
master
data

Backend Opera ons

eCom

BI
Tools

CRM

IoT

Retail
POS

Seman cLayer
Legacy
EDW

NOSQL
DB

ERP Pla orm

Figure 1: Core Diagram (left) and Corresponding Application Architecture (right)

B. Agile
Agile software development is based on the four tenets of the Agile
Manifesto [SCR]:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
These tenets underpin a methodology that deploys software in iterative sprints as
shown below [SCR]. Organizationally (FIGURE), Agile requires a Product Owner and a
Sprint Team. The Product Owner is the ultimate decision-maker, prioritizing and
approving the work of the sprint team. The Sprint team consists of software
architecture, development, and test resources. The Agile tenets, methodology, and
organization demonstrably improve the speed, quality, and cost-effectiveness of
software development.

Product Owner

Sprint Team

C. DevOps
Agile relies on an iterative development process, with the Product Owner and
the Sprint Team working together to prioritize, build, and test functionality.
Of course, deploying the results into production requires the Operations
team, who manage the infrastructure (servers, desktops, and network). Fast
and effective deployments, as well as post-launch maintenance in a complex
multi-tiered environment assumes tight collaboration between the Sprint
development and Operations teams, something that hasnt always occurred.
DevOps, the practice of operations and development engineers
participating together in the entire service lifecycle, from design through the
development process to production support. [AA], evolved to meet this
need.

Product Owner

Sprint Team

Opera ons

DevOps is based on a set of processes, Continuous Integration and

Continuous Delivery, both of which emphasize automation. Continuous
Integration ensures that development teams use version control, frequent
check-ins, daily builds, and automated testing in their activities. Continuous
Delivery adds automated and shared configuration of development,
integration, and pre-production, and production environments, as well as
automation of regression tests, release scheduling, and monitoring across all
applications. As shown in the figure below, Continuous Delivery links the
Development and Operations teams. Note that security, in both integration
and delivery practices is still required, giving rise to the practice of
DevSecOps (devsecops.org), with the goal of embedding security into the
DevOps practices.

Product Owner

Sprint Team
Con nuous
Integra on

Opera ons
Con nuousDelivery

Security

D. Cloud Computing
1. Definition
The National Institute of Standards and Technology defines cloud computing
as [NIST]:
a model for enabling ubiquitous, convenient, on-demand network access to
a shared pool of configurable computing resources (e.g., networks, servers,
storage, applications, and services) that can be rapidly provisioned and
released with minimal management effort or service provider interaction.
This cloud model is composed of five essential characteristics, three service
models, and four deployment models.
Cloud computing is comprised of three service models:
Infrastructure As A service (IAAS): server, desktop, and/or
networking resources
Platform As A Service (PAAS): application development stack
including languages, libraries, and tools. The underlying infrastructure
is owned and controlled by the service provider.
Software As A Service (SAAS): complete applications and/or
portfolios of applications. The underlying platform and infrastructure is
owned and controlled by the service provider.

Cloud computing services can be deployed as:

Public: customers share the infrastructure, platform, and/or
application with other entities. Public clouds are hosted by the service
provider.
Private: customers have their own instances of the infrastructure,
platform, and/or applications. Private clouds can be hosted at the
customer site, or at the service providers site.
2. Connection between application architecture and cloud
computing
Modern application architectures have a mix of legacy on-premise
applications as well as cloud environments including infrastructure (ex:
externally hosted disaster recovery servers), platforms (ex: AWS or Azure),
and software (ex: Salesforce). Determination of where to use which option
varies based on factors such as: company size, business requirements,
legacy software maturity, risk appetite, capital vs expense budgets, and IT
department capabilities. Given that the environment will contain a mix of
solutions, it is important to have an enterprise architecture [MITa] to guide
decision-making and to ensure that data can move seamlessly and securely
between environments.
3. Connection between Agile, DevOps, and Cloud Computing
Agile, DevOps, and Cloud computing can be used separately, but are
typically leveraged together. Cloud computing provides the infrastructure
and tools for the development, integration, pre-production, and production
environments used by the development and operations teams. DevOps
provides the continuous integration and delivery processes that leverage the
cloud infrastructure and tools. Agile provides the tenets and methodology
for collaborative and iterative software development that shapes the need
for DevOps.

E. Architecture and Implementation: how to get started

Implementing a hybrid on-premise and cloud-based multi-tier architecture
using Agile with DevOps raises the question of where and how to start.
Based on experience, we suggest starting with deploying Agile software
development. Agile will then drive the need for cloud-based environments.
Finally, as the cloud-based environments proliferate, DevOps provides the
processes to maintain and upgrade them and link them to on-premise or
cloud-based pre-production and production environments.
To deploy Agile development, start with the existing project list. Its best
NOT start with the largest, most difficult, and time-critical in-flight project.

The new project team will consist of a mix of Agile consultants, experienced
developers, and inexperienced (in-house) talent. The first Agile project(s)
will therefore include a time and cost learning curve. So its better to start
with smaller, less critical projects. Projects consisting of minor updates to
bespoke legacy systems should also be avoided, as the learning curve should
be from the in-house resources learning Agile, vs the experienced Agile
resources and consultants learning the idiosyncrasies of the legacy system
for a simple update.
Agile software teams will identify the need for development and integration
environments that can be rapidly configured and deployed in support of their
two-week sprints. This need is an ideal use case for cloud computing. Since
the environments are not used for production, the security team can provide
lightweight security and controls governance for them. The infrastructure
team can specify the cloud provider, implement the defined security model,
and then leave the management of the environments to the development
team, alleviating the need for infrastructure resource time and effort.
The first Agile projects will go to production using the traditional routethe
output from the sprints will be batched together into a bigger release, then
regression tested in the pre-production environment, and then moved to
production. As the organizational maturity with Agile grows, the following
issues will begin to arise:
The sprint teams (and business partners) will want to deploy new
capabilities to production at the end of every sprint vs. batching all of
the changes into a larger release
The sprint teams will want their development and integration
environments to mimic the production environment to avoid postrelease bugs related to differences between those environments
Regression tests need to be automated, again to avoid batch releases,
and deploy into production at sprint speed (usually every two weeks)
The infrastructure team cant keep up with the requests to deploy to
production faster, due to the points above
These issues are resolved by DevOps processes and tools. Given the
changes driven by DevOps, a good place to start is the interface between the
integration and pre-production environments. This interface is where the
development team must engage with security and infrastructure to deal with
the constraints of production environments. Examples of constraints include
segregation of duties, testing against production data in a secure way,
network configuration, storage and server limits, high availability, disaster
recovery, and release planning and coordination.
DevOps deployment
should be treated like a project, with a project manager and dedicated
security and infrastructure resources, working with a mature (at least 1
project or multiple sprints completed) agile development team.

References
[AA] https://theagileadmin.com/what-is-devops/
[MITa] Enterprise Architecture as Strategy: Creating a Foundation for
Business Execution, Harvard Business School Press, 2006.
[NIST] The NIST Definition of Cloud Computing, Special Publication 800-145,
Sept. 2011.
[SCR] http://www.screenmedia.co.uk/blog/2014/08/what-is-agiledevelopment-a-brief-introduction/