Scaling Agile Methods –

Ten Top Issues and Lessons Learned

Donald J. Reifer, University of Southern California

Frank Maurer, University of Calgary
Hakan Erdogmus, National Research Council Canada

On 20-21 February 2003, thirty-five concerned professionals met to discuss ways to resolve
issues associated with using agile methods on large systems at the First Invited Canadian
Workshop on Scaling Agile Methods in Banff, Alberta. Scaling agile methods addresses the
issue on how to produce lots of software functionality within a limited time frame with a larger
teams. If the functionality is small, a small team can create the software. If the time frame is
large, a small team can also deliver lots of functionality. Thus, scaling agile teams is only an
issue if having many developers working concurrently is the only option to meet the deadline for
the system delivery. Twenty delegates came from industry where most had been putting agile
methods to work (using XP, Scrum, DSDM, FDD, and a scaled down version of RUP).
Industrial delegates addressed a wide range of issues, notably:
• How they used a bottom-up sell approach on significant projects to convince
management of the benefits of agile approaches.
• How agile methods could be scaled to very large projects with barely sufficient upfront
planning and architectural work.
• How a federation of coordinated teams (each of which internally operates as an agile
team) can be used to scale up agile ideas.
• Which strategies can improve intra-team communication in larger teams that do not have
the privilege of daily face-to-face communication.
• How to apply agile methods within teams larger than a typical XP team.
• How they characterize the agile continuum through different project caricatures, ranging
from typical collocated XP project to large multi-team, multi-year project.
The delegates who came from academia shared their experiences and ideas on:
• using agile practices such as test-driven development and pair programming as
pedagogical tools in software engineering curricula;
• investigating the effectiveness of agile practices;
• reconciling agile methods with architectural paradigms; and
• using agile methods in research projects.
Some were newcomers looking to try agile methods in the classroom and on their research
projects.
The delegates from government were sponsoring research supportive of speeding the
introduction of agile methods into practice and have been actively involved in conducting

15. March 2003 1 Version 1.0

research evaluating the effectiveness of the underlying practices. They were supportive of
collaboration between industry, academia, and government to move forward in addressing the
adoption challenges
During the first day of the meeting, delegates discussed and reached consensus on the top ten
issues associated with scaling agile methods. The top ten issues raised were as follows:

1. How do we scale pure agile for non-pure agile?

Number of votes: 11
Discussion: Most delegates agreed that agile methods fit small projects where problems of scale
were minor. However, scaling these methods to fit large projects where teams of teams worked
together was difficult. Most delegates also agreed that transitioning organizations to the use of
agile methods involved using a mixed metaphor. In other words, agile must operate in a world
where both agile and traditional methods were used together. The issue then becomes how to
scale agile so that the underlying principles of the Agile Manifesto are not sacrificed. For
example, how do you evolve an emerging architecture when there is a mismatch with the
enterprise architecture? As another example, how do you scale collective ownership and what
do you do when one method allows it and another doesn’t?

2. Can we generate guidelines for non-sweet spot agile projects?

Number of votes: 9
Discussion: A sweet-spot agile project is typically characterized by a small, self-organizing,
collocated team of less than 20 developers and one or more on-site customers, together working
on a variable-scope application with unstable or emergent requirements, and the predominance
of an oral culture based on high-bandwidth face-to-face communication. Probably the most
debated topic was what a large project was and where did the sweet spots exist for use of agile
methods. Some argued that agile methods shouldn’t be used outside their original domain of fit.
Others suggested agile would be scaled to larger projects whether delegates liked it or not. All
agreed that guidelines for scaling them were needed.

3. What do we do to augment agile practices to fit large projects?

Number of votes: 7
Discussion: The issue of what additional practices were needed to scale agile methods came up
again and again especially when discussing non-sweet spot agile projects. For example, some
suggested that the concept of daily team meeting could be extended to include a daily intra-team
project meeting (Ken Schwaber’s “Scrum of Scrums”). Others discussed the issues associated
with scaling planning meetings and requirements engineering concepts. Ideas were exchanged,
but choosing proper practices and strategies is still an open issue.

4. How do we address integration of legacy, COTS, components, etc.

within an agile project?
Number of votes: 6
Discussion: Most delegates agreed that “green field” projects often mentioned in the literature
were idealistic. Most applications exploit existing architectures and use extensive amounts of
legacy software, COTS packages and components. The issue debated was how these reusable

15. March 2003 2 Version 1.0

components constrained the use of agile methods as elements of scale were introduced. It was
suggested that to accommodate legacy and COTS software, test-drivers be written to reveal all
relevant external behavior. Also raised was the issue of how to coordinate the use of standard
frameworks and components across teams of teams when agile methods and collective
ownership principles were used?

5. How do you scale agile within an enterprise across applications?

Number of votes: 6
Discussion: Coupled with the issues of legacy were those associated with product families and
product lines. It was agreed that applications did not exist in isolation within an enterprise. In
response, delegates pondered how to handle the communications barriers that exist across the
enterprise when building systems using agile methods. Most wanted to avoid stovepipes when
scaling a project as applications are produced using short iterative cycles. One approach that was
suggested for managing requirements in framework development was the systematic
generalization and aggregation of the user stories across similar applications to factored-out
framework stories.

6. How do you handle dispersed development within an agile project?

Number of votes: 6
Discussion: The size of the team and how to grow it so that it could operate in a dispersed (and
often geographically separated) environment was another issue discussed. While some had
experience growing teams of teams across geographic distances, little consensus was reached on
how to overcome the communications barriers. Some delegates suggested a team hierarchy and
a phased rollout model (e.g., architecture team first, then feature and integration teams). Others
thought that this approach would pose a serious threat to agility. Synchronizing the beats of the
individual teams and the possibility that the slowest or the weakest team would determine the
overall pace or fate of the project were perceived as major risks.

7. Who does integration testing as systems get bigger from a customer

viewpoint?
Number of votes: 5
Discussion: Most delegates agreed that the agile approach of having customers write acceptance
tests as before new features are implemented paid dividends. However, in a large project,
securing sufficient customer involvement was deemed unrealistic. It was mentioned that the task
of writing customer acceptance tests could be taken over by QA team members. The issue
becomes how to mechanize the agile approach without placing an extra burden on the customer
community?

8. What is agile (when polluted)?

Number of votes: 5
Discussion: The group debated whether an agile method that employed traditional practices was
really agile. Some purist argued “no.” Others disagreed.

15. March 2003 3 Version 1.0

9. What project management practices do we use for large agile
projects?
Number of votes: 4
Discussion: One of the most fun debates occurred when project management practices were
discussed. Some argued that management practices were needed as agile scaled to assess
progress, coordinate development and minimize risk. They suggested that self-directed teams
naturally tended to focus on what was right (e.g., delivering new functionality over analysis,
getting to the release over planning)). Others suggested that following such a course of action on
large projects would lead to chaos.

10. How do we respond to RFP’s when embracing agile methods?

Number of votes: 4
Discussion: The primary issue discussed dealt with the expectations mismatch between
customers and developers when dealing with agile methods. Contractors have a hard time
convincing buyers of a “fixed-price, variable-scope” tender up-front when using agile methods.
Because of their iterative nature and focus on time-boxing, they argue they can deliver
functionality sooner and can tolerate changes better than traditional approaches. But, changing
on the fly is not how contracting officers work. They want set functionality delivered per an
agreed-to budget and schedule.
We next discussed ways to resolve each of these issues. Although we didn’t quite achieve this
goal, we were able to capture our early adopters’ experiences relative to these issues primarily in
the form of lessons learned, both positive and negative. The top ten lessons learned by early
adopters were:

1. Scaling of agile methods will continue to happen whether you like or

not.
Discussion: While several participants, especially Martin Fowler in his keynote address, argued
that scaling of XP and agile projects was probably the last thing one would want to do, most
delegates agreed it would happen anyhow. Because of the communal push for scaling, it was
agreed that people would do it even when warned that it was the wrong thing to do.

2. Visibility for large projects can be increased via frequent planned

time-boxed releases. The shorter the cycles the better.
Discussion: Everyone agreed that spending time getting releases out per pre-established, short,
time-boxed schedules with whatever functionality they could was advantageous. While no
consensus was reached on the optimum duration, most agreed that two weeks to three
months(???) was in the proper range for iterations/releases. Agreement was reached on the value
of continuous integrations and automated build & test processes. Instead of talking about what
the product functionality would be, developers could show customers products that are
“potentially shippable”. This takes the subjectivity out of the discussions.

15. March 2003 4 Version 1.0

3. Communications on large projects can be improved using daily intra-
team meetings (Scrum of Scrums).
Discussion: The use of daily project meeting where team leaders got together to work issues,
identify desired functionality and coordinate release contents, iterations and progress was
deemed advisable for large projects.

4. Some up-front investment in architecture is recommended to

succeed when scaling agile methods.
Discussion: In a departure from current thinking, delegates who were working on large projects
agreed that some architectural development was needed prior to pushing ahead with iterations.
This could be done quickly using an architecture team. The members of the architecture could
then move on to seed the subteams of a large-scale agile project. While the architecture will still
continue to evolve over the project, the tendency will be to stabilize it and discourage any
significant changes because a stable architecture will provide multiple teams with the context for
coordinating contents of releases and iterations. However, this approach poses a threat to agility
because it might tip the scale more in favor of up-front planning and sticking to the plan over
allowing the architecture to emerge organically.

5. Scaling of on-site customers can be successfully handled via a

federation strategy.
Discussion: Several delegates suggested that a federation of on-site customers could be built to
provide developers with the feedback they needed on a daily basis. The key to success in such
an arrangement is to know who speaks for the customer on what issue. Establishing known
points of contact for specific items was therefore deemed essential in large projects. An issue that
might arise from multiple customer representatives might be that they do not speak with a single
voice anymore and that arriving at binding decisions might become slower.

6. Problems associated with scaling of collective ownership and code

integration can be reduced via branching, cloning and wrapping of
components.
Discussion: Some delegates recommended that integration problems could be minimized on
large projects by packaging components with wrappers that communicated essentials about their
features to those unfamiliar with them. Another strategy is to write test drivers for components
to reveal how they behave in a specific context. Because focus is placed on releases, this would
ease the communications problems and permit components to be used within their proper
context. However while wrapping cloned components for use in different contexts will reduce
commit clashes and alleviate integration problems in a collectively-owned code base, it will
increase redundancy and thus reduce maintainability.

7. When scaling agile methods, set expectations lower in terms of how

much change you can achieve and/or tolerate.
Discussion: It was agreed that larger teams were less toleratant to change. Adopters of agile
methods were warned to be extremely conservative when setting expectations for change
especially when trying to use these new methods on large projects.

15. March 2003 5 Version 1.0

8. When scaling large projects, empower your business analysts to be
the voice of the customer.
Discussion: One approach several delegates used to reduce the communications problems with
the customer was to use business analysts as surrogates for the customer or in a mediating role.
The analyst, however, should not function prohibit a direct communication between the customer
and the developer. The involvement of empowered analysts with a good understanding of the
business problems permitted federation to take place more easily and often was able to
compensate for the unavailability of an on-site customer.

9. We should try harder to take advantage of the lessons others have

learned when introducing agile methods to large projects.
Discussion: Needless to say, everyone agreed that there were models of change that others were
using that the agile community could exploit.

10. When scaling large projects, you can use a combination of agile and
traditional methods.
Discussion: Towards the end of our discussions, a few delegates agreed that there was more
compatibility between agile and traditional methods than most first thought of. While the debate
continued to rage, some hope for resolving the mismatches that emerged.
There was also some very profound advice offered by delegates. Martin Fowler suggested that
managers should try to do more with less by increasing their people’s skill sets. By emphasizing
use of better people, he argued, more could get done quicker and better. Ken Schwaber warned
that we don’t want to undermine the value of agile methods. Instead, we want to use them when
they make sense and when they add business value to our ventures.
The workshop was indeed valuable. It built bridges that allowed delegates from academia,
industry and government to discuss their experiences and concerns. Momentum to keep the
dialog flowing was probably the workshop’s most important output. Towards these ends, plans
for a second workshop on scaling of agile methods are being made for next year and the
Canadian Agile Network web site (http://can.cpsc.ucalgary.ca) is being augmented to provide
additional resources for those interested in putting agile methods into practice in industry,
academia or government. The organizers thank the delegates for a job well done. We learned a
great deal and are encouraged by the positive results of agile method usage.

List of participants
Dustin Aleksiuk (TransCanada Pipelines Ltd.), Jennitta Andrea (ClearStream Consulting, Inc.),
Thomas Chau (University of Calgary), Mark Davidson (BMW Financial Services), Scott Dick
(University of Alberta), Armin Eberlein (University of Calgary), Adrian Fiech (Memorial
University of Newfoundland), Martin Fowler (Thoughtworks, Inc.), Adam Geras (University of
Calgary), Daniel German (University of Victoria), Sean Goggins (Guidant Corporation), Janet
Gregory (Envista Technologies Inc.), Mike Grifffiths (Quadrus Development Inc.), James
Hoover (University of Alberta), Anatol Kark (National Research Council), Philippe Kruchten
(Rational Software Corporation), Dean Larsen (TransCanada Pipelines Ltd.), Jim Leask (Sybase
Canada), Jim MacDonald (Thoughtworks Inc.), Grigori Melnik (University of Calgary), Gerard
Meszaros (ClearStream Consulting Inc.), Miroslav Novak (togethersoft), Jonathan Rasmusson
(Thoughtworks, Inc.), Kristopher Read (University of Calgary), Dave Rooney (Mayford

15. March 2003 6 Version 1.0

Technologies), Ken Schwaber, Robert Sheehan (BMW Financial Services), John Shillington
(University of Alberta), Peggy Storey (University of Victoria), Ross Taylor (TransCanada
Pipelines Ltd.), Robert Wiebe (Synaptic Inc.), Ken Wong (University of Alberta)

Donald J. Reifer is a Visiting Associate with the Center for Software Engineering at the
University of Southern California and President of Reifer Consultants, Inc. Contact him at
dreifer@earthlink.net.
Frank Maurer is an Associate Professor in the Computer Science Department at the University of
Calgary. Contact him at maurer@cpsc.ucalgary.ca.
Hakan Erdogmus is a research officer at the National Research Council of Canada. Contact him
at hakan.erdogmush@nrc-cnrc.gc.ca

15. March 2003 7 Version 1.0