A Comparative Case Study of Teamwork in Norwegian Agile Software Development Projects

Viktoria Gulliksen Stray

University of Oslo 0047 93610848

stray@ifi.uio.no ABSTRACT
This paper presents the progress of my research for the 5th International Doctoral Symposium on Empirical Software Engineering (IDoESE 2010) Bolzano-Bozen, Italy. The main focus of my study is effective teamwork and coordination across teams in national and global software development projects. I investigate teamwork in projects doing agile software development (ASD). I want to develop an understanding about human and social factors and the challenges that arise in agile software projects. What makes the teamwork more effective, and what makes it less effective? It is increasingly common to have distributed teams, and this might add challenges to ASD: cultural challenges, communication challenges and knowledge sharing to name a few. My fieldwork will be in four companies, two consultancy companies and two companies that develop in-house software applications. All companies work with agile methods (Scrum, Lean, Kanban). The four companies are part of a research project that is funded by the Norwegian Research Council. So far I have been observing in two of the companies. I have done 8 interviews in one of them and distributed a questionnaire in the other. I used a preliminary instrument as a basis for conducting the interviews. This instrument [1] has 5 dimensions for diagnosing agile teamwork, and I added one dimension. I have found challenges in several dimensions, for instance in Team orientation, Redundancy and Learning. One challenge is that the team members have different perceptions of who their team members are. used an existing preliminary instrument for diagnosing and describing the status of agile teamwork [1]. This instrument consists of an interview guide and has open-ended questions along 5 dimensions: Shared Leadership, Team orientation, Redundancy, Learning and Autonomy. These dimensions are showed to be important in agile teamwork and has been used in several companies receiving good feedback [1]. Since the company I was conducting interviews in had a team in another country I decided to add questions about Trust because research has shown that Trust becomes increasingly important when you have global distributed team [4] [5]. Global distributed teams can also be called global virtual teams. Jarvenpaa et al. [4] define a global virtual team as a self-managed knowledge work team, with distributed expertise, that forms and disbands to address a specific organizational goal and that such teams engage in dispersed global work and, because for large time and space differences, communicate via e-mail and the Web. This definition fits well with Company As distributed teams. The do knowledge intensive work and collaborate and communicate by chat, e-mail and telephone conferences. I also added questions about culture and language since cultural differences may lead to additional challenges in distributed teams [5]. The existing team radar instrument was also indicated to be appropriate for tracing changes over time [1]. Since Company A was about to install professional video conferencing equipment I wanted to get at status of the teamwork before they installed it. I want to conduct interviews after they have used the equipment for a while to see how the teamwork changes over time I use the same definition for team and effectiveness as Cohen and Baily [6] :A team is a collection of individuals who are interdependent in their tasks, who share responsibility for outcomes, who see themselves and who are seen by others as a an intact social entity embedded in one or more larger social systems, and who manage their relationship across organizational boundaries. Cohen and Baily categorizes Effectiveness of Teams into three dimensions: (1) performance effectiveness assessed in terms of quantity and quality of outputs, (2) member attitudes, and (3) behavioral outcomes.

Categories and Subject Descriptors

D.2.9 [Software Engineering]: Productivity, Programming teams.

General Terms
Management, Performance, Design, Human Factors.

Keywords
Agile software development, Effective teamwork, Coordination across teams, Comparative case study, Agile, Scrum.

1.1 Issues to get advice on

My research questions are not fully defined. I need advice on how to narrow my scope and what to focus on. Effective teamwork is a broad topic, and it is challenging to go from this topic to specific research questions in my action research. I would also like to discuss different theories I can use.

1. INTRODUCTION
The use of agile as a methodology for software development continues to grow in industry, and teamwork is an important aspect of ASD[2, 3]. My overall research topic is what makes the teamwork effective in agile software projects. There are many dimensions in the context of ASD that are interesting to investigate. As a foundation for the interviews I

2. RELEVANT WORK
ASD was introduced as a set of principles in the agile manifesto, and is therefore open for a lot of interpretation [7]. As a result of this it is difficult to say exactly what is an agile methodology, and what is not, and this might contribute to agile becoming a buzzword. Many people claim that ASD introduced a whole new set of methods and techniques. As this to some degree might be true, many so-called agile methodologies existed before the term Agile Software Development was introduced in 2001. Already in 1987 Frederick Brooks wrote about the importance of iterative and incremental development and of showing something to the customer as early as possible (rapid prototyping) [8]. He also said that the challenges of software development are people, not technology and that the most important function that the software builder perform for the client is the iterative extraction and refinement of product requirements [8]. All of these are important aspects of ASD. Researchers have done several systematic reviews of ASD and claimed that there is a backlog of research issues in this area [9], [10] and [11]. One of them reported that there is a need for more empirical research on agile development methods [11]. My research will be based on empirical data, and it will be a comparative case study. Very few, if any, have done comparative case studies in the area of ASD [11]. Most studies are done on XP, as much as 76%, Scrum and Lean were found to be the area of study in only a few research articles [11]. My study will investigate the use of Scrum and Lean. There is also a research gap in research on mature agile teams [11]. The teams in Company A have used agile methods since 2000, and are seen as mature agile teams. There is limited research on the aspect of agile collaboration, except when it comes to pair programming [2, 3]. Relevant recent prior work includes [2], [12] and [13]. Appreciative Inquiry (AI) is a way of changing organizations that has shown to give measurable results [14]. AI is an approach within organization development and action research which focuses on the best of what is in a system, rather than problems to be solved, and works at engaging organizational members in envisioning and realizing its future [15]. Kent Beck, one of the authors of the agile manifesto says that we can make retrospective and other parts of agile development better by using the Appreciative Inquiry (AI) framework [16]. The past few years there has been a huge growth in the numbers of companies using AI, and that includes software development companies [17] [18]. The recent years many experience reports and important books describing AI have been published, however in 2002 there were only 20 published case studies of AI [17], and none in the Software development field. There exists research of use of AI in fields like education, psychology and organization development [19], but there is a research gap of using this framework in the software engineering area. I believe it is important to do research on what is popular in the industry.

principles of ASD are investigated to a great level, especially pair programming and early stages of ASD [20]. In my research I want to investigate gaps that need to be researched by practical experience.

3.1 Research questions

The final research questions remains to be defined. I will use action research to specify the research questions. What I want to focus on depends on preliminary investigations in the companies, and this will be a part of the foundation for defining the specific research questions. Two consultancy companies and two, partly global, companies that develop large, in-house software applications will be the hosts of my case studies. All companies work with agile methods. The four companies are a part of a research project that is funded by the Norwegian Research Council, which gives me access to several projects within these companies. The research project continues until 2013, which is the same as my Ph.D. study does. Challenges in the companies that are revealed in interviews will form the basis of my research questions. Possible research questions are: What are the challenges and benefits of teamwork in global distributed software development? o How does autonomy affect global distributed software development?

A literature review from 2004 indicates that more research should focus on global long-term virtual teams [5]. They also found no research answering the following questions and propose them as future research question [5]: o o Are autonomy and self-direction the team structure best suited for virtual teams? Under what circumstances does autonomy hinder team effectiveness in the virtual environment? Can a set of behaviors that promote effectiveness of a wide range of virtual teams be identified?

The five dimensions in the team radar instrument are shown to be important in agile software development [1], so it would be interesting to continue to do research on these dimensions: How do the dimensions autonomy, shared leadership, team orientation, redundancy and learning affect the teams (both co-located and distributed)? o How to establish and sustain shared leadership among the participants in the teams, roles in the team, and the general leadership in the company? How to achieve shared and overlapping competence to support flexibility in teamwork in a cost-effective way.

3. RESEARCH OBJECTIVES
Given the great interest in ASD in industry there is a clear need of extending the knowledge base on use of ASD, and given the importance of teamwork this is a relevant area to study. Some

Appreciative Inquiry has existed since 1985 as an approach to change [18]. Several Fortune 500 companies have used it with great success, and it is gaining more and more popularity[18]. It would be interesting to look closer at this by asking:

How will use of Appreciative Inquiry affect the teamwork in an agile software project? o How will use of Appreciative Inquiry affect the retrospective meetings?

project is staffed with approximately 15 people in Norway, and they also have some teams in another European country. After observing in Company A for several months, I agreed with the Technical Project leader from the project in Company A to interview the core resources in the project, which were 8 people from Norway. Two of them had been on the project since the start (10 years). The people I interviewed all attended the standup meeting every day. This is a meeting which is held every morning at 9 oclock where they discuss what they have done, what they are going to do until the next meeting, and challenges they have. I did not interview the people sitting in the other country or people sitting at another site (called help desk). The interviews were semi-structured to simplify the process of transcribing and analyzing the data. The interviews lasted between 30 and 60 minutes and were carried out in Norwegian. In the interviews I used most of the same questions as Moe et al [1], but some questions were reworded or deleted, and some were added (appendix A). The interviews were audio recorded and later transcribed.

Company A is very interested in looking at how the factor WorkIn-Progress (WIP) affects the project. A possible research question for this could be: When you do Kanban software development and lower the work-in-progress-limit, how does this affect the teamwork and the project?

4. RESEARCH APPROACH, EMPIRICAL STUDY DESIGN AND ARRANGEMENTS

My research is action research and I will follow the Cogenerative Action Research Model [21]. I have not yet found a suitable theory to apply as a frame of reference. I will combine existing theories in related disciplines and grounded theory. Possible theories could be: Adaptive structuration theory (AST) Agency theory Contingency theory Organizational learning theory

4.1 Definition of most important metrics

I will mostly gather qualitative data. My PhD-work has just started and I have not yet got to the point where I have metrics specified in full detail. In the interviews, in Company A, I asked questions along 6 dimensions: Shared leadership, Team orientation, Redundancy, Learning, Autonomy and Trust. After transcribing an interview I gave each dimension a score for that interview. The scores for all the interviews were aggregated and then I found that Team orientation and Redundancy are dimensions that got a low score, while Autonomy and Trust got a high score, see figure 1. I used the same definitions for the dimensions as Moe et al. [1]. Team orientation is giving priority to team goals over individual goals[1]. If the score is 0 this indicates that individual goals are more important than team goals. A 10 means the opposite. If the team members feel ownership to the project plans this also

The Dickinson and McIntyres teamwork model [22] is used in a case study of a scrum project, and the researchers found that it explains most of their findings, but that it should be extended with the components trust and shared mental models [23]. They recommend that in future work the extended model should be used in a mature agile development team, and a team using shorter sprints [23]. I can do this in my research; I have both the access to a mature agile project, and a project with sprints that are 2-3 weeks long. Table 1 shows an overview of the 4 companies that will be the hosts for my case studies. In company A I observes a project that develops, manage and enhance a custom-made web-based content management system for a Fortune 500 industrial company. The Table 1: Overview of the companies

indicates high Team orientation [1]. Redundancy means that team members are able to perform each others jobs and can substitute each other [1]. It is impossible for a team to have full redundancy, and it is unclear how much redundancy it is most effective to have [1]. A score of 0 means no redundancy, a 10 means full redundancy. Autonomy is defined as the influence of management and other individuals (outside the team) on the teams activities. [1] A score of 0 means that decisions made by the team are not respected (by managers outside the team), and 10 means that the decisions are respected. Self-managing teams should share the leadership and if everyone is involved in the decision making process this indicates that the team has shared leadership [1]. A 0 means that the team does not have shared leadership, while a 10 means that they have. Learning is indicated by giving feedback and constantly improving the work, and is important for effective teams [1]. A score of 0 means that they dont do this. A 10 means that team members give feedback to each other and continue to improve work methods. If the team members dont rely on their team members to do their work there is a lack of trust [4]. Trust can foster team effectiveness, but virtual teams can have difficulties with the trustdimension [5]. A score of 0 means that the team members dont trust the others, while a 10 means that they do.

4.3 Preliminary results

4.3.1 Team orientation
Team orientation refers to team members giving priority to team goals over individual goals, and is indicated by ownership to the plans and members relating to the tasks of individuals [1]. Team orientation got the lowest score of all the dimensions I asked in the interviews; it got the score 2. Quotes from the interviews that illustrate why they got a low score on this dimension are: Do all team members feel ownership to the project plans? No, Since I dont really know the long term plans I cant have any ownership to them The decisions about what to solve are not taken by the people working with it. That is communication between the project leader and the customer. We dont know what to do until we develop it. We just pick from the top of the list. So that means not everyone can be aware of the plans, or influence them.

Do team members show interest in other persons tasks? I would say partly. Personally I am not very interested in what others are doing. So I will say partly. Sometimes, and sometimes not. You can see it at the standup-meetings. Some people are not that interested. They wish they werent there.

4.2 Data analysis methods and techniques

The interviews conducted are transcribed. Data analysis will be based on Qualitative Data Analysis: An expanded Sourcebook [24]. I have used NVivo8 to analyze the data (www.qsrinternational.com), and the analysis is still an ongoing process. I have grouped and contrasted statements. Figure 1: Team radar result from Company A:

In [5] they propose the following research questions: Do virtual team members identify with their team as a social entity or do they remain tangential to it? Are virtual team members able to perform satisfactory even when they do not identify with the team? . I therefore asked in the interviews: Who do you see as your team members? I got the following answers: 1. That is a good question. It is not that clear. If I have to pick a group and call it my team I have to say it is the transitionteam. That is 3 people from Norway and 6 people from the other country That is everyone in Norway, including Help-desk That is me and 4 other from Norway I dont feel that we have a specific team inside this project. We used to have it, but not anymore. If I have to choose some people and call it my team members I would say me plus 5 people in Norway, or everyone in Norway sitting here at Company A That is me, only me That is the 10 people working with the Content Management part, not the people working with SharePoint The people I sit with are the easiest to perceive as my team members. But at the same time I also perceive some people in the other country as my team members more than other people there. I can also say that everyone in Norway are my team members. That can include Help desk when they are sitting here.

2. 3. 4.

5. 6. 7.

8.

That definitely depends on how you define it. That is either me and 4 other people or it is everyone sitting here at this site, that is 15 people.

about things that are not relevant to all Long meetings (especially the daily scrums) Some competence sit only with some people Sometimes different perceptions responsibility to reach goals The sprints are too short (only 2 weeks) Not everyone sits at the office, they work from home, which sometimes extends the time of the work process of personal

I think this is interesting, and in the definition I use of teamwork the team members should see themselves in an intact social entity. In the feedback session with the company I will discuss the different perceptions of teams, and ask them to discuss and agree on what teams they have in the project and which people belongs to the different teams. This might increase the team cohesion which is seen as fundamental for team effectiveness [5]. I have distributed a simple questionnaire to team members in company B. The questionnaire will be given later to the project I did interviews in to get additional metrics. In this questionnaire I used a 5-point scale. I got 15 respondents from 2 scrum teams. Most of the questions had a 5-point scale. 1 was Strongly disagree and 5 was Strongly agree. The questionnaire was in English, as were most of the answers. The results are shown in table 1 and table 2. It seems that team 2 is a little more satisfied with the retrospectives than team 1. Some quotes from the open-ended questions in the questionnaire are:

Positive statements that were listed about the teamwork: Good members. Reliable. Good co-operation Good atmosphere, supportive Everyone is good at helping, when I ask for it More efficient now It's a good feeling between the team members Very good skills make things easy Feeling of community Utilization of the persons strong points Not dependent on only one person Knowing others skills well

Negative statements that were listed about the teamwork: Fragmented knowledge Sometimes there is too much discussion in meetings

Table 2: Questions about the retrospective meetings

Feeling responsible for the outcome since others depends on it Good commitment

4.4 Validity threats and their control

It may be difficult to ensure good validity and generalizability since I am doing case studies, but my research will be based on the books [25] and [26] to ensure a good research design. I will also follow Guidelines for conducting and reporting case study research in software engineering given by [27]. I will present my results to the companies to get feedback and ensure that I have drawn the right conclusions. Triangulated sources and prolonged involvement will improve the validity of my research. The fact that I can compare 4 different cases and make it into a comparative case study can make it more reasonable to generalize the results.

their team. I have transcribed the interviews and will continue to analyze the data. So far the analysis shows that the dimensions Team orientation and Redundancy are dimensions that got a low score and this indicates that the team has challenges in these areas. I will discuss this with the team, and together we will see if there is something that we can do to score higher at these dimensions, and try to find out if this makes the teamwork more effective. I company B I have observed 2 Scrum-teams. They have filled out a questionnaire after a retrospective meeting. This questionnaire had 30 questions. A total of 15 people responded from 2 teams, including 2 scrum masters and the project owner. Fragmented knowledge and too long meetings are things they listed as negative about the teamwork. I want to conduct interviews and get a deeper understanding of what we can do to make the teamwork more effective. My next step will be to find a theory to use. I will also try out Appreciative Inquiry in retrospectives, and see how this affects the teamwork. I will carry out more interviews and distribute the questionnaire to more teams. When time allows I plan to observe in the two other companies, and conduct interviews there as well.

5. SUMMARY
I have done research in two of the companies since March 2010, company A and B. I have met with employees in company C and D, but I havent done any research there yet. I have conducted interviews in company A, and identified that there was a very diverse perception of teams. The team members had a lot of different opinions of which people they included in

Table 3: Questions about the teamwork

6. APPENDIX A. Interview guide

First the interviewees were told about the purpose of the interview, the approximate time for the interview, and that only I would have access to the transcripts. They were asked if I could audiotape the session and the purpose of this. At the end of the introduction I asked if they had any questions before we started. Warm-up: 1. 2. 3. 4. How old are you? Education? How long have you been working for this company? How long have you been working with the same technical problems and technology that you are working on now (in this or another company) ? How long have you been on this project? Are you full-time on this project? What is your role in the team? What do you do? Who do you see as your team members? Do you perceive the teamwork as effective? a. b. How effective do you think the meetings are? Who prepare the meetings? Are participants prepared for the meetings? the

Trust / Distributed teams: 7. 8. 9. What do you think about the teamwork? Do you rely on your team members to do what they are supposed to do? Do you rely on other teams to do what they are supposed to do?

10. What do you think about collaborating with the people in the other country? 11. What are the challenges of working with a team that is located far away? 12. On a scale from one to ten, how effective will you say that the teamwork with the other country is? 13. On a scale from one to ten, how good will you say that the communication in the project is? 14. How will you characterize the teamwork compared to other teams you have been a part of? 15. How will you characterize the work between teams in the project compared to other projects you have worked on? 16. Do you experience challenges? any languageor cultural

5. 6. 7. 8. 9.

Teamwork:

17. What do you think about the information flow in the project? Autonomy: 18. Does team loose resources to other projects/tasks? 19. Are the decisions made by the team respected by people and groups outside the team? 20. Do people and groups outside the team have influence over important operational decisions in the project? Learning: 21. What are the arenas where you give feedback on each others work? 22. How are software development problems identified, and do you improve the development method? 23. Do you keep what works well in your development process? 24. What do you do to learn? Closing: 25. Do you have any questions to me? 26. Are there any other aspects of teamwork that you would like to discuss, which were not covered by the questions asked?

10. Is there anything you think could improve the effectiveness of the teamwork or the project in general? Shared leadership: 11. Do you have clarified goals? How are the team goals defined? 12. How do you make decisions? Is everyone involved in the decision-making process? 13. Do team members make important decisions without consulting other team members? Team orientation: 14. How does the team take into account alternative suggestions in team discussions? 15. How does the team value alternative suggestions? 16. Do team members show interest in other individuals tasks? 17. Do all team members feel ownership to the project plans? Redundancy: 1. 2. 3. 4. 5. 6. How are tasks coordinated?) allocated? (How is the work

What do you do if someone in the team has too much to do or is away because of illness? What happens if someone gets stuck? How easy is it to complete someone elses task? Do you help others when they got problems? If someone leaves the team, how difficult or easy is it to substitute this person?

6. References
[1] Moe, N. B., Dingsyr, T. and Ryrvik, E. A. Putting Agile Teamwork to the Test An Preliminary Instrument for Empirically Assessing and Improving Agile Software Development. Springer Berlin Heidelberg, 2009.

[2] Sharp, H. and Robinson, H. Three 'C's of Agile Practice Collaboration, Co-ordination and Communication. SpringerVerlag, Berlin, Heidelberg, 2010. [3] Dingsyr, T., Dyb, T. and Moe, N. B. Agile Software Development: An Introduction and Overview. 2010. [4] Jarvenpaa, S. L., Shaw, T. R. and Staples, D. S. Toward Contextualized Theories of Trust: The Role of Trust in Global Virtual Teams. Info. Sys. Research, 15, 3 (2004), 250-267. [5] Powell, A., Piccoli, G. and Ives, B. Virtual teams: a review of current literature and directions for future research. SIGMIS Database, 35, 1 (2004), 6-36. [6] Cohen, S. and Bailey, D. What Makes Teams Work: Group Effectiveness Research from the Shop Floor to the Executive Suite. Journal of Management, 23, 3 (1997), 239-290. [7] Beck, K., Beedle, M., Bennekum, A. v. and Cockburn, A. Agile manifesto.2001. http://agilemanifesto.org [8] Frederick P. Brooks, J. No Silver Bullet: Essence and Accidents of Software Engineering. Computer, 20, 4 (1987), 1019. [9] Abrahamsson, P., Salo, O., Ronkainen, J. and Warsta, J. Agile software development methods: Review and analysis. VTT Publications, 478(2002). [10] Cohen, D., Lindvall, M. and P.Costa An introduction to agile methods. Advances in Computers, Vol. 62(2004), 1-66. [11] Dyb, T. and Dingsyr, T. Empirical studies of agile software development: A systematic review. Information and Software Technology, 50, 9-10 (2008), 833-859. [12] Abdullah, N. N. B., Sharp, H. and Honiden, S. Communication in Context : A stimulus-Response Account of Agile Team Interactions. Springer-Verlag Berlin Heidelberg, 2010. [13] Doraoraj, S., Noble, J. and Malik, P. Understanding the Importance of Trust in Distributed Agile Projects: A practical Perspective. Springer-Verlag Berlin Heidelberg, 2010. [14] KINNI, T. Exploit what you do best. Harvard Management Update, Vol. 8, 8 (2003).

[15] Coghlan, D. Appreciative Inquiry: Rethinking Human Organization toward a Positive Theory of Change. Leadership & Organization Development Journal, Vol. 21, 4 (2000). [16] Beck, K. Appreciating your way to XP.2010. http://www.threeriversinstitute.org/AppreciatingYourWayToXP.h tm [17] G.R. Bushe and Kassam, A. F. When Is Appreciative Inquiry Transformational? A meta-case analysis. The Journal of Applied Behavioral Science, 41, 2 (2005), 161-181. [18] Whitney, D. K. and Trosten-Bloom, A. The power of appreciative inquiry: a practical guide to positive change. Berrett-Koehler, 2010. [19] AI. Appreciative inquiry research 2010. http://appreciativeinquiry.case.edu/research/bibPublished.cfm [20] T. Dingsyr, T. Dyb and N.B. Moe Agile Software Development: Current Research and Future Directions. Springer, 2010. [21] Greenwood, D. J. and Lenvin, M. Introduction to Action Research: Social Research for Soctial Change. SAGE Publications, Inc, 2007. [22] Dickinson, T. L. a. M., R.M. A conceptual framework for teamwork measurement. Team performance assessment and measurement: Theory, methods, and applications (1997), 19-43. [23] Moe, N. B., Dingsyr, T. and Dyb, T. A teamwork model for understanding an agile team: A case study of a Scrum project. Information and Software Technology, 52, 5 (2010), 480-491. [24] Miles, M. B. and Huberman, A. M. Qualitative data analysis: an expanded sourcebook. Sage, 1994. [25] Robson, C. Real world research: a resource for social scientists and practitioner-researchers. Blackwell, 2002. [26] Yin, R. K. Case study research: design and methods. Sage, 2009. [27] Runeson, P. and Hst, M. Guidelines for conducting and reporting case study research in software engineering. Empirical Softw. Engg., 14, 2 (2009), 131-164.