Systemist, Vol. 28(2), Jun.

2006

Soft Systems Approach for Managing Free and Open Source Software Quality
Anas Tawileh, Wendy Ivins and Omer Rana

School of Computer Science, Cardiff University

5 The Parade, Cardiff CF24 3AA, UK {m.a.tawileh, w.k.ivins, o.f.rana}@cs.cardiff.ac.uk

Abstract
Quality in the free and open source software (F/OSS) community has been an explored area for a long while. This can be attributed to the lack of consensus about the notion of quality among different stakeholders. In this paper, we propose the use of Soft Systems Methodology as a radically different approach to software quality that could accommodate the systemic nature of the quality problem. The application of SSM was combined with UML to design and implement a supporting information system that would facilitate the integration of users' needs and expectations early in the development process.

Introduction
Software development practices have received considerable research attention and many initiatives have been proposed to improve the quality of their resultant artefacts. However, quality within relatively new development paradigms, such as free and open source software development, is still an area of ambiguity and concern. This can be attributed partially to the very nature of F/OSS development practices, which are based on the contribution and distributed collaboration of large numbers of developers from all over the world, and the highly informal information sharing structures of the community. However, we are of the opinion that it is necessary to reassess the very definition of quality associated with software artefacts, especially when viewed with reference to the F/OSS development process. The F/OSS community argue that the software they develop is technically superior to many commercial efforts and that their open and transparent development practices would provide for higher quality levels than those of proprietary software. End users, on the other hand, oppose these claims by describing the lack of appropriate end user participation mechanisms within the F/OSS community, and the consequent lack of regard of user needs and requirements. This issue is not exclusive to F/OSS software, which is evident in the increasing demand for more involvement of users in the development process -- proposed by recently evolving paradigms such as prototyping and extreme programming. However, the particular nature of open interaction and lack of authority in the F/OSS community magnifies the users feeling of alienation in the development process. The apparent lack of any consensus about the notion of quality at large further complicates the situation. In the literature, different definitions have been proposed to explain quality. The International Standards Organisation (ISO, 2002), for instance, defines quality as: "The totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs."

-1-

Soft Systems Approach for Managing Free and Open Source Software Quality
Another definition of quality is proposed by Juran as being "fitness for user" (Juran, 1979), and Crosby considers quality to be "conformance to requirement" (Crosby, 1979). Placing quality in the context of F/OSS software development provides even more difficulty to developing a coherent definition. As software has mixed characteristics of both a product and a service, and being developed in an open environment, the expectations and perspectives of different stakeholders are more susceptible to conflicts and collisions. In addition, the lack of end user involvement makes it difficult to claim quality in reference to requirements such as the definition proposed by ISO. It was suggested that in the context of software development, the process of defining quality is as problematic as the management of quality (Vidgen et al., 1993). A systemic approach to enhancing quality in the F/OSS community is proposed. We suggest that by investigating the systemic mechanisms underlying software quality, the perspectives and expectations of different stakeholders may be accommodated, and a set of mechanisms could be devised and implemented to i prove the problematic situation of software quality in an open environment. The m paper aims to test the validity of systems thinking ideas, and the Soft Systems Methodology (Checkland, 1999) in particular, in tackling ill defined and complex problem situations, and which contain humans with differing perspectives and free will. The use of Soft Systems Methodology was supplemented by other modelling techniques such as UML to compensate for the limitations in the methodology in developing an eventual solution. This approach also demonstrates the potential gain of combining both techniques (Bustard et al., 1999), along with mechanisms to evaluate the proposed approach.

Background
Development practices within the F/OSS community are fundamentally different from those adopted by the proprietary software industry. The main principle underlying the F/OSS movement is the freedom of the end user to access the software source code, modify it and redistribute it for third party use, as long as s/he respects the terms and conditions of the original license of the software thereby guaranteeing the transfer of similar modification privileges to any subsequent users. These principles encourage interested developers to engage in developing, promoting and initiating F/OSS software projects. The high levels of collaboration and communication required to tackle the complexity of software development projects, coupled with similar ideals of the participating volunteers, has resulted in the formation of the largest virtual community in human history. This community has several distinct characteristics; it contains different schools of thought, ranging from pure idealists to extreme pragmatists. It spans geographical and cultural boundaries more than any other human community or business enterprise. However, the benefits associated with this diversity come at a price. While the F/OSS community has developed novel mechanisms to effectively tackle coordination and collaboration difficulties resulting from its open, informal and highly distributed nature; some areas were, perhaps unintentionally, not given sufficient consideration. Members of the F/OSS community usually claim credit for producing, in a very informal and decentralised process, software that outperforms centrally coordinated, heavily financed counterparts. They believe that the evolved open and transparent development practices constitute a sufficient guarantee of the quality levels attainable by the community (Raymond, 2000). However, users of F/OSS software do not seem to agree much with such claims. They believe that the claimed open access to the community is restricted by barriers of difficult language and technical jargon. Users do

-2-

Tawileh, Ivins and Rana

not care much about the internal workings of a particular piece of software, as long as it satisfies their needs. Moreover, there is a lack of approaches for determining requirements of end users or procuring interest from end users in suggesting projects. Therefore, the different interpretations of what constitutes good software and how this could be judged exemplify the lack of consensus in defining software quality. Different frameworks have been proposed to measure the quality of F/OSS software. These initiatives include the Open Source Maturity Model (OSMM), developed by Navica (OSSM, 2003), CapGeminis Open Source Maturity Model (Duijnhouwer and Widdows, 2003) and the Business Readiness Rating (BRR) (OpenBRR, 2005). Although they provide a fairly reasonable way to determine free and open source software quality, these frameworks attempt to evaluate the quality of F/OSS using currently available information, such as the number of releases, number of security vulnerabilities, reference deployments, market penetration and others. These factors are highly subjective, and can be judged differently from different perspectives. They also fail to suggest measures that would improve the process of software development in the first place. Therefore, inquiry into the nature of the problem of software quality in a F/OSS context should take a different approach.

A Soft Systems Approach

In order to accommodate the different perspectives and interpretations of the notion of software quality by different stakeholders, a wider systemic approach needs to be considered. Systems thinking suggests that problems in the real world are characterised by complexity and contain human actors, who are very likely to have different, sometimes contradicting, viewpoints (Checkland, 1999). The problem at hand is characterised by complexity. The whole process is enacted by i teracting n human activities and aims to satisfy a particular need (the development of a specific artefact that fulfils a specific set of requirements). Existence of humans complicates the situation as humans have free will, and usually have genuine freedom of choice in selecting their actions (Checkland, 1999). Because every human actor has his own perception of the intended aims of the process, and his distinct expectations, this results in the whole process being perceived differently by each actor with sometimes contradicting viewpoints. The different viewpoints of users and members of the community discussed above is a striking example. These characteristics of the problem situation seamlessly fit within Peter Checkland's definition of a "purposeful" Human Activity System (HAS) (Checkland, 1999), for which he argues that the wellestablished methods of science can not provide appropriate tools for study. Checkland claims that problematic situations with human activity systems are highly unstructured. Building structure into the problematic situation itself is almost impossible, as the problem does not exist in the real world; it exists in the observer's perception of the real world, which is the main distinction between soft and hard systems. Therefore, any account of the problematic situation should be based on a specific perception (Weltanschuuang or world view). This description perfectly applies to the problematic situation under study, as the different perceptions of the various stakeholders result in diverse range of understanding of the problem (namely the definition and perceived value associated with software quality). Checkland proposes the Soft Systems Methodology (SSM) as "a general problem solving approach appropriate to human activity systems" (Checkland, 1999).

-3-

Soft Systems Approach for Managing Free and Open Source Software Quality Modelling
SSM suggests an intellectual construct called "Root Definition" (RD), which aims to provide a clear and unambiguous textual definition of the system to be modelled. It provides a way to capture the essence (root) of the purpose to be served (Checkland, 1999). Different root definitions could be developed to accommodate the different viewpoints or activities of the system. The following two root definitions were developed to capture the essence of the software quality situation from two different perspectives: end users and developers:

RD-1: A system to provide guarantees acceptable by end-users that developed artefacts within the F/OSS community will meet users' expectations by deriving appropriate mechanisms to assess these artefacts and produce such guarantees, while considering the resource constraints inherent in the F/OSS community.

RD-2: A system to ensure that artefacts developed within the F/OSS community will meet determined quality criteria by deriving mechanisms to evaluate and approve submitted contributions to these artefacts against the acceptance criteria.

Based upon the formulated root definitions, the conceptual model that represents the system described in these root definitions was developed. The conceptual model illustrates any required activities present within such systems, in addition to any logical relationships between these activities. These relationships represent logical dependencies between activities, and would allow one to determine the inputs and outputs of each activity. The resulting conceptual model is reproduced in Figure 1. One weakness of the Soft Systems Methodology is that although it structures highly complex and messy problems and provides valuable aid in investigating possible intervention in situations involving humans, it only provides insight into what should be done to improve the problematic situation. SSM does not provide any guidance on how to build the required support system or implement the proposed solutions. In order to describe the proposed solution in such a way that facilitates its implementation, different tools and techniques should be used in conjunction with the artefacts of SSM. Several studies have attempted to bridge this gap, including the work of Bustard, Zhonglin & Wilkie (1999) and Wade & Hopkins (2002) who argue that SSM lacks the detailed information required to develop the intended systems, and that the use of the Unified Modelling Language (UML) would compensate for this weakness and provide the level of detail required to facilitate the design and implementation of the desired systems. Mingers proposed embedding structured methods into SSM (Mingers, 1992). Two primary processes could be identified within the conceptual model, the boundaries of each process are illustrated in Figure 1: 1. The F/OSS development process. 2. The guarantees' provision process.

-4-

Tawileh, Ivins and Rana

Figure 1. SSM Conceptual Model

Each process was identified by grouping appropriate activities relevant to the achievement of the process' goals. Links between activities determine the logical progression of action and the required resources and expected results for the achievement of each activity. To enable implementation of any proposed interventions that resulted from the SSM analysis, it was decided that UML should be used. This was undertaken in order to create models that describe the proposed solutions in sufficient detail to enable subsequent design and implementation of the supporting information system. Using UML's standardised notation assures the clarity of representation and ease of communicating the designed systems. The synergy of complementing SSM and UML will enhance the final outcome, and provide a logically defensible development route from high level system analysis down into specific system implementation details (Wade and Hopkins, 2002) (Bustard et al., 1999). The UML activity diagram for each process can be developed by mapping each activity within the process boundary in the conceptual model (Figure 1) to an activity in the UML diagram using its standardised notation. Flows in the UML diagram should match the dependencies represented in the conceptual model and the logical sequence of activities. The activity diagram also illustrates the actor who should perform each activity in addition to the resource requirements and outputs generated by each activity. Using UML modelling makes actors and outputs more explicit and therefore provides the

-5-

Soft Systems Approach for Managing Free and Open Source Software Quality
basis for the design of the supporting information system. Figure 2 shows the UML activity diagram developed for the guarantees' provision process.

Figure 2: UML Activity Diagram of the guarantees' provision process

Atkinson (2000) proposes the SISTeM approach for informing debate about the development of an information system focusing on human/machine activities. The approach is based on the use of two "cycles": (i) the first deals with the overall strategic decision making; (ii) the second deals with translating some of these strategic decisions into operational actions that lead to some change within an organisation. The SISTeM approach is more comprehensive than SSM, taking account of social, political and market impacts on a problem situation, and involves the use of multipe forms of models: conceptual, expressive and matrix. As a consequence, the approach is also more complex to use, requiring a much more significant involvment of the actors and their understanding of the context in which they are evaluating a particular problem situation. The approach is therefore useful for understanding the likely impact an information system will have on an organisation, and the subsequent debate that is necessary to fully appreciate how the information system impacts a particular actor and his/her role. However, the usefulness of the approach to realise an information system is limited in scope. The second cycle in SISTeM, which focuses on particular operational decision making, also does not fully indicate the set of activities that are required to implement the eventual information system. This aspect of the final realisation of the information system is still open to the interpretation of a developer. The SISTeM approach therefore provides a useful starting point to better evaluate requirements and relate these to the wider context within which a F/OSS software may be deployed. Its usefulness as a means to translate these requirements into an information system is limited in scope.

Exploiting the Conceptual Model

The developed SSM conceptual model was further analysed to determine whether each activity currently exists, and if so, how effectively it is currently being performed. Evaluating each activity in the conceptual model identified missing activities that should be added, and activities that could be improved. This was performed by mapping each activity in the conceptual model to the real world of the F/OSS community, and for each activity the following questions were investigated (Wilson, 1984):

q q

Does the activity currently exist in reality? Is the activity in the real world done the way it should be done?

-6-

Tawileh, Ivins and Rana q

Is the activity the right thing to be done in the context of the system represented by the conceptual model?

This mapping leads to recommendations for possible actions that could be implemented by the F/OSS community to improve the uptake of F/OSS artefacts. The assessment of activities in the conceptual model in order to derive any relevant recommendations is presented in Table 1.

Activity

Is It Done Now user's No

By Whom

Outputs

Process

Inputs

Judgement

Recommendations

Determine expectations Determine artefact's requirements

End User

No formal exists

system

Implement measures to collect user's expectations Improve current requirements determination practices to consider user's expectations Implement measures to determine guarantees to be provided to customers based on their expectations and the artefact's requirements Implement measures to obtain user's acceptance of determined guarantees Improve the process of deriving mechanisms to produce guarantees to develop mechanisms that would produce acceptable guarantees for users Implement measures to enable users to assess different mechanisms for guarantees production Implement measures to enable users to select the appropriate guarantee production mechanism Implement measures to assess developed artefacts against the selected mechanisms Implement mechanisms guarantees selected produce

Partially

Maintainer

Current system does not consider user's expectations

Determine guarantees to be provided to users

No

Maintainer

No formal exists

system

Obtain user's acceptance of guarantees Guarantee Provision

No

End User

No formal exists

system

Derive mechanisms produce guarantees

to

Partially

M aintainer

Development of assurance mechanisms currently is based only on the maintainer's judgment

Assess derived mechanisms

No

End User

No formal exists

system

Select mechanism

appropriate

No

End User

No formal exists

system

Assess artefact selected mechanism

against

No

Maintainer

No formal exists

system

Produce guarantees

No

System

No formal exists

system

to

Table 1: Analysis results of the guarantee provision process

-7-

Soft Systems Approach for Managing Free and Open Source Software Quality
Analysis of the conceptual model indicates a significant lack of end user involvement within the current development processes within the F/OSS community. In order to improve the quality levels of the F/OSS artefacts, end user requirements and expectations should be considered from the early stages of the project development process. Quality may be perceived differently by different parties (Michlmayr, 2005), and our proposed approach suggests a systematic method to consider and integrate the different perspectives of developers and end users. Software development projects within the F/OSS community are initiated by a developer or a group of developers to fulfil their own personal requirements. This contrasts with the traditional software development process which is mainly initiated to satisfy specific customers' requirements. The significant share of server side applications within the F/OSS community in comparison to desktop and end user applications is a direct consequence of this characteristic. However, the different advantages provided by the F/OSS software stimulated the wide adoption of these applications, which in turn increased the size of both its users' and developers' communities. F/OSS community members were urged to adapt to the new situation by trying to involve the end users more in their software development process in order to meet their specific requirements and expectations. Nevertheless, the evolutionary nature of this change prevented the development of clear, systematic procedures to involve users and incorporate their needs and expectations. Many approaches were based on simple, unstructured web based communication and collaboration techniques such as mailing lists and discussion forums. Each developer, after initiating his project, will try to define the expectations from the project. In the current tradition within the F/OSS community, the developer usually determines the project's direction based merely on his own expectations and needs. The process suggested by the conceptual model analysis recommends implementing measures to include user's expectations in the early stages of the project development. Upon determining the maintainer's and user's expectations, the maintainer will try to decide whether the user's expectations are inline with his own. At this point, the maintainer may make some compromises on his own expectations in order to align them with those of the users or if there are substantial differences between the two, the maintainer may not alter his expectations, and the user might look for another project that better matches his expectations. After a balance is struck between the maintainer's and user's expectations, the maintainer will translate those expectations into detailed requirements. In the current processes, the maintainer translates his own expectations into requirements, without considering those of the user. This process will be improved by considering the user's expectations, in accordance with the findings of the process analysis. Requirements are descriptions of "required features of the proposed system". They may be functional (describing what the system should do) or non-functional (describing how a facility should be provided, or how well, or to what level of quality) (BSI, 1994). F/OSS projects usually lack an explicitly formulated requirements specification (Bartkowiak, 2005), because most of the projects depend upon the evolutionary development of requirements. Determining project's requirements does not conflict with this concept, and the recommendation simply suggests formulating the translated expectations in clearly defined requirements that the developed project should meet. The initial formulation may capture only the essential required functionality, and may advance through the same current evolutionary development mechanisms. For the developer to assure the users that the artefact she is developing matches their initial expectations, she has to devise and suggest a set of guarantees that would serve this function. Users

-8-

Tawileh, Ivins and Rana

then will decide upon which guarantees are accepted by them as a sufficient proof of her claims. Users' involvement in deciding on the acceptability of produced guarantees is the critical point that will increase users' trust in the developed artefact. If the artefact under development is taken to be a piece of software, the ultimate goal of the guarantee is to prove that the developed software will meet users' expectations. Software testing is the most obvious method to check the ability of any piece of software to satisfy its intended requirements and expectations. Consequently, it follows naturally that software testing results should provide an acceptable verification of the software's ability to perform as intended. These test results could be used as guarantees that end users may rely upon to assess software suitability to their requirements and expectations. Such tests should be reproducible and transparently performed. Software testing in general aims at measuring software quality. While software quality lacks a clearly defined notion, it is suggested that software has only to be as good as its users expects it to be (Bach, 1996). This means that for any application software to be (good enough) from its end user's perspective, it should satisfy the requirements upon which it was developed. Therefore, the existence of at least a requirement is inevitable for the evaluation of a software application's adherence to these requirements. Unfortunately, current F/OSS development processes do not require the availability of any requirements or specifications, nor do they encourage users' involvement in setting these requirements. However, the previously proposed actions suggest specific mechanisms to alleviate this problem and increase users' participation in determining their expectations from the F/OSS artefacts. The project maintainer should propose guarantees that warrant both the functional and non-functional requirements of his software. Afterwards, a reasonable number of users should approve the proposed guarantee and determine the adequate acceptance criteria. The project maintainer may propose different criteria for users to select from. Given that these criteria are based upon careful prioritisation of the previously created software requirements, which were themselves based on the users' and project maintainer's expectations, users and the project maintainer are very likely to strike an agreement on one of the proposed criteria. Once the project maintainer and the user have agreed on the proposed tests and acceptance criteria that will be used as guarantees to prove the software's compliance with their expectations, the mechanisms to produce these guarantees could be investigated and evaluated. Currently, project maintainers may develop such mechanisms for different, usually subjective reasons. However, in the majority of cases, they base their selection merely on their own judgment; they do not engage the user in deciding on which mechanisms would be acceptable from his point of view. In addition, the lack of clearly defined requirements for the majority of F/OSS projects will decrease the effectiveness and impact of such mechanisms. The proposed process suggests that the guarantees should be approved by users prior to the derivation of the appropriate mechanisms to produce them. This sequence makes sense in two aspects: firstly, it enables the project maintainer to focus on the derivation and proposition of mechanisms that will be appropriate to produce the intended guarantees. Secondly, it prevents the selection of irrelevant mechanisms which may not produce guarantees that would be accepted by users. If the ultimate goal of this process is enabling the end user to assess and select F/OSS artefacts based on his own preferences, then it becomes extremely important to obtain the user's acceptance of all the proposed procedures.

-9-

Soft Systems Approach for Managing Free and Open Source Software Quality
In the case of software, the guarantees to be provided are d efined as a set of functional and nonfunctional test results that should satisfy the established acceptance criteria. Therefore, the mechanism to produce these guarantees should provide appropriate test results as its output. If the provided test results satisfy the acceptance criteria, then the software has met the users' expectations, and those results could be used to guarantee this claim. Transparency and openness throughout the process will enable users to verify the guarantees by themselves, which will increase their level of confidence in the software under consideration and in the development process as a whole. Deriving appropriate mechanisms for this scenario then melts down into the selection of possible testing techniques that could provide the intended test results in a transparent and repeatable manner. Selection of appropriate testing techniques largely depends upon the implementation technology (programming language, operating system, etc) and the availability of testing tools for that partic ular technology. The selected tool should be able to perform tests that would generate results which can be validated against the approved acceptance criteria. The final activity in the process as illustrated by the SSM conceptual model is the assessment of the developed artefact against the selected validation mechanisms in order to ensure the proper functionality of the artefact and to produce guarantees that assure the end user of the conformance of the artefact to his expectations. If the project maintainer validates the artefact against the selected mechanism and the artefact passes the proposed tests, she can release the artefact with greater confidence that it will attract better adoption by end users. Test results and the acceptance criteria should be bundled with the released artefact to enable the authentication of the guarantees by any potential user and to ensure accurate linking between different releases and their associated guarantees. Although project maintainers can tweak their own projects' collaboration platforms to perform the proposed activities and processes, this will result in different implementations of the same concepts which might cause confusion to users. Building a central platform that provides these services in a standardised way and which project maintainers could use to host information about their projects would eliminate this confusion. This approach also offers the advantage of comparing different projects against each other to select the most suitable option. A simple prototype was developed to demonstrate the applicability of the proposed interventions and the functionality of the supporting information system that could support the implementation of the recommended courses of action. The prototype acts as an integrated pla tform to allow communication between developers and end users at the different stages of the project development lifecycle, and an easy and convenient way to communicate test results and quality information. It could be improved by integrating test automation into the system's interface. Results produced by the assessment methodologies proposed by previous initiatives can be supplemented by the results stored within the system to provide for better and more extensive evaluation of F/OSS artefacts.

Conclusions
Quality in the free and open source (F/OSS) community is a sensitive issue that has been the subject of much debate. Although the F/OSS community has proved to be highly effective in managing massively complex distributed development efforts, it has not provided proper consideration to various quality aspects of its development processes. The main issue of the quality problem lies in the lack of consensus on what software quality actually is, and whose interpretation of quality should be considered. Different initiatives were proposed to evaluate and measure quality in an open source

- 10 -

Tawileh, Ivins and Rana

context. However, the essential question of how to deal with the variety of perspectives of multiple stakeholders was not answered. A radically different approach towards the investigation of F/OSS software quality is being investigated in this work. Based on ideas form the systems thinking field, the problem was placed in a wider context considering the different stakeholders and their varying perceptions and interpretations of the notion of quality. The principles of the Soft Systems Methodology were used to stimulate inquiry into the problematic situation, and its intellectual tools provided a more elaborate understanding of the different factors and dynamics involved in determining software quality. By using the SSM conceptual constructs to analyse the problematic situation, various courses of actions to improve the problematic situation were derived. In order to implement the proposed courses of actions, a supporting information system was developed that put these recommendations into the service of the F/OSS community. A key aim of such an information system is to enable F/OSS developers to: (i) interact with end users to better assess their requirements, (ii) mechanisms of automated testing of software artefacts that result from an open source project; and (iii) mechanisms to allow end users to evaluate the usefulness of a particular software artefact, and subsequently steer a software development process in a particula r direction based on requirements that have been identified by the majority of end users interacting with developers. The use of SSM was combined with the Unified Modelling Language to facilitate the design and implementation of the supporting information system. SSM was useful as it allowed the development of a conceptual model that incorporated a developer's and a customer's viewpoints. A more extensive model could be developed by eliciting viewpoints from a number of developers and customers and using these to develop further root definitions and conceptual models. However, analysis of our conceptual model identified a significant number of activities that were not being addressed in the F /OSS community, which has been the focus of our further research. There is also some difficulty in moving from the conceptual model that explores the problem situation to developing a solution system. UML activity diagrams were found to be particularly useful as they can be used to map activities from the conceptual model into activities in processes, which form the basis for the behavioural design of the solution system. However, further activities need to be in place in the activity diagrams to ensure that the processes can map into real world activities. The combination of SSM and UML proved to be highly beneficial. While SSM aided the identification of required system activities, UML provided the means to link those activities to the final information system and communicate the system design in an elegant, standardised notation that is widely understood by software developers. This approach promotes higher levels of software quality within the F/OSS community by integrating users' needs and expectations from the early stages of the development process. Transparency and openness of the proposed process were emphasised during all stages of analysis in order to guarantee the production of a system that would be acceptable by the community as it adheres to its values. An open and transparent process will also increase users' confidence in the whole system and encourage them to trust its outcomes and participate in its development.

Acknowledgement
We would like to thank Steve McIntosh for his suggestions on the use of SSM in this work.

- 11 -

Soft Systems Approach for Managing Free and Open Source Software Quality References
Al-Humaidan, F. and Rossiter, N. (2004), 'Business Process Modelling with OBPM: Combining Soft and Hard Approaches', Workshop on Computer Supported Activity Coordination (CSAC), 6th International Conference on Enterprise Information Systems, Porto, 13-14 April, 2004, pp 253260. Atkinson C. J., "The Soft Information Systems and Technology Methodology (SISTEM): A Contingency Approach to Integrated Decision Making and Development", pp 71-76, Proceedings of First International Conference on Systems Thinking in Management, 2000. Bach, J. (1996), 'The Challenge of "Good Enough" Software', http://www.di.ufpe.br/~hermano/cursos/calcprog/good-enough-software.htm, ST Labs, (accessed April 7, 2006). Bartkowiak, I. (2005), 'Quality Assurance of Open Source Projects', Seminar - Open Source Software Engineering, Wintersemester 2005. BS 7738-1:1994 (1994), 'Specification for information systems products using SSADM (Structured Systems Analysis and Design Method). Implementation of SSADM version 4', British Standards Institute. Bustard, D. W., He, Z., and Wilkie, F. G. (1999). 'Soft Systems and Use-Case Modelling: Mutually Supportive or Mutually Exclusive?' Proceedings of the Thirty-Second Annual Hawaii international Conference on System Sciences, Volume 3 (January 05 - 08, 1999). HICSS. IEEE Computer Society, Washington, DC, 3055. Checkland, P. (1999) Systems Thinking, Systems Practice. Wiley, UK. Crosby P.B., (1979), Quality is Free: The Art of Making Quality Certain. McGraw-Hill. Duijnhouwer, F. and Widdows, C. (2003), 'Capgemini Expert Letter Open Source Maturity Model', Capgemini, http://www.seriouslyopen.org/nuke/html/modules/Downloads/htmldocs/osmm1.html, (accessed April 7, 2006). Juran, J. (1979), Quality Control Handbook, 3rd edition. McGraw-Hill. International Standards Organisation,(2002), http://www.iso.ch/iso/en/ISOOnline.frontpage, (accessed April 7, 2006). Michlmayr, M., Hunt, F. and Probert, D. (2005) "Quality Practices and Problems in Free Software Projects" in Proceedings of the First International Conference on Open Source Systems. Genova, Italy, 2005, pp 24-28. Mingers, J. (1992), 'SSM and Information Systems: An Overview', Systemist, Vol 14 (3), August 1992. Raymond, E. S. (2000), The Cathedral and the Bazaar, http://www.catb.org/~esr/writings/cathedralbazaar/cathedral-bazaar/, (accessed April 7, 2006). OpenBRR.org (2005), Business Readiness Rating for Open Source, http://www.openbrr.org/docs/BRR_whitepaper_2005RFC1.pdf, (accessed April 7, 2006). Open Source Maturity Model (2003), http://www.seriouslyopen.org/, (accessed April 7, 2006). Vidgen, R., Wood-Harper, T., and Wood, R. (1993). 'A Soft Systems Approach to Information Systems Quality'. Scand. J. Inf. Syst. 5 (Aug. 1993), pp 97-112. Wade, S. and Hopkins, J. (2002), 'A Framework for Incorporating Systems Thinking into Object Oriented Design', Seventh CAiSE/IFIP8.1 International Workshop on Evaluation of Modeling Methods in Systems Analysis and Design (EMMSAD02), Toronto, Canada, 27-28 May, 2002. Wilson, B. (1984): Systems Concepts, Methodologies, and Applications. Wiley, UK.

- 12 -

Tawileh, Ivins and Rana

- 13 -