Chapter 1: Software Development Life Cycles and Testing Chapter 2: Software Testing Overview Chapter 3: Test Requirements Chapter

4: Test Design Techniques Chapter 5: Working with a Test Case Management System Chapter 6: Software Error Chapter 7: Working with a Bug Management System Chapter 8: Reporting Progress: Status Report Chapter 09: Reading Technical Document and Creating Test Requirement Chapter 10: An Introduction to Automated Software Testing Chapter 11: An Introduction to Action-Based Testing Chapter 12: An Introduction to TestArchitect Chapter 13: The Big Picture of an Automation Project Chapter 14: TestArchitect Next Generation Best Practices Chapter 15: A Day in a Life of a New Test Engineer Chapter 16: Essentials Skills for Working Effectively in an Outsourced Team

CHAPTER 1: Software Development Lifecycles (SDLC) and Testing

SDLC and Testing Waterfall Model Spiral Model V-Model Concurrent Model Agile Model Other SDLC Models Testing Phases and Milestones
In this chapter, we want to give an overview of various software development life cycles (SDLC). This is essential to software testing because with each type of SDLC comes with a different style of how software is developed. With each style, it requires different method of communication and documentation. Therefore, it will affect the software testing activities, when what test activities will begin. Another very essential concept in this chapter is how test phases and milestone are defined and enforced. If the phases and milestone not well defined and rigorously enforced, then the SDLC is meaningless. More will be discussed later.

An introduction to Common Software Development Life Cycle (SDLC) practices and how software testing fits in those contexts. Understanding testing phases, milestones, checkpoints in a SDLC and their purposes
2

 Learn about how each SDLC choice affects software testing and its implementation
The objectives in this slide speak for themselves. Not only we will describe each SDLC but we will also stress how software testing fits in each development style. As indicated earlier, we have to clearly understand the phases and milestones concept (through clear and enforceable definitions). From there, we will institute checkpoints to qualify that the software is indeed reaches or passes the predefined phases and/or milestone. Then we will discuss the implication on each SDLC on software testing activities.

CHAPTER 1.1: SDLC and Testing

- Khi pht trin phn mm, ngi ta cn lm theo mt life cycle c th v: - Mt ng dng phn mm rt ln (nhiu source code), hay 1 software application cng c th integrate vi nhiu application khc => vic xy dng v qun l phn mm tr nn rt phc tp - M hnh pht trin phn mm c dng qun l cc ti nguyn v con ngi (mi ngi tham gia xy dng phn mm bit mnh ang u, cn lm g trong qu trnh pht trin phn mm) - Mi loi life cycle c mt c im ring, vic kim th phn mm trong cc life cycle v th cng khc nhau. - cc slide sau, chng ta s tm hiu vic kim th phn mm trong cc cc d n dng life cycles khc nhau th khc nhau nh th no.

How testing is directly affected by the SDLC choice: o Documentation availability to test against
3

o Time to test o Time to automate or produce effective automation o Knowledge and understanding of the application as we plan our tests o Amount of regression testing
Vi mi model khc nhau, vic testing s c thc thi khc nhau:

- Thi gian thc thi vic test khc nhau - Automate vic testing khc nhau - Kin thc cn bit v ng dng m ta cn test khc nhau - Regression and how to do it are different

CHAPTER 1.2: Waterfall Model

- Trong phn ny chng ta s tm hiu s cn thit ca vic kim th phn mm v cch kim th phn mm trong cc d n dng m hnh waterfall

REMINDING: What is waterfall model? The waterfall model is a sequential software development model which no overlapping phases. One phase does not start until the previous phase is completed and documents are stored in document control. How does it make transition between phases? Transition between phases is done by a formal review. When a phase is done, it cannot be modified anymore. What is the purpose of checkpoints? The review is a checkpoint to see that you are on the right track

- Testing is not inherent to every phase of the Waterfall model.

5

- Constant testing from the design, implementation and verification phases is required to validate the phases preceding them. - Testing phase: Only after coding phase, software testing begins. Different testing methods are available to detect the bugs that were committed during the previous phases. A number of testing tools and methods are already available for testing purposes.
Where does testing involve? - Testing phase is related with Coding and Deployment phases. The testing phase starts after code generation phase. As a tester in this phase, you use several testing methods and testing types to test the software and report your working progress to your QA - When a new version is released, the testing phase finds bugs of this version. This task is repeated with all versions until the application comes into deployment phase. - Testing also takes care the application when it comes into the deployment phase by user acceptance testing, testing the product system and maintenance. - We will learn more about these in later slides. - Actually in waterfall there is no feedback that is why it is theory and not done. - Document heavy process. It the SDLC is waterfall style, all/most testing is verification directly from documentation. Commonespecially in companies tied to test director/quality center. - Waterfall is back end testing. Expensive. Change is very risky. test in quality is bad. Build in Quality is good. - WHY IS TESTING NECESSARY IN WATERFALL MODEL?
6

- Cc phase trong waterfall do nhiu ngi m nhn v hot ng gn nh c lp vi nhau - Khi mt phase kt thc, phase sau s ly kt qu ca phase trc lm tip. Nu phase trc lm sai, vic sa cha gn nh l khng th - waterfall, vic testing c thc hin mt phase sau khi code, vic pht hin bug giai on ny l kh tr v v th sa rt tn km - Ngy nay, waterfall khng c p dng rng ri trong vic pht trin phn mm v khng ph hp vi bn cht ca vic xy dng phn mm vn nhiu bug, bug c th xy ra mi ni v cn c test cng sm cng tt. - Waterfall thch hp vi vic sn xut hardware hn do tnh cht ca vic sn xut hardware i hi tnh chnh xc cao v kh sa li

- Ngy nay, khi s dng waterfall, ngi ta ci tin n (cng nh ci tin cc loi m hnh khc) bng cch thm vo s xut hin ca QA. QA s qun l ton b quy trnh pht trin phn mm v mt cht lng, m bo tng phase trong m hnh lm ng chc nng ra nhm t c mc tiu cao nht l sn xut c phn mm ng vi yu cu, cht lng cao tha mn ng o ngi dng. - QA lm vic cht ch vi nhng ngi tham gia trong tng phase. - Ngy nay, ngi ta ci tin s hn ch ca waterfall (thiu s lin lc gia cc phase v t test ti cc phase u tin) c th s dng n hiu qu hn. QA does the constant testing from the design, implementation and verification phases is required to validate the phases preceding them. General opinions: o Advantages: The SDLC appears to be very structured. In theory, it will be easily manage through a lot of documentations and checkpoints. You cannot start the next phase until the current phase is complete. o Disadvantages: This style requires a lot of documentation up front. The more time you do documentation the less time you have for design, code and testing. It is also very critical to changes while we known that software will change, the design will changes, and coding will change as we learn more about the software while we progress through each step of the way.

CHAPTER 1.3: Spiral Model

REMINDING: - The spiral model is a software development process combining elements of both design and prototyping-in-stages, in an effort to combine advantages of top-down and bottom-up concepts. - Each cycle is completed by a review. This review covers all products developed during the previous cycle, including the plans for the next cycle and the resources required to carry them out. The reviews major objective is to ensure that all concerned parties are mutually committed to the approach for the next phase.

Notes: This model of development combines the features of the prototyping model and the waterfall model. The spiral model is intended for large, expensive and complicated projects. A typical cycle of the spiral Each cycle of the spiral begins with the identification of: 1. the objectives of the portion of the product being elaborated (performance, functionality, ability to accommodate change, etc.); 2. the alternative means of implementing this portion of the product (design A, design B, reuse, buy, etc.); and 3. the constraints imposed on the application of the alternatives (cost, schedule, interface, etc.). Each cycle is completed by a review. This review covers all products developed during the previous cycle, including the plans for the next cycle and the resources required to carry them out. The reviews major objective is to ensure that all concerned parties are mutually committed to the approach for the next phase The steps in the spiral model can be generalized as follows: 1. The new system requirements are defined in as much detail as possible. This usually involves interviewing a number of users representing all the external or internal users and other aspects of the existing system. 2. A preliminary design is created for the new system. 3. A first prototype of the new system is constructed from the preliminary design. This is usually a scaled-down system, and represents an approximation of the characteristics of the final product. 4. A second prototype is evolved by a fourfold procedure:

10

- evaluating the first prototype in terms of its strengths, weaknesses, and risks; - defining the requirements of the second prototype; - planning and designing the second prototype; - constructing and testing the second prototype. Advantages: The spiral model promotes quality assurance through prototyping at each stage in systems development. This is a improved version of waterfall that allows iterations of smaller phases. Disadvantages: Documentation is still heavily required in this style.

Planned and structured releases Usually has documentation to test against Each spiral iteration can be thought of as a mini-waterfall; there are defined testing phases Previous releases must be regression tested
Notice of the point that testing phases are defined. This is essential to testing time as indicated earlier. Also notice that as we progress through test testing phase, not only that we run new tests but we also have to rerun old tests. Rerun old tests by definition is regression testing.

11

CHAPTER 1.4: V-Model

REMINDING: - V-model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The V-Model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing. - The V-model deploys a well-structured method in which each phase can be implemented by the detailed documentation of the previous phase. Testing activities like test designing start at the beginning of the project well before coding and therefore saves a huge amount of the project time. 12

Note: - While this style is another improved version of waterfall, it brings in the clarity of Verification and Validation, both are quality engineering activities including testing.

Verification: Checks that a deliverable is complete (Contains all requires information, follows standards; verify a procedure). Validation: Checks that the deliverables satisfy requirements specified in the previous stage or an earlier stage, and that the business case is met (Validate a function or requirement). Testing: Ensures that the specification is properly assembled and implemented (Test to see if it works). In testing, these are important changes in software development from the V-model: Write the test cases at requirements review. Unit testing
Khc nhau gia verification and validation: - VD: Khi t hng, nhn hng s kim: - ng hng khng (verification) the right product is built - Hng c c lm ng cch hay khng (validation) the (right) feature is working correctly or not (finding bug) I. Verification Phase 1. Requirements analysis - In the Requirements analysis phase, the requirements of the proposed system are collected by analyzing the needs of the user(s). This phase is concerned about establishing what the ideal system has to perform. However it does not determine how the
13

software will be designed or built. Usually, the users are interviewed and a document called the user requirements document is generated. - The user requirements document will typically describe the systems functional, physical, interface, performance, data, security requirements etc as expected by the user. It is one which the business analysts use to communicate their understanding of the system back to the users. The users carefully review this document as this document would serve as the guideline for the system designers in the system design phase. The user acceptance tests are designed in this phase. See also Functional requirements, and Non-functional requirements 2. System Design - Systems design is the phase where system engineers analyze and understand the business of the proposed system by studying the user requirements document. They figure out possibilities and techniques by which the user requirements can be implemented. If any of the requirements are not feasible, the user is informed of the issue. A resolution is found and the user requirement document is edited accordingly. - The software specification document which serves as a blueprint for the development phase is generated. This document contains the general system organization, menu structures, data structures etc. It may also hold example business scenarios, sample windows, reports for the better understanding. Other technical documentation like entity diagrams, data dictionary will also be produced in this phase. The documents for system testing is prepared in this phase. 3. Architecture Design - The phase of the design of computer architecture and software architecture can also be referred to as high-level design. The baseline in selecting the architecture is that it should realize all
14

which typically consists of the list of modules, brief functionality of each module, their interface relationships, dependencies, database tables, architecture diagrams, technology details etc. The integration testing design is carried out in this phase. 4. Module Design - The module design phase can also be referred to as low-level design. The designed system is broken up in to smaller units or modules and each of them is explained so that the programmer can start coding directly. The low level design document or program specifications will contain a detailed functional logic of the module, in pseudocode - database tables, with all elements, including their type and size - all interface details with complete API references- all dependency issues- error message listingscomplete input and outputs for a module. The unit test design is developed in this stage. II. Validation Phases 1. Unit Testing - In the V-model of software development, unit testing implies the first stage of dynamic testing process. According to software development expert Barry Boehm, a fault discovered and corrected in the unit testing phase is more than a hundred times cheaper than if it is done after delivery to the customer. - It involves analysis of the written code with the intention of eliminating errors. It also verifies that the codes are efficient and adheres to the adopted coding standards. Testing is usually white box. It is done using the Unit test design prepared during the module design phase. This may be carried out by software testers, software developers or both. 2. Integration Testing - In integration testing the separate modules will be tested together to expose faults in the interfaces and in the interaction between
15

integrated components. Testing is usually black box as the code is not directly checked for errors. It is done using the integration test design prepared during the architecture design phase. Integration testing is generally conducted by software testers. 3. System Testing - System testing will compare the system specifications against the actual system. The system test design is derived from the system design documents and is used in this phase. Sometimes system testing is automated using testing tools. Once all the modules are integrated several errors may arise. Testing done at this stage is called system testing. 4. Final Implementation in Production

CHAPTER 1.5: Concurrent Model

16

REMINDING: - The concurrent development model is also called as concurrent engineering. - Concurrent process model can be represented schematically as a series of major technical activities, tasks and their associates states - Concurrent process model defines a series of events that will trigger transactions from one state to another for each of the activities in s/w engineering. Note: This model or style is introduced for completeness of the overall SDLC concept. Many software development teams are, nowadays, move to Agile as an improvement of the concurrent model. There is no need to worry too about this model/style.

Planning is concurrent to design; design is concurrent to developmenteverything is happening at the same time. The whole project is not well planned or well structured. Planning, design and development are most dynamic. Product is in constant change. Very difficult to test; impossible to effectively plan testing project. Often no documentation to test against. Testing is ad hoc. Coverage usually cannot be measured. Structured regression testing is impossible. Bugs will be missed because of so much change and so little planning. Risk analysis and reporting is crucial.
Note:

17

Due to the dynamic nature of how the software is defined, designed and developed, it becomes a challenge for testing to effectively involve during the development cycle. In this style, testing is often done at the backend as system testing.

CHAPTER 1.6: Agile Model

Agile is a newer SDLC concept based on high customer satisfaction, smaller, quicker and more frequent code release form small, focused teams. Project scope can change often. Daily, frequent and thorough communication is required between the team members and also between the development team and the business team.

18

Waterfall is necessary for get it right the first time but upfront heavy planning and design. It is the opposite of where the dev world is heading with Agile, in a way, doc at the end. Waterfall is predictive, agile is adaptive. Key Principles of Agile - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan Top priority is customer satisfaction - Welcome changing requirements, even late in the project - Deliver working software people should work together daily - Communicate face-to-face, whenever possible - Working software is the primary measure of progress - Teams constantly reflect on how to improve their processes and methods.

Delivery cycles are short. Development is very focused. Diagrams, use cases, user stories, index cards, or discussions of functionality serve as documentation to test against. Agile projects usually have more unit testing by developers. Dynamic nature of development needs structured regression testing. Testing is often ad hoc, but focused.

19

 Dynamic style or projects include much change but the change is discussed and side-effects noted. Note:
Do not confused developer-testing or unit testing as system testing. Developer-testing or unit testing is aimed to improve the quality of the code (at the coding level) delivered to QA. However, more testing must be done to validate the individual functionality as well as how the software works as a whole. Test must be designed, often before the working functionalities are available for testing. Testing cycles are short. While testing is exploratory or ad-hoc by nature, combining flexibility with structured testing and automation is essential. (Automated) Regression testing is another essential element of testing in Agile style. Automated regression testing is implemented at two levels: Unit testing (designed and implemented by developers, run and rerun by developers as well as testers) and system testing (designed, implemented, run and rerun by testers).

CHAPTER 1.7: Other SDLC Models - Rapid Application Development (RAD) Model - Test First - Rational Unified Process (RUP) - eXtreme Programming (XP) - the list goes on

20

Reality check: - As we are wrapping up the introduction of the SDLC models, it is essential to know that many software development organizations, they might say that the are applying certain model, in reality, they might be loosely apply the concept of that model. The most important fact is to determine how much time we have for testing, and when can we start doing what testing activities. That brings us to the next topic which is the most important topic as it affect testing time in each SLDC model.

CHAPTER 1.8: Testing Phases and Milestones - Development Phase: A time block within SDLC. There are a number of activities that have to be done in this time block. - Examples of some Test Phases: Unit Test, Integration Test, System Test, User Acceptance Test, Release Test and Maintenance Test. - Development Milestone: A significant event in the software developing process in which, the software moves from one phase to another. The milestone let us know where the application is in SDLC - Criteria: Built or created from the standard of the project.
REMINDING: Cc giai on v ct mc trong qu trnh pht trin phn mm
21

- VD v 1 phase: Qu trnh hc i hc l 4 nm (1 SDLC), th nm u tin s l 1st phraseTrong nm hc u tin bn s tri qua nhiu hot ng v d nh nghe ging bi, lm bi tp, nghin cu c th thi k thi kt thc nm hc th nht. Ci mc chuyn t nm th nht sang nm th 2 c xem nh 1 milestone. - Milestone gip chng ta qun l cc phases. - pass c 1 milestone, phn mm phi t mt chun no , c gi l criteria (Cng nh mun kt thc nm nht, pass c qua nm th 2, sinh vin phi c trn 5 tt c cc mn thi chng hn).

Examples Product Design and Specification Complete First Functionality Pre-Alpha or Alpha Candidate Alpha Pre-Beta or Beta Candidate Beta User Interface Freeze Pre-Final or Golden Master Candidate (GMC) Final Test Release or Golden Master

Qu trnh xy dng 1 ng dng phn mm thng i qua cc milestones nh trn Note: To testing the concept of Pre-<milestone> is very important. During this milestone, testing is done to qualify the software is indeed meets the requirements of the milestone. This is purely a project management activities, of quality activities.
22

This also stresses that testing for a particular milestone does not start until the software has reached the milestone. This is helpful to unsure that milestone testing does not cut into the software (quality) testing time.

Vi mi milestones trong development phase, c cc testing milestones v testing phase tng ng Chng ta thng test t c cc milestone, cc testing tasks thng c xc nh da vo cc tiu chun ca milestones (milestone criteria) Milestone gip chng ta qun l cc phase (VD: pass qua phase mi l integration, milestone unit test phi c lm xong) --------------------------------------------------------------------------------------Cc cng ty khc nhau khc nhau c th gi tn cc phases test khc nhau, VD:
23

Pre-Alpha Alpha Beta GM/Release Hoc l Unit Integration System User Acceptance --------------------------------------------------------------------------------------- Unit test c thc hin bi developer (cc bug tm c trong qu trnh unit test l private bug (developer tests his own bugs)) Integration: vic test c thc hin bng cch s dng/test phi hp nhiu function vi nhau/cng lc System test: test at run time, n khc integration test ch vic system test lc ny c thc hin trn 1 complete environment User acceptance test: validation, t verify --------------------------------------------------------------------------------------Most of our testing and automation happens during alpha/beta on Integration/System Test.

Example Tasks and Activities for the Test Team: Test Plan reviewed Test Plan finalized Requirements reviewed and base-lined Test system approved and built Bug database opened and available for bug entry
Mt v d v testing milestones criteria

24

In this examples, the stated criteria must be met in order to the software to be declared that it reaches the stated milestone.

The Test Plan covering all test phases: Source Level Testing, Application Testing, and Integration Testing are complete and approved. Developers testing sign-off document is complete Approved Unit Test Checklists are available for review Test Design/ Cases are complete and approved Build acceptance test is executed successfully Regression test has been defined Automation test has been defined (if applicable) The hardware and software necessary for the tests, as identified in the Test Plan, are available and ready. All test results have been documented All identified application testing has been executed Regression test has been executed Automation test has been executed (if applicable) All software fixes identified for resolution in this test phase are fixed, retested and closed Software is baselined Test report is completed All P1&P2 defects have been resolved and closed
25

More example: - Here, all the stated activities and artifacts must be complete in order for the software be qualified entering the stated phase.

Depending on the type of application to be built and or customers requirement, we choose a suitable SDLC model. There are several phases in the SDLC. In each phase, there are milestones and criteria for each milestone. The criteria, milestones and phases of the project help the product to be developed correctly, timely and effectively. Depending on the SDLC, the testing and its implementation is affected, usually in time, knowledge about the application, documentation and amount of regression test done.

26

 Testing Group Objectives of Testing Test Planning Framework Test Coverage Manual Testing vs. Automated Testing Key Lessons-Learned

An introduction to the concept of quality assurance (QA), quality control (QC), and software testing The roles of software testing groups Objectives of testing and some testing issues An introduction to Manual Testing and Automated Testing An introduction to LogiGear Software Test Planning Framework and possible test strategies, approaches, methods, types and techniques used in various software application development phases, for specific quality objectives
27

CHAPTER 2.1: Testing Group

Another key here for our type work is to mention that what we do is QC based on requirements/specs and testing/bug finding. We dont do QA Testing - Kim li phn mm: L qu trnh chy chng trnh v thc thi cc chc nng nhm mc ch tm li ca chng trnh. Quality Assurance Bo m cht lng phn mm: L qu trnh bo m cht lng cho ton b nhng sn phm v nhng quy trnh thc hin phn mm, bo m tt c phi tun theo nhng chun mc yu cu nht nh. Quality Control Kim tra cht lng phn mm: L qu trnh hot ng c thit k nhm mc ch nh gi cht lng sn phm phn mm.
28

Phn bit Testing, QC, QA: Testing: cha bit kt qu ca TCs, test by experiment , khng c g lm sample v chun mc ng v sai QC: Validation, c 1 sample so snh th no l ng, th no l sai (thng l c document so snh), validate against requirement, bit trc kt qu ca TCs ri, lm verify li (regression) QA: lm vic vi process, make the rule how to test, how to develop the software, thng thng project manager lm cng vic QA, hoc 1 nhm ngi lm cng vic QA Vic QA, QC lm cung cp thng tin v product QA: l ngi ra quyt nh/chu trch nhim v vic bug c c sa hay khng

Responsibility ca testing group l tm kim cung cp thng tin v quality ca product

29

Trong : Tester/QC thng lm cng vic Finding and reporting bugs and Identifying weak areas of the program Khi tm c bug, Testing thng Explaining findings in ways that help - Engineering solve or fix problem: Gii thch thng tin chi tit v bug developer c th gii quyt vn - Trong trng hp vn tm c khng th/khng c gii quyt, testing group c nhim v gii thch v cc cch phng trnh cng nh gii quyt tnh hung khi bug xy ra (troubleshooting) cho cc bn lin quan, c th l customer service staff h gip khch hng khi khch hng gp phi cc vn cha c gii quyt - Testing Group s chu trch nhim gii thch v bug, tm nh hng, mc nghim trng gip b phn qun l d n ra nhng quyt nh v bug (c nn sa hay khng, sa nh th no) QA thng lm cng vic Identifying high risk areas of a project

30

Nhim v chnh ca tester l tm bug Find problems: Trong phn mm, c 2 dng vn thng gp, mt l bugs, gy li cho phn mm (phn mm hot ng sai) v hai l design issues. Design issues khng lm cho phn mm hot ng sai nhng lm cho ngi dng cm thy bt tin. L mt tester, bn khng ch tm bugs m cn phi tm cc design issues. Communicate Problems: Khi tm c bugs hay design issues, tester c nhim v bo co vn mnh tm c. Trong qu trnh test phn mm, tester cng phi bo co tc /tnh trng ca qu trnh kim th cng nh cht lng ca phn mm. i vi nhng tester lu nm hoc l ngi gim st d n test (thng l test lead), h chu trch nhim ln k hoch/thi gian test c th cng nh lm cc cng vic qun l d n khc nh c lng khi lng cng vic, ti nguyn, thi gian v chi ph

CHAPTER 2.2: Objectives of Testing The programs specification: This program is designed to add two numbers, which you will enter Each number should be one or two digits
There are several points of this example that we want to hit: 1. Sometimes, even with specification, often, the documentation does not cover all angles. Therefore, as part of testing, we will discover more issues through exploring with the application under testing (AUT)
31

2. The massive volume of possible test cases, even with a very simple application. The reason we use this very simple example is to prove a point that testing can be very complex even with the simplest AUT. 3. While this chapter is not about test design technique, we will introduce the boundary condition analysis and equivalent class partitioning concept. 4. Most importantly, since we cannot complete testing the AUT due the too many possible test cases, the objective of testing is not to verify. The objective is to find problems. In turn, the discovered problems can be fixed, hence, the product quality will improve.

32

 design test case, chng ta tm kim cc boundary ca function. Boundary cn tm kim khng ch l ca input m cn l ca output. We always start with a few simple tests which we know clearly what the expected results are. We normally call these tests positive tests (as suppose to negative tests, design to break the software). We do this simple exercise to make sure that the AUT is stable enough and can handle a few simple tests. We also do this to learn more about the AUT through exploration.

33

 Note that since the specification does not say anything about disallowing negative values, we then assume that 2-digit negative values are possible. As you can clearly see now, there are 39,601 valid test cases and infinite number of invalid test cases to be executed.

34

 This on the next four slides are dedicated to teach Boundary Analysis and Equivalent Class Partitioning. We will walkthrough this test design technique. Then later, we be will come back to the objective of software testing point. In this slide, we go over the definition of equivalent class and boundaries. The objective of this technique is to help us focus on fewer powerful test cases in the interest of time.

Key concept: First, we identify the test cases that can be equivalent. In this example, we see the following classes: 1. The Value class. That is the actual value from -99 to 99 are valid values.
35

2. The number of characters. That is 3 since we assume that negative numbers can be valid. An example of a negative twodigit number is -99. By definition, two tests belong to the same equivalence class if the expected result of each is the same. Now, we will focus on the boundaries. For each boundary, we will typically have 3 test cases. 1. A case right on the boundary. 2. Boundary -1 3. Boundary +1 For valid values of this example, we have 2 boundaries, therefore, 6 test cases: -99, 99, -100, -98, 98, 100. For valid number of characters, we also have 2 boundaries. However, we have only 5 test cases because 2 is shared for both boundaries 1, 3, 0, 2, and 4. Now, we will have one invalid test case on each side of the boundary (that is not part of the boundary cases). 1. For value, we have two cases, one on each side. 2. For number of characters, we only have 1 case of the upper boundary, but nothing on the lower boundary (since you cannot enter less than 0 character).

36

This slide generalize and recap the discussion we had on the previous sllide

This is the step-by-step procedure to apply in this test design technique.

37

Here is another example to illustrate how this test design works.

Yes, another example.

38

Mc tiu ca Testing khng phi l xc minh chng trnh lm vic tt, bi v: Nu bn khng th kim tra ht tt c mi ngc ngch ca mt phn mm th bn khng th bit chc n c tht s hot ng tt hay khng. V trn thc t, bn khng th kim tra ht tng chc nng vi tng gi tr d liu t - n + c. L mt tester, sn phm u ra ca bn l cc thng bo Bug (Bug Report), hay ni cch khc, nhim v ca tester l tm Bug ch khng phi l chng minh chng trnh khng c Bug. Nu bn hy vng chng trnh s hot ng tt, bn s c khuynh hng b qua cc li chng trnh hn l nu bn c mun ph cho n hot ng khng tt.

39

Mc tiu ca Testing l tm Bug Tm li chng trnh chnh l ct li cng vic ca bn. L mt tester, bn s mun tm c cng nhiu li cng tt, v chnh l sn phm ca bn. V nhng li cng nghim trng s cng tt. The objective of testing is to find bugs. Note: Now, we are done with the test design technique. We are back reinforcing the philosophy: The objective of testing is to find bugs.

40

Mc tiu ca vic tm Bug l chng c sa cha (fixed) Mc tiu chnh l nng cao cht lng phn mm Ngi tester gii nht khng phi l ngi tm c nhiu Bug nht hay ngi gy rc ri cho nhiu lp trnh vin nht. Ngi tester gii nht l ngi c nhiu Bug c fixed nht.

41

CHAPTER 2.3: Test Planning Framework

Glass-Box Testing: Khi test theo phng thc hp knh, bn s dng kin thc ca mnh v source code to ra cc trng hp test (Test Case). Phng php ny thng dng trong Unit Test, khi cc lp trnh vin debug source code m h vit ra.

42

Black-Box Testing: Khi test theo phng thc hp en, bn chy chng trnh v thc thi cc chc nng m khng quan tm n source code bn di. Tng t nh vi mt ci hp mu en, bn a vo input v ly ra output, khng quan tm chuyn g din ra bn trong hp. y l phng php tip cn theo quan im ca khch hng ch khng theo quan im ca lp trnh vin.

43

Gray-Box Testing Test theo phng thc hp xm cng bao gm input/output nh blackbox testing, nhng y cc Test Case c thit k s dng nhng cng c test v kin thc v chng trnh cng nh mi trng tng tc ca n, nng cao hiu qu test, hiu qu tm v phn tch Bug.

44

Khi tester cng hiu nhiu v k thut xy dng nn phn mm, vic test cng hiu qu (do c th gray box tt). Tuy nhin, nu mang qu nhiu technical mindset khi test th tester s khng th suy ngh c nh 1 user bnh thng v do s b qua nhiu test case m 1 user bnh thng (khng c kin thc v phn mm) hay thc hin => b st li => vic testing s do bt hiu qu

45

 thc hin vic testing cho mt d n phn mm, ngi ta thng theo 1 dng testing planing framework nh trn.

46

load, stress, performance Test: mc ti, mc chu ng, run hiu qu - Volume l mt loi load Test, nhng kim tra v khi lng

DAST: Diagnostic Approach to Software Testing The difference between paradigm and approach?
47

Test type: The way we execute Test case, a group of test case together to reach an objective

48

CLI: Command-line Interface GUI: Graphic User Interface API: Application Programming Interface

Vn ny c ni 1 ln trong phn Development Phase

49

Test methods: How to write test case, develop TCs, create TCs

50

51

CHAPTER 2.4: Test Coverage

Chng ta khng th no test ht c 1 chng trnh, v: - Too many inputs (qu nhiu trng hp cho cc d liu u vo) - Limited time (khng thi gian) - Too many combination (qu nhiu s kt hp kh d) - Too many paths (qu nhiu ng i)

52

53

54

55

CHAPTER 2.5: Manual Testing vs. Automated Testing

56

CHAPTER 2.6: Key Lessons-Learned The Program Doesnt Work. Your task is to find errors. Be Methodical Complete Testing is Impossible Use Powerful Representatives of Tests Communication Must be Effective Change is Normal Domain knowledge Testing knowledge IT

Khng c chng trnh no hot ng tt hon ton: Tt c cc chng trnh u c li Bt c mt thay i no vi chng trnh u c th gy ra li, v bt c mt kha cnh no ca chng trnh cng c th b ph v.
57

Cng vic ca bn KHNG phi l xc minh chng trnh chy tt. Vic ca bn l TM LI ca n.

Khng bao gi c th test ht tt c cc trng hp ABCDEFGHIXEXIT

58

S lng tp hp ng dn gn nh l v hn. V d: i t A n Z, bn c th i t A qua B n Z hoc t A qua C qua D n Z, v.v S lng tp hp d liu nhp vo cng gn nh v hn. V d: test php tnh cng hai s hng, bn c th nhp tng gi tr d liu t - n + cho s hng th nht, t - n + cho s hng th hai, v tng s tp hp kt hp hai gi tr ny, nh 1+1, 1+2, 1+3, 2+1, 2+2, 2+3 l v hn. Bn khng bao gi c th test ht tt c cc trng hp. V vy, nhim v ca bn l tm Bug, nhng khng phi tm tt c cc Bug Bn s mun: Tm cng nhiu Bug cng tt Tm nhng Bug nghim trng nht Tm Bug cng sm cng tt ( lp trnh vin c thi gian fix Bug)

59

Vic thng bo li cn phi hiu qu Thng bo li (Bug Report) ca bn nhm mc ch thng bo vi mi ngi v nhng li chng trnh hin c. Bug Report ca bn c th nh hng n k hoch d n v doanh thu / chi ph ca cng ty Bug Report ca bn cng r rng th cng ty cng c th c nhng quyt nh kinh doanh hp l. V d: Nu gn n ngy pht hnh (release) m chng trnh vn cn c qu nhiu li nghim trng, c kh nng cng ty s quyt nh di ngy release li sa cc li ny, v nh th s nh hng n k hoch v chi ph ca cng ty. Hoc nu bn tm ra nhiu li nghim trng nhng khng thng bo r rng, cng ty s khng nh gi c mc nghim trng ca cc li chng trnh vo thi im v tip tc release mt sn phm khng tt, lm nh hng n uy tn cng nh doanh thu ca ton cng ty.

60

61

 Products Document What is Test Requirement (TR)? Test Requirement Essentials Requirement Attributes

An introduction to requirement, specification and design An introduction to test requirements and how to use test requirement to implement test case

62

CHAPTER 3.1: Products Document

A product requirements document (PRD[1]) is a document written by a company that defines a product they are making, or the requirements for one or more new features for an existing product. It is designed to allow people within a company to understand what a product should do and how it should work (Refer: http://en.wikipedia.org/wiki/Product_requirements_document)

63

CHAPTER 3.2: What is Test Requirement (TR)?

TR example: Verify user login with valid account FR example: Add Call Profiles Only XX users can add new calls. Initiation After selecting an active customer, the user selects a menu option to add a new call for that customer. Call Data Calls have the following associated information. Each customer can have multiple calls associated:
64

Date, Type... Processing The user is given a drop-down of existing Contacts for the Customer. New calls are transmitted to the host at the next connection. The addition is logged.

Look and feel example: username, password, login controls must be located nearly so that user fill information easily Boundary example: Verify user can not fill more than 20 characters to username Negative example: Verify user can not login with inexistent account

CHAPTER 3.3: Test Requirement Essentials

65

TR ID: TR-007 Test Requirement: Verify that Load dialog is appeared when user clicks Load button Traceability: Introduction to XX 1-15 Test case ID: TC_XX_001

CHAPTER 3.4: Requirement Attributes

Test requirement Attributes: - Stated positively (not absence of a negative) - Terms: known meaning, identifiable, consistent - Assumptions documented
66

- Stand on own vagueness/ambiguities

without

internal

contradictions

or

- Consistent with (suitable) objectives - Identifies major functions, limits - Alternative consequences defined [Magic words--must, shall, will (no TBD)]

67