Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Integration Testing
Mohammad Mousavi
Eindhoven University of Technology, The Netherlands

Software Testing, 2011

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Outline
Integration and (De)Composition Levels of Testing Composition MM-Graphs Conclusions

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Waterfall Model
Traditional Method
top-down functional decomposition

Drawbacks
huge gap between spec. and testable product no possibility of interaction on intermediate products

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Waterfall Model
Traditional Method
top-down functional decomposition

Drawbacks
huge gap between spec. and testable product no possibility of interaction on intermediate products

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

New Paradigms
Composition
a simple yet tangible basis add to it gradually

Contemporary Processes
incremental: alleviate work-force bottleneck evolutionary: to meet changing requirements spiral: evolutionary combined with rapid prototyping possibility of go/no-go feedback
Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Eect on Testing
Traditional Paradigm
testing bottleneck in the end complete requirement while testing

Contemporary Processes
changing / evolving requirements mainly: regression / progression testing

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Outline
Integration and (De)Composition Levels of Testing Composition MM-Graphs Conclusions

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Levels of Testing
Unit Testing
individual pieces of functionality: interleaving of program statements single source and sink nodes fairly easy to localize the faults
Unit (Function, Method)

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Levels of Testing

Integration Testing
detailed behavior interleaving of module port events crossing the unit boundaries design-based testing
Integrated Units

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Levels of Testing

System Testing
overall behavior: interleaving of input-output port events requirements-based testing
System

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Outline
Integration and (De)Composition Levels of Testing Composition MM-Graphs Conclusions

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Naive Approach
Big-Bang Integration Testing
compose everything treat the system as a unit

Drawbacks
wait till all modules are in place impossible to (nd and) localize the bugs need to extend the techniques

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Naive Approach

Solution: An Incremental Approach

test sessions: compose a few components each time assume correct components (unit testing done before) test the sub-systems (techniques learned before)

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Incremental Approach

Problems
dependencies beyond sub-system boundaries still need to extend the techniques (e.g., single source and sink assumptions, treat method calls in ow graphs)

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Stubs

Dependencies
cut the dependencies beyond the sub-system replace with an apparently correct piece of software: stub wrap the sub-system in the main test-bed: driver

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Stubs: Example

ValidatePIN(PIN pin): bool; begin if int(PIN) < 4444 then return true else return false end if end

ATM

Validate PIN

Screen Driver

Get PIN

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Stubs and Drivers: Example

Display(MSG msg); begin end main() begin repeat ... ValidatePIN(random); until(sucient) end

ATM

Validate PIN

Screen Driver

Get PIN

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Decomposition

structural decomposition (e.g., aggregation and composition): no information about dependencies functional/behavioral decomposition (e.g., association, uses relation): call-graphs
stub

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Integration: Top-Down

start with main and stubs as units test the logic of main integration: breadth-rst

Integration 2 Integration 1 ATM

Validate PIN

Drawback
many stubs many test sessions
Screen Driver Get PIN

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Integration: Bottom-Up

ATM

Drawbacks
many drivers many test sessions
Screen Driver

Validate PIN

Get PIN

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Integration: Sandwich

Basic Idea
big-bang on sub-trees less stub implementation

Drawback
more dicult to locate the fault

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Integration: Other Variants

pair-wise integration neighborhood integration check the book for details (not much, I am afraid)

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Outline
Integration and (De)Composition Levels of Testing Composition MM-Graphs Conclusions

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Basic Idea

edges: message (and return) entry (and exit) points for modules nodes: paths between message and return

r1 p1

p1 m1 p0 m0 r0

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Structural Testing: Integration Level

main m0

call m0

expand the ow-graph to MM-graph

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Structural Testing: Integration Level

main m0

call m0

call m0

expand the ow-graph to MM-graph,

ret m0

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Structural Testing: Integration Level

main m0

expand the ow-graph to MM-graph, apply the following reductions:

1. a chain of nodes not containing source, sink or call/ret. start/end node 2. a branch not containing source, sink or call/ret.: remove 3. a loop not containing source, sink or call/ret. start/end node
call m0

call m0

ret m0

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Structural Testing: Integration Level

main m0

expand the ow-graph to MM-graph, apply the following reductions:

1. a chain of unimportant nodes start/end node 2. a branch of unimportant nodes: remove 3. a loop of unimportant nodes start/end node
call m0

call m0

ret m0

apply structural techniques on the resulting graph

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Structural Testing: Example

test-session: sandwich on ValidatePin (with GetPin, PrintScreen and ContactBank) driver: calls ValidatePin exercise: generate the MM-Graph for: ValidatePin with GetPin, reduce it.

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Structural Testing: Example

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Outline
Integration and (De)Composition Levels of Testing Composition MM-Graphs Conclusions

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Conclusions

Naive (big-bang) integration: impractical Other integration paradigms:

top-down (many stubs), bottom-up (many drivers), sandwich (dicult to trace), neighborhood(controlled sandwich), pairwise

structural techniques extended to integration testing: MM-graphs and their reductions

Mousavi: Integration Testing

Integration and (De)Composition

Levels of Testing

Composition

MM-Graphs

Conclusions

Guest lectures (June 10)

Bart Knaack (Logica, Senior test manager): Testen in de Praktijk Machiel van der Bijl (Axini): Recent developments in Model-Based Testing

Mousavi: Integration Testing