Transcript
Testing Techniques Equivalence class analysis Cause effect graphs All pair testing
Equivalence classes
Equivalence classes
Equivalence classes
Triangle Problem • Read 3 integer values in range [1..200]. • These 3 values represent the length of the sides of a triangle. The programme displays a message which establishes that the triangle is isosceles, equilateral or scalene.
Triangle Problem • we are interested in 4 questions: – – – –
Is it a triangle? Is it an isosceles? Is it a scalene? Is it an equilateral?
• We may define the input test data by defining the equivalence class through “viewing” the 4 output groups: – – – –
input sides
do not form a triangle input sides form an isosceles triangle input sides form a scalene triangle input sides form an equilateral triangle
Valid Classes
Non Valid Classes
Non Valid Classes
Could you find other non valid classes ?
Cause effect graph
Methodology
Triangle example
Triangle Problem
Cause Effect Graph
Decision Table
Test cases
All pairs testing • Based on orthogonal Array • Drastically reduces number of test cases • A combination of worst cases and equivalence class testing • Well supported in the agile methods community • Supported by commercial and noncommercial products
What Does All Pairs Do? • Input is a set of equivalence classes for each variable. • Sometimes the equivalence classes are those used in Boundary Value testing (min, min+, nominal, max-, max) • Output is a set of (partial) test cases that approximate an orthogonal array, plus some pairing information. • Why partial? No expected outputs are provided. natural enough, but a tester still needs to do this step.
All Pairs Assumptions
• Variables have clear equivalence classes. • Variables are independent. • Failures are the result of the interaction of a pair of variable values.
Example • Suppose that we have three parameters. • For each parameter, there are two possible values. – Values are : • A, B for parameter 1. • J, K for parameter 2. • Y, Z for parameter 3.
• Degree of interaction coverage is 2. – We want to cover all potential 2-way interactions among parameter values.
Set of all possible test configurations Three parameters, two values for each.
23
There are = 8 possible test configurations.
A
J
Y
A
J
Z
A
K
Y
A
K
Z
B
J
Y
B
J
Z
B
K
Y
B
K
Z
Set of all possible degree 2 interaction elements There are
(32) × 2
2
= 12 possible interaction elements.
A
J
A
Y
J
Y
A
K
A
Z
J
Z
B
J
B
Y
K
Y
B
K
B
Z
K
Z
• Coverage measure: – Percentage of interaction elements covered.
Test configurations as sets of interactions One test configuration...
… covers 3 possible interaction elements.
A
J
A
J
A
Y
Y J
Y
Interaction test coverage goal A A B B
J K J K
A A B B
Y Z Y Z
J J K K
Y Z Y Z
A
J
Y
A
J
Z
A K Y A K Z B B
J J
Y Z
B K Y B K Z
Goal:
cover all interaction elements…
…using a subset of all test configurations.
Selection of test configurations for coverage of interaction elements Interaction elements
A A B B
J K J K
A A B B
Y Z Y Z
Test configurations
J J K K
Y Z Y Z
Degree 2 coverage: 3 / 12 = 25% Degree 3 coverage: 1 / 8 = 12.5%
A
J
Y
A
J
Z
A K Y A K Z B B
J J
Y Z
B K Y B K Z
Selection of test configurations for coverage of interaction elements Interaction elements
A A B B
J K J K
A A B B
Y Z Y Z
Test configurations
J J K K
Y Z Y Z
Degree 2 coverage: 6 / 12 = 50% Degree 3 coverage: 2 / 8 = 25%
A
J
Y
A
J
Z
A K Y A K Z B B
J J
Y Z
B K Y B K Z
Selection of test configurations for coverage of interaction elements Interaction elements
A A B B
J K J K
A A B B
Y Z Y Z
Test configurations
J J K K
Y Z Y Z
Degree 2 coverage: 9 / 12 = 75% Degree 3 coverage: 3 / 8 = 37.5%
A
J
Y
A
J
Z
A K Y A K Z B B
J J
Y Z
B K Y B K Z
Selection of test configurations for coverage of interaction elements Interaction elements
A A B B
J K J K
A A B B
Y Z Y Z
Test configurations
J J K K
Y Z Y Z
Degree 2 coverage: 12 / 12 = 100% Degree 3 coverage: 4 / 8 = 50%
A
J
Y
A
J
Z
A K Y A K Z B B
J J
Y Z
B K Y B K Z
Choosing the degree of coverage • In one experiment, covering 2 way interactions resulted in the following average code coverage: – 93% block coverage. – 83% decision coverage. – 76% c-use coverage. – 73% p-use coverage. – Source: Cohen, et al, “The combinatorial design approach to automatic test generation”, IEEE Software, Sept. 1996. • Another experience report investigating interactions among 2-4 components: – Dunietz, et al, “Applying design of experiments to software testing”, Proc. Of ICSE ‘97.
Online Mortgage Application (Thanks, Bernie Berger, STAREast)
NY
L
1 fam
A+
Pri
80%
Ye s Yes Ye s
Cust
Ye s Yes
NJ
M
2 fam
A
Va c
90%
No
Bank
No
FL
H
3 fam
A-
Inv
100%
TX
H+ 1
4 fam
B
CA
H+ 2
Coop
