Introduction
A New Method for Interoperability Test Generation
• Interoperability Testing: verifying that the implementations communicate properly while providing the expected services • Objectives of this study: – Interoperability formal definitions – A method for automatic test generation – Focusing on one-to-one interoperability context
Alexandra DESMOULIN and César VIHO IRISA / Université de Rennes 1
[email protected], http://www.irisa.fr 1
One-to-one interoperability testing architectures Tester (T1)
Test System (TS)
Interoperability testing • Verification of both interaction between the implementations and provided service
Tester (T2)
UT1
UT2
UP1
LT1
LT2
LP1
LP2
UP2
UI1
⇒Service: verification of outputs on upper interfaces ⇒Interaction: verification of both outputs and inputs on lower interfaces
UI2
IUT1
IUT2
LI1 Lower Interface
2
LI2 SUT(System Under Test)
3
4
Input’s observation issues
Global iop criterion iopG
• Output verification: comparison between observations and specification • Input verification:
Two conditions in the criterion: ¾ C1: after a trace of the interaction of the specifications, all outputs observed during the interaction of the implementations must be foreseen in the specification interaction ¾ C2: inputs corresponding to messages sent by one IUT must be effectively received by the other
– Problem: Inputs are not observable – The idea: searching outputs that can be observed if and only if the considered input was actually received. ¾ Method: Calculating causal dependencies between the considered input and the possible outputs to conclude on the execution of the input 9 Algorithm: based on a breadth-first search algorithm 5
6
1
Comparison between global and bilateral iop criteria
Bilateral iop criterion iopB Conditions describing the bilateral iop criterion: ¾ C1a: after a trace of S1 observed during the asynchronous interaction of the implementations, all outputs and quiescence observed in I1 must be foreseen in S1 ¾ C1b: same as C1a in the point of view of I2 / S2
9iopB ≡ iopG : same power of noninteroperability detection 9S1||S2 is calculated in test generation methods derived from global criterion iopG, not in methods using bilateral criterion iopB ⇒The idea: interoperability test generation based on bilateral criterion avoiding state-space explosion
¾ C2:inputs corresponding to messages sent by the other IUT must be effectively received and vice-versa 7
8
iopG-based interoperability test generation method
Interoperability test generation
Iop global test purpose (TP)
S1||S2
• Presented approaches – Classical approach (based on iopG) – New method: bilateral approach based on iopB
Iop test generation SUT(I1||I2)
• Application to a specification describing a connection request protocol
Iop global TC Test execution
Iop Verdict V
9
iopB-based interoperability test generation method S1 S2 S1
Comparing interoperability test generation methods
Iop global TP
Test purpose derivation
Iop unilateral TPS1 Iop unilateral TPS2 Conformance test tool + modifications
U1?CNR
l1!cnr 2
Test execution
l1?ack
Iop Verdict V2 Verdict V’ = V
l2!ack
l2?cnr
1 U1!ACK
Iop unilateral TC2
Iop Verdict V1
0
0
S2
Conformance test tool + modifications
SUT(I1||I2) Iop unilateral TC1 Test execution
10
11
3
U1!NACK
l1?nack
4
l2!nack 1
Specifications S1 and S2 12
2
Applying iopB-based method • Global TP: TP= l2?cnr.U1!ACK • Unilateral TPs: TP1: l1!cnr.U1!ACK TP2: l2?cnr.l2!ack
Applying iopG-based method
• Unilateral test cases Specifications S1 and S2: global test case 0
0 UT1!CNR 1 ?(l1!cnr)
?(l2?cnr) UT1!CNR 6 UT1?NACK
2 ?(l1?nack) ?(l1?ack) 4 3 UT1?ACK(PASS)
?(l2?cnr) 1
UT1!CNR ?(l1!cnr) ?(l2?cnr)
0
3 ?(l2!nack)
0 U?CNR
l?cnr 1
U!ACK
l!ack 5
l!cnr l?ack
3
U!NACK
2 l?nack 4
?(l2!ack)
4
3
?(l1?ack) UT1?ACK, PASS 11 5
?(l2!nack) ?(l1!cnr)
?(l2?cnr) 9 10
?(l2!ack) (PASS)
UT1?NACK
8
UT1!CNR
7
6 ?(l1?nack)
2
TC2
13
Application to another specification l!nack
2
5
TC1
Specification S
1
14
Conclusion • Equivalence between global and bilateral methods based on global iop criterion and bilateral iop criterion • Equivalence confirmed by experimentations • Bilateral method avoids state-space explosion problem • Future work: generalization to a context with N interacting implementations
With iopB-based method : Same TP, TP1 and TP2 TC1: 9 states/17 transitions TC2: 12 states/22 transitions
With classical method: • Interaction S||S: 47.546 states/114.158 transitions • Global TC: 54.456 states/120.443 transitions 15
16
3