n

Formalizing Interoperability for Test Case Generation Purpose

n n

Alexandra DESMOULIN & César VIHO IRISA / Université de Rennes 1 [email protected], http://www.irisa.fr

Classical approach Our approach Comparison

¾ ¾ ¾

Conclusion and future work

n

1

2

Interoperability testing in a context one-to-one

Formal definitions of interoperability testing and test generation n

Specification S1

Specification S2

Conformance testing I1-S1

Conformance testing I2-S2

Implementation Under Test (IUT)

n n

I2 Interoperability Testing I1-I2

I1

n

Interoperability : verification of the communication between the IUTs and of the provided services Formal definitions improves conformance testing Some existing attempts to formalize interoperability but no precise characterization. Interoperability test generation : no method based on formal definitions

3

4

Interoperability testing architecture

Model of IOLTS and notations

Test System (TS)

Tester (T1)

Specifications S1 and S2

Tester (T2)

UT1

0

UT2

UP1

LT1

UP2

LT2

LP1

LP2 UI2

IUT1

U!B

5

l?a

l!b

System S 2

l?b

LI2 SUT(System Under Test)

U!C

l!a

IUT2 Lower Interface

B C

l!c

Upper Interface U

1

UI1

LI1

A

0

U?A

3

1

l?c

a b c Lower Interface l

4 6

1

Global interoperability criteria iopG:

Interoperability definitions nBased on conformance notions: Conformance relation ioco I ioco S =∀σ∈Traces(Δ(S)) ⇒ Out(Δ(I),σ)⊆ Out(Δ(S),σ) nDifferent definitions of interoperability based on the different architectures are possible

iopG = two implementations are considered interoperable iff, after a trace of the asynchronous interaction of the specifications (and in I1||AI2), all outputs and quiescence observed during the (asynchronous) interaction of the implementations must be foreseen in the asynchronous interaction of the specifications. z Formally : iopG(I1, I2)= ∀σ∈Traces(S1||AS2) ⇒Out(I1||AI2, σ)⊆Out (S1||AS2,σ) z

IUT I

S σ

Out(I,σ)

σ

Out(S,σ)

7

8

Bilateral interoperability criteria iopB:

Example

iopB = after a trace of S1 observed during the asynchronous interaction of the implementations, all outputs and quiescence observed in I1 must be foreseen in S1, and the same in the point of view of I2 implementing the specification S2. n Formally : iopB(I1, I2)= ∀σ1∈Traces(Δ(S1)), σ∈Traces(I1||AI2), σ/ΣI1 = σ1 ⇒

0

n

Out

((I1||AI2)/ΣI1, σ)

∀σ2∈Traces(Δ(S2)), σ∈Traces(I1||AI2),

= σ1 ⇒

1

U!B

l?a

U!C

l?b

2 l?c

3

I1

1

U!C

l!a l?b 3

S1

2 l?c

iopB ≡ iopG (same power of noninteroperability detection)

n

S1||AS2 calculated in iopG, not in iopB Interoperability test generation based on iopB whithout the calculation of the interaction

⇒

0

0

U?A 1 U!B l!a

4 l?b

9

n

1

S2

4

I2

U?A U!C

l!c

l!b

Implementations

0

Out ((I1||A I2)/ΣI1, σ) ⊆ Out (Δ(S1),σ1).

Comparison of iopB and iopG

0

l!a

⊆ Out (Δ(S1),σ1) and σ/ΣI1

Specifications

U?A

3

0

l?a

l?a

l!c

l!b 1

1

I4

I3

2 10

Interoperability test generation based on iopG TP

S1||AS2

Interoperability test generation algorithm

SUT(I1||AI2)

TC Test execution

Verdict V 11

12

2

Interoperability test generation based on iopB (S1, S2) S1

TP

n

S2

Test purpose derivation

TPS2

TPS1

n

Conformance test generation algorithm

Conformance test generation algorithm

SUT(I1||AI2)

TC1

Comparison of the two methods

Test execution

n

TC2

n

Test execution

Verdict V1

Test Purpose Derivation : separating in TP events from S1 and S2 to obtain “unilateral” test purposes. No calculation of S1||AS2 with method based on iopB ⇒ no state-space explosion problem. Same verdicts with the two methods Test cases obtained with our method are unbiaised

Verdict V2 Verdict V’ = V 13

14

Exemple of test cases : TP=l1!a.U1!B 0

Conclusion and future work

Specifications

0

n

U?A 1

U!B

l?a

U!C

l!b

l!a l?b

2 l?c

3 0

U1!A

1

TC1 0

TC

S1 4

TPS1=l1!a.U1!B

1

S2

1

n

TPS2=l2?a.l2!b ?(l1?b)

?(l1!a)

3

2

U1?B Pass

0 ?(l2?a) 1 ?(l2!b) Pass

?(l2?a)

?(l1!a) 2

3

?(l2!c)

TC2

?(l1?c) Inc U1!A

l!c

?(l2!b)

?(l2!c) 15

4

?(l1?b)

5

U1?B

Inc

n

Proof of equivalence between global interoperability criterion iopG and bilateral interoperability criterion iopB Proposition of a method to generate interoperability test cases from iopB avoiding state-space explosion problem Future work : ¾ ¾

P

Experimenting the method on real protocols Studying the case of (N>2) implementations

I 16

3