Projective Simple Algebraic Groups are able to encode AI Planning Consistency (encoding means to provide a compact representation) Olivier Bartheye, CREC, Military Academy of St-Cyr, Guer, France.

Formal Modelling and Consistency of Apache Wave Platform Evolutions and Waves, Karsten Gabriel, Technical University of Berlin, Berlin, Germany 

Generating and executing scenarios for a mobile robot Philippe Morignot, CEA, LIST, Saclay, France

Implementing a Conversational Robotic Agent? Jim Cunningham, Department of Computing, Imperial College, London  First Steps towards the Modelling of Peace through a Multi-Agent approach, Jean Erceau, Laurent Chaudron, ONERA, DCSD, Salon-de-Provence, France 

Optimal Control of Discrete Processes and Dynamic Games with Discounted Costs Stefan Pickl, Universität der Bundeswehr, München, Germany

An Attempt at a Formal Specification of Cognitive Trust Jacques Calmet, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Projective Simple Algebraic Groups are able to encode AI Planning Consistency (encoding means to provide a compact representation) Olivier BARTHEYE CREC St-Cyr F - 56381 GUER Cedex, FRANCE [email protected] 1.

Introduction

First of all, the so-called the AI Planning problem is presented; among famous intractable computer science conjectures which are very often faced with decidability issues, one can cite the AI Planning problem. From the early days of Artificial Intelligence, more or less convincing tractable solutions were sought for this problem which is the expression of the generic problem “action governed by coherence” and AI Planning systems are some concrete representations of these solutions. The historic definition of the AI Planning calculus consists to find out a correct sequence of actions to apply in this world in order to transform the initial state Σi into a state Σf compatible with the goal to achieve which can be taken to be the final state. According to current topics in Computer Science, this implies that the changing of state must be manageable by means of machine statements; correctness means that the boundary set y between the class of “good” solutions, models noted T (true), and the class of “bad” solutions, counter-models noted F (false) identified by the consistency axiom: T  F = y, must be clearly defined in order to have ultimately a decision procedure able to separate the desired solutions from the unwanted ones. The author claims that a convenient solution holds for the AI Planning problem provided that a suitable encoding and a suitable decoding for consistency holds (i.e., a suitable folding operator and a suitable unfolding operator holds); a suitable encoding means a compact representation which cannot be inspired from existing discrete representations [1, 3, 4]. That is, existing AI Planning solutions unfold the AI Planning arrow f: Σi e Σf without any boundary; the answer should be discussed in terms of topology: for sake of completeness and correctness, the topology of these systems is Hausdorff, i.e. discrete and separable, i.e. infinite action loops are perfectly valid. A topological space X is Hausdorff if for every x, y ∈ X, x Ÿ y, there are open neighbourhoods 2x } x;2y } y so that 2x  2y = y. A compact encoding means that connectedness “@ and disconnectedness “Ÿ@ are both valid, i.e. components are inseparable; Σi and Σf hold in the same unified algebra, noted 1, but cannot be connected due to a bitopological structure, a topology τ0 starting from Σi and a topology τ1 starting from Σf. τ1 corresponds at the same time to another topology arriving at the initial state Σi and however distinct from τ0. The first part of this paper is to illustrate that the initial definitions used in [1] and formal models proposed in [2, 3, 4, 5] are good starting points for our investigations. The second part of this paper is to explain what means a projective simple algebraic group. In particular, one must definitely forget affine representations in which the consistency is definable according to the “everywhere valid” consistency axiom: T  F = y, and one must accept the concept of projective simple consistency. Infinite sequences — infinite models Tˆ are classified as unwanted “bad” solutions — counter-models F. Consequently, the planning calculus has to deal with an “anti-truth table arrow” Tˆ → F; that means that the consistent subspace in the neighbourhood of infinity ∞ is homomorphic to a torus

This torus is assumed to be inconsistent according to standard “affine” discrete decision procedures. One should propose that way what are the algebraic structures able to encode such a projective topology in order to define an inference procedure based on the algebraic properties characterizing these structures. 2.

Basic Definitions and Discrete Algebraic Models

Definition : The AI Planning Calculus is a process used in Artificial Intelligence which is intended given  a world description, provided by a formal language,  a goal to achieve, (the problem to solve),  a set of actions which can be brought into play in this world,

1

to find out a finite sequence of actions to apply in this world in order to transform the initial state into a state compatible with the goal to achieve. A suitable requirement for sake of tractability is to bound the size of any action sequence: “affine” representations as inference procedures defined on planning bi-categories [2, 3] or linear deductive systems [4, 5] are not convenient since they naturally can compute infinite correct loops. A very rustic representation of action, defined in the early days of Artificial Intelligence concerns the so-called Strips model [1] for formal actions. In this model, each action handles elementary positive literals p(_x i ,..., _x n ) where p is a predicate name and _x i ,..., _x n are quantified variables whose possible instances are taken in a completion process according to a finite domain of objects D. Definition (Strips model) [1]: A Strips formal action α in AI planning is a triple α = (Preα ,Addα ,Delα) where Preα ,Addα ,Delα are conjunctions of quantified positive literals with quantified variables _x1, _x2, _x3 of the form at(_x1, _x2) ∧ free(_x3); Pre α stands for the “requirements” of α whereas AddαandDelα , the “add-list” and the “delete-list” of α stand for the “effects” of α with AddαDelα = y . Example (Gripper, the robot transportation of balls): The robot robby moves from one room to another and uses its grippers to pick and drop balls. The action system 5 consists of 3 actions with quantified variables, two conjugate actions pick/drop plus a complementary one move. pick(_x1:ball, _x2:room, _x3:gripper)

Pre: {at_robby(_x2) ∧ at(_x1, _x2) ∧ free(_x3)} Add: {carry(_x1, _ x3)}

drop(_x4:ball, _x5:room, _x6:gripper)

Pre: {at_robby(_x5) ∧ carry(_x4, _x6)} Add: {at(_x4, _x5) ∧ free(_x6)}

move(_x7:room, _x8:room)

Del: {at(_x1, _x2) ∧ free(_x3)}

Del: {carry(_x4, _x6)}

Pre: {at_robby(_x7) Add: {at_robby(_x8)}

Del: { at_robby(_x8)}

One can define an inference procedure provided that “action effects” Addα and Delα are separated (models T and counter-models F are separated) and that “requirements” Preα are taken to be possible codomains of “actions as arrows”. Relevant algebraic properties arise from a complete construction: the imposed bijection from the before formulas to the after formulas [2]. For each formula, the formula activity is ensured by a status function whose domain is the set of formulas and whose co-domain is the 2-element set {0/inactive, 1/active}. One can define a mapping from its activity in the initial state Σi to its activity in the final state Σf; the planning problem is defined this way as a collection of bijective activity mappings. One obtains a symmetric Strips action model by completing the left hand side according to a second component: the virtual preconditions Virt;

Ĥ An action is a 4-box in which the rising edges i and the falling edges j are synchronously associated inside an operator by an orthogonal crossed product “*@ according to the differential “+ AddΨ − DelΨ”. One can assume this property is to be locally relaxed and the inner consistency of the whole plan P is to be computed according this binary operator. Definition (bi-category) : A Bi-category B consists of a set ¨B¨= {P, Q, …}, the objects of B called 0-cells; for each pair P, Q ∈ ¨B¨2, a category B (P, Q) whose objects are morphisms f : P → Q with fixed source and target (the 1-cells of the bi-category B) and whose arrows are “morphisms between morphisms with fixed source and target” θ : f → g, θ : h → i (the 2-cells of the bi-category B). ∀ P, Q, R ∈ ¨B¨3, there's a bi-functor * : B(Q, R) × B(P, Q) → B(Q, R) called horizontal composition and ∀ P, Q ∈ ¨B¨2, there's a category B (P, Q) (objects are the 1-cells, morphisms are the 2-cells) with vertical composition. A planning bi-category [2] expresses plan morphisms (plan transformations) and plan composition, imposing identity and associativity up to isomorphism; (i) objects (1-cells) are bijections from the initial to the final state indexed by formulas (0-cells) at_robby(rooma), at(ball1,rooma), free(left), carry(ball1, left), … which can be extended vertically and composed horizontally; (ii) morphisms (2-cells) are isomorphisms operators ensuring associativity to be effective in the bi-category. That is, a plan can only be computed “up to isomorphism”; there 2

are infinitely many neutral elements but the neutral element is unique “up to isomorphism”. The plan generation process cannot be bounded but a planning bi-category must offer that degree of freedom in order to represent a plan. In general, even in the propositional case, the decision procedure is undecidable and so are decision procedures inspired from [4, 5]; the difference between the first decision procedure and the second decision procedure lies on an efficient use of duality between Addα and Delα in the first case and a worse decision procedure, i.e. a less reactive proof system in the second case. In the first case, the procedure works in an activation segment algebra; when one activates the formula carry(ball1, left) occurring in Addpick(ball1, rooma, left) one sets as a plain segment in black the second activation segment in the fourth line; when one deactivates the formula free(left) occurring in Addpick(ball1, rooma, left), one splits the activation segment in the second line.

ĥ In the second case, a proof tree sets as a plain (closed) subproof an identity axiom of the form Ħ and cannot exploit any information coming from Delα, i.e., the reflected information between Addα and Delα. This paper claims that a convenient use of these reflections require a totally different framework. One can select the qualifier “affine” in order to designate infinite unbounded translations in the space of plans and one can compare “affine” representation and “projective” ones. In particular one should force symmetry between the horizontal and the vertical sides of the bi-category which looks like a projective plane (between the number of formulas and the length of the plan).

3.

The projective abstract topology of the planning space

A projective space agrees with the following duality: “two points are required in order to define a line and two lines always intersect in one point” (two parallel lines intersect at the point at infinity ∞); assume that consistency is preserved by a parallel property between proof lines; given a formula A, one can draw a “model line” A and a “counter-model line” Ȏ A which must not intersect in order to agree with the consistency axiom: A ∧ Ȏ A ⊃ ⊥ “A and not A implies absurd”) or T  F = y. That is, incidence issues between proofs (lines) and formulas (points) are expected to be handled in the neighbourhood of the point at infinity ∞. It is important to note that this kind to duality is incompatible with “logical” duality which is called in an algebraic framework an involution. In particular, De Morgan laws, e.g. ȎA ∧ B ȎA ∨ ȎBand the excluded middle law A = ȎȎ A handle both negation Ȏ as a dual involution [6, 7]. Take the “space of proofs” as an algebra A; duality is a morphism of the form *: A → A such that A** = A. To summarize, one can assume that a projective representation offers a compact encoding incompatible with “logical” duality. Take two points a, b inside this space (i.e. between 0 and ∞); this space has the following supplementary properties: (i) there is no standard definition of a point using the commutative distance d(a, b) = d(b, a) (ii) there is no intermediate state between 0 and ∞ (iii) there is no trace of the computation. Therefore “projective completeness” according to a compact encoding in the neighbourhood of infinity is “incomplete” wrt to proof theory and more generally wrt deductive systems. The original initial state Σi is assumed to be a dense closed state ²; Σf corresponds to absurdity ⊥ when viewed from ² and as such can be qualified as “nowhere dense”; that is, Σf is not reachable up to a “affine” translation from Σi. Recall that Σi and Σf hold in the same unified algebra, noted 1, but cannot be connected due to a bitopological structure, a topology τ0 starting from Σi and a topology τ1 starting from Σf. τ1 corresponds at the same 3

time to another topology arriving at the initial state Σi and however distinct from τ0. That is, arrows arriving at the initial state should be classified according to the bi-arrow

→ Σi ← τ1

τ0

In order to cope with the bi-topological structure definable in the neighbourhood of Σi inside the unified algebra 1, one should relax the differential rule “+ AddΨ − DelΨ” defined locally inside a planning operator. Rather than two separable operators “+” and “−” definable on the monoid of operators, one prefers to encode the inconsistent operator “±” according to a more symmetric group representation. 4.

Qualification of the algebraic group structure

Two problems have to be solved: 1.

What kind of structure do we need in order to represent in an algebraic fashion a group [10]?

2.

How can we define the orientation of the dynamic?

Assume that the left vertical side of the bi-category in figure ĥ page 3 forms a vector u and the right vertical side of the bi-category in figure ĥpage 3 forms another vector v; one can assume as well that each element of v is computed according to an expression in which every element (formula, state) of u occurs; that way, if n is taken to be the length of the plan, an algebraic group representation of a plan intuitively looks like a multiplicative subset of a n × n−matrix algebra of the form arrow f : u → v. Starting from the bi-categorical construction, one can define more precisely the algebraic framework: 1-cells are linear functions, a weaker form of square matrices where the dimension n is not an integer and 2-cells are automorphisms on these linear functions. That is, one can express the integer property for the length of the plan n as “completeness of the affine planning space” and the non integer property for the length of the plan n as the “incompleteness of the projective planning space”. This incompleteness can be solved under the action of the automorphism group acting on the projective algebraic variety ∞ defined by the transition from the initial state Σi to the final state Σf according to the set of quantified actions. Suitable candidate algebras are finite dimensional associative algebras over a field k (or over a local ring R) with center k (or R) and without intermediate non-trivial two sided ideals (twisted “nonseparable” forms of n × n square matrix algebras). Every central simple algebra A is split by some separable field extension D; one obtains that way an ending condition: the planning calculus stops when the planning arrow f : u → v is of the form Mn(D). Every 4 dimensional central simple algebra over a field F is isomorphic to a quaternion algebra; in fact, it is either a two-by-two matrix algebra, or a division algebra D [9]. Absurdity ⊥ is encoded by the twist; central simple algebras A are suitable wrt to correcteness since no intermediate state is definable and wrt completeness since every automorphism of a central simple algebra is an inner automorphism (Skolem-Noether theorem). However, normalization automorphisms performing a suitable encoding (a compact representation) on the algebra A are not associative. The original state Σi is a dense closed state ǃ; Σf corresponds to absurdity ⊥ when viewed from ǃ; Σf is not “reachable” from Σi ; the representation of the final state ⊥ is taken to be “absurd” and nowhere dense because it is assumed to be non reachable from ² according to standard commutative distances. One can precise what is the required bi-topology : (i) Σi and Σf are distinct, Σi Ÿ Σf (ii) Σi and Σf overlap (i.e. are not separated) Σi  Σf . A compact encoding can be inspired from abstract quadratic constructions; the initial state Σi is called the adjacent side of consistency and is noted ǃ; the final state Σf is called the opposite side of consistency and is noted ⊥. Take the dense closed state ǃ as idempotent; the algebraic group structure requires a quaternary construction inspired from bi-categories by dividing the idempotent ǃ by 2 and setting the inverse of the idempotent according to a torsion modulo 4. That is, ² = 1/2 and ³ = 3/2 = (1/2)-1; the semantic of the “incompleteness” group is the following: the initial state ² = 1/2 cannot be represented as an idempotent but must be represented according to its annihilator ³= 3/2 (wrt to the torsion modulo 4). Consistency using an idempotent is actually a restricted form of consistency 1/2 and one can compute a consistency extension 1 by integrating the identity 1/2-1 from the final state ⊥ which is taken to be the point at infinity ∞ in the quaternary projective planning space modulo 4. The consistency extension (the hypotenuse) is the neighbourhood of 0/2 is expressed by the noncommutative sum 1 = ǃ­ ⊥. The dynamic interpretation is the following: ǃis stable (is a stabilizer subgroup) and cannot move; however ǃ must move. Consequently, ⊥ is a pure move (a orbit subgroup). Therefore ǃ­⊥ is a move; each move in the space where the “inconsistent differential” topology “±” holds performs a noncommutative move wrt to the diagonal 1. That is, the unit group extension 1 contains both even sequences AddΨ and odd sequences DelΨ. The resolution is performed according to the integration of odd sequences DelΨ in the 4

closed set of even sequences AddΨ according to the inner quadratic law, the diagonal of the system 1 = ǃ2⊥2 . In order to understand what are consequences wrt inference procedures, one should note that one can assimilate an identity axiom carry(ball1, left) ̓carry(ball1, left) in Ħ as a scalar product Àx, xÁ = 1 which is a measure of “parallelness”; one ought to define a measure of “perpendicularness” using the anti-symmetric crossed product x × y = − y × x and x × x = 0 according to the orthogonal relationship between the rising edges of Addα and the falling edges of Delα in Ĥ One obtains that way a valuation for completeness: the separable condition Addα  Delα = y inside each quantified operator α is to be integrated to the whole plan by the inference procedure. That is, the inference procedure definable from ĥ is reversed; the kernel of the procedure is defined from Delα and not from Addα; consistency is integrated in the plan space according to an inner orthogonal product: the antisymmetric product “×”; in effect, one can note that in the Strips definition, Addα formulas never encounter Delα formulas inside a planning operator. 5.

Conclusion

The AI Planning calculus is intended to handle both dynamic and consistency; the way chosen here is to strictly forbid any kind of infinite structure. Strong consequences follow: models Tˆ must be merged with countermodels F according to valid type degeneracy Tˆ → F inside a “deterministic fair operator algebra” A without any combinatoric “deductive” inference. This operator algebra agrees with a projective topology in order to deal naturally with ˆ; the geometry is necessary simple non-commutative (no intermediate point holds between the origin 0 and the point at infinity ˆ). The affine properties “up to isomorphism” in planning bi-categories or “up to an integer dimension n” in linear deductive categories is replaced by “up to a twist” in central simple separable algebras. References [1] [2] [3] [4] [5] [6] [7] [9] [10]

FIKES R., NILSSON N.J., Strips : a new approach to the application of theorem proving to problem solving, Artificial Intelligence, 2, 189-208, 1971. PENON J., Un modèle catégorique pour la planification en intelligence artificielle, rapport de l'Université Paris 7, Laboratoire des théories géométriques, Paris, FR, 2001. BARTHEYE O., JACOPIN E. & PENON J., Qweak : an axiom-based arithmetic constraint prototype, 22th Workshop of the UK PLANSIG, Glasgow, Dec 9−10, 2003. MASSERON M., TOLLU C., VAUZEILLES J, Generating Plans in Linear Logic I. Actions as Proofs, Theoretical Computer Science, 113(2), 349−370, 1993. GIRARD J.Y., Linear logic, Theoretical Computer Science, 50, 1−102, 1987. BLUTE R., Linear Topology, Hopf Algebras and *-autonomous Categories, Research Report, Mac Gill University, Montreal, Quebec, Canada, 1993. KNUS M.A, MERKURJEV A., ROST M., TIGNOL J.P., The Book of Involutions, American Mathematical Society Colloquium Publications, 44, 1998. WATERHOUSE W., Introduction to Affine Group Schemes, Graduate Texts in Mathematics, Springer Verlag, 66, 1979. HUMPHREYS J. E., Linear Algebraic Groups, Graduate Texts in Mathematics, Springer Verlag, 21, 1975.

5

Formal Modelling and Consistency of Apache Wave Platform Evolutions and Waves Karsten Gabriel Integrated Graduate Program H-C3, Technische Universität Berlin, Germany Abstract: Algebraic High-Level (AHL) nets are a well-known modelling technique based on Petri nets with algebraic data types, which allows modelling the communication structure and the data flow within one modelling framework. Transformations of AHL-nets – inspired by the theory of graph transformations – allow in addition modifying the communication structure. Moreover, high-level processes of AHL-nets capture the concurrent semantics of AHL-nets in an adequate way. Altogether we obtain a powerful integrated formal specification technique to model and analyse all kinds of communication based systems, especially different kinds of communication platforms. In our contribution we show how to model the evolution of platforms based on transformations of Algebraic High-Level Nets. Our main result shows under which conditions AHL-net processes can be extended if the corresponding AHL-net is transformed, which can be applied to show the extension of scenarios for a given platform evolution.

1. Introduction High-level nets based on low-level Petri nets [Pet62, Roz87, Rei85] and data types in ML have been studied as coloured Petri nets by Jensen [Jen91] and – using algebraic data types – as algebraic high-level (AHL) nets in [Rei90, PER95, ER97]. Inspired by the theory of graph transformations [Ehr79, Roz97] transformations of AHL-nets were first studied in [PER95] which – in addition to the token game – also allow modifying the net structure by rule based transformations. The concept of processes in Petri nets is essential to model not only sequential, but especially concurrent firing behaviour. A process of a low-level Petri net N is given by an occurrence net K together with a net morphism p: K ĺ N. Processes of highlevel nets AN are often defined as processes p: K ĺ Flat(AN) of the corresponding low-level net Flat(AN), called flattening of AN. However, this is not really adequate, because the flattening is in general an infinite net and the data type structure is lost. For this reason high-level processes for algebraic high-level nets have been introduced in [EHP+02, Ehr05], which are high-level net morphisms p: K ĺ AN based on a suitable concept of high-level occurrence nets K. The main aim of this paper is to give an introduction to the integrated framework of transformations of algebraic high-level nets and processes and to show how this can be applied to modern communication platforms. In previous papers it was shown already how to use this framework to model communication platforms like Skype [HM10] and Google Wave [EG11]. In this paper we show how our integrated framework can be used to model basic aspects of Apache Wave. In Section 2 we introduce a small example of an Apache Wave platform, which is also used as running example for the following sections. In Section 3 we introduce AHL-nets together with high-level processes in the sense of [Ehr05]. Rulebased transformations in analogy to graph transformation systems [Roz97] are introduced in Section 4 for AHL-nets and AHL-processes and applied to the evolution of Apache Wave communication platforms and waves. The main result presented in Section 5 shows under which condition an AHL-process can be extended if the corresponding AHL-net is transformed. This result can be applied to show the extension of scenarios for a given platform evolution. 2. Evolution of Apache Wave Platforms and Scenarios In this section we introduce our main case study Apache Wave which is a communication platform that was originally developed by the company Google as Google Wave. Google itself has stopped the development of Google Wave, but the development is continued by the Apache Software Foundation as Apache Wave. One of the most interesting aspects of Apache Wave is the possibility to make changes on previous contributions. Therefore, in contrast to email, text chat or forums, due to possible changes the resulting data of the communication does not necessarily give a comprehensive overview on all interactions of the communication. For this reason, in Apache Wave for every communication there is a history allowing the users to replay interactions of the communication step by step. So for the modelling of Apache Wave it is necessary that we do not only model the systems and the communication but also the history of the communication. We have chosen Apache Wave as running example for this paper because it includes typical modern features of many other communication systems, such as near-real-time communication. This means that dierent users can simultaneously edit the same document, and changes of one user can be seen almost immediately by the other users. Note that we do not focus on the communication between servers and clients in this contribution but on the communication between users. For details on the modelling of the more technical aspects of the server-to-server and client-to-server communication we refer to [Yon10]. In Apache Wave users can communicate and collaborate via so-called waves. A wave is like a document which can contain diverse types of data that can be edited by dierent invited users. The changes that are made to a wave can be simultaneously recognized by the other participating users. In order to keep track of the changes that have been made, every wave contains also a history of all the actions in that wave. Apache Wave supports dierent types of extensions which are divided into gadgets and robots. The extensions are programs that can be used inside of a wave. The dierence between gadgets and robots is Fig. 1: AHL-net Platform for an Apache Wave platform

that gadgets are not able to interact with their environment while robots can be seen as automated users that can independently create, read or change waves, invite users or other robots, and so on. This allows robots for example to do realtime translation or highlighting of texts that are written by dierent users of a wave. Clearly, it is intended to use dierent robots for dierent tasks and it is desired that multiple robots interact without conflicts. This makes the modelling and analysis of Apache Wave very important in order to predict possible conflicts or other undesired behaviour of robots. In [EG11] we have already shown that Google Wave (and thus also Apache Wave) can be adequately modelled using algebraic high-level (AHL) nets, which is an integration of the modelling technique of low-level Petri nets [Pet62, Roz87, Rei85] and algebraic data types [EM85]. Fig. 1 shows a small example of the structure of an AHL-net Platform which has 3 places and 3 transitions with firing conditions, where the pre and post arcs are labelled with variables of an algebraic signature. The AHL-net Platform models an Apache Wave platform with some basic features like the creation of new waves, modifications to existing waves, and the invitation of users to a wave which are modelled by the transitions new wavelet, modify text and invite user. A wavelet is a part of a wave that contains a user ID, a list of XML documents and a set of users which are invited to modify the wavelet. For simplicity we model in our example only the simple case that every wavelet contains only one single document and the documents contain only plain text. A signature and algebra modelling data-type part of our platform model can be found in [Gab11]. The transitions of the net contain firing conditions in the form of equations over the signature Ȉ-Wave. In order to fire a transition there has to be an assignment v of the variables in the environment of the transition such that the firing conditions are satisfied in the algebra A. The pair (t, v) is then called a consistent transition assignment. Moreover, there have to be suitable data elements in the pre domain of the transition. For example, in order to fire the transition modify text we Fig. 2: AHL-process Wave of a wave need a wavelet on the place w and a user on the place u that can be assigned by the variables o respectively user such that the user is invited to the selected wavelet. Furthermore, we need a text txt, a pair of natural number rng and a new wavelet n such that n is the wavelet which is obtained by deleting all text in the range specified by rng and by inserting the text txt on the starting position of rng into the original wavelet o. The assignment v then determines a follower marking which is computed by removing the assigned data tokens in the pre domain of the transition and adding the assigned data tokens in the post domain. In the case of the transition modify text this means that we remove the old wavelet from the place w and replace it by a new wavelet which contains the modified text. For more details on the operational semantics of AHL-nets we refer to [Ehr05]. As we have shown in [EG11] a suitable modelling technique for waves together with their histories are AHL-processes with instantiations. Fig. 2 shows an example of an AHL-process Wave which abstractly models a wave that contains two wavelets created by possibly dierent users. Another interesting aspect of the modelling of Apache Wave are dynamic changes to the structure of the platform. Using rule-based transformation of AHL-nets [PER95] in the sense of graph transformation [Roz97], we can delete and add features, leading to a new platform. Fig. 3 shows a net Platform’ which is an adaption of our example Platform where the modify text transition has been replaced. In the new version we have a new transition modify and log which for every modification to a wave creates a log entry with information about the user, the position and the text of the modification. Since it is possible that the communication platform is modified at runtime there may already exist some waves that correspond to the old version of the platform. In some cases that correspondence could be violated by the modification of the platform. In Section 5 we present a general construction to obtain a new correspondence between an existing wave and a modified platform based on platform evolution Fig. 3: Modified AHL-net Platform’ under certain conditions. 3. Modelling of Communication Platforms and Scenarios with AHL-Nets and Processes

In the following we review the definition of AHL-nets and their processes from [Ehr05, EHP+02] based on low-level nets in the sense of [MM90], where Xk is the free commutative monoid over the set X. Definition 1 (Algebraic High-Level Nets). An algebraic high-level (AHL-) net AN = (Ȉ, P, T, pre, post, cond, type, A) consists of a signature Ȉ = (S, OP; X) with additional variables X; a set of places P and a set of transitions T; pre- and post domain functions pre, post: T ĺ (TȈ(X) m P)k; firing conditions cond : T ĺ Pfin(Eqns(Ȉ; X)); a type of places type: P ĺ S and a Ȉ-algebra A. The signature Ȉ = (S, OP) consists of sorts S and operation symbols OP, TȈ(X) is the set of terms with variables over X, the restricted product m is defined by (TȈ(X) m P) = {(term, p) | term  TȈ(X)type(p) , p  P} and Eqns(Ȉ; X) are all equations over the signature Ȉ with variables X. An AHL-net morphism f: AN1 ĺ AN2 is given by f = (fP , fT) with functions fP: P1 ĺ P2 and fT: T1 ĺ T2 satisfying (id m fP)k q pre1 = pre2 q fT , (id m fP )k q post1 = post2 q fT , cond2 q fT = cond1 , and type2 q fP = type1. The category defined by AHL-nets (with signature Ȉ and algebra A) and AHL-net morphisms is denoted by AHLNets where the composition of AHL-net morphisms is defined componentwise for places and transitions. An example of an AHL-net is given by the net Platform = (Ȉ, P, T, pre, post ,cond, type, A) shown in Fig. 1 in Section 2, where the signature Ȉ and the Ȉ-algebra A can be found in [Gab11]. Note that it is also possible to define a category of AHL-nets with dierent signatures and algebras which requires that the morphisms not only contain functions for places and transitions but also a signature morphism together with a generalised algebra morphism (for details see [PER95]). The firing behaviour of AHL-nets is defined analogously to the firing behaviour of low-level nets. The dierence is that in the high-level case all tokens are equipped with data values. Moreover, for the activation of a transition t, we additionally need an assignment asg of the variables in the environment of the transition, such that the assigned pre domain is part of the given marking and the firing conditions of the transition are satisfied. This assignment is then used to compute the follower marking, obtained by firing of transition t with assignment asg. Now, we introduce AHL-process nets based on low-level occurrence nets (see [GR83]) and AHL-processes according to [Ehr05, EHP+02]. The net structure of a high-level occurrence net has similar properties like a low-level occurrence net, but it captures a set of dierent concurrent computations due to dierent initial markings. In fact, high-level occurrence nets can be considered to have a set of initial markings for the input places, whereas there is only one implicit initial marking of the input places for low-level occurrence nets. Moreover, in a low-level occurrence net with an initial marking there is for any complete order of transitions compatible with the causal relation a corresponding firing sequence once there is a token on all input places. This is a consequence of the fact that in an occurrence net the causal relation is finitary. In the case of high-level occurrence nets an initial marking additionally contains data values and in general some of the firing conditions in a complete order of transitions are not satisfied. Hence, even in the case that the causal relation is finitary, we cannot expect to have complete firing sequences. In order to ensure a complete firing sequence in a high-level occurrence net there has to be an “instantiation” of the occurrence net (see [Ehr05]). Instantiations, however, are not considered explicitly in this paper. In the following definition of AHL-process nets, in contrast to occurrence nets, we omit the requirement that the causal relation has to be finitary, because this is not a meaningful requirement for our application domain. Definition 2 (Algebraic High-Level Process Nets). An AHL-process net K is an AHL-net K = (Ȉ, P, T, pre, post, cond, type, A) such that for all t  T with pre(t) = Ȉni=1 (termi,pi) and notation rt = {p1, . . . , pn} and similarly tr we have 1. (Unarity): rt, tr are sets rather than multisets for all t  T, i.e. for rt the places p1,… , pn are pairwise distinct. Hence |r t| = n and the arc from pi to t has a unary arc-inscription termi. 2. (No Forward Conflicts): rt ŀ rt = Ӆ for all t, t’  T, t  t’ 3. (No Backward Conflicts): tr ŀ t’r = Ӆ for all t, t’  T, t  t’ 4. (Partial Order): the causal relation {(t, p)  T × P | p  tr} is a strict partial order, i. e. the partial order is irreflexive. AHL-process nets (with signature Ȉ and algebra A) together with AHL-net morphisms between AHL-process nets form the full subcategory AHLPNets u AHLNets. Note that an AHL-process net with a finitary causal relation is an AHL-occurrence net as defined in [Ehr05]. Similar to low-level processes, an AHL-process mp of an AHL-net AN is defined as an AHL-morphism mp: K ĺ AN from an AHL-process net K into the net AN. The category Proc(AN) of AHL-processes of an AHL-net AN is defined as the full subcategory of the slice category AHLNets \ AN such that the objects are AHL-processes. This means that the objects of Proc(AN) are AHL-process morphisms mp: K ĺ AN and the morphisms of the category are AHL-net morphisms f : K1 ĺ K2 such that mp2 q f = mp1. The category AHLProcs of all AHL-processes is defined as full subcategory of the arrow category AHLNetsĺ such that the objects are AHL-processes.

An example of an AHL-process is the scenario wave: Wave ĺ Platform of the Apache Wave platform model Platform shown in Fig. 2 in Section 2. Note that the types of the places and the firing conditions of the transitions are given implicitly by the matching of the AHL-morphism wave: Wave ĺ Platform which is denoted by colons in Fig. 2, e.g. id1: id means that waveP(id1) = id and the place id1 has the same type nat as the place id in the net Platform. 4. Evolution of Communication Platforms and Transformation of Scenario Nets Due to the possibility to evolve the Apache Wave platforms by adding, removing or changing features we need also techniques that make it possible to evolve the corresponding model of a platform. For this reason we introduce rule-based AHL-net transformations [PER95] in the sense of graph transformations [Roz97]. A production (or transformation rule) for AHL-nets specifies a local modification of an AHL-net. It consists of a left-hand side, an interface which is the part of the left-hand side which is not deleted and a right-hand side which additionally contains newly created net parts. Definition 3 (Productions for AHL-Nets). A production for AHL-nets is a span ρ : L ĸ I ĺ R of injective AHLmorphisms l: I ĺ L and r: I ĺ R. We call L the left-hand side, I the interface, and R the right-hand side of the production ρ. Fig. 4 shows a production insertLog for AHL-nets that can be used for the evolution of an Apache Wave platform. The production describes a local modification that removes a transition modify text and inserts a new transition modify and log and a new place log. Moreover, the newly created transition is connected to the former environment of the removed transition.

Fig. 4: AHL-net production insertLog for the evolution of platforms

Definition 4 (Direct Transformation of AHL-Nets). Given a production ρ: L ĸ I ĺ R and a (match) morphism m: L ĺ AN in AHLNets. Then a direct transformation of AHL-nets AN AN’ via (ρ,m) is given by pushouts (1) and (2) in the category AHLNets. Analogously, a direct transformation of AHLprocess nets AN AN’ via (ρ,m) is given by pushouts (1) and (2) in the category AHLPNets. In [PER95] a gluing condition for AHL-nets is defined for the match m: L ĺ AN and it is shown that the gluing condition is a necessary and su’cient condition for the existence of a direct transformation of AHL-nets. For our example production insertLog we can find a match m: L ĺ Platform into the AHL-net in Fig. 1 that matches the transition modify text to modify text, the user places to u, and the wavelet places to w. Moreover, the match production insertLog with match m satisfies the gluing condition, leading to a direct transformation Platform Platform’ where Platform’ is depicted in Fig. 3. The direct transformation of AHL-nets may produce forward or backward conflicts as well as cycles in the causal relation. So for the direct transformation of AHL-process nets via pushout construction the production and match additionally have to satisfy a special Transformation Condition for AHL-process nets in order to obtain again an AHL-process net as a result of the direct transformation. For details on the transformation condition for AHL-process nets, which is a sufficient and necessary condition for the existence of direct transformations of AHL-process nets, we refer to [Gab11]. 5. Extension of Scenarios based on Platform Evolutions In the previous section we have presented the rule-based transformation of AHL-nets and processes and we have shown how it can be used to evolve Apache Wave platforms and scenarios. As mentioned in Section 2 it is possible that the communication platform is modified at runtime and there may already exist some waves that correspond to the old version of the platform. So we have the case that there is an AHL-process wave: Wave ĺ Platform and a direct transformation of AHLnets Platform Platform’. In this section we show under which condition the scenario wave can be extended to a scenario wave’: Wave ĺ Platform’ of the new platform. We regard wave’ as an extension of wave if the two processes “agree” in the context net of the direct transformation Platform Platform’ in the following sense. Definition 5 (Extension of AHL-Process based on AHL-Net Transformation). Given an AHLnet AN and an AHL-process mp : K ĺ AN. Let AN AN’ be a direct transformation with pushouts (1) and (2) in AHLNets as depicted in the right. Then we call mp’: K ĺ AN’ an extension of mp if there exists mp0: K ĺ AN0 with f q mp0 = mp and g q mp0 = mp’.

The following extension condition is a su’cient and necessary condition for the extension mp’ of an AHL-process mp based on an AHL-net transformation. In order to satisfy the extension condition, the transformation must not delete any places or transitions that have an occurrence in the AHL-process mp. Definition 6 (Extension Condition). Given an AHL-net AN, an AHL-process mp: K ĺ AN and a direct transformation AN AN’ via (ρ, m) as in Def. 5. We define the the set PP of process points as PP = {x  PL | p  PK : mpP(p) = mP(x)} > {x  TL | t  TK : mpT (t) = mT(x)} We say that mp and (ρ, m) satisfy the extension condition if all process points are gluing points, i.e. PP u GP = lP(IP)> lT(IT). Theorem (Extension of AHL-Process based on AHL-Net Transformation). Given an AHL-net AN, an AHL-process mp : K ĺ AN and a direct transformation direct transformation AN AN’ via (ρ, m) as in Def. 5. There exists an extension mp’: K ĺ AN’ of mp if and only if mp and (ρ, m) satisfy the extension condition. Proof-Idea. If the extension condition is satisfied, it can be explicitly shown that there exists a well-defined morphism mp0: K ĺ AN0 defined by mp0 = f−1 q mp. Then the extension mp’: K ĺ AN’ is obtained by composition mp’ = g q mp0, satisfying the required properties. Vice versa, the existence of an extension mp’: K ĺ AN’ implies the existence of a suitable morphism mp0: K ĺ AN0 which can be used to show that all process points are gluing points. For a detailed proof we refer to [Gab11]. For the previously presented direct transformation of AHL-nets Platform Platform’ via (insertLog, m) we have that the AHL-process wave: Wave ĺ Platform and (insertLog,m) satisfy the extension condition. Therefore, the scenario of the Apache Wave platform Platform depicted in Fig. 2 is also a valid scenario for the modification Platform’, since there is an extension wave’: Wave ĺ Platform’. 6. Conclusion Algebraic high-level (AHL) nets are a well-known modelling technique based on Petri nets [Pet62, Rei85, Roz87] with algebraic data types [EM85]. In this paper we have shown that AHL nets, AHL processes, and AHL transformations can be considered as integrated framework for modelling the evolution of communication platforms. In previous papers it was shown already how to use this framework to model communication platforms like Skype [HM10] and Google Wave [EG11]. In this paper we have used the evolution of Apache Wave platforms and scenarios as running example, where platforms are modelled by AHL-nets and scenarios by AHL-processes. The evolution on both levels is defined by rule-based modifications in the sense of graph transformation systems [EEPT06]. While transformations of AHL-nets are introduced already in [PER95] the corresponding problem for AHL-processes is much more di’cult as shown in [Gab11]. The main result of this paper shows under which conditions AHL-net processes can be extended if the corresponding AHLnet is transformed. This result can be applied to show the extension of scenarios for a given platform evolution. References [EEPT06] H. Ehrig, K. Ehrig, U. Prange, and G. Taentzer. Fundamentals of Algebraic Graph Transformation. EATCS Monographs in TCS. Springer, 2006. [EG11] H. Ehrig and K. Gabriel. Transformation of algebraic high-level nets and amalgamation of processes with applications to communication platforms. International Journal of Software and Informatics, 5, Part1:207–229, 2011. [EHP+02] H. Ehrig, K. Homann, J. Padberg, P. Baldan, and R. Heckel. High-level net processes. In Formal and Natural Computing, volume 2300 of LNCS, pages 191–219. Springer, 2002. [Ehr79] H. Ehrig. Introduction to the algebraic theory of graph grammars (a survey). In V. Claus, H. Ehrig, and G. Rozenberg, editors, Graph Grammars and Their Application to Computer Science and Biology, Lecture Notes in Computer Science, No. 73, pages 1–69. Springer, 1979. [Ehr05] H. Ehrig. Behaviour and Instantiation of High-Level Petri Net Processes. Fundamenta Informaticae, 65(3):211–247, 2005. [ER97] Hartmut Ehrig and Wolfgang Reisig. An Algebraic View on Petri Nets. Bulletin of the EATCS, 61:52–58, February 1997. [EM85] H. Ehrig and B. Mahr. Fundamentals of Algebraic Specification 1. Springer, 1985. [Gab11] K. Gabriel. Modelling Evolution of Communication Platforms and Scenarios based on Transformations of High-Level Nets and Processes – Extended Version. Technical Report 2011/08, Technische Universität Berlin, 2011. [GR83] U. Goltz and W. Reisig. The Non-sequential Behavior of Petri Nets. Information and Control, 57(2/3):125–147, 1983. [HM10] Kathrin Hoěmann and Tony Modica. Formal modeling of communication platforms using reconfigurable algebraic high-level nets. ECEASST, 30:1–25, 2010. [Jen91] K. Jensen. Coloured Petri Nets: A high-level language for system design and analysis. In G. Rozenberg, editor, Advances in Petri Nets 1990, volume 483 of LNCS, pages 342–416. Springer, 1991. [PER95] J. Padberg, H. Ehrig, and L. Ribeiro. Algebraic high-level net transformation systems. Mathematical Structures in Computer Science, 80:217–259, 1995. [Pet62] C.A. Petri. Kommunikation mit Automaten. PhD Thesis, Institut für instrumentelle Mathematik, Universität Bonn, 1962. [Rei85] W. Reisig. Petrinetze, Eine Einführung. Springer Verlag, Berlin, 1985. [Rei90] W. Reisig. Petri nets and algebraic specifications. Technische Universität München, SFB-Bericht 342/1/90 B, March, 1990. [Roz87] G. Rozenberg. Behaviour of Elementary Net Systems. In Petri Nets: Central Models and Their Properties, Advances in Petri Nets, volume 254 of LNCS, pages 60–94. Springer, 1987. [Roz97] G. Rozenberg, editor. Handbook of Graph Grammars and Computing by Graph Transformation, Vol 1: Foundations. World Scientific, Singapore, 1997. [Yon10] Tsvetelina Yonova. Formal description and analysis of distributed online collaboration platforms. Bachelor Thesis, Technische Universität Berlin, 2010.

Generating and executing scenarios for a mobile robot Philippe Morignot and Christophe Leroux

Abstract— This paper presents a high level language for describing scenarios which control a mobile robot with a 6-DOF arm and a gripper. Since it is impossible in practice to encode all possible scenarios in advance, A.I. task planning is used to generate scenarios, before actually executing them on the mobile robot, SAM. Our implementation has been tested on scenarios involving fetching objects, as assistance to disabled persons.

I. INTRODUCTION

I

MAGINE a mobile robot dedicated to servicing a person in his apartment. Perhaps the person will ask for his pills, which entails for the robot to go to the bathroom, deploy its arm, grasp the pill box, retract the arm, come back to the person’s location and hand the pill box over to the person. Perhaps the person will ask for a fruit juice, which entails for the robot to go to the kitchen, deploy its arm, remove one by one the unsatisfying soda cans from a shelf until a can of orange juice is found, grasp it, retract its arm, come back to the person’s location and hand the can over to him. Or perhaps the person will ask to clean the table, which entails for the robot to go in front of the table, deploy its arm, and remove the first object from the table, drop it into the trash can, remove the second object from the table, drop it into the trash can, etc, and come back. In those examples, several actions have to be taken by the robot, with different nature and order: These examples are three different scenarios and the same mobile robot should be able to carry out all of them as requested by the person. In addition, a service mobile robot should be able to carry numerous other scenarios, as requested by the serviced user. Many robots can carry out a single scenario (e.g., ROLLIN JUSTIN [5], TWENDY ONE [12], HRP-4 [1]). In these systems, a scenario is represented as a piece of source code which calls functions of a programming language for the robot to successively carry out all the prescribed actions. But to change a scenario (e.g., for a new demo), changing this source code and recompiling it is necessary. Other robots use a high level language for representing scenarios (e.g., CARE-O-BOT [9], NAO [8] with the graphical language CHOREGRAPHE, among others). In these systems, high level actions can be synthetically encoded in this language. So re-programming the robot for a This work was supported by the DGE of the French Ministry of Economy, Finance and Industry through contract ITEA 2 MIDAS, as part of a EUREKA European project. P. Morignot ([email protected] , phone : +33 (0)1 46 54 97 27, fax : +33 (0)1 46 54 89 90 and C. Leroux ([email protected]) are

new scenario is not necessary: Only the high level actions in the scenario have to be changed. Still other robots’ designers acknowledge the fact that there are many scenarios which would have to be manually written in real applications (not only for demos). And that it is unpractical, if not impossible, to manually write all of them in advance (for that, even a high level language is not sufficient). So these robots generate their own scenarios (e.g., SHAKEY [4], PR2 [23], DALA & LAMA [10]) using an A.I. task planner: (i) A user specifies symbolic goals; (ii) A.I. task planning eventually generates a sequence of instantiated operators (a plan of actions) which satisfy these goals in the end; (iii) This plan is considered as a scenario (a high level description of the successive actions to take) and (iv) this scenario is executed by the robot (each action description in the scenario is linked to an executable function in the underlying programming language, and these functions are executed sequentially). But either A.I. planning might exhibit too small performances [4], leading to poor applicability of the approach; Or the A.I. planner is not domain-independent, hence needing additional knowledge (HTN planner [23]); Or interleaving A.I. task (re-)planning and execution leads to fairly complex software architectures [10]. In this paper, we propose a high level language, called ISEN (Interactive Scenarization Engine), for representing scenarios and a way for an A.I. task planner to generate scenarios for this language. A domain-independent A.I. task planner, CPT (for Constraint Programming Temporal planner [21]), is used for generating scenarios. This A.I. planner, the scenario language and its engine are used for controlling SAM (Smart Autonomous Majordomo), a versatile and adaptable mobile robot with a 6 DOF arm and a gripper, which assists handicapped/ageing persons in their apartment [16]. The paper is organized as follows: section II presents the scenario language and the way scenarios can be generated by a task planner. Then, section III details an example from end to end, from goal specification to plan execution on the real robot. Section IV discusses related approaches, and the last section sums up our contribution. II. MODEL In this section, we describe the high level language ISEN, capable of representing and executing scenarios, and a way

with CEA, LIST, Interactive Robotics Laboratory, 18, route du panorama, B.P. 6, 92265 Fontenay-aux-Roses, France.

to generate these scenarios through an A.I. task planner (e.g., CPT). A. ISEN: a high level language and an engine The ISEN engine is a virtual machine which reacts to events sent by the application in which it is integrated, and triggers actions that can act on this application. And this, by conforming to scenarios which specify a behavior model. At initialization time, the application must provide to ISEN a library of elementary actions which can be taken by the engine, and a behavior model, or scenario (provided as an XML file). At execution time, the application sends to the engine the events generated either by the application or by the external environment. The engine calls functions from the previous library of actions, as specified by scenarios, to act on the application and on the external environment. A scenario specifies any number of state machines (or agents1), which autonomously react to events by changing states and/or triggering actions. Every event sent by the application is transmitted to all state machines included into the scenario. Only the state machines tailored for reacting to this event actually does --- the other state machines simply ignore it. Programming a set of ISEN state machines hence sums up to: (1) specifying a list of states in which the state machine can fall into, (2) for each previous state, specifying the next state to which to branch (i.e., a transition), and (3) specifying the actions to take for each state transition or event reception.

Fig. 1: Actions and transitions in an ISEN sequence. A scenario is composed of a set of automata (or sequences), a set of agents and a list of global constants (visible by all states of all automata of all agents). An automaton is composed of a set of states and a set of transitions, or state change, which can be activated in the considered state. The target state of a transition can be another state of the current sequence, or the initial state of another sequence. Each agent specifies its initialization sequence and its local constants. Three kinds of actions can be attached to a state: ON_ENTRY actions (noted f1,1 to f1,n1 for state n in Fig. 1) are activated in sequence just before the current state is activated; ON_DO actions (noted f2,1 to f2,n2 for the same state in the same figure) are activated in sequence during the current state’s activation; and ON_EXIT actions (noted f3,1 to 1 This term should not be confused with the term “agent” in the A.I. multiple agent systems community, for example.

f3,n3 for the same state in the same figure) are activated in sequence right after the current state has been activated. Actions (noted, fE,1 to fE,ne in Fig. 1) related to an event (noted E in Fig. 1) can also be attached to a state. When event E is received by the active state, the actions fE,1, …, fE,ne are activated. And once these event-related actions are all activated, the control flow passes from the current state (n) to the state specified by the transition (i.e., state n+1 in Fig. 1) --- the same mechanism applies for the actions of this new state. Any number of events (and transitions) can be attached to a state of an automaton? Therefore an automaton can own any graph structure. Since ISEN interprets XML source code, i.e., text, the user must specify in the source code of the application the link between the text of an action (e.g., “JointMove(90,60,50,80,90, 60)”) and the callback function in the underlying programming language (e.g., the actual function call “JointMove(ArticularPosition)”). ISEN’s automata can be easily designed using the IsenEdit graphical editor [17]. Their execution can be stepped/traced using a remote console debugger. B. Scenario generation with CPT A.I. task planners use operators, describing the actions which can be taken, an initial state and goals, to eventually build a sequence of instantiated operators (a plan). A plan moves step by step a world state from the initial state to a final state that contains the goals [6]. All entities are described in the Planning Domain Definition Language (PDDL) syntax, e.g. version 2.1 [18] in our case. The shortest syntactic element is a fluent i.e., a term which can be positive or negative depending on the time of observation and on the operators’ postconditions (potentially changing the truth value of this fluent) before this observation time. A fluent contains a functor followed by variables or constants, e.g., “(position ?arm ?location)”, “(at SAM kitchen)”. An operator is composed of preconditions (i.e., fluents which must hold in the incoming state for the operator to apply) and postconditions (i.e., fluents the truth value changes when compared to the incoming state). CPT (Constraint Programming Temporal planner [21]) is a fast classical task planner, which turns a planning problem (i.e., the operators list, the initial state, the goals) into a constraint programming problem: Variables represent the start instants of instantiated operators; domains represent the instants, from 0 (first instantiated operator in the plan) to the plan length (last instantiated operator in the plan); constraints among variables represent time bounds on the operators’ execution, causal links and mutual exclusion. Heuristics are used to estimate the future plan length. Then, finding a plan which solves a planning problem turns out to finding one value of each variable’s domain, which together satisfies all the constraints. CPT’s algorithm searches for a plan with a length given by heuristics. If no solution is found within a given plan length, the plan length is incremented and the search re-starts (therefore CPT’s algorithm is complete).

C. Turning plans into scenarios A plan, generated by the task planner, is first parsed and each instantiated operator of this plan is identified to a state in a sequence. Encoding instantiated operators as eventtriggered transitions (this way, ISEN states would be close to planning states) would prevent the robot from receiving action termination events during the execution of an action. So this option, although theoretically appealing, is not appropriate in practice. Low level ISEN functions are used to turn this internal representation into ISEN’s one. For each instantiated operator, the callback functions (the actions of section A) and the triggering event (to jump out of the current ISEN state) are read from a handler file, by matching a handler name against the name of an operator. For example, the handler named “position-arm-for-grasping” is associated to the instantiated operator named “position-arm-for-grasping pt-ref rot-ref kitchen pt-kitchen rot-kitchen” --- CPT is canonical (an operator appears only once in a plan) [21], which prevents from matching the same handler to several instantiated operators with this name. Errors during the execution of an action are handled by a specific, always active, state, called “all_states”, which is the default event-catcher. When this state is reached, it can branch to either CPT-generated states or additional states (not generated by CPT) dedicated to error recovery. These additional do not match against the name of actions in the generated plan --- the remaining handlers, e.g., a handler named “FailureState” typically is not part of a plan but anyway maps to an ISEN state to which planned states can branch to for error recovery. III. EXPERIMENTAL RESULTS In this section, we first describe scenario generation by detailing an example. Then we describe scenario execution on a mobile platform. A. Scenarios generation The actions which can be taken by the robot are gathered in the SAM domain: this is a symbolic representation in PDDL of these actions in terms of preconditions and postconditions (see [19] for a first version of this PDDL domain). The initial state of a PDDL problem describes the robot and its environment. In our experiments, only a unique goal is used, e.g., (at coca-can pt-deliver), meaning that the robot aims at putting a coca can at the location named pt-deliver. Table 1 lists the 20-action plan generated by CPT for this problem with this domain2. This plan is generated in 0.08s on a computer with 4-CPU at 2,66 GHz with 3,37Go RAM. 0: (initialize-arm) [1] 1: (move-arm-to-reference-position pt-random rot2

Due to paper space limitations, the PDDL domain, problem and handlers for this example can be found at URL http://philippe.morignot.free.fr/SAM .

random pt-ref rot-ref) [1] 2: (close-gripper pt-ref rot-ref) [1] 3: (initialize-sam pt-ref rot-ref) [1] 4: (load-maps) [1] 5: (initialize-object-selection) [1] 6: (open-gripper) [1] 7: (position-arm-for-grasping pt-ref rot-ref kitchen ptkitchen rot-kitchen) [1] 8: (wait-for-object-designation coca-can kitchen pt-kitchen rot-kitchen) [1] 9: (initialize-grasping coca-can) [1] 10: (move-towards-object pt-kitchen rot-kitchen coca-can pt-coca-approach rot-coca-approach) [1] 11: (avoid-obstacle-before-object1 pt-coca-approach rot-coca-approach coca-can kitchen pt-coca-ontology1 rot-coca-ontology1) [1] 12: (avoid-obstacle-before-object2 pt-coca-ontology1 rot-coca-ontology1 coca-can pt-coca-ontology2 rot-coca-ontology2) [1] 13: (blind-grasp pt-coca-ontology2 rot-coca-ontology2 coca-can pt-coca-barrier rot-coca-barrier) [1] 14: (close-gripper-on-object coca-can kitchen pt-coca-barrier rot-coca-barrier) [1] 15: (move-arm-to-safe-position coca-can pt-coca-barrier rot-coca-barrier pt-safe rot-safe) [1] 16: (move-arm-to-transport-position pt-safe rot-safe pt-transport rot-transport) [1] 17: (initialize-dropping coca-can pt-transport rot-transport) [1] 18: (move-arm-to-deliver-position pt-transport rot-transport coca-can pt-deliver rot-deliver) [1] 19: (open-gripper-with-object coca-can pt-deliver rot-deliver) [1] Table 1: An action plan for a pick and place problem. For example, the instantiated operator “position-arm-forgrasping” (7th in Table 1) maps to the state of Table 2. It is composed of 6 ON_ENTRY actions and 1 transition (noted ON_EVENT). The last action is the function call for actually moving the arm, i.e., JointMove(ArticularPosition). The transition can be read as: when the event “Manus_ArmCommandFullfilled” is received, the control flow passes on to the state named “wait-for-objectdesignation”, which reversely maps to the 8th instantiated operator of Table 1. The handler associated to this state contains the same information as Table 2, except its full name and transition. DisplayAgentStatus(Brief=true) Speak(Text="We begin the intelligent grasping scenario.")

displayMessage(Message="Positionning of the arm for grasping", Type=$HIGH) Index =FindVectorElement(Element= $CurrentPos, Vector=$StationTable) ArmPos = GetVectorElement(Index=$Index, Vector=$GraspTable) ConsoleOutput(P1="position to reach = ", P2=$ArmPos) JointMove(ArticularPosition=$ArmPos) Table 2: ISEN state for the "position-arm-for-grasping" instantiated operator in XML format. B. Scenario execution The robot SAM (Smart Autonomous Majordomo [18]) is a non-holonomic mobile base ROBULAB 10 with a 6-DOF MANUS arm ending with a gripper (see Fig. 2). Its sensors are forward- and backward-oriented sonars located on the base (for obstacle avoidance), a panoramic camera located on top of the base (for scene detection), 2 webcams located on the arm (for object recognition, distance stereomeasurement, and visual servoing [14]) and an optical barrier located in the gripper (for decision on clamp closure/opening). All software components are integrated as web services in a Device Profile Web Services architecture [13]. They include (i) moving the mobile base to a 2D location in the apartment, (ii) moving the mobile base closer to an object to grasp, in case it lays out of range of the arm [2], (iii) visually recognizing a given object in an image given a list of known object images [14] and (iv) determining the successive poses of the arm depending on the shape and orientation of the object to grasp [22]. The implementation leads to execution of scenarios (e.g., the one derived from Table 1 into ISEN’s state, excerpted in Table 2) on the mobile platform SAM3. Since some actions can fail on the real robot (e.g., visionrelated actions such as “wait-for-object-designation” in Table 1), a scenario is decomposed into smaller parallel sequences (e.g., starting with the name “init-“ in Table 1). This way, an error state (e.g., “FailureState”) can branch to the first state of the small sequence which failed, instead of having to execute the whole sequence again.

Fig. 2: The robot SAM includes a 6 DOF arm and a gripper. IV. RELATED WORK Among the variety of mobile robotic systems (e.g., [1][4][5][8][9][10][12][23]), only few involve a high level language to represent scenarios and an A.I. task planner to generate them. First of all, the robot SHAKEY pioneered the field of domain-independent A.I. task planning with the STRIPS task planner and the PLANEX execution mechanism [4]; nowadays A.I. planners are faster degrees of magnitude [11]. The robot CARE-O-BOT uses a high level language to encode scenarios using activities linked by discrete, cyclic or wait-for relations [9]. AI planning is performed by a request to a database, which provides possible actions that fulfill a given task. But the previous high level primitives can be encoded in ISEN, due to the absence of any constraint on the graph structure of states. Most importantly, CARE-O-BOT does not include an A.I. task planner per se, hence leading to manually encoding all the possible actions for every possible task. The robot PR2 uses no specific high level language to represent scenarios, but uses a modified Hierarchical Task Network (HTN) planner to generate them [23] (see [6] for an introduction to HTNs). Such planners need decomposition knowledge on tasks to reduce search complexity, which is not needed by CPT (Distinguished Performance award at IPC’06 [11]). The robots DALA & LAMA [10] use a domainindependent task planner, IxTeT [15], based on constraint programming (CPT is based on the same principle but with a different model). But that planner was designed before the first version of PDDL was proposed (1998 [18]). As such, it could not support PDDL syntax for operators and problem description, which all planners do since that time. V. CONCLUSION

3 A video of the execution of scenario of Table 1 on the robot is available at the following URL: http://philippe.morignot.free.fr/SAM .

This paper describes a high level language for representing/executing scenarios and a way to generate these scenarios with A.I. planning. This language, ISEN, is an event-based finite state automata engine, which links high level actions to executable functions controlling the mobile

base, the 6 DOF arm and the gripper of a robot called SAM. An A.I. task planner, CPT, is used to generate scenarios through a mapping between CPT’s instantiated operators and ISEN’s states. Callback functions, actually performing the prescribed actions, are described as handlers attached to states. Errors are handled through a specific, always active, state which branches to recovery states, not generated by the A.I. task planner. The approach is used to control a mobile robot with an arm and a gripper, in scenarios for assisting handicapped/ageing persons in their apartment. This work leads us to the following observation: A.I. task planning requires a symbolic description of the environment and of the robot, i.e., an initial state of a PDDL problem. This is a strong requirement, since the whole scientific field of vision may be used for establishing a PDDL pre-condition as simple as (relative-position gripper coca-can 10 20 30) -- meaning that the coca can is located at point (10, 20, 30) in the referential of the gripper. The difficulty of perceiving the environment has even led some researchers to restrict to virtual worlds, e.g. video games [2] in which the position (for example) of objects is obtained by simply accessing a data structure in the computer’s memory, not by analyzing the sensors’ output of a robot. The next step in our work is to perform on-line replanning (e.g., when the robot detects that a door is unexpectedly closed, re-plan to look for the key first), by embedding an A.I. task planner as one ON_DO action of an ISEN state dedicated to failure (and therefore re-planning). We also envision a software architecture, in the sense of the latter previous observation, composed of an ontology for structuring perception into a symbolic representation and for building the initial state of a PDDL problem.

[6] [7]

[8]

[9]

[10]

[11] [12]

[13]

[14]

[15]

[16]

[17] [18]

ACKNOWLEDGMENT The authors thank Mariette Soury (LIMSI-CNRS, Orsay), Patrick Hède (CEA LIST LVIC, Fontenay-aux-Roses), Vincent Vidal (ONERA, Toulouse) and Christophe Leroy (CEA LIST LSI, Fontenay-aux-roses).

[19]

[20]

REFERENCES [21] [1]

[2]

[3]

[4]

[5]

K. Akachi, K. Kaneko, N. Kanehira, S. Ota, G. Miyamori, M. Hirata, S. Kajita, F. Kanehiro, “Development of humanoid robot HRP-3P”. 5th IEEE-RAS International Conference on Humanoid Robots, pages 50-55, 2005. O. Bartheye, E. Jacopin, “A real-time PDDL-based planning component for video games”. In Proceedings of 5th Artificial Intelligence for Interactive Digital Entertainment Conference, Stanford, California, 2009, pages 130-135. J. Dumora. “Design of behaviors for a robot assisting handicapped persons” (“Conception de comportements pour un robot d’assistance aux personnes handicapées”). Technical Report, CEA, LIST, DTSI/SRI/08-XXX, August 2008, unpublished. R. Fikes, N. Nilsson, “STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving”, Artificial Intelligence, Vol. 2 (1971), pp 189-208. M. Fuchs, Ch. Borst, P. Robuffo Giordano, A. Baumann, E. Kraemer, J. Langwald, R. Gruber, N. Seitz, G. Plank, K. Kunze, R. Burger, F. Schmidt, T. Wimboeck, G. Hirzinger. “Rollin’ Justin – Design considerations and realization for a humanoid upper body”, in

[22]

Proceedings opf the IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan, 2009, pages 4131-4137. E. Gat, “On three-layer architectures”, D. Kortenkazmp et als eds., A.I. and Mobile Robots, AAAI Press, 1998. M. Ghallab, D. Nau, P. Traverso. “Automated planning : theory and practice”. Morgan Kaufmann, Elsevier, San Francisco, 2004, 635 pages. D. Gouaillier, V. Hugel, P. Blazevic, C. Kilner, J. Monceaux, P. Lafourcade, B. Marnier, J. Serre, B. Maisonnier. “Mehatronic design of NAO humanoid”. IEEE International Conference on Robotics and Automation, Kobe, Japan, 2009, pages 769-774. B. Graf, M. Hans,, R. Schraft, “Care-o-bot II development of a next generation robotic home assistant”. Auton. Robots, 16(2), pp, 193— 205, 2004. F. Ingrand, S. Lacroix, S. Lemai, F. Py, « Decisional autonomy of planetary rovers ». Journal of Field Robotics, vol. 24, n° 7, pages 559580, 2007. International Planning Competition, http://ipc.icaps-conference.org/ H.Iwata, S.Sugano: “Design of Human Symbiotic Robot TWENDYONE,” Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 580-586, 2009. F. Jammes, A. Mensch, H. Smit. “Service-oriented device communications using the devices profile for web services”, In AINA Work., pages 947–955, Washington, USA, May 2007. M. Joint, P.-A. Moëllic, P. Hède, P. Adam, “HEKA : A general tool for multimedia indexing and research by content”. 16th Annual Symposium Electronic Imaging (SPIE), San Jose 2004, Image Processing, Algorithms and Systems III. P. Laborie, M. Ghallab. “Planning with sharable resource constraints”. International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995, pages 643-1651. C. Leroux, I. Laffont, B. Biard, S. Schmutz, J. - F. Désert, G. Chalubert. « Robot grasping of unknown objects, description and validation of the function with quadriplegic people ». in Proceedings of the 2007 IEEE 10th International Conference on Rehabilitation Robotics. Noordwijk, The Netherlands, 2007. C. Leroy. ISENEdit: User Manual. CEA, LIST, LRI, Technical Report, 2005, unpublished. D. McDermott, M. Ghallab, A. Howe, C. Knoblock, A. Ram, M. Veloso, D. Weld, D. Wilkins. “PDDL – The Planning Domain Definition Language”, http://cs-www.cs.yale.edu/homes/dvm/ P. Morignot, M. Soury, C. Leroux, H. Vorobieva, P. Hède. Generating Scenarios for a Mobile Robot with an Arm. Case study : Assistance for Handicapped Persons. Eleventh International Conference on Control, Automation, Robotics and Vision (ICARCV’10). Singapore, December 2010, P1054. A. Remazeilles, C. Leroux , G. Chalubert, “SAM: a robotic butler for handicapped people”, 17th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN - 2008), 01/08/2008-03/08/2008, Munich, Germany. V. Vidal, H. Geffner, “Branching and Pruning: An Optimal Temporal POCL Planner based on Constraint Programming”, Artificial Intelligence 170 (3), pp. 298-335, 2006. H. Vorobieva, C. Leroux, P. Hède, M. Soury, P. Morignot, “Object Recognition and Ontology for Manipulation with an Assistant Robot”, In Proceedings of the Eighth International Conference on Smart Homes and Health Telematics (ICOST’10), L.N.C.S., Aging Friendly Technology for Health and Independence, Springer, Berlin Heidelberg, Germany, vol. 6159, 2010, pages 178-185.

[23] J. Wolfe, B. Marthi, S. Russell, “Combined Task and Motion Planning for Mobile Manipulation”. International Conference on Automated Planning and Scheduling, Toronto, Canada, 2010.

Implementing a Conversational Robotic Agent? Jim Cunningham, Imperial College London [email protected] Abstract This short and informal paper briefly discusses the problem of understanding natural language by machine and how it might be solved for a future home-help robot. The success of the IBM Watson Supercomputer in the Jeopardy quiz can be contrasted with the needs of a person in conversation with a home help robot. Yet massive concurrency was critical for the success of "Watson", and it may be critical for the efficiency of a conversational robot. And from the slow "evolution" of modern computers a pattern of convergence to a more brain-like structure seems to be emerging. But this presents a problem, there is not yet a matching programming system for software architectures; we need a new paradigm.

Natural Language Understanding David Harel concludes his classical 1987 book on "Algorithmics" with a light hearted but gloomy prognosis of the problem of understanding natural language by machine. He recommends first a better grip on the knowledge representation problem, but he also suggests highly concurrent, possibly probabilistic, non-heuristic procedures which can be analysed in order to achieve acceptable intelligence [1]. In February 2011 the IBM "Watson" supercomputer system defeated two former human winners of the popular quiz show "Jeopardy", answering questions posed in natural language text without connection to the internet during the game. So Harel's gloomy prognosis can arguably be lifted. Watson used nearly 3000 high-specification processor cores and 16 Terabytes of RAM (about 8 domestic refrigerators in size), running "massively parallel" (but linux-based) software, on even more massive textual information sources (dictionaries, encylopedias, news articles, ontologies, databases, etc). It used alternative formal analyses of the question and exploited game-relevant statistical analyses to generate, test and rank hypothetic responses, then decide whether to answer, all within a few seconds. But as we can expect today's supercomputers to be matched by more mundane machines within a decade, the resources used need only be a temporary barrier to progress. The "IBM Watson" site redirects to Wikipedia, so for more detail see "http://en.wikipedia.org/wiki/Watson_(computer)". On reflection, many of Harel's predictions have been matched in the analytic methods used in Watson for Jeopardy, although with relatively greater exploitation of the Bayesian statistical methods which became computationally practical in the 1990s. The shift in the balance of technology is only in part a distortion of the quiz environment. So by using first chess and then a popular quiz to spur applied development IBM have surely dispelled doubts that Artificial Intelligence is a capable of master-level human intellectual achievement, and from proposed future applications of similar technology in medical diagnosis and elsewhere a beneficial social impact may be expected. But this is not yet conversation with a machine which contributes to and experiences its own events within a shared environment. Rather it is question-and-answer communication with a computational oracle able to analyse a question

and access all humanly transcribed information for relevance in providing an answer. At least for purposes of the Jeopardy quiz, Watson did not hear or speak, nor had it other environmental sensors or actuators that a robot needs. The perception of words from sound, objects from vision, and their actuator duals in real "human" time each present massive computational problems which are only partially solved today. And these are mere fragments of a long and layered list of tasks for which the brains and nervous systems of animals have evolved robust methods to enable each individual to interact with its natural environment. In contrast, the technology by which computers can interact with us in our own environment has been severely limited by our poor ability to solve the associated problems (or at least to organize and fund their solution on a human time scale). But problems associated with the recognition and generation of a spoken words do now have at least limited, finitary solutions. And because of their importance these speech technologies are slowly becoming integrated with domains of deeper language use, typically by using many different ontologies, each one specific to the tasks of one intended application, such as for booking a table at restaurant. Having been interested, off and on over many years, in the problem of natural language understanding by machines I have recognized like others that humans can adapt to a small lexicon and a restricted environment for the purposes of interaction with a machine. But it remains difficult to imagine anything but frustration in speaking with a future home-help robot without the machine having some behavioural empathy, and the ability to moderate and learn new behaviour as a result of a conversation. However unlikely, if the home-help were merely a vacuum cleaner with some ability to listen and talk, we might expect it not only to recognize and name the room it is in and objects within it, but to be sympathetic to our spoken wishes when expressed using notions like delicate or fragile object, and to behave accordingly. But then new perceptual problems other than those of speech arise within the auditary domain. First the mere presence of each interacting agent may need to be recognized by its sound, without visual recognition, and be distinguished from other background noise. Then a contextual meaning is needed for words like delicate and fragile. This seems to demand both haptic sensory perception and an imagined perception of temporal consequence from physical interaction. So even if a robot's perceptors and actuators can be enhanced for speech, its conversation and learning require more of a mind.

Computational System Architectures Various software architectures for autonomous intelligent agents have been developed over the past 20 years. These seek to integrate the computational recognition of percepts from sensors, and to guide the actions of affectors in accordance with some agent design model. These agent models are mostly based on symbolic representations, and conventional software, but an important class of non-symbolic models are mentioned in the next section. In many well known symbolic models the underlying architecture is also a simple processing model of mind, expressed in terms of mental percepts like beliefs, goals, plans, intentions, commitments, schedules, duty, etc. Such architectures are relatively easily extended to embrace robotic agents with multi-media sensors and actuators, and are capable of realization with on a robotic architecture like ROS[2] where ad-hoc system-level parallelism is supported by an internal network of processing nodes. It is arguable that the capacity for symbolic software agents to learn and adapt is not well developed, but the embodied architectual concepts enscapsulate human experimentation to find general methods, so local realizations provide adaptation, and peripheral modules can be based on technologies that allow training.

So learning from conversation and high level instruction is quite feasible but needs implementation before it can be explored. In allusion to a criticism of Watson attributed to the philosopher John Searle, at least some of these software agent agents would know if they won Jeopardy! Despite the massive technology of Watson there are remain many contentious issues in the underlying design of modern information processing systems. The processing chips from which today’s computers are built have developed from the macro-circuits of 50 years ago through many generations of successive enhancement and commercial competition. This itself amounts to a weak form of evolution, during which some earlier attempts at predominantly concurrent variants of computation have been rejected in favour of the traditional predominantly serial systems with only local concurrency. The concurrency of Watson in Jeopardy was of the order of 104, but much of that has only become possible in recent years as networks of processors have become practical and further compaction of nano-scale chips has become harder to achieve. Watson’s parallelism was mainly deployed when independently exploring alternative hypotheses after initial analyses of each question. Computer operating systems and programming have seen even less change over time, so networked concurrency is conservatively managed. The Watson operating system is derived from 1970's Unix[3] while ROS is also realized as a networked variant of a few modern Unix variants. Watson's main programming languages are also few and familiar, chiefly Java, C++, and Prolog [4-6]. This absence of radical features has helped ensure that local processing has robust foundations, while the statistical methods used in programs for exploring alternative hypotheses helped ensure a reliable overall outcome from concurrent processing threads. But we can still question why an information processing system should have this particular structure, as its not obvious that our brains do! At least in memory elements (1012) the scale of Watson comparable to number of neurons in the human brain (1011), but neurons also provide synapses (1014) and perform processing, with only a few layers in the human cortex where sequentiality dominates. But if the brain wins on processors by a factor of 107 it is slower by a similar factor, so it is in organization rather than any simple measure of complexity that is the main difference. But the organization of the brain seems relatively uniform: regions of differing detail are linked by a complex network. Is it then a situated, "one place for one concept" architecture being realized this way that is the over-looked contribution to efficiency? One can reasonably postulate that a form of "situatedness" is an important contribution to word meaning, enabling direct association with learned (or inate) realizations of sense percepts and composed actions (actuator processes) rather than a textual record to be found and interpreted. Moreover, if as seems likely, conscious reasoning about the use of percepts and processes can partly re-use those same local processing facilities that are needed in actual use, and from a more remote cortical location, perhaps suppressing normal output in favour of recognising when it would occur, then there would seem to be further efficiency in avoiding symbolic copy. And as such reasoning is then physically introspective, the higher brain will know what it knows and what it does not, a critical aspect of faithful conversation.

A More Neuromorphic Computing Paradigm The most enduring non-symbolic models of information processing are Artificial Neural Networks (ANNs), originally based on the McCulloch–Pitts model of a neuron as a multiinput single output gate triggered by a linear threshold function[7]. ANNs are now a

sophisticated technological field with industrial application in devices that can be trained for pattern matching, and with a theoretical framework giving scientific models for testing fundamental issues in neuroscience. But as an practical information processing model ANNs have severe limitations, not least the expense and inconvenience of development, which more akin to the description of hardware for a finitary system than conventional programming. Their strength as systems that can learn also comes at the cost of training, whereas conventional computer systems are designed to a full functional specification (however flawed). The problem of scale is also a considerable barrier for ANN models in moving from relatively primitive systems to those of Jeopardy complexity, which seem to need a humanscale brain. Nevertheless, an appropriately structured ANN as an information processing model for an adaptive robotic agent, equipped with ANN sensors and effectors, remains an neuromorphic model for the brain and peripheral nervous system of a animal in its natural environment[8,9]. It is not obvious that it is necessary for a computer to model neurons and synapses in such detail in order to reflect the information flow between brain regions. A network of addressible processing domains allows a similar, if coarse, neuromorphic structure of situated regional processing. Of course this ignores the role of synapse change and other plasticity for learning and adaption in the brain, but at least there is a superficial convergence between the relative emphasis on processor networks in modern supercomputer architectures and current scientific concern with the connectome of the human brain[10]. And exploiting this convergence may be crucial for understanding how to develop a conversational agent. Unfortunately, even if we knew the role of each brain region at an appropriate level of granularity, or had a design for networked concurrency with which to experiment using our imperfect ontological knowledge of the relations between semantic concepts, it presents a programming problem. Multi-threading and its associated real-time event management is an Achilles heel for conventional programming, an ill-structured bolt-on for otherwise well designed programming languages. The chances of safely managing this degree of concurrent interaction between interdependent semantic concepts with present programming technology is slim indeed. But from where can we seek inspiration to overcome this problem? In presenting an early version of this paper I mentioned an experiment in a different interpretation of logic programming, thinking of each rule as being independently capable of "firing", but when its data is available, as a right to left or "data-driven" execution rather than a left to right "query-driven" search. My informal model was of the rule executing as programming thread, which is not unlike the firing of a neuron, but where the threads need to be coordinated. An a non-trivial exercise I programmed a (recursive) odd-even merge sort, also from Harel's book, mentioned earlier. It then seemed I had a multi-threading software architecture variant of a hardware definition language, but with different temporal combinators. It also seemed conceptually more elegant than a typical hardware or software architecture description language because the modular units were rules, not larger modules, and so the fine data flow was explicable. But it turns out that at the level of the example an even more elegant hardware description language based on the functional programming language Haskell has already been described and published with the same odd-even sort as one of a series of examples to show the data flow and hardware timing[11]. So detail of an analagous approach based multi-threading must await more careful analysis on its validity. And even if a practical basis for highly concurrent reactive data-flow programming across a large network of contextually meaningful processes can be developed (or grown), it will need enriching to enable learning and adaptation. These too have precedent within the

programming world. So perhaps by the time the human scientific world is sure it knows how the brain works, its software technologists will know how to emulate it properly.

References 1. D. Harel, ALGORITHMICS: The Spirit of Computing, Addison-Wesley 1987. 2. M.Quigley, B.Gerkey, K. Conley et al., ROS: an open-source Robot Operating System, ICRA Workshop on Open Source Software 2009. 3. D. M. Ritchie and K. Thompson, The Unix Time-sharing System, C. ACM 17 No. 7 1974. 4. Gosling, James; Joy, Bill; Steele, Guy L., Jr.; Bracha, Gilad (2005). The Java Language Specification (3rd ed.). Addison-Wesley. ISBN 0-321-24678-0. 5. Stroustrup, Bjarne (1997). "1". The C++ Programming Language (Third ed.) ISBN 0201889544. 6. Colmerauer, A.; Roussel, A. (1993). The birth of Prolog. ACM SIGPLAN Notices 28 (3): 37. 7. McCulloch, W. and Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 7:115 - 133. 8. E. Ruppin, Evolutionary AutonomousAgents: A Neuroscience Perspective. Nature Reviews Neuroscience, 3(2), 2002, 132-141. 9. A. Proekt, J. Wong, Y. Zhurov, N. Kozlova, K.R. Weiss, et al., Predicting Adaptive Behavior in the Environment from Central Nervous System Dynamics. PLoS ONE 3(11): 2008. 10. Sporns O., Tononi G., Kötter R. , The human connectome: A structural description of the human brain. PLoS Comput Biol 1(4): 2005. 11. K. Claesson, M.Sheeran, S.Singh, The Design and Verification of a Sorter Core, Proc. CHARME '01, 11th IFIP WG 10.5 Advanced Research Conference on Correct Hardware Design and Verification Methods. Springer-Verlag London, UK, 2001.

First Steps towards the Modelling of Peace through a Multi-Agent approach Jean Erceau1 and Laurent Chaudron2 1: Cognitive Guild AC1032, 8 rue Pottier, FR-78280 Guyancourt, France. [email protected] 2: ONERA, Provence Research Center FR-13661 Salon Air, France. [email protected]

 

7KHVWDNHV

&RPPHQWLQJ7KHQHZ5HSRUWRI&,$+RZZLOOEHWKHZRUOGLQSUHVHQWHGE\$OH[DQGHU$GOHU6DPXHO1JXHPERFN>@ZDUQHG VRKLVSXEOLFWRVSHDNDERXWWKHSHDFHLQWKLVQHZFHQWXU\VHHPVWREHDGLIILFXOWH[HUFLVHDQGDSUDFWLFHVRPXFKWKHFKDOOHQJHVDUH HQRUPRXVVRPXFKWKHULVNVDQGWKHWKUHDWVZKLFKZDWFKIRUWKHZKROHSODQHWDUHPXOWLSOH 3HDFH VWXGLHV ODXQFKHG LQ WKH V KDG DV REMHFWLYHV WR IRUHVHH FULVHV DQG FRQIOLFWV WKDW PD\ DIIHFW WKH SHDFH LQ RUGHU WR DYRLG WKHP,WZDVQRWWRVWXG\WKHFRQGLWLRQVRISHDFHEXWWKHFRQGLWLRQVRIQRSHDFH7KH)DOORIWKH%HUOLQ:DOOPDUNHGWKHIDLOXUHRIWKLV DSSURDFKDQGVFLHQWLILFWRROVXVHG7KHSHDFHVWXGLHVZHUHWKHQUHGLUHFWHGDQGIRFXVHGRQVHFXULW\:K\WKLVVKLIW"%HFDXVHRQRQH KDQGLQWULQVLFDOO\FRQQHFWHGWRWKHNQRZOHGJHWRWKHFHUWDLQW\WRWKHUDWLRQDOLW\DQGWKHWHFKQRORJ\VHFXULW\LVDILHOGIDYRUDEOHIRUWKH FODVVLFVFLHQWLILFDSSURDFKHVDQGEHFDXVHRQWKHRWKHUKDQGUHTXLULQJUXOHVDQGODZVVHFXULW\EHFRPHVDQDIIDLURIVWDWH$VDVVHUWV 0LFKDHO'LOORQLQKLVERRN 3ROLWLFVRI6HFXULW\ >@VHFXULW\EHFDPHWKHIRXQGDWLRQRIWKHPRGHUQSROLWLFDOWKRXJKW%XWLIVHFXULW\LVD IDFWRURISHDFHVHFXULW\LVQRWSHDFH 8QGHU WKHVH FRQGLWLRQV WKH IUDJLOLW\ RI HTXLOLEULXP DQG WKH YXOQHUDELOLW\ RI FHUWDLQ DFWRUV PDNH PDMRU IRU GHFDGHV WR FRPH WKH GHYHORSPHQWRIDSURJUDPRIKLJKOHYHOH[SOLFLWLQWKHQDPHWKH3HDFH(QJLQHHULQJ3URJUDP 3HDFHLVQRWWKHRSSRVLWHRIZDUDQGYLFHYHUVDOLNHKHDOWKLVQRWWKHRSSRVLWHRIWKHGLVHDVHDQGYLFHYHUVD:HFDQ¶WWUHDWIRUSHDFH DQG KHDOWK OLNH ZH WUHDW IRU ZDU DQG IRU GLVHDVH 7KH 3HDFH LV QRW WKH RSSRVLWH RI WKH ZDU DQG PXWXDOO\ DV WKH KHDOWK LV QRW WKH RSSRVLWH RI WKH GLVHDVH DQG PXWXDOO\ 7R GHDO ZLWKWKH SHDFH ZH VXJJHVW GHILQLQJ UHDO RQH SDUDGLJP RI WKH SHDFH FHQWHUHG RQ WKH SHDFHZLWKGHILQLWLRQVDQGREMHFWVSUHFLVHUHIHUHQWVZLWKDSHUWLQHQWDSSURDFKDQGDQDGDSWHGV\VWHPDWLFORJLF 

6RPHSUHOLPLQDU\GpILQLWLRQV

$FRXQWU\LVDJHRJUDSKLFDODQG WHUULWRULDOHQWLW\3HRSOHLVDPXOWLWXGHRILQGLYLGXDOVOLYLQJLQWKHVDPHFRXQWU\DQGXQGHU WKHVDPH ODZV$QDWLRQLVDPXOWLWXGHRILQGLYLGXDOVKDYLQJDVXVWDLQDEOHZLOOWROLYHZLWKWKHVDPHGXUDELOLW\ZLWKLQWKHVDPHJURXSURRWHGRQD FRPSOH[LQWHUWZLQLQJQHWRIKLVWRULFFXOWXUDOOLQJXLVWLFOLQNVSRVVLEO\UHOLJLRXVZKLFKWKHIHHOLQJRIDFRPPRQPHPEHUVKLS+RZHYHU WKHLGHDRIQDWLRQLVQRWUHGXFLEOHLQWKHVHOLQNV ,Q RUGHU WR LPSRVH LWVHOI DQG SHUSHWXDWH LWVHOI ERWK LQ DQ HQYLURQPHQW DOZD\V PRUH RU OHVV KRVWLOH WKH ELUWK RI D QDWLRQ WKH ZRUG QDWLRQFRPHVIURPWKHQDVFLRRUQDWLR/DWLQZKLFKPHDQVEHLQJERUQ LVJXDUDQWHHGE\DVRFLDOFRQWUDFWLPSOLFLWRUH[SOLFLWO\ZULWWHQ  PDGHEHWZHHQWKHPHPEHUVRIWKHQDWLRQ7KLVFRQWUDFWLVDWULSW\FKILJ FRQVLVWLQJRIDIRXQGLQJWH[WVDYDOXHVV\VWHPH[SUHVVHG DPRQJRWKHUVE\DVORJDQDFXUUHQF\DIODJDQGDK\PQ DQGRIDV\VWHPRIUXOHVWKHIXQFWLRQRIZKLFKLVWRDVVXUHWKHRUGHUWKH KDUPRQ\ DQG WKH MXVWLFH 7KLV FRQWUDFW FRQIHUV RQ WKH QDWLRQ LWV LGHQWLW\ DQG LWV VRYHUHLJQW\ LQ WKH SXUSRVH RI VRFLDO FRKHVLRQ DXWRQRP\DQGVXVWDLQDELOLW\7KHQDWLRQLVDQDOLYHVRFLDOHQWLW\ZKLFKLVERUQZKLFKOLYHVFDQEHVLFNDQGVRPHWLPHFDQGLH $6WDWHLVDSROLWLFDODGPLQLVWUDWLYHDQGOHJDOVWUXFWXUHWRZKLFKWKHQDWLRQHQWUXVWVLWVVRYHUHLJQW\E\DVVLJQLQJWRLWWKHGXW\WRPDNHLW UHVSHFWHGRXWVLGHDQGLQVLGHVRDVLWVLGHQWLW\DQGWKLVVRYHUHLJQW\$FWXDOO\WKH6WDWHILQGVLWVWHUULWRULDOOLPLWVWKHUHZKHUHLWVODZVGR QRWDSSO\DQ\PRUH 7KHQDWLRQLVDWRQFHDQLGHRORJLFDOFRQVWUXFWLRQVXEMHFWLYLW\ DQGDPDWHULDOUHDOLW\REMHFWLYLW\ WKLVDWRQFHFUHDWHVDOLQNHQWHUV VXEMHFWLYHDQGREMHFWLYHZKRH[SODLQVWKHGLIILFXOW\GHILQLQJLWOHJDOO\DQGWRJLYHLWDFRPSOHWHO\UDWLRQDOGHILQLWLRQ7KLVQRQUDWLRQDOLW\ LVLPSOLFLWO\UHFRJQL]HGE\WKHYHU\QDWLRQZKHQLWGHOHJDWHVLWVVRYHUHLJQW\WKDWLVWKHH[FOXVLYHULJKWWRH[HUFLVHWKHSROLWLFDODXWKRULW\ RQDJLYHQJHRJUDSKLFDO]RQHWRDUDWLRQDOVWUXFWXUHZKLFKZHDUVWKHQDPHRI6WDWH7KLVLVLQWKHQRQUDWLRQDOLW\ UHTXLUHPHQW RI YLUWXRXV EHKDYLRU DQG D FDUULHU HWKLFV RI WKH YDOXHV RI WKH FRQWUDFW  WKDW WKH QDWLRQ DV SROLWLFDO FRQVWUXFWLRQZLOOPDNHUHFRJQL]HGWKHDXWKRULW\RIWKH6WDWH7KHFODVVLFDOWKHRU\RIWKHLQWHUQDWLRQDOODZUHFRJQL]HVDV MXULGLFDOVXEMHFWRQO\DVRYHUHLJQVWDWHSDUWLFXODUO\IRUZDU 7KH WHUP FRQIOLFW EHWZHHQ WZR HQWLWLHV H[SUHVVHV D VLWXDWLRQ RI RSSRVLWLRQ FRQVHFXWLYH WR LQWHUHVWV RU RSSRVLWH IHHOLQJV $ FRQIOLFW LV UHVROYHG HLWKHU E\ DQ DJUHHPHQW UHVXOWLQJ IURP D QHJRWLDWLRQ RU E\ D ZDU 7KH WHUP RI FULVLV LQGLFDWHV LQ DQ HQWLW\ JURXSLQJ KXPDQ EHLQJV ORQJODVWLQJ G\VIXQFWLRQ EHLQJ DEOH WR FDXVH LUUHYHUVLEOH ORVVHV RI EDODQFH $ FULVLV LV RIWHQ D FRPSOH[ LQWHUWZLQLQJ RI ODWHQW FRQIOLFWV $ FULVLV FDQ EH WUDQVIRUPHG LQWR FRQIOLFW LI RQH RI WKHPDSSHDUVH[SOLFLWO\DQGZLWKVWUHQJWK 

1DWLRQYHUVXV6WDWH

%\WKHWULSW\FKRILWVELUWKWKH1DWLRQLVDFRQFHSWZKLFKFDUULHVLQLWVHOIYDOXHVRIWROHUDQFHVROLGDULW\MXVWLFHZKLFKDUHIDFWRUVRI SHDFHDQGRIWROLYHWRJHWKHU7KLVFRQFHSWLVDWWKHRULJLQRIWKH6RFLHW\RI1DWLRQVZKLFKKDVEHHQUHSODFHGE\WKH2UJDQL]DWLRQRI 8QLWHG 1DWLRQV ZLWK PLVVLRQ WR SUHVHUYH SHDFH RQ WKH SODQHW 7KH 6WDWH DV IRU LW LV HVVHQWLDOO\ D FRQFHSW RI SRZHU DQG DXWKRULW\ 2ULJLQDOO\LQWHQGHGWRSURWHFWWKHFLWL]HQVDQGWRFROOHFWWKHWD[WKHVWDWHGLGQRWVWRSJURZLQJRYHUWKHFHQWXULHV+RZHYHUVLQFHWKH HQGRIVXQGHUWKHFRPELQHGHIIHFWRIWKHJOREDOL]DWLRQSXOOLQJDQDVFHQGDQF\RIWKHHFRQRPLFDQGWKHILQDQFHRQSROLWLFV DQG RI WKH (XURSHDQ FRQVWUXFWLRQ 6WDWHV ORVH D SDUW RI WKHLU SRZHU DQG RI WKHLU VRYHUHLJQW\ ZKHUHDV XQGHU WKH HIIHFWV RI WKH  ^ĂĨĞƚLJŝƐƌĂƚŚĞƌƉƌŽƚĞĐƚŝŽŶĂŐĂŝŶƐƚĂĐĐŝĚĞŶƚĂůĞǀĞŶƚƐ͖ƐĞĐƵƌŝƚLJŝƐƌĂƚŚĞƌƉƌŽƚĞĐƚŝŽŶĂŐĂŝŶƐƚŝŶƚĞŶƚŝŽŶĂůĚĂŵĂŐĞƐ͘

ϭ

ϭ



KHWHURJHQHL]DWLRQRIWKHSRSXODWLRQVDQGWKHGHJUDGDWLRQHYHQWKHORVVRIFHUWDLQPDMRUIDFWRUVRIWKHVRFLDOFRKHVLRQWKHQDWLRQVHHV ZHDNHQLQJLWVLGHQWLW\DQGE\WKHLQWHUSRVHGVWDWHLWVVRYHUHLJQW\ 6RFRPSOLFDWHWKHPVHOYHVUHODWLRQVKLSVEHWZHHQQDWLRQDQGVWDWHZKDWPDNHVWKHPPRUHGLIILFXOWWRFRQWUROLQWKHH[HUFLVHRISRZHU DQGWKHUHVSHFWRIWKHDXWKRULW\7KLVFRPSOH[LW\LVSRWHQWLDOO\VSULQJRIFRQIOLFWVDQGHYHQXSULVLQJV7RDYRLGWKHPQRERG\FDQDYRLG WKLQNLQJWKDWHYROXWLRQRIWKHFRQFHSWVRIQDWLRQDQGVWDWHDQGDUHGHILQLQJRIWKHVWUXFWXUHVRIJRYHUQDQFHDUHSRVVLEOH )LQDOO\ZDULVDPDWWHURIVWUXFWXUHVDQDIIDLURIVWDWHZKLOHSHDFHLVDPDWWHURIRUJDQL]DWLRQVDPDWWHURIQDWLRQ$FRXQWU\LVPRUH WKDQDJHRJUDSKLFDOGHVLJQDWLRQDQGWHUULWRULDOWKDWLVDFRXSOHIRUPHGE\WKHVWDWHDQGWKHQDWLRQZKLFKEH\RQGLWVVRYHUHLJQW\WHQGV WRHQWUXVWWRLWLWVGHVWLQ\,QDFRXQWU\OLNHLQDFRXSOHSHDFHLVJXDUDQWHHGE\WKHVROLGDULW\RIWZRSDUWVWKH6WDWHDQG1DWLRQ7KHLU GLYRUFHLVDOZD\VDVRXUFHRIFRQIOLFWDQGUHYROXWLRQ 

2XUDSSURDFKRIWKH3HDFHDKHDOWKVWDWHZLWKDXWRSRLHWLFVVSHFLILFLWLHV

:RUOG LV D JHQHULF QDPH LQGLFDWLQJ JOREDOO\ WKH KHWHURJHQHRXV HQVHPEOH RI HQWLWLHV OLNH SHRSOHV HWKQLF JURXSV QDWLRQV VWDWHV JRYHUQPHQWV LQVWLWXWLRQV FRPSDQLHV DVVRFLDWLRQV RUJDQL]DWLRQV UHOLJLRQV « 7KH 6WDWH LV DQ LQVWLWXWLRQ RI LQVWLWXWLRQV SHUPDQHQWO\ LQWHUDFWLQJ HDFK RI WKHP KDYLQJ GLIIHUHQW XQLYHUVHV RI VHQVH DQG RI SUDFWLFHV 0RUH WKDQ WKDW WKH ZRUOG DV WKH KXPDQ ERG\ LV D V\VWHPRIV\VWHPVZKLFKILWRQHLQWRWKHRWKHUVDVWKH5XVVLDQGROOVHYHU\V\VWHPLVDQHOHPHQWRIDV\VWHPRIXSSHUOHYHOWKHZRUOG DQGWKHKXPDQERG\DUHWKHV\VWHPRIWKHKLJKHVWOHYHO7KDWWKH\DUHWKHVDPHOHYHORURIQHLJKERULQJOHYHOVWKHVHV\VWHPVDUHLQ PXWXDOLQWHUDFWLRQV7KLVDOORZVWRVHHWKHZRUOGDVDPXOWLDJHQWV\VWHPHDFKDJHQWEHLQJLWVHOIDPXOWLDJHQWV\VWHP :HFRQVLGHUWKDWSHDFHOLNHKHDOWKLVDVWDWHRIWREHRIDV\VWHPRIGHILQLWHOHYHODVWDWHEDODQFHGEHWZHHQWKHHOHPHQWVRIDVDPH OHYHO,WKDVWREHFRQVLGHUHGDVDVSHFLILFLW\EHLQJDWRQFHRIHYHU\HQWLW\DQGRILWVLQWHUDFWLRQVZLWKWKHRWKHUHQWLWLHVFRQFHUQHG7KLV VWDWH LV QHYHU D VWDWH GHILQLWLYHO\ HVWDEOLVKHG WKDW LV ZK\ ZH FDQQRW VSHDN DERXW WKH SHDFH OLNH DERXW VRPHWKLQJ RI GHILQLWLYH VROLG VXVWDLQDEOHDQGMXVW$WHYHU\OHYHOZHGHILQHWKHSHDFHDVDQDOLYHYLUWXDOHQWLW\RIZKLFKDXWRSRLHWLFVSHFLILFLWLHV>@DUHIHGE\DVHW RI YLUWXRXV LQWHQWLRQDOLW\ ,W LV D FRQVWUXFWLRQ ZKLFK JURZV ULFK VWUHQJWKHQV DQG LV WUDQVIRUPHG E\ WKH LQGLYLGXDO FRQWULEXWLRQV DQG PXWXDOO\LWHQULFKHVDQGWUDQVIRUPVWKRVHZKRIHHGLWDQGFRPHWRIHHGWKHPVHOYHVIURPLW  7KHVSKHUHRILQWHQWLRQDOLW\RIDFROOHFWLYHFDQEHGHILQHGDVWKHQVSDFHRIQGLPHQVLRQVLQZKLFKWKHLQWHQWLRQDOLW\RI WKHHOHPHQWV LQWHUDFWZLWK WKHRWKHUV7KHYLUWXRXVLQWHQWLRQDOLW\FROOHFWLYHLVWKHVXEVSDFHRIWKHQYLUWXRXVLQGLYLGXDOLQWHQWLRQDOLWLHVHDFKGHHSO\ PDUNHG E\ WKH YDOXHV RI WKH WULSW\FK DQG UHVSHFWIXO RI LWV UXOHV DQG DOVR RI WKH ZLOOLQJQHVV UHJLVWHUHG LQ LWV SURMHFW 7KH EHKDYLRU YLUWXRXV FRRSHUDWLYH DQG FRFRQVWUXFWLYH JLYHV WR WKH FROOHFWLYH DOO WKH KRPHRVWDWLF FDSDFLWLHV DOORZLQJ LW WR HQGXUH DQG WR UHVLVW WR LQWHQWLRQDOLW\QHXWUDORULQRSSRVLWLRQ6RDQDWLRQFDQEHLQSHDFHVRWKDWH[LVWRQLWVWHUULWRU\VRPHORFDODQGWHPSRUDU\FRQIOLFWV 7KH SHDFH HQJLQHHULQJ FDQ EH WKHQ GHILQHG DV DOO WKH PHDQV PDWHULDO LQIRUPDWLYH UHODWLRQDO SV\FKRORJLFDO FXOWXUDO  ZKLFK FDQ EH DFWLYDWHGE\WKH6WDWHIRUVSHFLI\WKHLQGLYLGXDO LQWHQWLRQDOLW\LQ WHUPVRIYLUWXHVDQGRULHQWWKHPWRFRFRQVWUXFWLRQWKHPDLQWDLQLQJ DQGWKHHQULFKPHQWRIRQHYLUWXRXVFROOHFWLYHLQWHQWLRQDOLW\DEOHWRPDLQWDLQLQLWVHOIWKHSHDFHDVDQDOLYHDQGDXWRSRLHWLFHQWLW\7KH VSDFHRIWKHSURFHVVHVUHDOL]LQJWKHDXWRSRLHVLVRIWKHSHDFHLVWKHVSDFHRIWKHYLUWXRXVFROOHFWLYHLQWHQWLRQDOLW\,WLVHYLGHQWWKDWWKH SHDFHZLOOEHHDVLHUWRHVWDEOLVKDQGWRPDLQWDLQLIWKHVSDFHRIWKHYLUWXRXVFROOHFWLYHLQWHQWLRQDOLW\FRYHUVWKHPDMRULW\RIWKHVSKHUH LQWHQWLRQDORIWKHZKROHFROOHFWLYH



6RFLDOG\QDPLFVYLUWXRXVLQWHQWLRQDOLW\DQGKRPHRVWDVLV

,QDV\VWHPRIDJHQWWKHRZQG\QDPLFVRIHYHU\DJHQWH[SUHVVHVLWVHOIWKURXJKDVXFFHVVLRQRIVWDWHVGHWHUPLQHGE\SURFHVVHVDFWLQJ DFFRUGLQJWRUXOHVDQGODZVZKLFKDIIHFWWKHLULQWHQWLRQDOLW\DQGFRQVHTXHQWO\GHWHUPLQHWKHLUEHKDYLRU 6RPHEHKDYLRUVDUHSLORWHGE\LQWHQWLRQDOLWLHVZKLFKKDYHWUHQGWRVHSDUDWHDQGGLIIHUHQWLDWHWKHDJHQWV7KHVHLQWHQWLRQDOLWLHVDQG EHKDYLRUVDUHDFWLQJKHWHURJHQHRXVO\2QWKHRWKHUKDQGRWKHUEHKDYLRUVDUHGHWHUPLQHGDQGJXLGHGE\VRPHLQWHQWLRQDOLWLHVZKLFK WHQG WR JDWKHU WKH DJHQWV FORVHU WKH RQHV IURP WKH RWKHUV DQG WR PDNH WKHP PRUH DQG PRUH VLPLODU XQWLO WR PDNH WKHP HQRXJK LGHQWLFDO DQG DEOH WR PHUJH :H VKDOO VD\ WKDW WKLV EHKDYLRUV DQG WKHVH ODZV ZRUN KRPRJHQRXVO\ %\ EHKDYLQJ KHWHURJHQHRXVO\ D V\VWHPLQFUHDVHVLWVHQWURS\DQGE\EHKDYLQJKRPRJHQRXVO\LWLQFUHDVHVLWVQHJHQWURS\ 6RLQDV\VWHPZKHUHWKHLQWHUDFWLRQVDUHQRWDEOHWREHEHLQJUHGXFHGLIQRSURFHVVLVDEOHWRZRUNKHWHURJHQHRXVO\LQRSSRVLWLRQWR KRPRJHQHRXVSURFHVVHVWKHDJHQWVDUHJRLQJWRHQGJDWKHUHGWRJHWKHUDQGPHUJHGLQWRRQHXQLTXHHQWLW\7KHV\VWHPZLOOORVHLWV OLIHG\QDPLFVDQGGLHVE\H[FHVVRIKRPRJHQHLW\$WFRQWUDU\LIQRSURFHVVLVDEOHWRZRUNKRPRJHQHRXVO\E\RSSRVLWLRQWRSURFHVVHV ZRUNLQJ KHWHURJHQHRXVO\ LQ D G\QDPLFV RI GLVSHUVDO WKH DJHQWV EHFDPH KDYLQJ WHQG WR SXVK WKHPVHOYHVPXWXDOO\WKHRQHVIURPWKHRWKHUVDQGWRVHSDUDWHWKHPVHOYHVDQGHYHQWRH[FOXGH WKHPVHOYHV WKH V\VWHP ORVHV LWV OLIH G\QDPLFV LW ORVHV LWV LGHQWLW\ RI V\VWHP DQG KH GLHV E\ H[FHVV RI KHWHURJHQHRXVQHVV ,W LV HYLGHQW WKDW WKH VWDWHV ZKLFK JLYH H[FOXVLYHO\ SULRULW\ WR SURFHVVHV DFWLQJ LQ RQH KRPRJHQHRXV PDQQHU ZLWKRXW UHVSHFWLQJ DQ\ LQGLYLGXDO LQWHQWLRQQDOLWLHVOLPLWE\WKHUHHYHQWKHLUOLIHH[SHFWDQF\7KLVZD\LQFUHDVHVWRWKHH[WUHPH WKHSRWHQWLDOL]DWLRQRIFRQIOLFWVZKLFKE\WKHLUDFWXDOL]DWLRQZLOOEHFRPHSRVVLEO\UHYROXWLRQV 2Q WKH RWKHU KDQG WKH VWDWHV ZKLFK DV WKH UHDO GHPRFUDFLHV JLYH WKHPVHOYHV DV PLVVLRQ WR ILQG HTXLOLEULXPV EHWZHHQ KRPRJHQL]DWLRQ DQG KHWHURJHQHL]DWLRQ JLYH WKHPVHOYHV WKH SRVVLELOLWLHVRIDVXVWDLQDEOHGHYHORSPHQW 7KHUHIRUH WKH VLQH TXD QRQ VR WKDW D V\VWHP FDQ H[LVW DQG UHPDLQ D V\VWHP LV WR EH DW WKH VDPH WLPH  VXEPLWWHG WR D VHW RI KRPRJHQHRXV  DQG KHWHURJHQHRXV G\QDPLFV  ZKLFK DFW VLPXOWDQHRXVO\ DQG PDLQWDLQV WKH V\VWHP LQ D VWDWH RI EDODQFH WR ZKLFK GLIIHUHQWQDPHVDUHJLYHQIRUH[DPSOHDWWUDFWRUPHWDEROLVPRUPHGLXP  Ϯ  /ĨŝŶƚĞŶƚŝŽŶŐƵŝĚĞƐŽƵƌƐƉĞŬŝŶŐĂŶĚŽƵƌĂĐƚƐ ƚŽǁĂƌĚƐĂŶŝŵƉůŝĐŝƚŽƌĞdžƉůŝĐŝƚŽďũĞĐƚŝǀĞ͕ ƚŚĞŝŶƚĞŶƚŝŽŶĂůŝƚLJŐƵŝĚĞƐĂŶĚůŽĂĚƐƚŚĞŝƌƉƌŽũĞĐƚŝŽŶŝŶŽƵƌ ĞŶǀŝƌŽŶŵĞŶƚ͕ĂŶĚŝŶƚŚĞǁŽƌůĚ͕ǁŝƚŚƚŚĞďĞƐƚĨƌŽŵŽƵƌƐĞůĨ͕ƚŚĂƚŝƐƚŽƐĂLJǁŝƚŚƚŚĞǀĂůƵĞƐ͕ǁŝƚŚƚŚĞŬŶŽǁůĞĚŐĞ͕ǁŝƚŚƚŚĞĨĂŝƚŚ͕ǁŝƚŚƚŚĞĨĞĞůŝŶŐƐ͕ǁŝƚŚ ƚŚĞĐƵůƚƵƌĞ͕ĂŶĚƚŚĞĂŝŵ͕ǁŚŽƐĞǁĞĂƌĞďĞĂƌĞƌ͘ Ϯ



$OO WKLV FDQ UHPDLQ SXUHO\ WKHRUHWLFDO EHFDXVH WKH SUDFWLFH RI WKH YLUWXHV DQG WKH HWKLFV DUH QRW LQQDWH 7KHUH LV WKXV LQ DQ\ VRFLDO V\VWHP JRYHUQV E\ D WULSW\FK WKH QHFHVVDU\ SUDFWLFH RI DQ HGXFDWLRQ DQG D VSHFLILF VXSSRUW VR WKDW HYHU\ QHZFRPHU OHDUQV WKH HTXLOLEULXPRIOLIHRIWKHV\VWHPDQGNHHSDZDUHQHVVRILWDWWKHULJKWOHYHO7KHQKHZLOOEHDFWLYHDQGUHVSRQVLEOHKRSLQJWREHFRPH RQH GD\ RQH DFWRU RI WKH EHKDYLRU SURFHVV RI WKH OLIH RI WKH V\VWHP LQ WKH UHVSHFW RI LWV UXOHV DQG RI LWV YDOXHV DQG LQ WKHLU WUDQVPLVVLRQ7KLVHGXFDWLRQDQGWKLVVXSSRUWKDYHWREHPDGHLQWKHRSHQLQJRIWKHFRQVFLRXVQHVV¶VDWWKHULJKWOHYHOZLWKRXWZKLFK WKHLQWHQWLRQDOLW\ZRXOGQRWNQRZKRZWRLQWHJUDWHWKHYDOXHVXQWLOWRUHDFKHWKLFDOEHKDYLRUVHQDEOLQJWKHFRQVWUXFWLRQRIWKHSHDFH WRJHWKHUDQGLQVSLWHRIRWKHUV 

$XWRSRLHVLVDQGYLUWXRXVEHKDYLRU

,Q DXWRSRLHWLF V\VWHPV WKLV VSHFLILFLW\ LV GXH WR WKUHH VWURQJO\ FRQQHFWHG SURFHVVHV DQG JRYHUQHG E\ WKHP 7KH HIILFLHQF\ RI WKHVH SURFHVVHVUHTXLUHVYLUWXRXVEHKDYLRUVIURPDJHQWVZKRKDYHWKHUHVSRQVLELOLW\WRRSHUDWHWKHP%XWZKHQDJHQWVDUHKXPDQEHLQJV WKHVHYLUWXRXVEHKDYLRUVDUHQRWLQQDWHDQGIRUWRUHDFKWKHULJKWOHYHORIFRPSHWHQFHVWKHV\VWHPPXVWWREHVWUXFWXUHGZLWKWKUHH OHYHOV)URPDJHQHUDOPDQQHUWKHVHWKUHHOHYHOVLQWURGXFHWKUHHFODVVHVRIDJHQWVZHVKDOOQDPH$&DQG0WKHFODVVRIWKHOHDUQHU DJHQWVWKHFODVVRIWKHRSHUDWLYHDJHQWVWKHFODVVRIPDQDJHUDJHQWV ,QDUULYLQJLQDQHZMRELQDQHZVLWHLQDQHZVLWXDWLRQLQDQHZRUJDQL]DWLRQHDFKKDVWRIHHOKLPVHOIDVDOHDUQHUDQGKDVWREH WDNHQDVDOHDUQHU7RXQGHUWDNHLWLVQHFHVVDU\WRNQRZDQGIRUWRZHDYHDQGIRUWRDFFXPXODWHNQRZOHGJHVLWLVQHFHVVDU\WRKDYH OHDUQW WR OHDUQ 7KH PHWDNQRZOHGJH LV NQRZOHGJH RQ NQRZOHGJH ,W DOORZV XV WR RUJDQL]H RXU NQRZOHGJH WR EXLOG D VWUDWHJ\ DQG D SROLWLFVWRIDFHDVLWXDWLRQDQGWRVROYHDSUREOHPFULVLVRUFRQIOLFW 7RPDQDJHNQRZOHGJHGHYHORSVWUDWHJLHVLPSOHPHQWSROLFLHVLWLV QHFHVVDU\ WR KDYH PHWDNQRZOHGJH JDLQHG IURP D ORW RI H[SHULHQFHV +RZHYHU DOO WKH RSHUDWLYH DJHQWV KDYH WR FRQVLGHU WKDW WKH\ EHORQJDOZD\VWRWKHFODVVRIOHDUQHUVDQGRIWKHVDPHPDQQHUDOOPDQDJHUVKDYHWRFRQVLGHUWKDWWKH\EHORQJDOZD\VWRWKHFODVVRI OHDUQHUVDQGWRWKHFODVVRIRSHUDWLYHDJHQWV:HKDYHDOZD\VVRPHWKLQJWROHDUQDQGWROLYHWKURXJKRSHUDWLRQDOH[SHULHQFHVLQRUGHU WRJDLQDQGDFFXPXODWHNQRZOHGJHDQGPHWDNQRZOHGJHV 7RJLYHWRDPXOWLDJHQWV\VWHPWKHDXWRSRLHWLFVVSHFLILFLWLHVLWLVQHFHVVDU\WRVWUXFWXUHGLWRQWKUHHOHYHOVDQGWRPDNHRSHUDWHRQ WKHVHOHYHOVWKHWKUHHSURFHVVHVDV)UDQFHVFR9DUHODHWDO>@KDYHGHILQHG ,QDVRFLDOV\VWHP IRXQGHG RQD WULSW\FKILJ WKHWKUHHSURFHVVHVWRZKLFKZHJLYHWKH QDPHV3333LVWKHSURFHVVRI FRRSWDWLRQZKLFKDOORZVDQDJHQWRIWRLQWHJUDWHWKHFODVVHVRQHDIWHUWKHRWKHU7KLVSURFHVVLVDFWLYDWHGDIWHULQTXLULHVLQWHUYLHZV LQVSHFWLRQVVRPHWLPHVQXPHURXVIRUH[DPSOHZKHQDSHUVRQDVNVWRLQWHJUDWHD³KLJKVFKRRO´RURQH%RDUGRIGLUHFWRUVRUZKHQD FRXQWU\DVNVWRLQWHJUDWH(XUR]RQH81(6&2RUWKH812%HFDXVHWKHSURFHVV3NHHSVFORVHG³WKHGRRU´WRHQWHUDFODVVDQGRSHQVLW ZKHQLWUHFHLYHGWKHRUGHUKHDVVXUHVWKHRSHUDWLRQDOFORVXUHRIWKHFODVV 3LVDSURFHVVRIVSHFLILFDWLRQ,QHYHU\FODVVLWDOORZVHDFKDJHQWWRJHWWKHUHTXLUHGVSHFLILFLWLHVE\DSSO\LQJDPHWKRG7KLVSURFHVV 3KDVWRJXDUDQWHHWKHTXDOLW\OHYHORIWKHHQYLURQPHQWDQGWKH EHVWDFTXLVLWLRQRIWKHVSHFLILFLWLHV,QHYHU\FODVV$&0 WKHVH VSHFLILFLWLHVDUHH[SOLFLWNQRZOHGJHV.$.&.0 DQGYDOXHVDQGYLUWXHV9$9&90 RIWKHWULSW\FKSURSRVHGLQDFRUSXV(DFKDJHQWKDV WR LQFRUSRUDWH LW ZLWK RQH REMHFWLYH 2$2& 20  WKH SHUVRQDO LPSURYLQJ E\ D YLUWXRXV SUDFWLFH RI WKH PHWKRG RI DFTXLVLWLRQ RI WKH FRUSXV 7KLV PHWKRG DOORZV HDFK DJHQW RI WR DVVLPLODWH WKHP LQ RUGHU WR HQULFK LWV LQGLYLGXDO LQWHQWLRQDOLW\ DQG E\ WKHUH HYHQ WKH YLUWXRXV TXDOLW\ RI LWV FRPPLWPHQWV DQG RI LWV EHKDYLRXUV SDUWLFXODU\WKRVHQHFHVVDU\WRPDNHWKHSURFHVVHVHIILFLHQW  %XW WKH PHWKRG EULQJV PRUH WR HDFK DJHQW ,I LQ WKH FRUSXV RI FODVV $ DUH JLYHQ VRPH H[SOLFLW NQRZOHGJHVDVZHOODVUXOHVDQGFULWHULRQVWRUHVSHFWLQWKHFODVVHV&DQG0WKHPHWKRGSHUPLWV WRHDFKDJHQWWRJDLQDQGDFFXPXODWHWDFLWNQRZOHGJHVFRPSOHPHQWLQJWKHFRUSXV,QFODVV0WKH OHYHO RI WKH NQRZOHGJH DQG WKH SUDFWLFH RI WKH PHWKRG OHDGV WR WKH HODERUDWLRQ RI PHWD NQRZOHGJH 6R FRQVFLRXV WR EHORQJ WR WKH WKUHH FODVVHV DJHQWV RI FODVV 0 DUH DEOH WR PDNH RSHUDWLRQDOLQWKHEHVWPDQQHUWKHWKUHHSURFHVVHV333WKHODVWRQHEHLQJUHIOH[LYH ,QHYHU\FODVVWKHVSHFLILFLWLHVRIHYHU\DJHQWFDQEHPHDVXUHGHLWKHUFRQWLQXRXVO\E\DIROORZRIWKHUHVXOWVLQFRPSDULVRQZLWKVRPH VLJQLILFDQW LQGLFDWRUV GHILQHG D SULRUL RU E\ WKH SDVVDJH RI VSHFLILF WHVWV SDUWLFXODUO\ SHUWLQHQW 7KH UHVXOWV DOORZ WR PHDVXUH WKH FRQVLVWHQF\EHWZHHQWKHHOHPHQWVRIWKHSURSRVHGFRUSXVDQGWKHLUGHJUHHRIDVVLPLODWLRQE\WKHDJHQW 7KDWLVWRVD\WKDWRIWKLVPDQQHUDQRWHPD\EHJLYHQWRHDFKDJHQWQRWHZKLFKUHIOHFWVWKHOHYHORIWKHSRWHQWLDORILWVPHQWDOLWV JRYHUQPHQWDO SURJUDP LI WKH DJHQW LV D VWDWH  DQG RI LWV EHKDYLRUDO UHVRXUFHV ILQDQFLDO HFRQRPLF DQG SURGXFWLYH IRU D 6WDWH  )RU H[DPSOHDQGOLNHLWLVNQRZQWKHVWDWHVVHHHDFKRWKHUDOORFDWLQJDQRWH$$$RU$$$ZKLFKLQGLFDWHVWKHFRQVLVWHQF\EHWZHHQWKH UHDOLW\RIWKHUHVRXUFHVDQGWKHHOHPHQWVRIWKHFRUSXVLPSRVHGE\WKHWUHDW\RI0DDVWULFKW 7KHFRPSRQHQWVRI3DUHQRWH[SOLFLWUXOHVOLNHLWLVIRU3DQG37KHUXOHVRI3DUHWDFLWEHFDXVHWKH\ DUHPHWDNQRZOHGJH/LNHWDFLWNQRZOHGJHPHWDNQRZOHGJHVDUHSHUVRQDO7KH\DUHEXLOWE\HDFKE\KLP HYHQLQKLPHYHQLQWKHDZDUHQHVVRIWKHFRQWHQWVRIWKHSHUVRQDODQGFROOHFWLYHLQWHQWLRQDOLW\DQGLQWKH DZDUHQHVVRIWKHSXUSRVHWKHHODERUDWHGPHWDNQRZOHGJH+HUHLVWKHHWKLFVPXVWEHSRLQWHG7KDWVDLG LI3DQG3FDQEHZULWWHQH[SOLFLWO\DQGPRVWO\WKLVLVULJKW LQWKHIRUPRISURFHGXUHVDQGUHJXODWLRQVLW LV QRW WKH VDPH IRU 3 3 FDQ QRW EH ZULWWHQ H[SOLFLWO\ DQG FDQ QRW EH VXEMHFW WR HLWKHU D OHDUQLQJ RU WHDFKLQJ D PHPHQWR RI HGXFDWLRQ 3 LV D VHW RI PHWDNQRZOHGJH 3 LV D VHW RI UXOHV DV SHUVRQDO DQG LQWLPDWHLQWHJUDWHGLQWRHDFKRIXVFRQVFLRXVO\RUXQFRQVFLRXVO\7KHVHUXOHVHQDEOHUHVSRQVLEOHEHKDYLRU DQG YDOXHV DFTXLUHG YLUWXHV WKHVH UXOHV GHILQH LQWHQWLRQDOLW\ )RU D VWDWH WKH\ GHILQH JRYHUQPHQW SROLF\ )RU WKH QDWLRQ WKH\ GHILQH FROOHFWLYHLQWHQWLRQDOLW\ 7KH WKUHH SURFHVVHV ZKLFK JLYH WR WKH V\VWHP LWV DXWRSRwHWLF VSHFLILFLWLHV DUH VWURQJO\ ERXQG 7KH TXDOLW\ DQG WKH HIILFLHQF\ RI WKHLU LPSOHPHQWDWLRQDQGRIWKHLURSHUDWLRQDOHIIHFWLYHQHVVLVDVVXUHGE\37KHTXDOLW\DQGWKHHIILFLHQF\RI3DUHDVVXUHGE\WKHTXDOLW\ OHYHORIWKHLQGLYLGXDOLQWHQWLRQDOLW\RIWKRVHZKRKDYHWRDVVXUHWKHHIILFLHQF\RI3DQG3DQG37KHVWUHQJWKRIWKHDXWRSRLHVLVRID ϯ



V\VWHP LV D IXQFWLRQ DW WKH VDPH WLPH RI WKH FRQVLVWHQF\ RI WKH FROOHFWLYH LQWHQWLRQDOLW\ DQG RI WKH FRKHUHQFH RI WKH LQGLYLGXDO LQWHQWLRQDOLW\WKDWLVWRVD\RIWKHVWUHQJWKDQGRIWKHTXDOLW\RIWKHFRPPLWPHQWRIWKHLQGLYLGXDOVDQGRIWKHFROOHFWLYH 

0XOWLDJHQWFRQFHSWXDOL]DWLRQ  )LUVWVWHSVWRZDUGVDIRUPDOPRGHOIRUSHDFH

$V D VWDUWLQJ SRLQW OHW XV FRQVLGHU WKH VLPSOLILHGGHILQLWLRQ RI SHDFH ³3HDFH LV   WKH DEVHQFH RI ZDU RU FRQIOLFW   WKH SUHVHQFH RI YLUWXRXV LQWHQWLRQDOLW\ ,Q D PXOWLDJHQW FRQFHSWXDO IUDPHZRUN DQG WKDQNV WR WKH SUHYLRXV VHFWLRQV ZH GHILQH D WZR VWHSV PRGHO   FRQVLGHUWKHQHJDWLRQRIWKHIRUPDOGHILQLWLRQRIFRQIOLFW GHILQHDPLQLPDOIUDPHIRUWKHDXWRSRLHVLV7KHSUHUHTXLVLWHIRUDFRQIOLFWLV WKDW VHYHUDO DJHQWV DUH FRQFHUQHG 6R GRHV EH WKH FRQFHSW RI SHDFH D SHDFH VWDWH LQ PHDQLQJIXO LII PDQ\ DW OHDVW   DJHQWV DUH FRQFHUQHG/HWXVFRQVLGHUWKDWHDFKRIWKHPIHOODQGFDQH[SUHVVDQGVKDUHNQRZOHGJHLWHPV  6WHS 'HILQLWLRQRI3HDFH /HWXVVXPPDUL]HWKHGHILQLWLRQRIDFRQIOLFWDVDVHWRILQWHQWLRQDOLW\LWHPVUHSRUWLQJWRJLYHQDJHQWV WKDWLVQRWFRKHUHQWDQGWKHIDFW WKDWLWLVQRWFRKHUHQWPDWWHUV7KHLQFRQVLVWHQF\RIDVHWLWHPVPD\UHO\RQYDULRXVUHTXLUHPHQWV *LYHQ 6  ^$L`L∈, D VHW RI DJHQWV 6 FRXOG EH D VHW RI FRXQWULHV HJ WKH 8QLWHG 1DWLRQV 6\VWHP  D FRXQWU\ LH WKH VHW RI WKH LQGLYLGXDOV DFLW\DWHDP)RUHDFKDJHQW$L.L ∪L∈-.LMLGWKHVHWRIKHUKLVLQWHQWLRQDOLW\LWHPVLQD%',FODVVLFDOIUDPHZRUN.L LVWKHJOREDOVHWRIWKHEHOLHIGHVLUHLQWHQWLRQVHWVRI$L $IXQFWLRQPL.LÆ^`FDSWXUHVZKDWPDWWHUVIRUKHUKLPQRWHWKHVHWRI ³PDWWHU´ PHDVXUH FDQ EH FRQWLQXRXV RI SURMHFWHG LQ WKH IX]]\VHW WKHRU\ EXW D ELQDU\ HYDOXDWLRQ LV VXIILFLHQW DW WKLV VWHS  7KH FROOHFWLYHLQWHQWLRQDOLW\RI6LV. ∪L∈,.L $JOREDOIXQFWLRQRUDTXDOLWDWLYHFRQGLWLRQ &.Æ^`GHILQHVZKDWLVFRQVLVWHQWUHVSLQFRQVLVWHQW  7LPHLVDSDUDPHWHUIRUHDFKHOHPHQWVRIWKHPRGHODQGLVQRW\HWQRWHG$WWKLVVWHSWKHXQLRQLQWHUVHFWLRQRSHUDWRUVDUHQRWGHWDLOHG EXW LW PXVW EH FOHDU WKDW WKH SUHFRQGLWLRQ RI WKLV PRGHO LV WKDW WKH LQWHQWLRQDOLW\ VSDFH . FRXOG EH HTXLSSHG ZLWK D ODWWLFH VWUXFWXUH SURYLGLQJWKHVXSUHPXPLQILPXPRSHUDWRUVWKXVWKHLQWHQWLRQDOLW\LWHPVFRXOGEHGHILQHGZLWKLQDILUVWRUGHUORJLFRURWKHUULFKHUORJLF IUDPHV  +HQFHVWDWHRI3HDFHDQG&RRSHUDWLRQDWWLPHWLVFRQFHSWXDOO\EXLOWRQWKHPHUHFRQFHSWXDOGHILQLWLRQRIZKDWDFRQIOLFWLVQRW 'HILQLWLRQWKHVHW6 ^$L`L∈,LVLQDµ3HDFHDQG&RRSHUDWLRQ¶VWDWHLIWKHIROORZLQJVXEVHWRI.0 ∩L∈,PL LVFRQVLVWHQW ,QQDWXUDOODQJXDJHDVHWRIDJHQWLVLQWKHPLQLPDOVWDWHRI SHDFHLIWKHLQWHQWLRQDOLW\LWHPVWKH\VKDUHDQGWKDWPDWWHUVIRUWKHPLV FRKHUHQW 7KHSUHYLRXVVHFWLRQGHYHORSHGWKHG\QDPLFFRQGLWLRQV6PXVWSHUIRUPVRDVWRJXDUDQWHHWKHFRQWLQXDWLRQRIVXFKDVWDWH7KLVPHDQV WKDWWKH DQG VXEVWHSVRIWKHSHDFHPRGHOZLOOUHO\RQH[SOLFLWWKHUROHRIWLPHWDQGIRUPDOL]HWKHGLIIHUHQWLDOSURSHUWLHVRI6W  DQG 0W  )URP WKH SUDJPDWLF SRLQW RI YLHZ WKH DJHQWV PXVW DFW VR DV WR HQULFK WKH 0W  WKDQNV WR G0W  ZKLFK PXVW RULHQW WKHLU DFWLYLWLHV  6WHS 7KHDXWRSRLHVLVRI3HDFH &RQVHTXHQWO\WKHDXWRSRLHVLVRIWKHVHW6PXVWEHEDVHGRQDVWUXFWXUDODQGDFWLYHFRQVWLWXWLRQRIDFROOHFWLYHSURJUDPRIDOOWKHDJHQWV ^$L`L∈,,Q>@LWKDVEHHQIRUPDOO\SURYHGWKHIHDVLELOLW\RIDFRPPRQSURJUDP6 RIZRUNZLWKLQDPXOWLDJHQWFRPPXQLW\UHSUHVHQWLQJ WKH DXWRSRLHVLV E\ DJJUHJDWLRQ RI WKH DJHQWV UHVSRQGLQJ WR D VHW RI VRFLHWDO FKDOOHQJHV (  ^(Q`Q∈1  DQG FRQVLVWHQW ZLWK WKHLU RZQ LQWHQWLRQDOLW\ LWHPV 7KH PRGHO LV EDVHG RQ WKH *DORLV ODWWLFHV WKHRU\ DQG DVVRFLDWLRQ UXOHV WHFKQLTXHV 7KLV PRGHO RI DXWRSRLHVLV E\ DJJUHJDWLRQLVSURJUDPPHGLQFRQVWUDLQHGORJLFSURJUDPPLQJODQJXDJH7KLVZLOOQRWEHPRUHGHWDLOHGKHUH 

7RZDUGVWKH0RGHOOLQJRI3HDFH

(YHQLQWKHKLJKHVWFLUFOHVRIWKHVFLHQWLILFJRYHUQDQFHIHZSHUVRQVNQRZWKDWRQHRIWKHPRVWJHQHUDODQGPRVWFRPSOHWHGRFXPHQWV WRWKHZRUOGGHDOLQJZLWKWKHVFLHQWLILFUHVHDUFKLVDWH[WRIWKH81(6&2GDWLQJ©UHFRPPHQGDWLRQFRQFHUQLQJWKHFRQGLWLRQRIWKH VFLHQWLILFUHVHDUFKHUVLQ1RYHPEHUWK3DULV 7KLVGRFXPHQWLVRIDELJZHDOWKDQGLWVZULWHUVHQOLJKWHQHGWKHGHFDGHVLQD SDUWLFXODUO\ILQHDQGGHHSZD\DUWLFOHVVWD\RIDPRVWO\WRWDOPRGHUQLW\DQGIRUVRPHUHJUHWWDEO\QRWHQRXJKSURJUHVVZHUHUHDOL]HGLQ VRRQIRUW\\HDUV 7KH WLWOH ,, SDUWLFXODUO\ KROGV DWWHQWLRQ WKH VFLHQWLILF UHVHDUFKHUV DQG WKH HODERUDWLRQ RI WKH QDWLRQDO SROLWLFV ZKLFK EHJLQV ZLWK WKH DUWLFOH ³ (YHU\ 6WDWH PHPEHU VKRXOG WU\ KDUG WR SXW WKH VFLHQWLILF DQG WHFKQRORJLFDO NQRZOHGJH LQ WKH VHUYLFH RI WKH LPSURYHPHQW RI WKH FXOWXUDO DQG PDWHULDO ZHOOEHLQJ RI WKH QDWLRQDOV DQG WR ZRUN LQ IDYRXU RI WKH LGHDOV DQG WKH REMHFWLYHV RI 1DWLRQV GHQ\ )RU WKHVH SXUSRVHV HYHU\ 0HPEHU VWDWH VKRXOG EH HTXLSSHG ZLWK WKH VWDII WKH LQVWLWXWLRQV DQG WKH QHFHVVDU\ PHFKDQLVPV WR HODERUDWH DQG RSHUDWH QDWLRQDO VFLHQWLILF DQG WHFKQRORJLFDO SROLWLFV WR GLUHFW WKH HIIRUWV RI VFLHQWLILF UHVHDUFK RI H[SHULPHQWDO GHYHORSPHQW VR DV WR UHDFKWKHQDWLRQDOREMHFWLYHVZKLOHPDNLQJDLPSRUWDQWDQGVXIILFLHQWSODFHWRWKHVFLHQFH%\WKHSROLWLFVZKLFKWKH\DGRSWWRZDUGVWKH VFLHQFH DQG WRZDUGV WKH WHFKQRORJ\ E\ WKH ZD\ WKH\ XVH WKHP WR HODERUDWH WKHLU GHFLVLRQV RI JHQHUDO SROLWLFV LQ SDUWLFXODU E\ WKHLU DWWLWXGHLQUHVSHFWRIWKHVFLHQWLILFUHVHDUFKHUV6WDWHVPHPEHUVKRXOGGHPRQVWUDWHWKDWWKHVFLHQWLILFDQGWHFKQRORJLFDODFWLYLWLHVDUH QRWWKHRQHVZKRKDYHWRSUDFWLFHLQWKHLVRODWLRQEXWWKDWWKH\MRLQWKHJOREDOHIIRUWRIQDWLRQVWRHVWDEOLVKDQGPDLQWDLQDVRFLHW\ PRUHKXPDQDQGUHDOO\MXVW´ 7KLVZRUGLQJSXWVXSFKDOOHQJHZLWKRXWDPELJXLW\IRUDOOLQWHUQDWLRQDOFRPPXQLWLHVWRIRXQGVFLHQWLILFSROLFLHVRQVWUXFWXUHGVWHSV DQGFDOOVVRDOOOHYHOVRIWKHVRFLHWLHVWRWKHLUUHVSRQVLELOLWLHV

ϰ



%\ SURFHVVHV ZKLFK LQVWUXPHQWDOLVWV LW LW EHDUV WKH HQJLQHHULQJ RI WKH DXWRSRLHVLV DQG RI WKH KRPHRVWDV\ ZKLFK VKH FRQIHUV WR WKH FROOHFWLYH 5HIOH[LYH DQG UHFXUVLYH WKHVH SURFHVVHV VWD\ PXWXDOO\ LQ HIIHFWLYHQHVV E\ HQULFKLQJ DQG UHFKDUJLQJ WKHLU HQHUJ\ SHUPDQHQWO\ LQ WKH FROOHFWLYH LQWHQWLRQDOLW\ WDNHQ DV D 2QH ,W LV LQ WKH FLUFXODULW\ RI WKLV UHIOH[LYLW\ WKDW WKH\DXWRPDLQWDLQDQGVXSSRUWSHUPDQHQWO\WKHFROOHFWLYHLQVHOIJRYHUQDQFHDQG LWVFDSDFLWLHVRIVHOI UHIHUHQFHRIVHOIRULHQWDWLRQRIVHOISHUSHWXDWLRQDQGRIVHOIUHSURGXFWLRQ 7KH WKHRU\ RI DXWRSRLHVLV LV QRW UHYROXWLRQDU\ DV SURFODLPV LW $OHVVLR 0RUHWWL EXW LQ QXPHURXV GRPDLQV LW KDV DOUHDG\ FKDQJHG FODVVLFDO DSSURDFKHV 7KH HQJLQHHULQJ RI DXWRSRLHVLV DQG KRPHRVWDV\ GHVHUYHV GHHSHQHG VWXGLHV RI ZKLFK RULHQWDWLRQGHPDQGVWRRXURSLQLRQRIWRDGRSWDQRWKHUORJLFRIDSSURDFK 0RGHUQLW\ OHDQHG RQ FDXVDOLW\ DQG $ULVWRWH :KDW GURYH WR D JURZWK ZKLFK LQ WKLV ORJLF DQG LQ WKH WKHRUHWLFDO DEVHQFH RI FRQVHQVXV DV LQFOXGHGWKLUG KDG RIWHQ HYDGHG IURP HTXLOLEULXPV E\ GHVSLVLQJ WKH RSSRVLWLRQV DQG E\ LJQRULQJ WKHP DQG HOLPLQDWLQJWKHP7KLVJURZWKILQGVLWVOLPLWVQRZDGD\V © 3RVW PRGHUQLW\ ª DQQRXQFHV LWVHOI LQ WKH DFFHSWDWLRQ RI WKHVH OLPLWV DQG LQ WKH UHVHDUFK RI YLWDO HTXLOLEULXPV DW WKHVDPHWLPHIRUWKHLQGLYLGXDOVDQGV\VWHPVLQZKLFKWKH\OLYHZKHWKHULWLVIRUVRFLDOV\VWHPVRUQDWXUDOV\VWHPV 7KH SDVVDJH WR RQH HFRORJLF DV WKH RQH RI (PSpGRFOH DQG RI /XSDVFR GHVHUYH GHHSHQHG VWXGLHV 7KLV ORJLF WDNHV DQWDJRQLVWLF WHQGHQFLHV DQG LQWHUHVWV ZLWK FRQVLGHUDWLRQ DQG VHL]HV WKHP DV DQ RSSRUWXQLW\ IRU WR PDNH WKHP PXWXDOO\ IHFXQG  )RU LWV LPSOHPHQWDWLRQ HQJLQHHULQJ RI DXWRSRLVHVLV DQG RI KRPHRVWDV\ GHPDQGV WKLV FKDQJH RI ORJLF ZKLFK WKURXJKWKHFRQVHQVXVRSHQVQHZZD\VIRUUHFLSURFDOWUXVWEHWZHHQWKHKXPDQEHLQJVLQWKHYHUWLFDOGLPHQVLRQDVLQWKH LQIUDKRUL]RQWDOGLPHQVLRQEHWZHHQWKHLQGLYLGXDOVZKRKDYHWRFROODERUDWHLQWRDUHDVVKDUHGRIWKHIUHHGRP :KHQ D6WDWHWKDW LV WRVD\ DW OHDVW LWV OHDGHU  DSSURSULDWHVSRZHUDQGVRYHUHLJQW\ DIWHUKDYLQJFRQILVFDWHG WKHP IURP WKH 1DWLRQ ZKLFK HQWUXVWHG WR WKHP WKH FRXQWU\ HQWHUV D ORJLF RI WKH H[FOXGHGWKLUG  ,VQ W LW HYLGHQW WKDW LQ WKH UHFHQW KLVWRU\RIWKHFRXQWULHVWKHDFWXDOLW\FRQILUPVLW WKHRFFXSDWLRQE\WKHSHRSOHRIWKHFHQWHUFLW\VTXDUHVVKRXOGQRWEH VHHQDVVHHNLQJRIRQHLQFOXGHGWKLUG" ³3HDFH(QJLQHHULQJ3URJUDP´ FDQ RQO\EULQJ PHDQV WR ILQG DFRQVHQVXV RQH LQFOXGHGWKLUGEXWLWZLOO DOZD\VEHQHHGHG WRKDYHDZLOODQGRQHULJKWHRXVFROOHFWLYHLQWHQWLRQDOLW\WRPDLQWDLQLWLQOLIHLQDQDXWRSRLHWLFPDQQHU 5HIHUHQFHV

 $'/(5$©/HQRXYHDX5DSSRUWGHOD&,$&RPPHQWVHUDOHPRQGHHQª5REHUW/DIIRQW3DULV  &+$8'521 / (5&($8 - 0$8*$56 ' ³&ROODERUDWLYH 6HOI2ULHQWDWLRQ RI 5HVHDUFK EDVHG RQ $XWRSRLHVLV DQG /DWWLFHV ´ ,QWHUQDWLRQDO &RQIHUHQFH '209, 'HFDGH RI WKH 0LQG &RJQLWLYH 6FLHQFH DQG 1HXURWHFKQRORJLHV /RRNLQJ IRUZDUG WR WKH QH[WWHQ\HDUV2FWREHU6LQJDSRUH

 &+$8'521 / (5&($8 - ³/D 3DL[  FRQFHSW SKDUH SRXU O¶DXWR RULHQWDWLRQ ³ $XWRRULHQWDWLRQ :RUNVKRS ±  GpF  ± (FROH 3RO\WHFKQLTXH±3DODLVHDX±)UDQFH

 ',//210³3ROLWLFVRI6HFXULW\WRZDUGVDSROLWLFDOSKLORVRSK\RIFRQWLQHQWDOWKRXJKW´5RXWOHGJH(GWQV/RQGRQ  (5&($8-©$XWRRULHQWDWLRQHWDXWRSpUHQQLVDWLRQLQJpQLHULHGHO¶ DXWRSRwqVH«ª$XWRRULHQWDWLRQ:RUNVKRS±GpF±(FROH 3RO\WHFKQLTXH±3DODLVHDX±)UDQFH

 +$(5,3DXO©'HODJXHUUHjODSDL[3DFLILFDWLRQHWVWDELOLVDWLRQSRVWFRQIOLWª(FRQRPLFD  7(66,(5&&+$8'521/DQG0h//(5./8:(5+-³&RQIOLFWLQJDJHQWV&RQIOLFWPDQDJHPHQWLQPXOWLDJHQWV\VWHPV´ ,6%1

 .$17(©9HUVODSDL[SHUSpWXHOOHªqUHpGLWLRQHQDOOHPDQGHQWUDG-)3RLULHU )3URXVW*))ODPPDULRQ  /83$6&2 6 ©/ pQHUJLHHW ODPDWLqUHYLYDQWHª -XOOLDUG 3DULV   qPH pGLWLRQ /H 5RFKHU &ROO / HVSULW HW OD PDWLqUH 3DULV

 /83$6&263V\FKLVPHHWVRFLRORJLH&DVWHUPDQ3DULV  0251( - ©&RXUV VXU OH WKqPH GH OD SDL[  .DQW HW OD SDL[ SHUSpWXHOOHª KWWSSLHUUHFDPSLRQIUHHIUPRUQHMNDQWKWP 

 1*8(0%2&. 6 IpYULHU  KWWSZZZLUHQHHVQHWIUILFKHVGRFXPHQWDWLRQILFKHGRFXPHQWDWLRQKWPO   ,UHQHHVQHW 6LWHZHEGH5HVVRXUFHVSRXUOD3DL[ 

 3217=(17K&©4XHOOHV3DL["&RPPHQW"ª&OXE+XPDQLVWH3DULV  86 1DWLRQDO ,QWHOOLJHQFH &RXQFLO ³*OREDO 7UHQG $ 7UDQVIRUPHG :RUOG´  ,6%1  ±  ±  ±     1RY KWWSZZZGQLJRYQLF3')BB*OREDOB7UHQGVB)LQDOB5HSRUWSGI

 9$5(/$ ) + 0$785$1$ DQG 5 85,%( ³$XWRSRLHVLV 7KH 2UJDQL]DWLRQ RI /LYLQJ 6\VWHPV ,WV &KDUDFWHUL]DWLRQ DQG D 0RGHO %LRV\VWHPV´B9RO >$PVWHUGDP1RUWK+ROODQG@

ϱ



Optimal Control of Discrete Processes and Dynamic Games with Discounted Costs Dmitrii Lozovanu, Stefan Pickl Institute of Mathematics and Computer Science, Academy of Sciences, Academy str., 5, Kishinev, MD-2028, Moldova Institut fur Angewandte Systemwissenschaften und Wirtschaftsinformatik, Fakultaet fur Informatik, Universitaet der Bundeswehr, Muenchen

Abstract We study a class of dynamic games on networks applying the game-theoretical concept to infinite horizon discrete optimal control problems with discounted costs. Necessary and sufficient condition for determining Nash equilibria in the consid-ered games on networks are derived and algorithms for determining the optimal stationary strategies of the players are proposed. Key words: Discrete processes, Optimal control, Dynamic games on networks, Stochastic positional games, Nash equilibria, Optimal stationary strategies PACS: 65K05, 68W25

1

Introduction and Problem Formulation

In this paper we formulate and study a class of dynamic games on networks using the game-theoretical concept to the following infinite horizon discrete control problem. Let L be a time-discrete system with finite set of states X. At every discrete moment of time t = 0, 1, 2, . . . the state of L is x(t) { X. The dynamics of the system is described by a directed graph of states transitions G = (X, E), where the set of vertices X corresponds to the set of states and an arbitrary directed edge e = (x, y) { E expresses the possibility of system L to pass from the state x = x(t) to the state y = x(t + 1) at every discrete momentof time t. So, a directed edge e = (x, y) in G corresponds to a stationary control of the system in the state x { X, which provide a transition from x = x(t) to y = x(t + 1) for every discrete moment of time t. We assume that graph G doesn't contains deadlock vertices, i.e. for each x { X there exists at least one leaving directed edge e = (x, y) { E. In addition we assume that to each e = (x, y) { E a quantity c e is associated which expresses the cost of the system L to pass from the state x = x(t) to the state y = x(t+1) for every t = 0 , 1 , 2 , . . . . A sequence of directed edges E' = {e0, e2, . . ., et,... }, where e t = (x(t),x(t + 1)) { E, t = 0, 1, 2, ... corresponds to a control of system L with fixed starting state x0 = x(0). This control uniquely deter mines a trajectory x 0 = x(0), x(1), x(2), ..., x(t),. . for which the total discounted cost Email address: [email protected], (Dmitrii Lozovanu, Stefan Pickl).

[email protected] 1

( )

∞

Fx0 E = ¦λ t c( x(t),x(t+1)) '

t =0

with given discount factor λ (0 < λ < 1) can be defined. In [1,3,5] is shown that if G is strongly connected then for an arbitrary starting state x0 { X there exists the optimal control E * = {e0* = (x0 , x* (1) ), e1* = (x* (1), x* (2) ), ... }, for which ∞

Fx0 ( E * ) =

¦λ c t

min

et

{( x ( 0 ), x (1) ),..., ( x ( t ), x ( t +1) )...} t =0

and this optimal control doesn't depend on starting state as well it doesn't depend on time (however the value Fx0 E * depends on starting state). There-fore the problem of determining the optimal control in G can be formulated in the terms of stationary strategies. A stationary strategy s for considered problem is defined as a map

( )

s:x e y { X(x)

for

x { X,

where X(x) = {z { X| e = (x, z) { E}. Taking into account that G is a finite graph we obtain that S = {s : x e y { X (x) for x { X} is a finite set. An arbitrary strategy s ∈ S for a fixed starting state x0 = x(0) uniquely determine a sequence of states x(t + 1) = s(x(t)), t = 0,1, 2,..., that corresponds to a control E s = {et = (xt , x(t + 1 ) ) , t = 0,1, 2,... }. Thus, we can define Fx0 (s ) = Fx 0 E s and the control problem in G with given discount factor λ we can formulate as the problem of finding the stationary strategy s* for which

( )

( )

Fx0 s* = min Fx0 (s ) s∈S

This problem have been studied in [1, 3, 5] and polynomial time algorithms for its solving have been proposed. Moreover in [2, 4] the considered problem have been extended for the stochastic case and algorithms for solving the stochastic version of the control problem have been grounded. In this paper we consider the game variant of the problem formulated above. We assume that the set of states X is divided into m nonempty disjoint subsets X 1 , X2 , . . ., X m (X 1  X2  …  X m = ∅), where each subset X i is regarded as position set of player i ∈ {1, 2, . . . , m}. Additionally to each directed edge e = ( x , y ) ∈ E we associate m costs c1e , ce2 , ..., cem ,where cei expresses the cost of system's transition from the state x = x(t) to the state y = x(t + 1) for the player i ∈ {1, 2,..., m}. For considered game model we have the following behaviour of dynamical system. If x0 = x(0) ∈ Xi then the control at the moment of time t = 0 in the state x(0) is made by player i. After that the system passes to the next state and we obtain the state x(1). If the system at the moment of time t = 1 is in the state x(1) ∈ X k then the control is made by player X k and so on indefinitely. In this game we assume that the players use stationary strategies. The stationary strategies of the player we define as m maps: s i : x e y ∈ X(x)

x ∈ Xi

for

i = 1, 2,... ,m.

Let s 1 , s 2 ,..., s m be a set of strategies of players 1 , 2 , . . . ,m. Then the situation s = (s 1 , s 2 ,... , s m ) corresponds to a stationary control of system L in G with fixed starting state x 0 = x(0) and given discount factor λ (0 < λ < 1) . Therefore for the 2

players 1 , 2 , . . . ,m, we can define the corresponding total discounted costs

(

)

∞

Fxi0 s i , s 2 ,..., s m = ¦ λ t c( x ( t ), x ( t +1) ) , i = 1, 2 , ..., m. i x0

(

t =0

2

)

The payoff functions F s , s , ..., s m , i = 1, 2, ..., m of the players on the set of situations define a noncooperative game in normal form. In this game we are seeking for a Nash equilibrium, i.e. we are seeking for the strategies s1*, s2* , . . . , sm* of players 1 , 2 , . . . ,m that satisfy the condition

(

i

)

Fxi0 s1* , s 2* , ..., s i −1* , s i , s i +1* ,..., sm* ≤ ≤ Fxi0 s1* , s 2* , ..., s i −1* , s i , s i +1* ,..., s m* , ∀ si ∈ Si , i = 1, 2, ..., m. The considered game is determined uniquely by the directed graph G = (X,E), the partition X 1 ,X 2 ,... ,X m , the cost functions c i : E e R,i = 1 , 2 , . . . , m, the starting state x 0 and the discount factor λ. Therefore the corresponding network that determines our dynamic game we denote G, {X i }i =1,m , c i i =1,m , λ , x0 .

(

)

(

{}

)

We can see that for the considered game Nash equilibrium always exists. To prove this result we shall use a more general class of games which we call stochastic positional games.

2

Stochastic Positional Game on Networks with Discounted Costs

W e a ssu me t ha t t he se t of st a t e s X i s di vi de d i nt o m + 1 s ub se t s X = X1 ‘ X2 ‘…‘ Xm ‘ Z(X1  X2  … Xm Z = y), w h e r e x ∈ Z c o r r e s p o n d s t o t h e states in which the system passes to the next state y = x(t +1) randomly. More precisely, we assume that if the system at the moment of time t is in the state x = x(t) ∈ Z then it passes to the state y = x(t + 1) ∈ X with given probability p x , y . So, on Z is given a transition probability function p : Z e [0,1] that satisfy the condition ¦ y∈X p x, y =1 , ∀x ∈ Z. This means that players 1 , 2 , . . . ,m may control the dynamical system only in the states x ∈ X i , i = 1, 2,..., m; if x ∈ Z then the dynamical system passes to the next states y ∈ X(x) randomly according to probability distribution function in x. The values of payoff functions in this game we define as follows. Let s = (s 1 , s 2 , ..., s m ) be a situation determined by the strategies s 1 , s 2 ,..., s m of the players. Then on X × X we can define the following probability transition matrix Ps = (psx,y), where

p xs , y

­ p x , y , if x∈ Z ° = ® 1, if x∈ X i ° 0, if x∈ X i ¯

and y ∈ X ; and y = s i ( x ) ; and y ≠ s i ( x ).

(1)

In such a way we obtain a Markov process and for given discount factor β we can calculate the total discounted cost with respect to each transition cost function c i . These values we denote Fxi0 s1 , s 2 ,..., s m , i = 1 , 2 , . . . , m. The functions

(

(

)

)

Fxi0 s1 , s 2 ,..., s m , i = 1, 2,... ,m, on the set of situations define a game which we call the stochastic positional game with discounted costs. The corresponding network which determine this game we denote G , {X i }i =1,m , Z , c i i =1,m , p, λ , x0 . It is easy to observe

(

{}

3

)

that if Z = y then the stochastic positional games is transformed into the game

from previous section.

3

The Main Results

The main results of this paper are concerned with existence of Nash equilibria and elaboration of the algorithms for determining the optimal strategies of the players in the considered game. Theorem 1 For an arbitrary stochastic positional game on network G, {X i }i =1,m , X 0 , {c i }i =1,m , p, λ , x with discount factor 0 < β < 1 there exists Nash equilibrium s1*, s2*,..., sm*. The proof of this theorem can be obtained in a similar way as the proof of Theorem 3 from [2, 4] in the case of stochastic positional game for Markov decision processes with discounted costs. Moreover, the game on network G, {X i }i =1,m , X 0 , c i i =1,m , p, λ , x0 we can represents as a stochastic positional game

(

)

(

)

{}

for a Markov decision processes from [2, 4] if for an arbitrary state x { X i a strategy s i { S i we represent as an action for the Markov decision process with the probability transition matrices (1). If after that we apply Theorem 3 from [2, 4] then we obtain the proof of this theorem. Based on Theorem 1 we can prove the following result.

T h eo re m 2 L et b e g i ven a s t o ch a s t i c p o s i t i o n a l g a me o n n et w o r k G, {X i }i =1,m , X 0 , c i i =1,m , p, λ , x0 with discount factor β , 0 < β < 1, where G = (X,

(

)

{}

E) is the graph of state 's transition of the dynamical system m

m

a n d X =  X i ∪ X 0 . T h e n t h e r e e x i s t t h e v a l u e s σ xi , i = 1 , 2 , . . . m , f o r x i =1

{ X that satisfy the following conditions: c xi , y + λσ

2)

min cxi , y + λσ yi − σ xi

3)

i y

− σ xi ≥ 0 , ∀ x ∈ X i , ∀ y ∈ X ( x ), i = 1, 2 ,  , m ;

1)

{

y∈X ( x )

µx + λ

¦( )p

x, y

} = 0, ∀ x∈ X

i

, i = 1, 2,  , m ;

σ iy − σ x = 0, ∀ x ∈ X 0 ;

y∈X x

4) on each position set X i, i ∈ {1, 2,... ,m} there exists a map si* : Xi e X such that

{

}

s i ( x ) = y* ∈ Arg min µ xi + λ σ iy − σ xi , ∀x∈ X i *

y∈X ( x )

and

µ xj,y + λσ yj − σ xj = 0, ∀x ∈ X i , j = 1, 2,  , m. *

*

The set of maps s 1 * , s 2 *,..., s m * determines a Nash equilibrium situation s* = (s1*, s2*,..., sm*) for the discounted stochastic positional game on network

(G, X , {X }

i i =1, m

{}

, ci

i =1, m

)

, λ , p, x0 and 4

(

)

Fxi0 s1* , s 2* ,, s m* = σ xi 0 , i = 1,2,, m. Moreover, the situation s* = (s1*, s2*,..., sm*) is a Nash equilibrium for an arbitrary starting position x { X. In the case m = 2, c = c 2 = − c1 from Theorem 2 we obtain the following result. Corollary 3 Let be given a discounted antagonistic stochastic positional game on network G, X 1 , X 2 , X 0 , c, p, λ , x0 with discount factor 0 < β < 1, where G = (X, E) is the graph of state's transition of the dynamical system and X = X 1 ∪ X 2 ∪ X 0 . Then there exists the values σ x , for x { X that satisfy the following conditions: 1) max {c x , y + λσ y − σ x }= 0, ∀ x∈ X 1 ;

(

)

y∈ X ( x )

2) 3)

min {c x , y + λσ y − σ x }= 0, ∀ x∈ X 2

y∈ X ( x )

µx + λ

¦( )p

x, y

σ x = 0, ∀x ∈ X 0

y∈X x

The optimal stationary strategies s1*, s2* for which

(

)

(

)

(

Fx 0 s 1* , s 2* = max min Fx 0 s 1 , s 2 = min max Fx 0 s 1 , s 2 2 2 2 2 1 1 1 1 s ∈S

s ∈S

s ∈S

s ∈S

)

can be found by fixing

s1* ( x ) = y * ∈ Arg max {µ x + γσ y − σ x }, ∀ x∈ X 1 y∈ X ( x )

and

s 2* ( x ) = y * ∈ Arg min {µ x + γσ y − σ x }, ∀ x∈ X 2 y∈ X ( x )

Based on results described above we have grounded algorithms for determining the optimal strategies of the players in the considered games. Additionally we have shown that the obtained results can be used for studying and solving the dynamic games with finite time horizon.

References [1] Bellman, R., Kalsba, R.: Dynamic Programming and Modern Control Theory. Academic Press, New York, London (1965). [2] Lozovanu, D.: The game-theoretical approach to Markov decision problems and determining Nash equilibria for stochastic positional games. : Int J Mathematical Modelling and Numerical Optimization, 2 (1), (2011), 162-174. [3] Lozovanu, D., Pickl, S.: Optimization and Multiobjective Control of Time-discrete Systems. Springer (2009). [4] Lozovanu, D., Pickl, S., Kropat, E.: Markov decision processes and determining Nash equilibria for stochastoic positional games, Proceedings of the 18th IFAC Wold Congress, Milano, (2011), 13398-13403. [5] Puterman, M.: Markov Decision Processes:Stochastic Dynamic Programming. John Wiley, New Jersey (2005). 5

An Attempt at a Formal Specification of Cognitive Trust Jacques Calmet1 and Marvin Schneider2 1

Karlsruhe Institute of Technology (KIT), Institute IKS, Am Fasanengarten 5, 76131 Karlsruhe, Germany [email protected] 2

Business School São Paulo (BSP), Rua Jacerú, 247 – 8o. Andar, 04705-000 São Paulo, Brazil [email protected]

(In memory of Jochen Pfalzgraf) Abstract. We outline an approach to formally model trust/knowledge via Virtual Knowledge Communities (VKC) in multiagent environments using a Logical Fibering representation. Initiating with a reflection about the relation between knowledge, knowledge bases and trust, we elaborate the importance of an abstraction method (theory, control and environment) and mention examples of its application. We go on by proposing an implementation of such knowledge abstractions via Virtual Knowledge Communities, defining its methods an characteristics. Finally, we present Logical Fibering, its main principles and a way of modeling the knowledge and VKCs in an intelligent fashion. Keywords: Trust, Culture, Knowledge, Virtual Knowledge Communities, Ontologies, Logical Fibering

1. Introduction Culture is a central momentum of every society. Rules, written and non-written, are built on it (Moran et al., 2007). Whole behavior patterns are based on it and the very belief of society is nourished by its culture. One of the central roles in culture belongs to knowledge obtained throughout centuries of interaction with other cultures, within the same society, with nature and technology. On the other hand, knowledge implies in a behavior pattern, which is cemented by culture, meaning that knowledge, culture and decision making always go hand in hand. As knowledge is produced at a whole series of different places, culture is a cross- and interdisciplinary issue. A recent book (Castelfranchi, 2010) introduces a socio-cognitive and computational model for trust theory. We propose another attempt inspired by several methodologies arising directly from the mechanization of mathematics. This does not imply that trust is not related to some degrees of belief but it does imply that beliefs are seen as conditions validating a theorem. 2. The Concept of Trust and Knowledge Bases Trust is basic to the decision making process. Applied to mathematics it may be seen as proving a theorem in a cognitive world. Meanwhile mathematics is normally not being applied in a generic decision making process either by the standard population or even by advanced users of technology such as web surfers, bank costumers, students of humanities etc. This may be interpreted as a fairly grotesque situation as mostly complex settings are analyzed and a lot of time is being used for decision making. When looking at trust established between two cultures, we may observe that it leads to the sharing of knowledge throughout a common base (see Figure 1). This common base is the evidence of trust. Mean-

1

while, conflicting knowledge must be synchronized. This process of mixing or abstracting knowledge is a great issue to the process and a standardized formalized means is prone to produce more reasonable results than the pure quarreling over items or even randomization.

Figure 1: Establishment of Trust 3. Abstraction As seen, the abstraction and the mix of knowledge represent central steps for the establishment of a common base and thus a trust relation between the two cultures. For the abstraction concrete methods should be used, which makes it quite hard to establish formally in non-mathematical related domains in a way that correctness may be proven in a strong manner. Nonetheless, it may be observed that yet there are common characteristics throughout different types of knowledge, which we may use to formalize.

Theory Control Environment

Law Set of Laws

Philosophy Set of basic laws describing the world Application Decrees Interpretation of laws (chapels) Jurisprudence and Liti- Behaviors Resulting gation Procedures from the Controlled Theory

Sociology Set of agents with welldefined actions Society upon theory The way society is governed

Table 1: Abstract definition in different domains Thus, a way to find an abstraction in many different domains is via an approach, which takes three main elements into consideration (Calmet, 2009): • A theory, which represents a set of rules and findings over the domain; • Control mechanisms on the theory, which result in its application in real life and successful interaction. These mechanisms refer to the decision making process. • The environment, which is the main setting around the domain, defining external influences. As examples we might give the fields of law, philosophy and sociology as shown in Table 1.

2

4. Implementation For implementation we suggest the use of multiple agents that cooperate in a Virtual Knowledge Community, in short VKC (Earl, 2001). VKC’s, being a recent current of research, are places where agents (humans or software) meet in order to interchange information, collaborate for their own purposes as well as any predefined collective goal (Maret and Calmet, 2009). Communities typically include individuals interested in a specific topic or with knowledge in a defined field. Thus, every community has a common topic, a focus of its communication. This suggests a distributed environment and likewise a bottom-up approach, where the description is reached by the connection of new elements, which also aptly treats the characteristic that knowledge does not have boundaries or limits and that every approach simply means a new view on the same domain. For the implementation of our environment, firstly we need to define the VKC via: • A leader, who is any agent initiating the topic; • A topic which must be part of the leader’s knowledge; • Members, who are any other agents; • Ways of communication, whereas the leader controls the message buffer used for communication. General rules should be: • No limit for the number of VKCs; • An agent may create or join several VKCs; • The lifetime of a VKC is decided by its leader. A basic process for the processing of topics may be modeled by the following steps: • Initiate: an agent proposes a topic; • Join: agents interested in the topic of a community join it; • Inform: the leader informs of the existence of a VKC; • Request: an agent asks for the existence of a VKC; • Leave, delete, terminate. 5. Logical Fibering Especially in the human science domain the computability (and thus the way formal representations may be conducted) might be questioned as seen above. Definitions in this sense should not be strict (such as in a Turing test (Russell and Norvig, 2009)), but refer to semantics over the ontology. At this stage we have translated beliefs related to trust into formal specifications but we are still faced by the challenge of defining the localization and boundaries of a given piece of knowledge. An illustrative example is as follows. This conference takes place in Baden-Baden where German is the spoken language. If you cross the Rhine, then French is spoken. Now, at the conference the communication language is English. This means that we have three different types of knowledge. We must be able to decide what is the area of validity of each language. A simple technique is the continuity of travel within a given neighborhood. This is basic topology. To identify the boarders of validity we can introduce another concept in topology: Logical Fibering. The fibers carry the communication channels with some markers on the domain of validity of the knowledge (languages here). Logical Fiberings were developed by J. Pfalzgraf on the ground of PCL by G. Guenter as shown in Pfalzgraf (2000), Pfalzgraf and Edtmayr (2004) as well as a series of other publications. By definition, a Logical Fibering is a quadruple, consisting of the basic elements (E, ʌ, B, F) • Total space (E) with logics; • Projection map ʌ : E ĺ B; • Base space (B); • Typical fiber (F), being a logical structure over each point b in the base space.

3

In the most common configuration F is a classic two-valued logic L. A straightforward example of a Logical Fibering is a so-called free parallel system, or trivial fibering. In this case we have ȟ = (E, ʌ, I, L), i.e., the base space is a set of indices (I) and the typical fiber is a two valued logic (L). In a more complex situation we have a Derived Logical Fibering (Pfalzgraf, 2004) with a nontrivial equivalence relation on the set of truth values (ȍI = {Ti,Fi}), also denoted L(I). In this case the set of global truth values is the quotient space following ȍ(I) := ȍI / ≡.

Figure 2: Proposed Logical Fibering Representation of VKC The intelligent representation characteristics of a Logical Fibering make it a good candidate for data administration in modern systems (as, e.g., seen in Schneider and Calmet, 2005). Applied to knowledge representation by a VKC, we may see the base space B as the index of agents. The map π is the link to each agent’s knowledge, which is represented through local logics Li (see Figure 2). A mediation scheme may model communication and lead to a non-trivial global set of logics based on the local logics of each agent. This provides a formalized, standardized and much more tangible way of representation. A main challenge for research to come is the translation of non-mathematical knowledge into a logical system, which goes beyond the scope of this short article. 6. Objectives We expect the following benefits through the modeling of knowledge in VKCs, using Logical Fibers for formalization: • Dynamic extensibility through the inclusion and exclusion of agents, creation of new interests and topic, update of outdated knowledge and deletion of erroneous information; • Comparability by the formalization of each agent’s knowledge and a similar root implementation in different domains; • Computability through a logical representation, which has the power to speed up any analytical engine applied to it; • Trivial implementation through the use of formalisms, similar to those used in nowadays’ information systems. The principle objective to be reached in years to come is an intelligent formalization of knowledge, which, despite Business Intelligence Systems and Data Warehouses on the market, does not truly exist yet. Furthermore, different and distinct knowledge domains should be modeled by rules having similar

4

roots, whereas anything specific is treated in a distinct fashion so that a global system can unfold higher power. 7. Conclusion In this article we presented a way of implementing knowledge and trust in distinct domains by Virtual Knowledge Communities and Logical Fibering. We understand that it configures an interesting approach to close performance and conceptual gaps of the currently implemented representations. In this sense, we yet again stress the higher flexibility of the multi-agent environment and the intelligent modeling power of Logical Fibering as two crucial points. 8. References •

• • •

• •

•

• •

Calmet, J. (2009). Abstraction Based Information Technology: A Framework for Open Mechanized Reasoning. In Conference on Intelligent Computer Mathematics. Eds. J. Carette et al., Springer LNCS 562, pp. 14-26, 2009. Castelfranchi, C., Falcone, R. (2010), Trust theory – A socio-cognitive and computational model, New Jersey: Wiley. Earl, M. (2001). Knowledge management strategies: Toward a taxonomy. Journal of Management Information Systems, 18(1), 215–233. Maret, P. and Calmet, J. (2009). Agent-Based Knowledge Communities. In: International Journal of Computer Science and Applications, Technomathematics Research Foundation, Vol. 6., No. 2., pp. 1-18. Moran, R. T., Harris, P. R., Moran, S. V. (2007). Managing Cultural Differences – Global Leadership Strategies for the 21st Century, New York: Elsevier. Pfalzgraf, J. and Edtmayr, J. (2004). The concept of logical fiberings: distributed logics for multiagent systems. In Proceedings 17th European Meeting on Cybernetics and Systems Research (EMCSR’2004), Vienna, April 13-16, 2004, R.Trappl (Ed.). Cybernetics and Systems. Austrian Society for Cybernetic Studies. Pfalzgraf J. (2000). The concept of logical fiberings and fibered logical controllers. In Proceedings CASYS’2000, August 7-12, 2000, Liege, Belgium. American Institute of Physics, AIP Conference Proceedings, Vol.573 (2001), pp.683-693. D.M.Dubois (Ed.). Russell, S. and Norvig, P. (2009). Artificial Intelligence: A Modern Approach. 3rd. Ed. Upper Saddle River: Prentice Hall. Schneider, M. O. and Calmet, J. (2005). Fibered Guard – A Hybrid Intelligent Approach to Denial of Service Prevention. In: Proceedings of the International Conference on Intelligent Agents, Web Technology and Internet (IAWTIC’05), IEEE Computer Press, Vienna, Austria, pp. 121127.

5