Parallel Discrepancy-based Search - Université Laval

21 févr. 2014 - Il s'agıt d'une course entre les processeurs pour obtenir une solution. Chaque processeur recherche une solution avec un algorithme qui lui est.
7MB taille 4 téléchargements 346 vues
Parallel Discrepancy-based Search T. Moisan, J. Gaudreault, C.-G. Quimper Universite´ Laval, FORAC research consortium

February 21th 2014

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

1 / 57

Upsides of Parallelization Parallel computing is a growing domain. Governments and industries are intensively investing in it. It is a huge opportunity for constraint programming. It can lead to unpreceded performances.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

2 / 57

Downsides of Parallelization Parallelization is hard. Speedup can stall very fast due to communication. It is difficult to reproduce in a parallel environment the search strategies that work well sequentially.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

3 / 57

Outline 1

Constraint programming

2

Depth-first Search (DFS)

3

Limited Discrepancy Search (LDS)

4

Parallel computing in constraint programming

5

Parallel Discrepancy-based Search (PDS)

6

Theoretical and statistical analysis

7

Experimentations with an industrial case

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

4 / 57

Constraint programming

Uses variables with a finite domain; Uses specific constraints to modelize the problem: I I

All-different: variables must have different values; Regular: variables must follow a defined regular expression.

Each constraint includes a specialized algorithm to filter the search space; Can include an objective function.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

5 / 57

Exemple

0ZQZ 3 L0Z0 2 0Z0L 1 ZQZ0 4

x1

x2

x3

dom(x1 ) = {1, 2, 3, 4} dom(x2 ) = {1, 2, 3, 4} dom(x3 ) = {1, 2, 3, 4} dom(x4 ) = {1, 2, 3, 4} T. Moisan, J. Gaudreault, C.-G. Quimper

x4

0L0Z Z0ZQ 2 QZ0Z 1 Z0L0 4 3

x1

x2

x3

x4

xi 6= xj ∀i < j xj − xi 6= j − i ∀i < j xj − xi 6= i − j ∀i < j Parallel Discrepancy-based Search

February 21th 2014

6 / 57

Espace de recherche Ensemble des assignations partielles des variables I

´ Cet espace est typiquement represent e´ par un arbre n-aire. 0Z0Z Z0Z0 0Z0Z 1 Z0Z0 4

0Z0Z 3 Z0Z0 2 0Z0Z 1 L0Z0 4

x1

x2

x3

3 2

x1

x2

x4

x3

x4

0Z0Z Z0Z0 QZ0Z 1 Z0Z0 4 3

0L0Z Z0Z0 QZ0Z 1 Z0Z0 4

2

x1

x2

x3

3 2

x4

x1

0Z0Z ZQZ0 2 0Z0Z 1 L0Z0

0L0Z Z0Z0 2 0Z0Z 1 L0Z0

4

4

3

3

x1

x2

x3

x4

x1

x2

x3

x

x

x

x

x

0L0Z Z0Z0 QZ0Z 1 Z0L0 2

x4

0L0Z 3 Z0Z0 2 0ZQZ 1 L0Z0 x

T. Moisan, J. Gaudreault, C.-G. Quimper

x

x2

x3

x

x4

3

4

x1

x3

4

x1

x

x2

x

x4

x

Parallel Discrepancy-based Search

x

x2

x3

x4

x

0L0Z Z0ZQ QZ0Z 1 Z0L0 4 3 2

x1

x2

x3

x4

February 21th 2014

7 / 57

Outline 1

Constraint programming

2

Depth-first Search (DFS)

3

Limited Discrepancy Search (LDS)

4

Parallel computing in constraint programming

5

Parallel Discrepancy-based Search (PDS)

6

Theoretical and statistical analysis

7

Experimentations with an industrial case

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

8 / 57

Depth-first Search (DFS)

The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one.

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

9 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

10 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

11 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

12 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

13 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

14 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

15 / 57

Depth-first Search (DFS)

1

T. Moisan, J. Gaudreault, C.-G. Quimper

2

3

4

5

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

16 / 57

Outline 1

Constraint programming

2

Depth-first Search (DFS)

3

Limited Discrepancy Search (LDS)

4

Parallel computing in constraint programming

5

Parallel Discrepancy-based Search (PDS)

6

Theoretical and statistical analysis

7

Experimentations with an industrial case

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

17 / 57

Limited Discrepancy Search (LDS) Harvey and Ginsberg (1995)

The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one. A discrepancy is a deviation from the first choice of the value ordering heuristic.

0

1

1

2

1

2

2

3

LDS proposes to visit the leaves in increasing order of discrepancy.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

18 / 57

Limited Discrepancy Search (LDS) Harvey and Ginsberg (1995)

The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one. A discrepancy is a deviation from the first choice of the value ordering heuristic.

0

1

1

2

1

2

2

3

LDS proposes to visit the leaves in increasing order of discrepancy.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

19 / 57

LDS Iterations 0 (0 discrepancy)

0

1

T. Moisan, J. Gaudreault, C.-G. Quimper

1

2

1

Parallel Discrepancy-based Search

2

2

3

February 21th 2014

20 / 57

LDS Iterations 1 (1 discrepancy)

0

1

T. Moisan, J. Gaudreault, C.-G. Quimper

1

2

1

Parallel Discrepancy-based Search

2

2

3

February 21th 2014

21 / 57

LDS Iterations 2 (2 discrepancies)

0

1

T. Moisan, J. Gaudreault, C.-G. Quimper

1

2

1

Parallel Discrepancy-based Search

2

2

3

February 21th 2014

22 / 57

LDS Iterations 3 (3 discrepancies)

0

1

T. Moisan, J. Gaudreault, C.-G. Quimper

1

2

1

Parallel Discrepancy-based Search

2

2

3

February 21th 2014

23 / 57

LDS Overhead versus DFS

Nodes in a binary tree with n variables: 2 · 2n − 1 Node visits with a DFS: 2 · 2n − 1 Node visits with a LDS: 4 · 2n − n − 3 By the time DFS visits the entire tree, LDS will visit half of the leaves. The leaves that LDS visits are those that have fewer than n/2 discrepancies. If the value ordering heuristic is as good as a random heuristic, LDS finds a solution by the time DFS completes.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

24 / 57

Outline 1

Constraint programming

2

Depth-first Search (DFS)

3

Limited Discrepancy Search (LDS)

4

Parallel computing in constraint programming

5

Parallel Discrepancy-based Search (PDS)

6

Theoretical and statistical analysis

7

Experimentations with an industrial case

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

25 / 57

Portfolio

Il s’agˆıt d’une course entre les processeurs pour obtenir une solution. Chaque processeur recherche une solution avec un algorithme qui lui est propre ´ Les processeurs peuvent s’aider en echangeant des informations. ´ Il est difficile d’etendre a` un nombre arbitraire de processeurs. I I I

´ ` Il faut definir des parametres de recherche pour chaque processeur. ` ´ Ces parametres sont souvent etablis manuellement. ´ ` ´ La parametrisation est fortement lie´ au probleme a` resoudre.

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

26 / 57

Work-stealing Xie et Davenport Processeur 1

Processeur 2

A

B

X

Y

C

D

Z

V

E

T. Moisan, J. Gaudreault, C.-G. Quimper

W

Parallel Discrepancy-based Search

February 21th 2014

27 / 57

Work-stealing Processeur 1

Processeur 2

A

B

X

Y

C

D

Z

V

E

T. Moisan, J. Gaudreault, C.-G. Quimper

W

Parallel Discrepancy-based Search

February 21th 2014

28 / 57

Work-stealing

Processeur 1

Processeur 2

A

B

X

Y

C

D

Z

V

E

T. Moisan, J. Gaudreault, C.-G. Quimper

Je n'ai plus de travail.

W

Parallel Discrepancy-based Search

February 21th 2014

29 / 57

Work-stealing Processeur 1

Processeur 2 C

A

D

B

X

Y

E

Z

V

W

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

30 / 57

Work-stealing Processeur 1

A

Processeur 2

B

X

Y

E

Z

V

C

W

D

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

31 / 57

Work-stealing Processeur 1

A

Processeur 2

B

X

Y

E

Z

V

C

W

D

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

32 / 57

Division de l’espace de recherche

` courante Approche tres ´ ´ Divise theoriquement la charge de travail egalement ˆ Peu probable que les sous-arbres soient de meme taille I I

ˆ Ne peut pas etre utilise´ sur un grand nombre de processeurs ` Probleme NP-Difficile

ˆ Si les sous-arbres ne sont pas de meme taille I I

ˆ Les processeurs n’auront pas la meme charge de travail ´ totale de la tache ˆ La duree sera plus longue

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

33 / 57

Outline 1

Constraint programming

2

Depth-first Search (DFS)

3

Limited Discrepancy Search (LDS)

4

Parallel computing in constraint programming

5

Parallel Discrepancy-based Search (PDS)

6

Theoretical and statistical analysis

7

Experimentations with an industrial case

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

34 / 57

Our Objectives

Having a search strategy that performs well sequentially, we want to parallelize it with four main goals in mind: I I I I

Search strategy preservation Workload balancing Robustness No communication

T. Moisan, J. Gaudreault, C.-G. Quimper

Parallel Discrepancy-based Search

February 21th 2014

35 / 57

Intuition behind PDS Each leaf is assigned to a processor. The leaves are implicitly assigned in a round-robin fashion. To split the task, let’s say that p1 takes odd leaves and p2 the even leaves.

1

2

T. Moisan, J. Gaudreault, C.-G. Quimper

3

5

4

6

Parallel Discrepancy-based Search

7

8

February 21th 2014

36 / 57

Intuition behind PDS Each leaf is assigned to a processor. The leaves are implicitly assigned in a round-robin fashion. To split the task, let’s say that p1 takes odd leaves and p2 the even leaves.

p1 p2

1

2

T. Moisan, J. Gaudreault, C.-G. Quimper

3

5

4

6

Parallel Discrepancy-based Search

7

8 February 21th 2014

37 / 57

PDS overview

p1 p2 p 1 & p2 1

2

3

T. Moisan, J. Gaudreault, C.-G. Quimper

5

4

6

7

Parallel Discrepancy-based Search

8

February 21th 2014

38 / 57

PDS overview

p1 p2 p 1 & p2 1

2

3

T. Moisan, J. Gaudreault, C.-G. Quimper

5

4

6

7

Parallel Discrepancy-based Search

8

February 21th 2014

39 / 57

PDS overview

p1 p2 p 1 & p2 1

2

3

T. Moisan, J. Gaudreault, C.-G. Quimper

5

4

6

7

Parallel Discrepancy-based Search

8

February 21th 2014

40 / 57

Implementing PDS p : its processor id

ρ : the number of processors n : the number of variables in the subtree k : the number of required discrepancies l : the processor id implicitly assigned to the leftmost leaf The processor will branch in the subtree if (p − l ) mod ρ