Parallel Discrepancy-based Search T. Moisan, J. Gaudreault, C.-G. Quimper Universite´ Laval, FORAC research consortium
February 21th 2014
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
1 / 57
Upsides of Parallelization Parallel computing is a growing domain. Governments and industries are intensively investing in it. It is a huge opportunity for constraint programming. It can lead to unpreceded performances.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
2 / 57
Downsides of Parallelization Parallelization is hard. Speedup can stall very fast due to communication. It is difficult to reproduce in a parallel environment the search strategies that work well sequentially.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
3 / 57
Outline 1
Constraint programming
2
Depth-first Search (DFS)
3
Limited Discrepancy Search (LDS)
4
Parallel computing in constraint programming
5
Parallel Discrepancy-based Search (PDS)
6
Theoretical and statistical analysis
7
Experimentations with an industrial case
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
4 / 57
Constraint programming
Uses variables with a finite domain; Uses specific constraints to modelize the problem: I I
All-different: variables must have different values; Regular: variables must follow a defined regular expression.
Each constraint includes a specialized algorithm to filter the search space; Can include an objective function.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
5 / 57
Exemple
0ZQZ 3 L0Z0 2 0Z0L 1 ZQZ0 4
x1
x2
x3
dom(x1 ) = {1, 2, 3, 4} dom(x2 ) = {1, 2, 3, 4} dom(x3 ) = {1, 2, 3, 4} dom(x4 ) = {1, 2, 3, 4} T. Moisan, J. Gaudreault, C.-G. Quimper
x4
0L0Z Z0ZQ 2 QZ0Z 1 Z0L0 4 3
x1
x2
x3
x4
xi 6= xj ∀i < j xj − xi 6= j − i ∀i < j xj − xi 6= i − j ∀i < j Parallel Discrepancy-based Search
February 21th 2014
6 / 57
Espace de recherche Ensemble des assignations partielles des variables I
´ Cet espace est typiquement represent e´ par un arbre n-aire. 0Z0Z Z0Z0 0Z0Z 1 Z0Z0 4
0Z0Z 3 Z0Z0 2 0Z0Z 1 L0Z0 4
x1
x2
x3
3 2
x1
x2
x4
x3
x4
0Z0Z Z0Z0 QZ0Z 1 Z0Z0 4 3
0L0Z Z0Z0 QZ0Z 1 Z0Z0 4
2
x1
x2
x3
3 2
x4
x1
0Z0Z ZQZ0 2 0Z0Z 1 L0Z0
0L0Z Z0Z0 2 0Z0Z 1 L0Z0
4
4
3
3
x1
x2
x3
x4
x1
x2
x3
x
x
x
x
x
0L0Z Z0Z0 QZ0Z 1 Z0L0 2
x4
0L0Z 3 Z0Z0 2 0ZQZ 1 L0Z0 x
T. Moisan, J. Gaudreault, C.-G. Quimper
x
x2
x3
x
x4
3
4
x1
x3
4
x1
x
x2
x
x4
x
Parallel Discrepancy-based Search
x
x2
x3
x4
x
0L0Z Z0ZQ QZ0Z 1 Z0L0 4 3 2
x1
x2
x3
x4
February 21th 2014
7 / 57
Outline 1
Constraint programming
2
Depth-first Search (DFS)
3
Limited Discrepancy Search (LDS)
4
Parallel computing in constraint programming
5
Parallel Discrepancy-based Search (PDS)
6
Theoretical and statistical analysis
7
Experimentations with an industrial case
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
8 / 57
Depth-first Search (DFS)
The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one.
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
9 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
10 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
11 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
12 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
13 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
14 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
15 / 57
Depth-first Search (DFS)
1
T. Moisan, J. Gaudreault, C.-G. Quimper
2
3
4
5
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
16 / 57
Outline 1
Constraint programming
2
Depth-first Search (DFS)
3
Limited Discrepancy Search (LDS)
4
Parallel computing in constraint programming
5
Parallel Discrepancy-based Search (PDS)
6
Theoretical and statistical analysis
7
Experimentations with an industrial case
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
17 / 57
Limited Discrepancy Search (LDS) Harvey and Ginsberg (1995)
The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one. A discrepancy is a deviation from the first choice of the value ordering heuristic.
0
1
1
2
1
2
2
3
LDS proposes to visit the leaves in increasing order of discrepancy.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
18 / 57
Limited Discrepancy Search (LDS) Harvey and Ginsberg (1995)
The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one. A discrepancy is a deviation from the first choice of the value ordering heuristic.
0
1
1
2
1
2
2
3
LDS proposes to visit the leaves in increasing order of discrepancy.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
19 / 57
LDS Iterations 0 (0 discrepancy)
0
1
T. Moisan, J. Gaudreault, C.-G. Quimper
1
2
1
Parallel Discrepancy-based Search
2
2
3
February 21th 2014
20 / 57
LDS Iterations 1 (1 discrepancy)
0
1
T. Moisan, J. Gaudreault, C.-G. Quimper
1
2
1
Parallel Discrepancy-based Search
2
2
3
February 21th 2014
21 / 57
LDS Iterations 2 (2 discrepancies)
0
1
T. Moisan, J. Gaudreault, C.-G. Quimper
1
2
1
Parallel Discrepancy-based Search
2
2
3
February 21th 2014
22 / 57
LDS Iterations 3 (3 discrepancies)
0
1
T. Moisan, J. Gaudreault, C.-G. Quimper
1
2
1
Parallel Discrepancy-based Search
2
2
3
February 21th 2014
23 / 57
LDS Overhead versus DFS
Nodes in a binary tree with n variables: 2 · 2n − 1 Node visits with a DFS: 2 · 2n − 1 Node visits with a LDS: 4 · 2n − n − 3 By the time DFS visits the entire tree, LDS will visit half of the leaves. The leaves that LDS visits are those that have fewer than n/2 discrepancies. If the value ordering heuristic is as good as a random heuristic, LDS finds a solution by the time DFS completes.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
24 / 57
Outline 1
Constraint programming
2
Depth-first Search (DFS)
3
Limited Discrepancy Search (LDS)
4
Parallel computing in constraint programming
5
Parallel Discrepancy-based Search (PDS)
6
Theoretical and statistical analysis
7
Experimentations with an industrial case
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
25 / 57
Portfolio
Il s’agˆıt d’une course entre les processeurs pour obtenir une solution. Chaque processeur recherche une solution avec un algorithme qui lui est propre ´ Les processeurs peuvent s’aider en echangeant des informations. ´ Il est difficile d’etendre a` un nombre arbitraire de processeurs. I I I
´ ` Il faut definir des parametres de recherche pour chaque processeur. ` ´ Ces parametres sont souvent etablis manuellement. ´ ` ´ La parametrisation est fortement lie´ au probleme a` resoudre.
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
26 / 57
Work-stealing Xie et Davenport Processeur 1
Processeur 2
A
B
X
Y
C
D
Z
V
E
T. Moisan, J. Gaudreault, C.-G. Quimper
W
Parallel Discrepancy-based Search
February 21th 2014
27 / 57
Work-stealing Processeur 1
Processeur 2
A
B
X
Y
C
D
Z
V
E
T. Moisan, J. Gaudreault, C.-G. Quimper
W
Parallel Discrepancy-based Search
February 21th 2014
28 / 57
Work-stealing
Processeur 1
Processeur 2
A
B
X
Y
C
D
Z
V
E
T. Moisan, J. Gaudreault, C.-G. Quimper
Je n'ai plus de travail.
W
Parallel Discrepancy-based Search
February 21th 2014
29 / 57
Work-stealing Processeur 1
Processeur 2 C
A
D
B
X
Y
E
Z
V
W
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
30 / 57
Work-stealing Processeur 1
A
Processeur 2
B
X
Y
E
Z
V
C
W
D
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
31 / 57
Work-stealing Processeur 1
A
Processeur 2
B
X
Y
E
Z
V
C
W
D
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
32 / 57
Division de l’espace de recherche
` courante Approche tres ´ ´ Divise theoriquement la charge de travail egalement ˆ Peu probable que les sous-arbres soient de meme taille I I
ˆ Ne peut pas etre utilise´ sur un grand nombre de processeurs ` Probleme NP-Difficile
ˆ Si les sous-arbres ne sont pas de meme taille I I
ˆ Les processeurs n’auront pas la meme charge de travail ´ totale de la tache ˆ La duree sera plus longue
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
33 / 57
Outline 1
Constraint programming
2
Depth-first Search (DFS)
3
Limited Discrepancy Search (LDS)
4
Parallel computing in constraint programming
5
Parallel Discrepancy-based Search (PDS)
6
Theoretical and statistical analysis
7
Experimentations with an industrial case
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
34 / 57
Our Objectives
Having a search strategy that performs well sequentially, we want to parallelize it with four main goals in mind: I I I I
Search strategy preservation Workload balancing Robustness No communication
T. Moisan, J. Gaudreault, C.-G. Quimper
Parallel Discrepancy-based Search
February 21th 2014
35 / 57
Intuition behind PDS Each leaf is assigned to a processor. The leaves are implicitly assigned in a round-robin fashion. To split the task, let’s say that p1 takes odd leaves and p2 the even leaves.
1
2
T. Moisan, J. Gaudreault, C.-G. Quimper
3
5
4
6
Parallel Discrepancy-based Search
7
8
February 21th 2014
36 / 57
Intuition behind PDS Each leaf is assigned to a processor. The leaves are implicitly assigned in a round-robin fashion. To split the task, let’s say that p1 takes odd leaves and p2 the even leaves.
p1 p2
1
2
T. Moisan, J. Gaudreault, C.-G. Quimper
3
5
4
6
Parallel Discrepancy-based Search
7
8 February 21th 2014
37 / 57
PDS overview
p1 p2 p 1 & p2 1
2
3
T. Moisan, J. Gaudreault, C.-G. Quimper
5
4
6
7
Parallel Discrepancy-based Search
8
February 21th 2014
38 / 57
PDS overview
p1 p2 p 1 & p2 1
2
3
T. Moisan, J. Gaudreault, C.-G. Quimper
5
4
6
7
Parallel Discrepancy-based Search
8
February 21th 2014
39 / 57
PDS overview
p1 p2 p 1 & p2 1
2
3
T. Moisan, J. Gaudreault, C.-G. Quimper
5
4
6
7
Parallel Discrepancy-based Search
8
February 21th 2014
40 / 57
Implementing PDS p : its processor id
ρ : the number of processors n : the number of variables in the subtree k : the number of required discrepancies l : the processor id implicitly assigned to the leftmost leaf The processor will branch in the subtree if (p − l ) mod ρ