The Joint Block Diagonalization (JBD) problem: a tensor framework.
Dimitri Nion & Lieven De Lathauwer K.U. Leuven, Kortrijk campus, Belgium E-mails:
[email protected] [email protected]
TDA 2010, Monopoli, Italy, September 13-17, 2010
1
Joint-Block-Diagonalization (JBD): model I
D1K
XK
L1
=
X1
LR
D11 I
0
AR
A1
0
D RK
A1T
L1
A RT
LR
0 D R1
X k AD k AT ( N k ) or X k AD k A H ( N k ), k 1,..., K JBD is a generalization of JD (Joint Diagonalization)/INDSCAL
DK
XK X1
0 0
R
= I
A
0 D1
c1
0
AT
=
a1 a1
cR + … +
aR aR 2
JBD : ambiguities D1K
L1
XK
X1
=
IA
1
LR
AR
D11
Z
Z-1
0
0
I
0
A1T
L1
A RT
L
D RK
D R1
Z-T
ZT
R
=
~
A
K {D k }k 1
~T
A
Observation: if you choose Z arbitrary, you lose the JBD structure. Question: what is the structure of Z such that the JBD model is still valid?
3
JBD : essential uniqueness
X
K k k 1
The JBD of
Λ Π
is said essentially unique if
Z ΛΠ
an arbitrary block-diagonal matrix, an arbitrary block-wise permutation matrix. 1 0
Λ1 ~
~
~
A1 A 2
A3
=
A1
A2
A3
Λ2 Λ3
1
0
0
1
0 1
1 0
0 1
Solving a JBD problem Estimation of {Span(Ar)}r=1,…,R in an arbitrary order 4
JBD: State of the art JBD is becoming popular signal processing tool in applications such as: Blind Source Separation (BSS) of convolutive mixtures in the time-domain,
Independent Subspace Analysis. Two approaches in the literature: Approach 1: Unitary-JBD [Abed Meraim and Belouchrani, 2004] « A is a square unitary matrix » (AT A = I ) Approach 2: Non-Unitary JBD Approach 2.1: [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010] « A is tall and full column-rank. » Indeed, their approach only works if A is a square non-unitary matrix. Approach 2.2: This talk « A can be a tall, square or fat non-unitary matrix » JBD is a particular instance of Block-Component-Decompositions Computation by a gradient-based algorithm In the square case, better performance than 2.1
5
Joint-Block-Diagonalization : state of the art (1) Approach 1: Unitary-JBD [Abed Meraim and Belouchrani, 2004] A is square unitary matrix (ATA = I )
Xk = A Dk AT (+ Nk ) K
max bdiag( A X k A) T
A
k 1
AT Xk A = Dk + (A Nk AT ) 2 F
K
or min offbdiag( A X k A) T
A
k 1
2 F
XK AT
X1
= A 6
Joint-Block-Diagonalization : state of the art (2) Approach 2: Non-Unitary-JBD [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010] A is tall and full column-rank (Let B = A , then BA = I )
Xk = A Dk AT (+ Nk )
BXk BT = Dk + (B Nk BT )
K
max bdiag(BXk B ) T
B
k 1
2 F
K
or min offbdiag(BXk B ) T
B
k 1
2 F
XK B
X1
= BT 7
Joint-Block-Diagonalization : state of the art (3) Approach 2: Non-Unitary-JBD [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010] 2 gradient-descent based algorithms, JBDOG and JBDORG to solve K
minoff offbdiag(BXk B ) T
B
k 1
2 F
(1)
Drawbacks of approach 2: B=0 is a trivial minimizer Under-determined case (A fat, IN) is not successfully handled either because if B [B1T , BT2 ]T A is solution of an exact JBD problem, i.e.,
offbdiag (BX k BT ) offbdiag (BAD k AT BT ) offbdiag (Dk ) 0, k ~
then
B [( A )T , BT2 ]T
because
0 BA X ~
is also solution of (1) but not solution of the JBD problem
0 is not full rank. Idem for ~ B [B1T ( A )T , BT2 ]T X
8
Joint-Block-Diagonalization : our contributions Starting point: the gradient-descent based algorithms JBDOG and JBDORG of [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010] for Non-Unitary JBD can only handle the case where A is square (I=N). Main motivation: Propose a novel technique to solve non-unitary JBD problems that can handle exactly-, over- and under- determined cases (i.e., A may be square, tall or fat). Main contributions: Formulate JBD as a tensor decomposition fitting problem; Build a Conjugate Gradient (CG) algorithm to compute the tensor decomposition; In the over-determined case, build a good starting point for any JBD algorithm; Application: blind source separation via Independent Subspace Analysis (ISA).
9
JBD in tensor format : link to BCD XK
X1
L1
=
D1K
LR
I A1
AR
D11
0
0
0
A1T
D RK
A RT
D R1
X R
=
r 1
Lr
I Ar
A rT
Dr
R
X Dr 1 A r 2 A r r 1
JD (or X
R
* D A A r 1 r 2 r r 1
in case of hermitian symmetry)
particular case of candecomp-parafac
JBD particular case of BCD-(Lr, Mr, .), (« BlockComponent-Decomposition in rank-(Lr, Mr, .) terms ») 10
JBD in tensor format : conditions for essential uniqueness BCD-(Lr , Mr , .) :
R
X Dr 1 A r 2 B r r 1
where A=[A1,…,AR] is I by N B=[B1,…,BR] is J by Q
Theorem [De Lathauwer, 2008]: Suppose that rank(A)=N, rank(B)=Q, K>2 and that the tensors {Dr}r=1,…,R are generic, then the BCD-(Lr , Mr , .) of X is essentially unique (Sufficient condition). R
JBD :
X Dr 1 A r 2 A r
where
A=[A1,…,AR] is I by N
r 1
The same theorem can be invoked (the proof still holds with A instead of B) In summary, it means that JBD is generically unique if K>2
and rank(A)=N
This is only a sufficient condition: uniqueness still holds but is harder to prove in several cases where the condition is not satisfied. For instance, uniqueness may still hold when rank(A)=I (A fat, I
