framework for neurosphere growth modelling under phase

/48. FRAMEWORK. Analysis. Micro sco p e. Ima g e. Analysis Module: Extraction of relevant information from image. - neurosphere detection. - image restoration.
7MB taille 1 téléchargements 101 vues
FRAMEWORK FOR NEUROSPHERE GROWTH MODELLING UNDER PHASE CONTRAST MICROSCOPE Stéphane U. Rigaud! Mars, 10th 2014 www.ipal.cnrs.fr

NEURAL STEM CELL ‣

"Undifferentiated biological cell with the following properties: selfrenewal, multi-potency, and tissue (re)generation" (wikipedia)!

"2 /48

NEURAL STEM CELL ‣

"Undifferentiated biological cell with the following properties: selfrenewal, multi-potency, and tissue (re)generation" (wikipedia)!



Neural stem cell history:! -

first described in 1989 (Temple, 89)!

-

first isolated in 1992 (Reynolds and Weiss, 92)!

"2 /48

NEURAL STEM CELL ‣

"Undifferentiated biological cell with the following properties: selfrenewal, multi-potency, and tissue (re)generation" (wikipedia)!



Neural stem cell history:!



-

first described in 1989 (Temple, 89)!

-

first isolated in 1992 (Reynolds and Weiss, 92)!

Clinical application of NSC for regenerative medicine (Lindwall et al., 2004)

"2 /48

NEURAL STEM CELL ‣

Free-floating spherical cluster of neural stem cells and progenitor stem cells

confocal

x4 x6

"3 /48

NEURAL STEM CELL ‣

Free-floating spherical cluster of neural stem cells and progenitor stem cells

confocal

x4 x6

"3 /48

NEURAL STEM CELL ‣

Neurosphere Formation Assay (Reynolds and Weiss, 92)

"4 /48

NEURAL STEM CELL ‣

Neurosphere Formation Assay (Reynolds and Weiss, 92)

a - cell extraction from mouse brain

"4 /48

NEURAL STEM CELL ‣

Neurosphere Formation Assay (Reynolds and Weiss, 92)

b - floating culture in serum-free solution with growth factor

"4 /48

NEURAL STEM CELL ‣

Neurosphere Formation Assay (Reynolds and Weiss, 92)

c - cell proliferation under incubation conditions (controlled T°, CO2, etc.)

"4 /48

NEURAL STEM CELL ‣

Neurosphere Formation Assay (Reynolds and Weiss, 92)

d - new generation from cultured cells! e - introduction of differentiation factor into the culture "4 /48

NEURAL STEM CELL ‣

Neurosphere Formation Assay (Reynolds and Weiss, 92)

f - differentiation of the cells into specialised brain cells

"4 /48

NEURAL STEM CELL

"5 /48

NEURAL STEM CELL

"5 /48

NEURAL STEM CELL

"5 /48

NEURAL STEM CELL

5 days observation at x10

7 days observation at x4

"5 /48

NEURAL STEM CELL

"6 /48

NEURAL STEM CELL



Enhanced monitoring of the proliferation sequence of the NFA

"6 /48

NEURAL STEM CELL



Enhanced monitoring of the proliferation sequence of the NFA



Gather possible relevant data on cells configuration in the sphere

"6 /48

NEURAL STEM CELL



Enhanced monitoring of the proliferation sequence of the NFA!



Gather possible relevant data on cells configuration in the sphere!



Enhanced observation of drug effects over cells proliferation

"6 /48

NEURAL STEM CELL

Improve knowledge and control on neural stem cell for clinical application

"6 /48

FRAMEWORK ‣

Determine cells configuration over time of the neurosphere growth!

"7 /48

FRAMEWORK ‣

Determine cells configuration over time of the neurosphere growth!



Limitations of the experiments! -

phase contrast microscopy!

-

no bio-marker!

-

low temporal resolution (15 min/img)!

-

dimensional difference between 3-D culture and 2-D modality

"7 /48

FRAMEWORK

"8 /48

Microscope Image

FRAMEWORK

1 3 2 4 n

t site 1

"8 /48

t

t site 2

. . .t site 3

site n

Microscope Image

FRAMEWORK

Analysis

Analysis Module: Extraction of relevant information from image! -

neurosphere detection!

-

image restoration!

-

cell detection "8 /48

Microscope Image

FRAMEWORK

Analysis

Synthesis

Prior Knowledge

Synthesis Module: Generation of multiple possible cell configuration! -

prior knowledge!

-

information from analysis module

"8 /48

Microscope Image

FRAMEWORK

Analysis Selection: Registration - Optimisation Synthesis

Prior Knowledge

Selection Module: Rate and rank all the models based on the image! -

3-D to 2-D registration

"8 /48

Analysis Selection: Registration - Optimisation Synthesis

Prior Knowledge

Visualisation: Merging of both model and image

"8 /48

Visualisation

Microscope Image

FRAMEWORK

Selection: Registration - Optimisation Synthesis

Prior Knowledge

"8 /48

Best models

Analysis

Visualisation

Microscope Image

FRAMEWORK

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"9 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"9 /48

ANALYSIS MODULE ‣

Analyse and extract information from the microscope image! !

"10 /48

ANALYSIS MODULE ‣

Analyse and extract information from the microscope image! !



Three processes pipeline Neurosphere Detection

Phase Contrast Restoration

"10 /48

Cell Detection

ANALYSIS MODULE Neurosphere Detection

"11 /48

ANALYSIS MODULE Neurosphere Detection



Temporal detection using ∑ - ∆ filter (Manzanera and Richefeu, 07)

"11 /48

ANALYSIS MODULE Neurosphere Detection



Temporal detection using ∑ - ∆ filter (Manzanera and Richefeu, 07)

"11 /48

ANALYSIS MODULE Neurosphere Detection



Temporal detection using ∑ - ∆ filter (Manzanera and Richefeu, 07)

"11 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Phase contrast image restoration (Yin et. al., 12)

"12 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Phase contrast image restoration (Yin et. al., 12)

"12 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Phase contrast image restoration (Yin et. al., 12)

"12 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Phase contrast image restoration (Yin et. al., 12)

"12 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Phase contrast image restoration (Yin et. al., 12)

"12 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Phase contrast image restoration (Yin et. al., 12)

"12 /48

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Detection using circle fitting process

"13 /48

Cell Detection

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Detection using circle fitting process

"13 /48

Cell Detection

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Cell Detection

Detection using circle fitting process Neurosphere Detection

ROI

Phase Contrast Restoration

Level Set

Centroids Heat Map

"13 /48

Circle Filter

Output Detection

ANALYSIS MODULE Neurosphere Detection



Phase Contrast Restoration

Cell Detection

Detection using circle fitting process Neurosphere Detection

ROI

Phase Contrast Restoration

Level Set

Centroids Heat Map

"13 /48

Circle Filter

Output Detection

ANALYSIS MODULE raw

restored

process

"14 /48

results

ANALYSIS MODULE



Precision

Recall

F-score

0.874

0.898

0.886

19 time lapse sequences from 130 to 300 images

"15 /48

ANALYSIS MODULE Precision

Recall

F-score

0.874

0.898

0.886



19 time lapse sequences from 130 to 300 images



Process limits! -

dust!

-

large cell deformation

"15 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"16 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"16 /48

SYNTHESIS MODULE ‣

Generates a set of possible cells configurations using! -

analysis module!

-

prior knowledge!

"17 /48

SYNTHESIS MODULE ‣



Generates a set of possible cells configurations using! -

analysis module!

-

prior knowledge!

Two generation processes! -

evolution algorithm!

-

primal / dual mesh

"17 /48

SYNTHESIS MODULE PRIOR KNOWLEDGE



Three major knowledge

"18 /48

SYNTHESIS MODULE PRIOR KNOWLEDGE



Three major knowledge -

proliferation speed

cell division speed

"18 /48

SYNTHESIS MODULE PRIOR KNOWLEDGE



Three major knowledge -

proliferation speed

-

structural constrains

minimal distance between cells! maximal compression of a cell

"18 /48

SYNTHESIS MODULE PRIOR KNOWLEDGE



Three major knowledge -

proliferation speed

-

structural constrains

-

visual appearance

phase contrast intensity

"18 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM



Search algorithm inspired from biological evolution

"19 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM



Search algorithm inspired from biological evolution Problem Representation

Best Solution

Initial Population

Population Evaluation

Yes

Termination Criteria

Mutating

No

Mating new population generation

"19 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM



Search algorithm inspired from biological evolution Problem Representation

Best Solution

Initial Population

Population Evaluation

Yes

Termination Criteria

Mutating

No

Mating new population generation



Definition of mutation and mating process, based on individual representation

"19 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM

"20 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM

Mutation : local translation

p1

p2

p5

o1 p4 p3

"20 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM

Mutation : local translation

Mating : rand cross mix

P

p1

p1

o1 o2

o1 o5

p4

o4 o3

p3

"20 /48

p4

p5

p2

p5

p2

p3

Q

q1 q2 q5

q4

q3

SYNTHESIS MODULE EVOLUTION ALGORITHM

"21 /48

SYNTHESIS MODULE EVOLUTION ALGORITHM



GPU Implementation: Selection - Generation of population Threads

Model 1

Fitness 1

Threads

Model 2

Selection - Generation of population Threads

...

Fitness 2

Model n

...

Model 1

Model 2

Threads

Threads

Threads

Fitness 1

Fitness 2

Fitness n

Fitness n

Termination decision / New loop

Termination decision / New loop

image-wise

pixel-wise "22 /48

Model n

SYNTHESIS MODULE EVOLUTION ALGORITHM



GPU Implementation:

"23 /48

SYNTHESIS MODULE DELAUNAY MESH



Hypothesis of a sphere packing problem! ! ! !

"24 /48

SYNTHESIS MODULE DELAUNAY MESH



Hypothesis of a sphere packing problem! ! ! !



Use of a Quad Edge Mesh structure (Guibas and Stolfi, 85)! -

constant radius!

-

favor horizontal proliferation before vertical "24 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



Horizontal layer proliferation z d x depth 2r

Depth calculation

"25 /48

SYNTHESIS MODULE DELAUNAY MESH



New layer proliferation

new layer cell layer 2

layer 1 7 cells

"26 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"27 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"27 /48

SELECTION MODULE ‣

Evaluate each generated model based on similarity with the current microscope observation! !



3-D to 2-D registration problematic

"28 /48

SELECTION MODULE Fix Data Y

Fitness Function

Optimisation

Moving Data X

Interpolation

Transform

"29 /48

SELECTION MODULE Fix n-D Data Y

Fitness Function

Optimisation

Interpolation

Moving m-D Data X

Dimensional Relation

"29 /48

Transform

SELECTION MODULE Fix n-D Data Y

Fitness Function

Optimisation

Interpolation

Moving m-D Data X



Dimensional Relation

Dimensional relation in registration (Markelj et. al.,12)

"29 /48

Transform

SELECTION MODULE Fix n-D Data Y

Fitness Function

Optimisation

Interpolation

Moving m-D Data X



Dimensional Relation

Dimensional relation in registration (Markelj et. al.,12)

2D silhouette

3D model

"29 /48

Transform

SELECTION MODULE Fix n-D Data Y

Fitness Function

Optimisation

Interpolation

Moving m-D Data X



Dimensional Relation

Transform

Dimensional relation in registration (Markelj et. al.,12)

2D silhouette

2D silhouette

3D model

3D model

"29 /48

SELECTION MODULE Fix n-D Data Y

Fitness Function

Optimisation

Interpolation

Moving m-D Data X



Dimensional Relation

Transform

Dimensional relation in registration (Markelj et. al.,12)

2D silhouette

2D silhouette

3D model

2D silhouette

3D Reconstruction

3D model

3D Data

"29 /48

SELECTION MODULE m1 p1 E1

E2

Obs I

m2 p2

Ei

mi pi

"30 /48

SELECTION MODULE m1 p1 E1

E2

Obs I

m2 p2

Ei

mi pi

"31 /48

SELECTION MODULE

m1 p1 E1

E2

Obs I

m2 p2

Ei

mi pi

"31 /48

SELECTION MODULE Shape

Texture

Distance

Obs (I) m1 p1 E1

E2

m2

Model (m)

p2

d

d d

Obs I

Ei

mi pi

d

d Registration

"31 /48

d

SELECTION MODULE Fix n-D Data Y

Fitness Function

Optimisation

Interpolation

Moving m-D Data X



Dimensional Relation

Gradient descent optimisation process! !



Rigid geometrical transform! !



Linear interpolation "32 /48

Transform

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"33 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"33 /48

RESULTS AND VISUALISATION ‣

Application of the framework on different datasets! ! ! ! !

"34 /48

RESULTS AND VISUALISATION ‣

Application of the framework on different datasets! ! ! ! !

"34 /48

RESULTS AND VISUALISATION ‣

Application of the framework on different datasets! ! ! ! !



Observation of the results! -

over an entier sequence!

-

at different time of the proliferation "34 /48

RESULTS AND VISUALISATION raw

restored

process

"35 /48

results

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"36 /48

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"36 /48

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"36 /48

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"37 /48

RESULTS AND VISUALISATION PRIMAL DUAL DELAUNAY

"38 /48

RESULTS AND VISUALISATION PRIMAL DUAL DELAUNAY

"39 /48

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"40 /48

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"40 /48

RESULTS AND VISUALISATION EVOLUTION ALGORITHM METHOD

"40 /48

RESULTS AND VISUALISATION PRIMAL DUAL DELAUNAY

"41 /48

RESULTS AND VISUALISATION PRIMAL DUAL DELAUNAY

"41 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"42 /48

CONTENT ‣

Introduction!



Analysis Module!



Generation Module!



Selection Module!



Results and Visualisation!



Conclusion

"42 /48

CONCLUSION ‣

Modelling framework for the observation of neural stem cell proliferation in vitro culture! !

Selection: Registration - Optimisation Synthesis

Prior Knowledge

"43 /48

Best models

Analysis

Visualisation

Three modules based framework

Microscope Image



CONCLUSION ‣

Methodology for neurosphere observation! -

visualisation modality!

-

structure monitoring! !



A push toward the promotion of 3-dimensional culture

"44 /48

CONCLUSION ‣

Two modelling methods

"45 /48

CONCLUSION ‣

Two modelling methods -

evolution algorithm

High flexibility Low control! Low dynamical information

"45 /48

CONCLUSION ‣

Two modelling methods -

evolution algorithm

-

primal dual mesh structure

High flexibility

High control! High dynamical information

Low control! Low dynamical information

Low flexibility

"45 /48

CONCLUSION PERSPECTIVES



Middle ground between EA and Delaunay generation process

"46 /48

CONCLUSION PERSPECTIVES



Middle ground between EA and Delaunay generation process



Cross validation of both models! Iterative Mesh Approach

!

t (time step) mesh 1

mesh 2

!

mesh n

new mesh n

mesh n+1

evo n

evo n

evo n+1

...

! evo 1

evo 2

t (time step)

Evolution Model Approach

"46 /48

CONCLUSION PERSPECTIVES



Middle ground between EA and Delaunay generation process



Cross validation of both models! Iterative Mesh Approach

!

t (time step) mesh 1

mesh 2

!

mesh n

new mesh n

mesh n+1

evo n

evo n

evo n+1

...

! evo 1

evo 2

t (time step)

Evolution Model Approach



Merge both processes toward a mesh structure generated using evolution algorithm

"46 /48

CONCLUSION PERSPECTIVES

!



Build a model database! !



Analysis on! -

possible patterns of configuration!

-

drug effects on proliferation and configuration

"47 /48

CONCLUSION PERSPECTIVES



Apply on new microscopy modalities! -

light sheet microscopy!

-

Integrated Autonomous Microscope System project (A*STAR JCO grant)

"48 /48

! ‣ International Conferences! - S. U. Rigaud, C.-H. Huang, S. Ahmed, Joo-Hwee Lim, and D. Racoceanu. "An analysis- synthesis approach for neurosphere modeling." EMBC, pp 1–6, 2013.! - S. U. Rigaud, N. Loménie, S. Sankaran, S. Ahmed, Joo-Hwee Lim, and D. Racoceanu. "Neurosphere fate prediction: An analysis-synthesis approach for feature extraction". IJCNN, pp 1–7, 2012.! - S. U. Rigaud and Nicolas Loménie. "Neural stem cell tracking with phase contrast video microscopy". SPIE Medical Imaging, pp 796230–796236, 2011.!

! ‣ White Journals! - H. Irshad, S. U. Rigaud, and A. Gouaillard. "Primal/dual mesh with application to triangular/simplex mesh and Delaunay/Voronoi". Insight Journal, pp 1–14, 2012.! - S. U. Rigaud and A. Gouaillard. "Incremental Delaunay triangulation". Insight Journal, pp 1–5, 2012.! - S. U. Rigaud and A. Gouaillard. "Walk in a triangulation : Straight walk". Insight Journal, pp 1–3, 2012.!

! ‣ Peer-reviewed Journals! - In preparation!

!

"49 /48

Stéphane Ulysse Rigaud

Doctoral student UPMC CNRS A*STAR! [email protected]

Image & Pervasive Access Lab

International joint research unit - UMI CNRS 2955! www.ipal.cnrs.fr



Temple, S. “Division and differentiation of isolated CNS blast cells in microculture”. Nature, 340(6233), pp 471 – 473, 1989.!



Reynolds, B. and Weiss, S. “Generation of neurons and astrocytes from isolated cells of the adult mammalian central nervous system”. Science, 255(5052), pp 1707 – 1710, 1992.!



Manzanera, A. and Richefeu, J. C. “A new motion detection algorithm based on ∑ − ∆ background estimation”. Pattern Recognition Letter, 28(3), pp 320 – 328, 2007.!



Yin, Z. et. al. “Understanding the phase contrast optics to restore artefact-free microscopy images for segmentation”. Medical Image Analysis, 16(5), pp1047 – 1062, 2012.!



Guibas, L. J. and Stolfi, J. “Primitives for the manipulation of general subdivisions and the computation of voronoi diagrams”. ACM Transaction on Graphics, 4(2), pp 75 – 123, 1985.!



Markelj, P. et. al. “A review of 3D/2D registration methods for image-guided interventions”. Medical Image Analysis, 16(3), pp 642 – 661, 2012.!

"51 /48