combination of shape-constrained and inflation deformable models

In the second step, the parametric transformations are op- timized by minimizing ... The edge-based energy, described in the section [?] as the "ex- ternal energy" ...
1MB taille 6 téléchargements 269 vues
COMBINATION OF SHAPE-CONSTRAINED AND INFLATION DEFORMABLE MODELS WITH APPLICATION TO THE SEGMENTATION OF THE LEFT ATRIAL APPENDAGE Pol Grasland-Mongrain ∗ Ecole Normale Supérieure de Cachan France

ABSTRACT This paper introduces a method for the flexible model-based segmentation of the whole heart from 3D CT images. The novelty of the approach is the combination in a single framework of two types of deformable models. The anatomical structures with well defined shapes (like the cardiac chambers) are segmented with deformable models constraining the deforming surface to stay close to some shape prior. On the other had, structures with highly variable shapes are extracted by locally inflating the deforming surface without making assumption on the shape of the object to segment. The proposed method has been applied to the segmentation of the heart of 17 patients. Cardiac chambers and major vessels were segmented using shape-constrained deformable models while the left atrial appendage (LAA) was extracted using the mesh inflation. Qualitatively, the mesh resulting from the inflation adapts well to the difficult shape of the LAA with some difficulties to reach the very tip of this elongated structure. These results are numerically confirmed with manually generated reference segmentations. Index Terms— model-based segmentation, deforming surface, mesh inflation, cardiac model, left atrial appendage 1. INTRODUCTION The accurate segmentation of the heart, i.e., the process of assigning labels to region in the image, is an important process in the diagnosis cardiovascular diseases. Even if the segmentation could be manually done, it is practically impossible in daily clinical routine and much efforts have been spent on the development of semi- or fully automatic approaches. In particular, deformable models [1] have been widely used for this purpose [2]. For the purpose of the chamber segmentation, shape priors showed to be useful to constraint the model deformation. A high degree of automatization could be achieved with application to computed tomography (CT) (see e.g. [3, 4]) or magnet resonance imaging (MRI) (see ∗ [email protected][email protected]

Jochen Peters, Olivier Ecabert† Philips Research Europe–Aachen Weisshausstr. 2, 52066 Aachen Germany e.g. [5, 6]). However, the constraints imposed by the shape prior may be too strong for substructures with highly variable shape like the left atrial appendage (LAA), which is the purpose of this work. The LAA is a substructure of the heart above the left ventricle and connected to the left atrium. It is involved in various heart diseases, including thrombosis building, cardiac fibrillation. The LAA has an highly variable shape, often tubular, hooked and with a few lobes. Its size varies from 1 to 19 cm3 [7]. Although the function of the LAA is not perfectly known, most physicians think today that it serves as a reservoir for the atrium, and helps the maintenance and the regulation of the heart function. It has been observed that its elimination causes various heart problems [8]. The method presented in this article builds upon a multistep framework for the automatic segmentation of the whole heart and the major vessels in CT images introduced in [3]. Once the multi-compartment heart model is adapted to the patient’s anatomy, a high resolution surface at the interface between the left atrium and the LAA is inflated into the LAA under the action of region forces without making explicit assumptions on the shape of the object being segmented. The novelty of the approach is that both the shape-constrained and the inflation deformable models are integrated into a single framework. This paper is structured as follows. Section 2 briefly outlines the existing framework for whole heart segmentation [3] for completeness (blue components in Fig. 1). The new inflation algorithm (red component in Fig. 1) is described in Section 3 and evaluated in Section 4. Segmentation Chain

New Image

1. Heart Detection

2. Parametric Adaptation (Similarity)

3. Parametric Adaptation (Piecewise Affine)

4. Deformable Adaptation

Segmented Image

5. Left Atrium Appendage Inflation

Fig. 1. Chain of modules combining the shape-constrained and inflation deformable models for whole heart segmentation.

2. HEART SEGMENTATION WITH SHAPE-CONSTRAINED DEFORMABLE MODELS Automatic whole heart segmentation of the chambers is achieved in several steps which are briefly summarized below. More details can be found in [3]. 2.1. Heart Detection In the first step, a modified Generalized Hough Transform is used to roughly localize the heart in the images and adapt the size of the model [9, 3]. 2.2. Parametric Adaptation The second and third steps in Fig. 1 adapt the model to the image by optimizing the parameters of a parametric transformation. In step 2 pose misalignment is compensated by a similarity transformation, while in step 3 a multi-affine transformation is optimized where each of the anatomical regions (left and right ventricles, left and right atria, and trunk of the great arteries) is assigned an affine transformation to capture changes in size and rotation of the chambers between patients. Practically, this adaptation is performed iterating two steps until the mesh reaches some steady state. In the first step, candidate points are detected in the image maximizing a boundary detection function, which is evaluated for each triangle along its normal vector. The point with the highest response is kept as the target point xtarget . i In the second step, the parametric transformations are optimized by minimizing the following external energy: Eexternal =

T X i=1

 wi

2 ∇I(xtarget ) target i · (xi − ci ) (1) k ∇I(xtarget )k i

where the sum is performed over the mesh triangles. In this energy, the triangle centers ci are attracted towards target target target points xi . The projection of (xi − ci ) onto the normal vector ∇I/ k ∇I k at the target point makes the energy invariant to movements of the triangle within the object tangent plane, avoiding that the point keeps stuck at the target position. The weights wi are large for reliably detected target points and small otherwise. 2.3. Deformable Adaptation In the fourth step of Fig 1, each vertex v i is allowed to move freely and the mesh adaptation performed minimizing an energy function made of two contributions. The external energy introduced above is still used to attract the model towards the image boundaries while the vertex displacements are constrained by an internal energy which penalizes deviations of the deformed model from the reference shape E = α · Eexternal + Einternal ,

(2)

with α a weighting factor and Einternal =

V X X

((v i − v j ) − (T [mi ] − T [mj ]))2 , (3)

i=1 j∈N (i)

with N (i) the set of indices of the neighbor vertices of vertex v i , and mi the vertex coordinates of the reference model undergoing the multi-affine transformation T [.]. As in the previous section, mesh adaptation is performed by iterating the boundary detection step and the minimization of the Eq. (2). 3. LEFT ATRIAL APPENDAGE SEGMENTATION WITH INFLATION DEFORMABLE MODEL The developed algorithm follows the previous steps described above. In this way, the position of the LAA is known before the beginning of the algorithm, and the gray values around this location are known. The principle of the algorithm is to have an high resolution mesh at the base of the LAA, which is inflating through the LAA. This inflation is made by following two types of energy : one external, which adapts the mesh to the image, and one internal, which constrains the mesh to avoid odd shapes and behaviours, all with a fixed topology. In this way, “leakages” of the mesh in other organs than the LAA are avoided. We have tried two different external energies and four different internal. We could use one or any combination of the mentionned energies, with at least one external energy and one internal energy. 3.1. External Energies: Edge-based and Region-based The edge-based energy, described in the section [?] as the "external energy", can’t be properly used in our case. The borders of the LAA are indeed not a priori known, so the edge-feature detector was not reliable enough. That’s why we tried another energy, called region-based. This energy use a grayscale threshold between the LAA and the background. This threshold is done by minimizing the grayscale classification error between the voxels of the left atrium (same gray values as the LAA) and the myocardium (roughly same gray values as the background around the LAA), these substructures being already segmented by the global segmentation method. This energy looks then at each center of triangle. It looks at the gray value at this location, and determine thus if the center is inside or outside the LAA. If this center is inside, the algorithm will search for a target point outside the mesh (inflation of the mesh) ; if it is outside, the algorithm will search for a target point inside the mesh (deflation of the mesh). These target points are computed along the normal of the triangle. Then the algorithm searches for target points farther until either (1) it reaches the maximum number of target points allowed, (2) the gray

value indicates that we have gone through the interface between LAA and background, or (3) the target point is already segmented, and belongs to another substructure. In our tests, we choose target points every 1 millimeter along the normal of the triangle, and 3 maximum target points.

applies to them a smoothing energy. Then, the intersecting faces are again detected, and further inflation are allowed only if this number is small enough. We determined that the best smoothing energy was the Mesh Reference energy. However, the repair is not efficient enough, and the loop are always reappearing at the same place at the next iteration.

3.2. Internal Energies: Triangle Regularization, Curvature, N-Gon Regularization, Mesh Reference

4. MAIN RESULTS

The first energy, described in the section [?] as the "internal energy" and that we called Mesh Reference, penalizes any deformation of the model. There one difference however: in the LAA-inflation algorithm, the reference mesh is regularly updated to the the current mesh. That is, at each iteration, the LAA grows, and the mesh created becomes the new reference mesh. As our mesh is intended to be highly deformed, we tried too other internal energies that we called Triangle Regularization, N-Gon Regularization (which are quite similar) and Curvature. Although these energies were not working as well as the Mesh Reference, we present them quickly above. The Triangle Regularization tries to approximate all the triangles into equilateral triangles. For this, it looks for all triangles, and for each triangle, an equilateral triangle is created. This equilateral triangle is rotated and isotropically scaled to minimize the distance between the corners of the two triangles. The vertices of the initial triangle are thus subjected to foreces which "pull" them into an equilateral triangle. Instead of considering triangles, the N-Gon Regularization looks at the vertices. We call here the vertices N-Gon, with N the number of neighbours of the considered vertex. The N-Gon Regularization tries to approximate all the NGon into regular N-Gon - that is to say, regularly distributed vertices at an equidistance from a reference vertex. For each N-Gon, we count the number of neighbours, and we create a regular N-Gon. This regular N-Gon is then rotated and scaled to minimize the distance between the two N-Gons. The vertices of the initial N-Gon are thus subjected to forces which "pull" them into a regular N-Gon. The Curvature energy will smooth the mesh and remove the peaks. For this, it looks for all vertices, and for each vertex, it looks for the neighbours. It computes then a plane which goes through them ; if there are more than 3 neighbours, the algorithm computes the best fitting plane. The initial vertex will then be "pulled" along the normal of the plane going through itself.

The algorithm has been tested on a Intel Xeon at 2,4 Ghz, with 3 GB of RAM. The whole method takes about 50 seconds, half of it for the presented algorithm. The region-based energy seems to be reliable though, and gives better results than the edge-based which doesn’t have specific edge detection features. For the internal energies, the mesh reference energy has been shown surprisingly efficient, for both of the growing and the loop correction. The step-bystep inflation, with the update of the reference mesh at each iteration, could explain this success. The best parameters experimentally found was to use the weights α between the external over the internal energy at 0.2, 1, 2 and 5. There are 5 iterations and 1 loop repair at every change of weight. Fig. 2 shows an example of inflation. Numerical results for 17 patients are presented under. The first bar, in blue, is the Sensitivity (or Recall), and represents the procentage of voxels belonging to the LAA which have been segmented by the algorithm. Wen can calculate it with the formula [?] : Sensitivity =

T rueP ositive T rueP ositive + F alseN egative

(4)

The second bar, in red, is the Precision (or Positive Predictive Value), and represents the procentage of voxels segmented by the algorithm which really belong to the LAA. We can calculate it with the formula [?] : P recision =

T rueP ositive T rueP ositive + F alseP ositive

(5)

The main comment about these results is the difference between the two bars. The mesh doesn’t actually grow far enough, until the end of the LAA (low specificity), but almost all the voxels which are segmented really belong to the LAA (high quality). There is mainly one major fail, the patient 14 (very low specifity and quality), and three or four patient don’t show satisfying enough results. These fails are mainly due to segmentation errors in the global segmentation method.

3.3. Loop Repair During Growing 5. CONCLUSION AND FUTURE WORK The growing makes appear some peaks, which are degenerating into loops, around the mesh. To deal with this problem, the intersecting faces are detected, with a method created by [10]. It searches then for its neighbours to the N-th order, and

The current main method in Philips is done in four steps : heart detection, global and semi-global parametric adaptation, and deformable adaptation. The method is quite efficient,

Left Atrial Appendage Inflation Results Specificity = True Pos. / (True Pos. + False Neg.) Quality = True Pos. / (True Pos. + False Pos.) 100 80 60 40

20 0

Fig. 3. Numerical evaluation of the segmentation of the LAA by the algorithm.

(c) Before growing

(d) α = 0.2

but can have small modelization errors. The Left Atrial Appendage, highly deformable substructure of the heart, is segmented after applying the global segmentation method. The developed algorithm inflates a flat mesh through the LAA, following an internal energy called Mesh Reference and an external energy called region-based. The results are satisfying in the majority of cases, but they can be improved, especially on the specificity. The algorithm could give better results with a another loop correction. 6. ACKNOWLEDGEMENTS We would like to thank H. Lehmann and R. Kneser from Philips Research Europe–Aachen for their support. In addition, we thank P. Cignoni and his team for creating the MeshLabs software. Finally, we wish to cite a teacher, H. Delingette, who created the connection between us! 7. REFERENCES [1] T. McInerney and D. Terzopoulos, “Deformable models in medical image analysis: a survey,” Medical Image Analysis, vol. 1, no. 2, pp. 91–108, 1996.

(g) α = 1

(h) α = 2

Fig. 2. Example of inflation. First picture : 2D slice with the LAA contours in green. Second pictures : 3D view of the mesh.

[2] A. F. Frangi, W. J. Niessen, and M. A. Viergever, “Three-dimensional modeling of functional analysis of cardiac images: A review,” IEEE Transactions on Medical Imaging, vol. 20, no. 1, pp. 2–25, 2001. [3] O. Ecabert, J. Peters, H. Schramm, C. Lorenz, J. von Berg, M.J. Walker, M. Vembar, M.E. Olszewski, K. Subramanyan, G. Lavi, and J. Weese, “Automatic ModelBased segmentation of the heart in CT images,” IEEE Transactions on Medical Imaging, vol. 27, no. 9, pp. 1189–1201, September 2008. [4] Y. Zheng, A. Barbu, B. Georgescu, M. Scheuering, and D. Comaniciu, “Four-chamber heart modeling and au-

tomatic segmentation for 3d cardiac CT volumes using marginal space learning and steerable features,” IEEE Transactions on Medical Imaging, 2008. [5] J. Lötjönen, S. Kivistö, J. Koikkalainen, D. Smutek, and K. Laurema, “Statistical shape model of atria and ventricles and epicardium from short- and long-axis MR images,” Med. Image Analysis, vol. 8, pp. 371–286, 2004. [6] H. C. van Assen, M. G. Danilouchkine, A. F. Frangi, S. Ordás, J. J. M. Westenberg, J. H. C. Reiber, and B. P. F. Lelieveldt, “SPASM: A 3D-ASM for segmentation of sparse and arbitrarily oriented cardiac MRI data,” vol. 10, pp. 286–303, 2006. [7] J.P. Veinot, P.J. Harrity, and F. Gentile et al., “Anatomy of the normal left atrial appendage: a quantitative study of age-related changes in 500 autopsy hearts: implications for echocardiographic examination,” Circulation 96, vol. 9, pp. 3112–3115, 1997. [8] C. Stöllberger, B. Schneider, and J. Finsterer, “Elimination of the left atrial appendage to prevent stroke or embolism?,” Chest 124, vol. 6, pp. 2356–2362, dec 2003. [9] D. H. Ballard, “Generalizing the Hough transform to detect arbitrary shapes,” Pattern Recogn., vol. 13, no. 2, pp. 111–122, 1981. [10] P. Guigue and O. Devillers, “Fast and robust triangletriangle overlap test using orientation predicates,” Journal of graphics, gpu, and game tools, vol. 8, no. 1, pp. 25–42, 2003.