COMBINATION OF SHAPE-CONSTRAINED AND INFLATION DEFORMABLE MODELS WITH APPLICATION TO THE SEGMENTATION OF THE LEFT ATRIAL APPENDAGE Pol Grasland-Mongrain ∗ Ecole Normale Supérieure de Cachan France

ABSTRACT This paper introduces a method for the flexible model-based segmentation of the whole heart from 3D CT images. The novelty of the approach is the combination in a single framework of two types of deformable models. The anatomical structures with well defined shapes (like the cardiac chambers) are segmented with deformable models constraining the deforming surface to stay close to some shape prior. On the other hand, structures with highly variable shapes are extracted by locally inflating the deforming surface without making assumption on the shape of the object to segment. The proposed method has been applied to the segmentation of the heart of 17 patients. Cardiac chambers and major vessels were segmented using shape-constrained deformable models while the left atrial appendage (LAA) was extracted using the mesh inflation. Qualitatively, the mesh resulting from the inflation adapts well to the difficult shape of the LAA with some difficulties to reach the very tip of this elongated structure. These results are numerically confirmed with manually generated reference segmentations. Index Terms— model-based segmentation, deforming surface, mesh inflation, cardiac model, left atrial appendage 1. INTRODUCTION The accurate segmentation of the heart, i.e., the process of assigning labels to region in the image, is an important process in the cardiovascular diseases diagnosis. Even if the segmentation could be manually done, it is practically impossible in daily clinical routine and much efforts have been spent on the development of semi- or fully automatic approaches. In particular, deformable models [1] have been widely used for this purpose [?]. For the purpose of the chamber segmentation, shape priors showed to be useful to constraint the model deformation. A high degree of automatization could be achieved with application to computed tomography (CT) (see e.g. [2, 3]) or magnet resonance imaging (MRI) (see ∗ [email protected] † [email protected]

Jochen Peters, Olivier Ecabert† Philips Research Europe–Aachen Weisshausstr. 2, 52066 Aachen Germany e.g. [4, ?]). However, the constraints imposed by the shape prior may be too strong for substructures with highly variable shape like the left atrial appendage (LAA), which is the purpose of this work. The LAA is a substructure of the heart above the left ventricle and connected to the left atrium. It has an highly variable shape, often tubular, hooked and with a few lobes. Its size varies from 1 to 19 cm3 [5]. It has some notable functions including the maintenance and the regulation of the heart function, and is involved in various heart diseases like thrombosis building, cardiac fibrillation, etc [6].

Fig. 1. Position of the LAA in the heart. Here, only the base of the LAA is highlighted, in red. The method presented in this article builds upon a multistep framework for the automatic segmentation of the whole heart and the major vessels in CT images introduced in [2]. Once the multi-compartment heart model is adapted to the patient’s anatomy, a high resolution surface at the interface between the left atrium and the LAA is inflated into the LAA under the action of region forces without making explicit assumptions on the shape of the object being segmented. The novelty of the approach is that both the shape-constrained and the inflation deformable models are integrated into a single framework. This paper is structured as follows. Section 2 briefly outlines the existing framework for whole heart segmentation [2] for completeness (blue components in Fig. 2). The new inflation algorithm (red component in Fig. 2) is described in Section 3 and evaluated in Section 4.

Segmentation Chain

New Image

1. Heart Detection

2. Parametric Adaptation (Similarity)

3. Parametric Adaptation (Piecewise Affine)

4. Deformable Adaptation

Segmented Image

5. Left Atrium Appendage Inflation

Fig. 2. Chain of modules combining the shape-constrained and inflation deformable models for whole heart segmentation.

2.3. Deformable Adaptation In the fourth step of Fig 2, each vertex v i is allowed to move freely and the mesh adaptation performed minimizing an energy function made of two contributions. The external energy introduced above is still used to attract the model towards the image boundaries while the vertex displacements are constrained by an internal energy which penalizes deviations of the deformed model from the reference shape E = α · Eexternal + Einternal ,

2. HEART SEGMENTATION WITH SHAPE-CONSTRAINED DEFORMABLE MODELS Automatic whole heart segmentation of the chambers is achieved in several steps which are briefly summarized below. More details can be found in [2]. 2.1. Heart Detection In the first step, a modified Generalized Hough Transform is used to roughly localize the heart in the images and adapt the size of the model [?, 2]. 2.2. Parametric Adaptation The second and third steps in Fig. 2 adapt the model to the image by optimizing the parameters of a parametric transformation. In step 2 pose misalignment is compensated by a similarity transformation, while in step 3 a multi-affine transformation is optimized where each of the anatomical regions (left and right ventricles, left and right atria, and trunk of the great arteries) is assigned an affine transformation to capture changes in size and rotation of the chambers between patients. Practically, this adaptation is performed iterating two steps until the mesh reaches some steady state. In the first step, candidate points are detected in the image maximizing a boundary detection function, which is evaluated for each triangle along its normal vector. The point with the highest response is kept as the target point xtarget . i In the second step, the parametric transformations are optimized by minimizing the following external energy: Eexternal =

T X i=1

 wi

2 ∇I(xtarget ) target i · (x − c ) (1) i i k ∇I(xtarget )k i

where the sum is performed over the mesh triangles. In this energy, the triangle centers ci are attracted towards target target target points xi . The projection of (xi − ci ) onto the normal vector ∇I/ k ∇I k at the target point makes the energy invariant to movements of the triangle within the object tangent plane, avoiding that the point keeps stuck at the target position. The weights wi are large for reliably detected target points and small otherwise.

(2)

with α a weighting factor and Einternal =

V X X

((v i − v j ) − (T [mi ] − T [mj ]))2 , (3)

i=1 j∈N (i)

with N (i) the set of indices of the neighbor vertices of vertex v i , and mi the vertex coordinates of the reference model undergoing the multi-affine transformation T [.]. As in the previous section, mesh adaptation is performed by iterating the boundary detection step and the minimization of the Eq. (2). 3. LEFT ATRIAL APPENDAGE SEGMENTATION WITH INFLATION DEFORMABLE MODEL Thanks to the four precedent steps, (1) the position of the interface LAA - Left Atrium is known, and (2) the surrounding substructures of the LAA (myocardium, left atrium...) are already segmented. The proposed method inflates a high-resolution mesh iteratively by minimizing the sum of two energies: one external energy, which inflates the mesh through the LAA, and one internal energy, which constrains the mesh, as described by the equation (4) : E = α · Eexternal + Einternal ,

(4)

3.1. Region-Based External Energy We called the external energy region-based. This energy is based on the gray values of the voxels. This energy moves the triangles close to the boundary by using a gray value threshold between LAA and background. This threshold is computed once at the beginning of the proposed method. To find the threshold, we applied a method which uses substructures surrounding the LAA: the left atrium and the myocardium. Indeed, the LAA, rather bright, has almost the same gray value as the left atrium, while the background, darker, has almost the same gray value as the myocardium. We have thus two grayscale histograms, respectively of left atrium and of myocardium, simply calculated by counting the number of left atrium and myocardium voxels of each gray value. The threshold is so the gray value which has the

smallest number of darker left atrium voxels and brighter myocardium voxels. Then, for each step, the energy attracts the triangles centers ci to target point xtarget . This target point depends on the i gray value at the location of the triangle center. If this gray value is: 1. above the threshold, ci is supposed inside the LAA, and is set along the normal of the triangle pointing xtarget i outside ; 2. under the threshold, ci is supposed outside the LAA, is set along the normal of the triangle pointand xtarget i ing inside ; 3. almost equal to the threshold, ci is supposed on the boundary, and xtarget is equal to ci . i In the first two cases, xtarget is set at a distance of 1, then 2, i and finally 3 millimeters from ci , if at each of these positions, the point doesn’t belong to another substructure and is not on the other side of the boundary from ci .

(c) Before growing

(d) α = 0.2

(g) α = 1

(h) α = 2

3.2. Mesh Reference Internal Energy We called the internal energy Mesh Reference. This energy is the one presented in [2] and reminded in section 2.3: it penalizes the deformations of the mesh. There is one particularity however: after each step, the reference mesh is updated to the current shape of the mesh. 3.3. Loop Repair During Growing During the inflation, some loops often appear. In a loop, some triangles are intersecting each other, the triangles between are pointing inside the mesh, and it can degenerate if it is not removed. The algorithm uses consequently a method created by [7] which detects all the intersecting triangles. It selects then the neighbours until the N-th order of the intersecting triangles, in order to select all the loop triangles, and applies a smoothing energy. Finally, the algorithm counts a second time the number of intersecting triangles. Further inflations are allowed only if this number is small enough. During our tests, we determined that the best smoothing energy was the Mesh Reference energy, that the 3d order was sufficient to select all the loop triangles, and the inflation continues with less than 10 intersecting triangles. However, the repair is not efficient enough, and new loops are always reappearing at the same places at the next iterations. 4. MAIN RESULTS The algorithm takes about 50 seconds, with about 20 seconds for the proposed method, on a Intel Xeon at 2,4 Ghz with 3 GB of RAM.

Fig. 3. Example of inflation. First picture: 2D slice with the LAA contours in dark green and left atrium in bright green (myocardium not visible). Second picture: 3D view of the mesh. Fig. 3 shows an example of inflation. In the proposed method, the weight α in the equation (4) is successively set to 0.2, 1, 2 and 5, with 5 iterations and one loop repair after each change. The external energy becomes thus stronger and stronger in order to help the mesh to reach the far boundaries of the LAA. These parameters have been experimentally chosen to be the best among about 5 selected patients. Numerical results computed by comparing manually segmented voxels (ground truth) and algorithm-segmented vox-

Left Atrial Appendage Inflation Results Specificity = True Pos. / (True Pos. + False Neg.)

MeshLabs software. Finally, we wish to cite a teacher, H. Delingette, who created the connection between us!

els for 17 patients are presented Fig. 4.

Quality = True Pos. / (True Pos. + False Pos.)

100 80

7. REFERENCES

60

[1] T. McInerney and Demetri Terzopoulos, “”Deformable models in medical image analysis: a survey”,” Medical Image Analysis, vol. 1, no. 2, pp. 91–108, 1996.

40

20 0

Fig. 4. Numerical evaluation of the LAA segmentation for 17 patients. The blue bar is the sensitivity and the red bar the positive prediction value. The blue bar is the Sensitivity: it represent the procentage of voxels belonging to the LAA which have been segmented by the algorithm, and is calculated by the formula (5): Sensitivity =

T rueP ositive T rueP ositive + F alseN egative

(5)

The red bar is the Positive Predictive Value (PPV): it represent the procentage of voxels segmented by the algorithm which really belong to the LAA, and is calculated with the formula (6): T rueP ositive (6) T rueP ositive + F alseP ositive The main interpretation is that the mesh has some difficulties to reach the tip of the LAA, as illustrated by the sometimes low sensitivity, but there are very few segmentation errors, with a good adaptation to the shape of the LAA, as shown by the high PPV. We can observe one major fail, the patient 14 (low sensitivity and PPV), and three not enough satisfying patients with a sensivity smaller than 60%. These fails are mainly due to small segmentation errors occuring near the base of the LAA in the method from [2]. PPV =

5. CONCLUSION AND FUTURE WORK This article presents a method based on two types of deformables models integrated in a common framework: one, more robust, stays close to the a priori shape, as described in [2], and the other one, more flexible, inflates a high resolution mesh, as introduced in this article. The loop repair is currently not enough reliable and could be improved. The inflation method could be applied to other highly variable substructure of the heart. 6. ACKNOWLEDGEMENTS We would like to thank H. Lehmann and R. Kneser from Philips Research Europe–Aachen for their support. In addition, we thank P. Cignoni and his team for creating the

[2] Olivier Ecabert, Jochen Peters, H. Schramm, C. Lorenz, J. von Berg, M. Walker, M. Vembar, M. Olszewski, K. Subramanyan, G. Lavi, and Jürgen Weese, “”Automatic Model-Based Segmentation of the Heart in CT Images”,” IEEE Transactions on Medical Imaging, vol. 27, sept 2008. [3] Y. Zheng, A. Barbu, B. Georgescu, M. Scheuering, and D. Comaniciu, “”Four-Chamber Heart Modeling and Automatic Segmentation for 3D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features”,” IEEE Transactions on Medical Imaging, 2008. [4] J. Lötjönen, S. Kivistö, J. Koikkalainen, D. Smutek, and K. Laurema, “”Statistical shape model of atria and ventricles and epicardium from short- and long-axis MR images”,” Medical Image Analysis, vol. vol 8, pp. 371–286, 2004. [5] John P. Veinot, Phillip J. Harrity, Federico Gentile, Bijoy K. Khandheria, Kent R. Bailey, Jeffrey T. Eickholt, James B. Seward, A. Jamil Tajik, and William D. Edwards, “”Anatomy of the normal left atrial appendage: a quantitative study of age-related changes in 500 autopsy hearts: implications for echocardiographic examination”,” Circulation 96, vol. 9, pp. 3112–3115, 1997. [6] Claudia Stöllberger, Birke Schneider, and Josef Finsterer, “”Elimination of the Left Atrial Appendage To Prevent Stroke or Embolism?”,” Chest 124, vol. 6, pp. 2356– 2362, dec 2003. [7] Philippe Guigue and Olivier Devillers, “”Fast and Robust Triangle-Triangle Overlap Test using Orientation Predicates”,” Journal of graphics, gpu, and game tools, vol. 8, no. 1, pp. 25–42, 2003.