CRACK DETECTION Carlos Becker, Rocio Cabrera, Guillaume Lemaitre, Ali Mirzai, Mojdeh Rastgoo and Alexandru Rusu
Presentation Outline 2
Introduction: cracks, crack detection & groundtruth generation
Detection using clustering methods (Unsupervised)
Detection using SVMs and local features (Supervised)
Further work
Conclusion
Introduction 3
Crack detection is of great importance in the wood industry to determine whether a piece of wood can be sold or not for a given application
False positives or false negatives are both undesired and lead to financial loss, and crack detection is not an easy task!
Introduction 4
Appropriate and correct ground truth is essential in order to classify and assess results properly
However, the provided data was not labelled and the location of wood cracks was provided separately in Word files This required an important amount of work to generate proper ground truth
There are still many doubts about what is crack/non-crack and the final ground truth is likely to be biased, with many incorrect labels
Introduction Available data and channels 5
Each wood acquisition is made of 4 different images (not necessarily aligned)
IR Image
SC Image
RD Image
3D Image
Detection through clustering 6
K-means, wavelet features & color histograms
Sensor fusion with thresholding techniques
Sobel and Gabor filters
Fuzzy c-means & phase-congruency
Hessian and Hough Transform
K-means classifier
7
Wavelet features
8
Color histogram
Information regarding to the cracks can be differentiated bye the information of color and intensity 1- RGB Lab 2- Define a neighboring windows around any pixel n*n 3- Calculate the color histogram 4- Find the local maximum with hill-climbing 5- Use k-means to classify the pixels
9
Color histogram
If Ir, Sc., Rd. channels are merged before this step and RGB image is used for processing the result will be much more better. (before any region cropping)
10
Result
11
Discussion
Analysis based on wavelet is time consuming and can detect crack with very small size and thin, therefore it is not suitable for real-time processing On the contrary, analysis based on color histogram is really fast and can detect most of the medium and large size crack but week in tiny crack detection thus it is suitable for real-time processing
12
Crack detection using thresholding and morphological operations 13
Popular methods: Histogram analysis Adaptive thresholding Gaussian modeling Morphological operations
Disadvantages: False detections Highly dependent on the choice of parameters
Scope:
Fast accuracy/computation speed accuracy(industrial applications)
A priori knowledge 14
SC Image
IR Image
Complemented IR Image
Thresholding and merging 15
Using Otsu’s optimal threshold
SC Binary Image
IR Binary Image
Merged Image
Initial results 16
Ground truth
Bounded Cracks
Highlighted crack pixels
Possible improvements Connect adjacent regions Eliminate small regions Use adaptive thresholds Scan line method Hough Transform
Final results 17
Ground truth
Bounded Cracks
Highlighted crack pixels
Conclusions Simple method Improved accuracy Less false positives Fast computation speed
Crack Segmentation – Sobel Filter 18
Generalized Sobel filter was used to detect the cracks Background Detection
Sobel Filter with Wsize 5
Image Thresholding – Otsu threshold Morphological Operations
Considering the intensity of the pixels based on majority of pixels in neighborhood of 3
Removing the noise by eliminating the areas with less connectivity than 10 pixels
Filling the holes in the crack
Image thresholding
Morphological Operation
Detected Crack in the image
Crack Segmentation – Sobel Filter 19
Result – Sensor1- crak-face2- 3d-croppedL
Drawbacks
Original Image
Sobel Filtered Image
Crack Detection
Useful on 3D images from the sensors
Accuracy = 0.9
Crack Segmentation – Gabor Filter 20
Linear filter for edge detection . Gaussian kernel function modulated by a sinusoidal plane wave. Orientation of the gabor filter
Frequency of sinusoidal function
x and y variance
Crack Segmentation – Gabor Filter 21
Gabor Filter
Combining the results Image thresholding
Detected crack in the image
Crack Segmentation – Gabor Filter 22
Result – Sensor1- crak-face2- 3d-croppedL
Original Image
Ir Iamge
Rd Image
Sc Image
Results from Gabor Filter
Accuracy = 0.95
3D Image
Crack Detection
Fuzzy c-means clustering 23
Original Image
Vessel cluster using fuzzy c-means
Tolias, Y. A. & Panas, S. M., « A fuzzy vessel tracking algorithm for retinal images based on fuzzy clusering Transactions on Medical Imaging, Volume 17, pages 263-273, 1988
», IEEE
Fuzzy c-means clustering 24
Fuzzy c-means clustering 25
Algorithm overview:
Original image
After pre-processing
Cracks cluster from fuzzy c-means
Ground truth
Fuzzy c-means clustering 26
Results:
3D sensor
Ir sensor
Rd sensor
Sc sensor
Ground truth
Fuzzy c-means clustering 27
Problem to determine the « cracks » cluster automatically
Pre processing phase can failed hence clustering too
Phase congruency 28
In the frequency domain, Fourier components are in phase at the point of step (square signal) and peaks (triangle signal)
P. Kosevi, « Image features from phase congruency », Videre: A journal of Computer Vision Research, MIT Press, Volume 1, Number 3, Summer 1999
Phase congruency 29
Phase congruency is illumanition and contrast invariant measures and allows to detect « solid » edges
Original image
Image features from phase congruency
Edge detection using Canny
Phase congruency 30
Phase congruency 31
Algorithm overview:
Original image
Features extraction
Otsu’s thresholding
Label image
Phase congruency 32
Results:
3D sensor
Ir sensor
Rd sensor
Sc sensor
Ground truth
Phase congruency 33
Phase congruency 34
Algorithm & Results:
3D sensor
Sc sensor Combine Image Ir sensor Pattern Recognition Project - Crack Detection 1/12/2011
Ground truth
Phase congruency 35
Results and Improvements:
Accuracy: 0.94 Precision: 0.63 (maybe due of the Parkinson disease of the person making the ground truth) Improvements regarding the combination of the segmented images
Hessian & Hough Transform Analysis Hessian Image 36
Frangi et al. [1] developed an approach for vessel enhancement on angiographic images.
Examined a multi-scale second order local structure of an image approach in order to enhance tubular structures
Second order image expansion around the neighbourhood of a point xo
Gradient
Hessian
Differentiation is defined as a convolution with a Gaussian derivative
Hessian & Hough Transform Analysis Hessian – Eigenvalue Analysis 37
Eigenvalue analysis of the Hessian.
Smallest eigenvalue corresponds to the direction (eigenvector) along the tubular structure
In a 3D image, the remaining eigenvectors form an orthogonal plane
No need to compute the filter response in several orientations, as directional information is implicit.
Hessian & Hough Transform Analysis Hough Transform for Crack Location 38
Hough Transform was used to extract lines from the previous filtered
Filtered image is transformed into a parameter space (rho and theta)
HT peaks are computed and the corresponding lines are superimposed on the original image
Cracks are expected to be formed of linear segments
Exact location of crack can be determined
Hessian & Hough Transform Analysis Sample Result and Final Observations 39
Result for crack_face_2
Original Image
Frangi Filtered Image
Crack Detection
Advantages
Simple concept
Fast computation
Exact crack location
Disadvantages
False Positive Rate
Small or low-contrast cracks are not detected
Supervised Detection: Line Operator Analysis 40
Convolution with a set of “line filters” of different orientations
Parameters:
Number of orientations (angle discretization) Filter size How to merge information from all filters
Previously used in many applications, such as retina segmentation & mammographic image analysis
Finding oriented line patterns in digital mammographic images – Zwiggelaar et al. Retinal blood vessel segmentation using line operators and support vector classification – Ricci et al.
Line Operator Analysis Patch generation 41
Patches generated from ground-truth (positive) and randomly from the non-crack parts of image (negative)
Patch position is random and overlap is allowed and preferred
Line Operator Analysis 42
Classification approach:
Patch Patch Patch Patch Patch Patch Patch (100x100)
Patch Patch Patch Patch Patch Patch Generated Descriptor
Extracted patches (Labelled)
Descriptors (Labelled)
SVM Classifier
Line Operator Analysis Merging different channels: Descriptor generation 43
Proposed method:
Apply the line operator to all channels and discretized orientations with expected polarity (ie: cracks are darker on Ir images and lighter on Sc)
Generate histogram for each channel
Concatenation Rotational Invariance: Rotate all w.r.t. maximum and normalize
Line Operator Analysis Results 44
Patch classification with SVM:
AUC 0.98, Avg. Precision 0.93 and at that point Recall 0.82
When applied to images to analyze every patch (50% overlap), it seems to capture adequate information for crack detection: Positive samples
Negative samples
Line Operator Analysis Final observations 45
Advantages
Disadvantages
Simple concept, can be applied to small patches and merged with other descriptors easily It seems to be appropriate for wood crack detection
Could result slow to compute on large images Looks exactly for straight lines, may not adapt to non-straight segments Has many tuneable parameters
Further work
Erroneous ground truth could have an important impact on SVM classification and should be verified Merge line operators with multiple scales (line size/thickness) to capture more irregular cracks Include a measure of edge or regularity of the patch to avoid mis-detections because of noise or slight line patterns Analyze the final results formally to obtain figures that represent the performance of the classifier with respect to every crack in the images
Further Work 46
Extend the Line Operator Analysis to work at different scales
The Hessian and/or Sobel operator could be used to generate orientation histograms as well, but faster and more efficient than the Line Operator Results should be evaluated properly with a correct measure using the groundtruth in order to provide meaningful figures
Conclusion 47
Different methods for wood crack detection were investigated, showing that crack detection is not an easy task A single method is not enough to provide enough accuracy and a fusion of several of them is needed Supervised local methods seem promising if the right features are identified
