32nd European Conference on Visual Perception
August 24th - 28th 2009 in Regensburg, Germany
University of Regensburg
A Computational Model of Visual Search Saliency for Road Signs Diagnostic Ludovic SIMON, Jean-Philippe TAREL, Roland BREMOND, Henri PANJO Laboratory for Road Operation, Perception, Simulation and Simulators LEPSIS, INRETS-LCPC, Université Paris EST
[email protected]
Context ●
Visual Saliency: Degree to which an object attracts attention with a given background, when the observer is performing a given task.
●
Driving: involves visual exploration of the road environment. One important driving subtask is to look for road signs.
●
Road Signs: Act on road safety and traffic control (driver's guidance, alert, information).
●
Accident Risk: Road signs are designed to attract driver's attention but not all of them are seen by all drivers
●
Application: Road managers need to diagnose road signs saliency along a road network from a vehicle with a digital camera.
●
Challenge: A complete model of saliency is not available to date. The diagnostic need to be automatic, i.e. to give estimation only on road signs.
saliency problem. IRCAN2: Road Imaging by Digital Camera MLPC equipment
State of the Art ●
Road Sign Saliency Estimator: must be correlated to the Driver's Visual Attention.
●
Computational Model of Visual Attention (Itti, 1998; Le Meur, 2006) : ➔ Based on the low levels of the Human Visual System. ➔ Works for a Bottom-Up Task (image memorization). ➔ Not valid for a Top-Down Task (search for an object). ➔ Impossible to take into account the task biases (ex: driving subtask).
●
Model of Visual Search: ➔ Mainly theoretical rather than computational (Navalpakkam, 2005; Sundsted, 2005) ➔ Computational ones are limited to laboratory situation (Gao, 2004; Wolfe, 2007).
Are the road signs salient enough for a driver ?
Itti's Saliency Map
Our Proposal : A SVM-Based Visual Search Saliency Model ➢ Relying on statistical learning algorithm, using Support Vector Machine (SVM). ● ●
Road Signs Learning: Capture the priors a driver learns about the appearance of any (set of ) road signs. Computation of a classification function C(x) by learning positive samples (road signs) and negative ones.
➢ Detection of the road signs and estimation of their saliency: ● ●
The saliency of the road signs (SCS) is define as a function of the confidence C(x) of the detector results. Intrinsical Computed Saliency (ICS): ➔ Maximum confidences values C(x) over various sliding windows sizes. ➔ Subtraction of the mean over a background-windows, 2° of visual angle around each detected sign.
●
Proposed Visual Search Saliency Map
Size-dependent Computed Saliency (SCS):
SCS(i): Visual Search Saliency of the road sign i. ICS(i): Intrinsical Saliency of the road sign i. A(i): Area of the road sign i (in pixels). 4
SCS i= ICS i× A i Map of C(x) with Windows Scale 40 x 40
Resulting ICS Map In green, the detected > signs bordered by their background-windows
Map of C(x) with Windows Scale 10 x 10
Validation of the proposed model
●
Phase I: ➔ Subject's task: count for « no entry » signs. ➔ Objective Evaluation of the saliency: Subject's eye fixations (foveal center). D i , j = 1 if sign i is noticed by subject j. HDR i : human detection rate of sign i. Fixation's durations on a relevant object Phase II: ➔ Subject's task: rate the saliency of each « no entry » signs. ➔ Subjective Evaluation of the saliency: subjects' scoring on a scale from 0 to 10. score i , j : rate for subject j and sign i.
SSS i , j=
References: ● ● ● ● ● ●
score i , j − E i score i , j i score i , j
5
Human Detection Rate (HDR) as a function of the Subject Standardized Score (SSS).
Size-dependent Computed Saliency (SCS) as a function of the Subject Standardized Score (SSS).
Typical scan-path of one subject searching for « no entry » sign.
HDR and SSS are correlated. Signs which are not well noticed are above a score threshold of 4.5. ● There is a linear link between SSS and SCS, our computational model. ● Our computational model of visual search saliency of road signs is correlated to Human Detection Rate. ● The proposed model (SCS) explains 56% of the variance between signs. ●
700
700
Mean of Fixation Duration (ms)
●
Psycho-Visual experiment: ➔ Remote Eye Tracker. ➔ 40 images taken from a car. ➔ 76 « no entry » signs. ➔ Various saliency levels. ➔ 32 subjects. ➔ Two phases, context close to driving.
Statistical Analysis:
Mean of Fixation Duration (ms)
●
●
650
600
550
500
450
650
600
550
500
450 [1 - 4,55[
[4,55 - 5,50[
[5,50 - 7[
[7 - 9,5[
Class of SSS
Mean duration of subjects' fixations as a function of the Subject Standardized Score (SSS).
[1,70 - 5[
[5 - 5,58[
[5,58 - 6,3[
[6,3 - 8,4[
Class of SCS
Mean duration of subjects' fixations as a function of the Size-dependent Computed Saliency (SCS).
Psycho-visual lab: the display room photometrically controlled
The higher the model estimates the saliency, the lower the fixation duration on the road signs. ● The same relation is observed between the Score and the fixation duration on the road signs. ● We find the score threshold of 4.5 again. ● Our computational model of visual search saliency is correlated to the fixation duration in a search task. ●
D. Gao and N. Vasconcelos, Discriminant saliency for visual recognition from cluttered scenes, Advances in Neural Information Processing Systems (NIPS’04). vol.17, Vancouver, British Columbia, Canada, december 13-18, 2004. L. Itti, C. Koch and E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence. vol.20, pp.1254-1259, 1998. O. Le Meur, P. Le Callet, D. Barba and D. Thoreau, A coherent computational approach to model bottom-up visual attention, IEEE Transactions on Pattern Analysis and Machine Intelligence. vol.28, no.5, pp.802-817, may 2006. V. Navalpakkam and L. Itti, Modeling the influence of task on attention, Vision Research. vol.45, pp.205-231, january 2005. V. Sundstedt, K. Debattista, P. Longhurst, A. Chalmers and T. Troscianko, Visual attention for efficient high-fidelity graphics, Spring Conference on Computer Graphics (SCCG 2005). pp.162-168, Psychology Press, may 2005. J.M. Wolfe, Guided search 4.0: Current progress with a model of visual search., In W. Gray (Ed.), Integrated Models of Cognitive Systems. pp.99-119, New York: Oxford, 2007.
