A Computational Model of Visual Search Saliency for Road Signs

32nd European Conference on Visual Perception August 24th - 28th 2009 in ... Application: Road managers need to diagnose road signs saliency along a road ...
2MB taille 4 téléchargements 119 vues
32nd European Conference on Visual Perception

August 24th - 28th 2009 in Regensburg, Germany

University of Regensburg

A Computational Model of Visual Search Saliency for Road Signs Diagnostic Ludovic SIMON, Jean-Philippe TAREL, Roland BREMOND, Henri PANJO Laboratory for Road Operation, Perception, Simulation and Simulators LEPSIS, INRETS-LCPC, Université Paris EST [email protected]

Context ●

Visual Saliency: Degree to which an object attracts attention with a given background, when the observer is performing a given task.



Driving: involves visual exploration of the road environment. One important driving subtask is to look for road signs.



Road Signs: Act on road safety and traffic control (driver's guidance, alert, information).



Accident Risk: Road signs are designed to attract driver's attention but not all of them are seen by all drivers



Application: Road managers need to diagnose road signs saliency along a road network from a vehicle with a digital camera.



Challenge: A complete model of saliency is not available to date. The diagnostic need to be automatic, i.e. to give estimation only on road signs.

saliency problem. IRCAN2: Road Imaging by Digital Camera MLPC equipment

State of the Art ●

Road Sign Saliency Estimator: must be correlated to the Driver's Visual Attention.



Computational Model of Visual Attention (Itti, 1998; Le Meur, 2006) : ➔ Based on the low levels of the Human Visual System. ➔ Works for a Bottom-Up Task (image memorization). ➔ Not valid for a Top-Down Task (search for an object). ➔ Impossible to take into account the task biases (ex: driving subtask).



Model of Visual Search: ➔ Mainly theoretical rather than computational (Navalpakkam, 2005; Sundsted, 2005) ➔ Computational ones are limited to laboratory situation (Gao, 2004; Wolfe, 2007).

Are the road signs salient enough for a driver ?

Itti's Saliency Map

Our Proposal : A SVM-Based Visual Search Saliency Model ➢ Relying on statistical learning algorithm, using Support Vector Machine (SVM). ● ●

Road Signs Learning: Capture the priors a driver learns about the appearance of any (set of ) road signs. Computation of a classification function C(x) by learning positive samples (road signs) and negative ones.

➢ Detection of the road signs and estimation of their saliency: ● ●

The saliency of the road signs (SCS) is define as a function of the confidence C(x) of the detector results. Intrinsical Computed Saliency (ICS): ➔ Maximum confidences values C(x) over various sliding windows sizes. ➔ Subtraction of the mean over a background-windows, 2° of visual angle around each detected sign.



Proposed Visual Search Saliency Map

Size-dependent Computed Saliency (SCS):

SCS(i): Visual Search Saliency of the road sign i. ICS(i): Intrinsical Saliency of the road sign i. A(i): Area of the road sign i (in pixels). 4

SCS  i=  ICS  i× A i Map of C(x) with Windows Scale 40 x 40

Resulting ICS Map In green, the detected > signs bordered by their background-windows

Map of C(x) with Windows Scale 10 x 10

Validation of the proposed model



Phase I: ➔ Subject's task: count for « no entry » signs. ➔ Objective Evaluation of the saliency:  Subject's eye fixations (foveal center).  D i , j = 1 if sign i is noticed by subject j.  HDR i : human detection rate of sign i.  Fixation's durations on a relevant object Phase II: ➔ Subject's task: rate the saliency of each « no entry » signs. ➔ Subjective Evaluation of the saliency:  subjects' scoring on a scale from 0 to 10.  score i , j : rate for subject j and sign i. 

SSS i , j=

References: ● ● ● ● ● ●

score  i , j − E i  score  i , j   i  score i , j 

5

Human Detection Rate (HDR) as a function of the Subject Standardized Score (SSS).

Size-dependent Computed Saliency (SCS) as a function of the Subject Standardized Score (SSS).

Typical scan-path of one subject searching for « no entry » sign.

HDR and SSS are correlated. Signs which are not well noticed are above a score threshold of 4.5. ● There is a linear link between SSS and SCS, our computational model. ● Our computational model of visual search saliency of road signs is correlated to Human Detection Rate. ● The proposed model (SCS) explains 56% of the variance between signs. ●

700

700

Mean of Fixation Duration (ms)



Psycho-Visual experiment: ➔ Remote Eye Tracker. ➔ 40 images taken from a car. ➔ 76 « no entry » signs. ➔ Various saliency levels. ➔ 32 subjects. ➔ Two phases, context close to driving.

Statistical Analysis:

Mean of Fixation Duration (ms)





650

600

550

500

450

650

600

550

500

450 [1 - 4,55[

[4,55 - 5,50[

[5,50 - 7[

[7 - 9,5[

Class of SSS

Mean duration of subjects' fixations as a function of the Subject Standardized Score (SSS).

[1,70 - 5[

[5 - 5,58[

[5,58 - 6,3[

[6,3 - 8,4[

Class of SCS

Mean duration of subjects' fixations as a function of the Size-dependent Computed Saliency (SCS).

Psycho-visual lab: the display room photometrically controlled

The higher the model estimates the saliency, the lower the fixation duration on the road signs. ● The same relation is observed between the Score and the fixation duration on the road signs. ● We find the score threshold of 4.5 again. ● Our computational model of visual search saliency is correlated to the fixation duration in a search task. ●

D. Gao and N. Vasconcelos, Discriminant saliency for visual recognition from cluttered scenes, Advances in Neural Information Processing Systems (NIPS’04). vol.17, Vancouver, British Columbia, Canada, december 13-18, 2004. L. Itti, C. Koch and E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence. vol.20, pp.1254-1259, 1998. O. Le Meur, P. Le Callet, D. Barba and D. Thoreau, A coherent computational approach to model bottom-up visual attention, IEEE Transactions on Pattern Analysis and Machine Intelligence. vol.28, no.5, pp.802-817, may 2006. V. Navalpakkam and L. Itti, Modeling the influence of task on attention, Vision Research. vol.45, pp.205-231, january 2005. V. Sundstedt, K. Debattista, P. Longhurst, A. Chalmers and T. Troscianko, Visual attention for efficient high-fidelity graphics, Spring Conference on Computer Graphics (SCCG 2005). pp.162-168, Psychology Press, may 2005. J.M. Wolfe, Guided search 4.0: Current progress with a model of visual search., In W. Gray (Ed.), Integrated Models of Cognitive Systems. pp.99-119, New York: Oxford, 2007.