VISION-BASED SAFE MANEUVERS WITH DETECTION OF 10CM HEIGHT OBSTACLES

C. Vestri, D. Tsishkou, F. Abad, S. Wybo, S. Bougnoux, and R. Bendahan Vision Group, IMRA Europe S.A.S. 220 Rue Albert Caquot, B.P. 213, 06904 Sophia-Antipolis Cedex, France TEL: +33-493-957-369, [email protected]

ABSTRACT An increasing number of new vehicles are being equipped parking and backing-up assistant systems. Those systems both help the driver in conducting difficult maneuvers and in detecting the presence of obstacles to prevent collisions. We present a stereovision system that detects and monitors obstacles inside a 6m area in front or behind a vehicle. We use wide-angle cameras to detect side-appearing obstacles. We aim at detecting pedestrian and cars, but also curbs, stopping blocks and small rocks which are ignored by most of other systems. This paper presents the low height obstacle (about 10cm) detection algorithm used in this system. The biggest challenge is to separate those obstacles from ground surface because the height variation with the ground is small and they correspond to very few pixels in the image (mainly with wide angle camera). We have developed a real-time wide angle stereo algorithm which detects obstacles in three stages: first, a plane that approximates the ground surface separates main obstacles such as pedestrians and cars from the ground. Second, a specific algorithm detects low height obstacles and small (10cm*10cm) objects by searching for small height transitions on the ground surface. Third, a free-space algorithm gathers information to compute the area where vehicle can safely drive. The low height obstacle detection algorithm is the main contribution of the paper. We show detection results under typical urban situations and evaluate its detection rate. Drivable surface is correctly marked and low height obstacles are detected up to 6 meters.

INTRODUCTION A number of systems for parking and backing-up assistance have been commercialized over the past few years. Camera-based systems help the drivers by displaying the scene behind the driver without any blind spot. Such systems can facilitate parking maneuvers. Combined with an obstacle sensor they can also help the drivers to prevent obstacle collisions and save human lives in other situations such as backing-up. Motivated by such applications, we 1

developed a stereo-camera-based obstacle sensor which aims at detecting dangerous obstacles on paths of potential collision: a 6m area in front or behind a vehicle in movement. This sensor not only detects pedestrians and cars, but it also focuses on curbs, stopping blocks and small rocks which are often ignored by other sensors. The stereo processing stage of this sensor has been presented in (1) and pedestrian/child recognition is presented and discussed in (2). This paper presents the generic obstacle detector of our vision sensor dedicated to small and low-height obstacles on the road. Separating low height obstacles (about 10cm tall) and small size objects (10cm*10cm) from the ground is challenging for two main reasons: first because they have the dimension of system accuracy (difference of height between the ground and those objects is small). Second, because those objects correspond to very few pixels in the image, mainly with wide angle camera (10cm width is 4pixels at 6m with a 97° horizontal Fov camera and an image of 512*384 pixels). This paper has two main contributions for small obstacle detection: •

The first contribution is a new algorithm that does a full 3D height analysis of raw data. The main advantage is that we perform detection directly in the image (height image) with complete raw 3D data. The obstacle detection algorithm is not sensitive to uncertainty of 3D reconstruction and there is no data smoothing. This algorithm is not only able to detect low height obstacles such as curbs, but also small objects on the ground such as stones.

•

The second contribution is a free-space computation which combines both 3D and photometric data. This combination allows to monitor the full 6 meters area, even in areas where stereo fails.

Stereovision obstacle sensors are already proposed such as (3)(4)(5)(6) and (7), but none of them is able to detect accurately 10cm height obstacles such as curbs, stopping blocks and very small objects such as stone. Previous approaches failed for two main reasons: •

The first reason is that most of the approaches smoothes ground data for removing false alarm and/or completing the missing 3D data. Smoothing the data reduces the height difference between low height and small obstacles with the ground; it is more difficult to detect them. In (3) a data propagation stage smoothes the Digital Elevation Model (DEM). It rejects the small clusters (peaks) and completes missing data for ensuring a good classification by their density measures. In (4) the detection stage uses u- and v-disparity image. Both are smoothed images where small objects cannot be seen. In (5) Wedel uses probabilistic occupancy grids. Data are smoothed when updating the probabilities of the cells. Finally in (6) Wellington uses an MRF model which is an approach that also smoothes data. 2

•

The second reason of failure of other approaches is that most of them use 3D data in their reasoning. Unfortunately, in the 3D space, ground and obstacle points are locally mixed because of depth uncertainty. It is difficult to recover the small amount of data that corresponds to the small objects. In (3) Nedevschi compresses 3D data in the DEM by keeping max height of projected 3D points. DEM is corrupted by depth uncertainty and the choice to only store the maximum height of the 3D points of the cell. In (5), (6) and (7) all authors use 3D points data which are corrupted by depth uncertainty.

To avoid these failures, we propose a low-height obstacle detection algorithm that works at pixel level. The algorithm is not sensitive to uncertainty of 3D reconstruction because 3D points remain ordered in image space. This advantage is illustrated in Figure 1. On the left we can see the two ways for separating the inaccurate 3D data points: using 3D position or using image point of view. On the right is presented the average height level obtained by averaging data on both sides of the separation axes. With the 3D-based separation, ground and obstacle data are mixed and average height difference is small. With image-based separation, they remain separated and the difference is larger. Image-based separation maximizes the estimation of height variation. Our detection algorithm operates in the height image. This image is built with the complete set of raw 3D data. Low-height obstacle detection is achieved by detecting meaningful height transitions in this image. We avoid the two main reasons of detection failure previously mentioned by estimating height transitions using image separation principle with raw height data. This algorithm is the main contribution of the paper.

Figure 1: Advantage of using image-based separation. Ground and obstacle points are mixed if they are separated using their 3D position. Separation by image allows correct data clustering which maximize
h estimation.

3

FREE-SPACE DETECTION There are various ways to detect obstacles. Computing the free-space surface is one of them. It combines detection of both the ground and the obstacles. We used it because it also has the interesting particularity to represent the safe-path: the ground surface where we can safely drive. Left image

Right image

Stereo Matching

Free-space algorithm

Obstacle/Ground separation

3D points Ground plane tracking Ground/obstacle separation

3D obstacle points Density-based filtering

First stage 3D ground points

Second stage

Underground points

10cm detection

3D obstacle points

Free-space surface computation Second stage

3D obstacle points

3D ground points

Underground points

Obstacle pointbased radial growing

Ground pointbased radial growing

Error points

Color-based radial growing

Fusion Hull computation FS hull

Figure 2: Overview of the free-space surface computation algorithm

The free-space surface is computed in three stages as presented in Figure 2. This surface is obtained from the set of 3D points from the stereo camera presented in (1). The two first stages classify the 3D points. First, a plane that approximates the ground surface separates 3D points that belong to main obstacles such as pedestrians and cars from those which belong to 4

the ground. Second, a specific algorithm analyzes local transitions in the height image of the scene. It recovers 3D points that correspond to low height obstacles and small objects located on the ground. The third stage computes the free-space surface from the classified 3D points. The surface grows along the ground and stops at objects, at holes or at the limit of the monitored area. These three stages are detailed in the next sections.

MAIN OBSTACLES/GROUND SEPARATION Main obstacle separation is achieved by computing a model of the ground surface. With this model, we can automatically separate obstacle, ground and underground pixels by measuring their distance to this model. We use a plane to model ground (which is usually the road). Since the vehicle is pitching while moving, we decided to track this ground plane for compensating pitching. Ground plane is initially estimated by applying a classic Least Median of Square (LMedS) estimation algorithm onto low height (-10cm