Reconstruction of “Next-generation” 3D Digital Road Model By Using Air-borne Three Line Scanner Imagery

Reconstruction of “Next-generation” 3D Digital Road Model By Using Air-borne Three Line Scanner Imagery

SHI Yun
Shibasaki Lab. The Department of Civil Engineering, University of Tokyo

Ryosuke SHIBASAKI
Center for Spatial Information Science, University of Tokyo

Abstract
Nowadays, for digital map, new demands emerge that cannot be fulfilled by traditional digital maps, especially in many fields like ITS, pedestrian support system and so on. For example, in Advanced Cruise-Assist Highway System (AHS, One of key issues of ITS), the typical services should be “Support for prevention of collisions with forward obstacles”, “Support for prevention of collisions with pedestrians crossing streets”, “Support for prevention of lane departure” and so on.

Actually, an integration of real-time sensing and map reference from vehicles would be very effective to achieve a complete Cruise-Assist system such as what can make driver avoid traffic accident. However, existing digital road maps such as network data or simple 3D data for vehicle navigation cannot support Cruise-Assist well, in particular, in urban area. In fact, road features such as Zebra, Lane Line, and Boundary are required for accurate map reference, and more, those features should have higher precision and more detail information. Therefore, an effective 3D road data reconstruction method, which provides high representative road features, is required.

This research carries out a study on reconstruction of this kind of higher precision and more detailed 3D digital road map (we call it as “Next-generation 3D Digital Road Map”, NDR-Map) using high-resolution Three Line Scanner (TLS) imagery. A robust road feature extraction schemes from TLS Imagery, as well as a novel method for Reconstruction of high-precision 3D digital road model are presented.

The results achieved with our approach show that it is possible and economic to reconstruct the High-precision 3D Digital Road Map for ITS by using TLS imagery.

1. INTRODUCTION

1.1 Background
Nowadays, for digital map, new demands emerge that cannot be fulfilled by traditional digital maps, especially in many fields like ITS, pedestrian support system and so on. For example, in Advanced Cruise-Assist Highway System (AHS, One of key issues of ITS), the typical services should be “Support for prevention of collisions with forward obstacles”, “Support for prevention of collisions with pedestrians crossing streets”, “Support for prevention of lane departure” and so on.

Actually, an integration of real-time sensing and map reference from vehicles would be very effective to achieve a complete Cruise-Assist system such as what can make driver avoid traffic accident. However, existing digital road maps such as network data or simple 3D data for vehicle navigation cannot support Cruise-Assist well, in particular, in urban area. It needs more detailed shape information and more accuracy position information of Road Feature (Like Zebra, Lane Line, and Boundary), which can make driver avoid traffic accident through Cruise-Assist system. So method for developing the higher precision and more detailed 3D digital road map have become issues of significant interest and high importance.

As a background of this research, it is necessary to review the related researches. The existing approaches of building digital map from image cover a wide variety of strategies, and different resolution aerial or satellite images. Knowledge based on image analysis for road constructing has been developed by Zhang 2001; Automatic methods usually extract reliable hypotheses for road segments through edge and line detection and then establish connections between road segments to form road networks; Automatic method based on actual and complex contextual road model.

But all of existing approaches focus on data acquisition or data update for GIS, but none of them can meet demand of the Map mentioned above, because of limit in image resolution they used, and difference of data use. In this paper, we are focusing on extracting High-precision 3D Digital Road Map for ITS using, which need data with high precision and high accuracy.

1.2 Objectives
Based on this urgent demand of new high-precision road map, this research carries out a study on how to develop this kind of high-precision road map efficiently and economically. As we all know, photogrammetry is one powerful tool for reconstruct digital city model, but resolution and precision of traditional aerial image is limited, which cannot satisfy so accuracy road map reconstruction. But fortunately, a new and revolutionary air-borne remote sensor have been developed, which is Three Line Scanner (TLS). TLS is composed of three linear CCD arranged in parallel, and it can acquire three images of each direction (forward, nadir and backward) at the same time, and it have high image resolution (ground resolution about 3cm) and high position precision (about 10cm).

In this research, the author try to reconstruct the higher precision and more detailed 3D digital road (we call it as “Next-generation 3D Digital Road Map”, NDR Map) using the above mentioned high-resolution Three Line Scanner (TLS) imagery. A robust road feature extraction schemes from TLS Imagery, as well as a novel method for Reconstruction of high-precision 3D digital road map are presented.

1.3 TLS
TLS (Three Line Scanner) is an optical sensor for aerial survey. TLS is composed of three linear CCD arranged in parallel, and it can acquire three images of each direction (forward, nadir and backward) at the same time. Orienting it on an aircraft perpendicularly to flight direction, and scanning a ground plane, a triple stereo image of a ground object can be acquired (See Figure 1). As a result, occlusion area can be extremely reduced. Using two images of the three, it is also possible to get 3D coordinates by stereo matching.

Because of its principle of data acquisition, it have the following advantages: (1) high resolution digital image can be collected with longer CCD linear sensor by push-broom mode; (2) seamless image strips for linear ground objects without mosaic processing; (3) the same ground area is covered three times by push broom mode with 100% coverage; (4) precise geometric positioning by modern GPS technology and simple aerial triangulation with few GCPs; (5) last but not least it have low cost of data acquisition.

Figure 1. TLS data acquisition
2. Recontrucion strtegy and methodology

2.1Conceptual Model of High-precision 3D Digital Road Map
The traditional road map always can give user only road boundary information or net connecting information. So this kind of road cannot meet the demands of the new LBS (Location Based Service), which need high precision digital road with main road features such as road lane line and zebra. In this paper, we bring forward a conceptual model of High-precision 3D Digital Road Map (See Figure 2). In this model, the road is described as a 3D surface with necessary road features such as center line, lane line, stopping line and so on, which are necessary in LBS and ITS system.

Figure2: Conceptual Model

Figure3: Main Flowchart
2.2 The Outline of our Methodology
As shown in Figure3, there are four steps for reconstructing our needed road map: Semi-automatic feature extraction, Automatic stereo matching, Road surface interpolation and Road segment integration. The presented Semi-automatic feature extraction schemes mainly include the two key methods: Edge based Template Matching for road marking, which conquers the noise disturbing of just using image edge information and low speed of just using template matching, and integrates the merits of edge approach and template matching approach; Active Contour Models based road boundary fitting, whose initial points can be offered based on extracted road marks.

One of key points of road map reconstruction is firstly to build the road smooth surface based on road surface characteristic using the some of road feature points, whose 3D information is gotten through stereo image matching and evaluating, and then to reconstruct road map using the built road surface and extracted road features.

2.3 Road Features Extraction from Images
In this model of High-precision 3D Digital Road Map, two kinds of features are important and indispensable, that are road marking and road boundary. As follow, we will introduce extraction methods of the two kinds of features.

Road markings such as zebra crossing, turning arrows are good indications of the existence of roads. They are generally found on main roads and roads in urban areas. Both of them have distinct color (usually white or yellow). In TLS images used in our project, road markings are white thin lines with a certain width. Based on the especial shape, we have developed a method: Edge based Template Matching. This method conquers the noise disturbing of just using image edge information and low speed of just using template matching, and integrates the merits of edge approach and template matching approach. (See Figure4)

Figure4: Edge based Template matching

Figure5: Matching Constrain Condition
Road Boundary defines the shape of road. Based on direction of road marking, and width of zebra crossing, using SNAKE model, we can fit the boundary. (About Snake method, you can refer in [4] and [5]). It is difficult and necessary that we should give relative accuracy initial points of boundary when using SNAKE model. In our method, because we firstly extract the road features, which can give the necessary initial points, it became efficient and useful that we use SNAKE as our boundary fitting method.

When road features have been extracted from TLS images, we should do stereo matching to get height information of these features so that the 3D road can be reconstructed.

2.4 3D Feature Reconstruction using Stereo Matching
There are two different points with traditional photogrammetry and useful cues for stereo matching of road reconstruction using TLS imagery: (1) TLS images are triple for any area, so it can be used to check the correctness of image matching using two different pairs of image matching; (2) road surface is always relatively flat, which means the height of surface will change smoothly, so we can narrow our searching range using neighbor matched points. As the figure5 show, we project a feature point to object space with approximate searching range, and then back-project these potential 3D points to forward and backward images, so we can get the epipolar line of this point. After we calculate the correlation coefficient of image, we can decide the conjugate of this feature point, based on the higher gray coefficient and checked by the two matching cues, which are mentioned above.

3. EXPERIMENTS AND RESULTS
In this experiment, we use TLS image data to reconstruct High-precision 3D Digital Road Map based on our conceptual model. In this method, the difficult and creative work is about road feature extraction, such as road marks and road boundaries, so here author would show results of our study area, Kinshicho of Tokyo. The following result shows our extracted 2D road features. Green line is boundary of road, and yellow one is road features. (See Figure6)

Figure6: Successful Case of Extraction

Figure7: Failure Case of Extraction
Of course, like the following case (Figure 7), because this kind of road feature is partly covered, and has very serious noise around it, it is difficult to detect its shape automatically. But fortunately, this kind of case is very few, and also it can partly solved by improving ratio of S/N of TLS image.

Figure7: Overall Evaluation of Study Area
From the overall evaluation of road feature extraction method in our study area (See Figure7), the success ratio is 85% for zebra, 80% for lane and so on. The following is our reconstructed High-precision 3D Digital Road Map using our approach. (See Figure8: Example of Reconstructed Road)

Figure8: Example of Reconstructed Road
4. Conclusions
In this thesis, Based on the urgent demand of new high-precision and more detailed road map, the author carries out a study on how to develop this kind of road map efficiently and economically using a new and revolutionary air-borne sensor, Three Line Scanner (TLS), which is composed of three linear CCD arranged in parallel, and which can acquire three images of each direction (forward, nadir and backward) at the same time, and which have high image resolution (ground resolution about 3cm) and high position precision (about 10cm). At first, the author presents the conceptual model of this kind of road map, which is called the “Next-generation” 3D Digital Road Map (NDR-Map). And then, the NDR-Map is reconstructed using the above mentioned high-resolution TLS imagery. Here a robust road feature extraction schemes from TLS Imagery, as well as a novel method for Reconstruction of NDR-Map are presented.

From the results of road reconstruction in KINSHICHO area of Tokyo, Japan, it is found that the semi-automatic road feature extraction schema is very efficient and robust for road feature extraction of TLS imagery. And also it is very clear that the schema of stereo matching of extracted road feature point is robust to get 3-D information of feature point for road surface interpolation.

So the final road reconstruction result show that it is possible and efficient to reconstruct this kind of so high-precision and so detailed NDR-Map using Three Line Scanner Imagery. But the success ratio also should be increased specially in some road area, whose road lanes arrange irregularly, and which is in shadow with noise. And also there are some necessary road features cannot be seen from airborne sensor TLS, like under the bridge. Those problems will be considered in my future study.

5. Acknowledgements
The authors would like to express the appreciation to the STARLAB Co. Ltd and their staffs for TLS images distribution and necessary related knowledge about TLS system for this research, and want to thank specially to Mr. Nakagawa, Mr. Nakamura, Dr. Guotao and Mr. Sompoch for their help and suggestion and our laboratory members.

6. References

Zhang Chunsun, Baltsavias E., Gruen A., 2001. Knowledge-based on image analysis for 3D road Reconstruction. Asian Journal of Geoinformatics, Vol. 1, No. 4 June 2001, pp, 3-14.
Hinz S., Baumgartner A., 2003. Automatic Extraction of Urban Networks from Multi-view Aerial Imagery. In: International Journal of Photogrammetry and Remote sensing. 58/1-2, pp 83-98.
Barzohar, m .D. B. Cooper. Automatic finding of main roads in aerial images by using Geometric Stochastic models and estimation, IEEE trans. PAMI, vol.18,no.7 , pp. 707- 721,July,1996