3D feature extraction by integrating Laser Scanner and CCD Sensor with IMU for an unmanned Helicopter Platform

3D feature extraction by integrating Laser Scanner and CCD Sensor with IMU for an unmanned Helicopter Platform

Masahiko Nagai, Ryosuke Shibasaki, Dinesh Manandhar, Huijing Zhao
Center for Spatial Information Science, The University of Tokyo
Cw-503, IIS, 4-6-1, Komaba, Meguro-ku, Tokyo 153-8505, Japan
Tel: +81-3-5452-6417 Fax: +81-3-5452-6414
Email: nagaim@iis.u-tokyo.ac.jp

Hideo Kumagai, Shintaro Mizukami
Tamagawa Seiki Co., Ltd. 1879 Ohyasumi, Iida-City, Nagano 395-8515, Japan

Abstract
Three dimension data are in great demand for the various applications such as 3D GIS, navigation, digital archive, simulation, computer games and so on. In order to represent 3D space and moving objects in details, it is indispensable to acquire 3D shape and texture together efficiently. However, there still lack a reliable, quick and cheap method for acquiring three dimension data of objects with higher resolution and accuracy in outdoor environment. In this research, we propose a combination of a digital camera and a laser scanner with low-cost IMU (Inertial Measurement Unit) for unmanned helicopters.

3D shape is acquired by a laser scanner as point cloud data, and images are acquired by CCD sensor from the same platform simultaneously. Positioning data is acquired by GPS and IMU.

All the sensors are synchronized with movement or attitude of the platform. Calibration of laser scanner and CCD camera is conducted to know the relative position and attitude of each sensor against GPS and IMU. Through Kalman filter, an optimal estimate of the sensor position and attitude are estimated from GPS and IMU. Using the CCD images, bundle block adjustment conducted to aid Kalman filter by initialization of position and attitude. The combination of bundle block adjustment and Kalman filter for precise positioning is proposed in this research. Using this measurement system, sensor position and attitude for mapping are basically acquired by GPS/IMU and their errors are complemented by CCD images using tie points. Colors are assigned to laser points based on the CCD image color.

Points cloud model is produced for rendering objects with rich shape and detailed texture for feature extraction. Geometric shape, which is acquired by leaser scanner, represents features.

Color information, which is acquired by CCD sensor, details those features. That is, more detail extraction is possible using both 3D shapes and colors.

1. Introduction
The mobile mapping has been developed since late 1980's. The development of the mobile mapping system becomes possible due to the availability of GPS and IMU (Kumagai, H., Kubo, Y., 2002). In this research, the combination of a CCD sensor and a small (cheap) laser scanner with an inexpensive IMU and GPS for mobile platform are proposed. Utilization of mobile platform is very important for acquiring data effectively (Zhao, H., Shibasaki, R., 2001). That is, it is necessary to consider the method to get to develop the high precision positioning system in moving environment. In this research, direct geo-referencing is achieved automatically from mobile platform without any ground control points. Here, a "direct" geo-referencing mean geo-referencing that does not require ground control points. The methods of data acquisition and digital surface modelling are developed with the method of direct geo-referencing. This leads to rendering objects with rich shape and detailed texture automatically. Using these data, feature extraction is conducted. More detailed extraction is possible using both laser range data and CCD image.

2. System Design
In this research, an unmanned helicopter, RPH2, which is made by SUBARU, is used as a platform as shown in Figure 1. The size of RPH2 is 4.1m length, 1.3m width, and 1.8m height.

Figure 1 RPH2
All the sensors, digital camera, laser scanner, GPS, and IMU are tightly fixed on the unmanned helicopter. Specifications of the sensors are shown in Table 1. All the sensors are synchronized. In the experiment, the unmanned helicopter flies about 1.8m/s of speed for acquiring laser point data with sufficiently fine resolution and sequential digital camera images with sufficient overlaps. An unmanned helicopter is an ideal platform to measure wide area effectively and safely.

Table 1 Specification of Sensors

3. Overview
Figure 2 shows the overview of data processing. In this research, the following data are acquired; base station GPS data, remote station GPS data, IMU data, CCD images, and laser range data. Though the data are acquired in different frequencies, they are synchronized. At first, kinematic GPS post processing is conducted. Secondly, processed GPS and IMU are integrated with Kalman filter. Thirdly, bundle block adjustment of CCD image is made with the support of GPS/IMU. Finally, GPS/IMU and bundle block adjustment of CCD images are combined for generating a high precision and time-series position and attitude data. This hybrid positioning data is used for coordinate conversion of laser range data and construction of digital surface model. Then, these acquired data are utilized for feature extraction.

Figure 2 Overview of Data Processing
3. Direct Geo-Referencing
The positioning and attitude of sensors are decided by integration of GPS/IMU, as well as CCD images. One of the main purposes of this research is to integrate sensors for developing the high precision positioning system in all circumstances. Integration of GPS, 1Hz, and IMU, 200Hz, has to be made with Kalman filter for geo-referencing of laser range data with the frequency is 18Hz. Positioning accuracy of GPS/IMU is around 10-30 cm, because it is limited by the accuracy of RTK-GPS. On the other hand, positioning and attitude can be estimated very high accurately with bundle block adjustment of CCD images, though the images are taken every 10 seconds.

Therefore, the combination of bundle block adjustment and GPS/IMU by Kalman filter is conducted to achieve higher accuracy. The result of bundle block adjustment assumed to be true position values. GPS/IMU is initialized for Kalman filter using the result of bundle block adjustment. Namely, after every computation of bundle block adjustment, GPS/IMU and its error are corrected. Figure 3 is the strapdown navigation algorithm for integrating GPS/IMU with the result of bundle block adjustment. The combination of GPS/IMU and bundle block adjustment is called hybrid IMU.

Figure 3 Strapdown Navigation Algorithm
As the result, GPS/IMU is corrected with bundle block adjustment, trajectory of hybrid IMU can assure good geo-referencing accuracy for the CCD images. This is one of the advantages of this system.

The point cloud data acquired by laser scanner is geo-referenced by hybrid IMU data. This hybrid IMU data is based on the common world coordinate system, which is the same as the coordinate of CCD images and GPS. Texture data are overlaid on geo-referenced point cloud data. The integrated point cloud data shows a good matching with image data because hybrid IMU data is initialized by the result of bundle block adjustment of sequential image data. Figure 4 shows the concept of construction of DSM (Digital Surface Model). Each point of geo-referenced laser point data corresponds to a pixel of geo-referenced CCD image is corresponded to a point of geo-referenced range data in the same coordinate system in the same coordinate system. In this research, 3D point cloud data takes on a color from the corresponding image pixels for textured digital surface model.

Figure 4 Construction of DSM
4. Feature Extraction
Feature can be detected from both laser range data and image data. The range data can be classified to several groups, such as ground surface, horizontal plane, vertical plane, scatter points, and line feature (Manandar, D., Shibasaki, R., 2002). Usually, scattered points come from natural feature like trees or weeds while man-made features tend to have some geometric patterns like horizontal and vertical aligned points. However some man-made features with very complicated shape such as bicycles or decorated object may have scattered range points. In this point of view, image data are used to complement range data for further details. Figure 5 shows the feature extraction procedure of the acquired data. At first, laser range data are used to roughly extract features. Then, segmentation is conducted using color and edge of images for each features, because image data has higher resolution than range data. Finally, the range data and image data are integrated for more detailed extraction. List extracted feature is shown in Table 2. In this research, only several kinds of main features are extracted, because the test site of this research is limited. This detailed extraction can be applied to monitoring damages of earthquake disaster, volcanic eruption, urban mapping and so on.

Figure 5 Feature Extraction Procedure
Table 2 Extracted Features

5. Conclusion
Laser scanner, digital camera, IMU and GPS are integrated to construct digital surface model. A method of direct geo-referencing by the combination of bundle block adjustment and Kalman filter is developed. By providing the result of bundle block adjustment to Kalman filter, geo-referenced range data and CCD images are geometrically registerd correctly. Feature extraction from both laser range data and CCD images are more effective than feature extraction from image data alone. In this research, all the sensors and equipments are assembled on an unmanned helicopter. This paper focus on how integrate these sensors with mobile platform and there utilization for effective feature extraction.

Reference

Kumagai, H., Kubo, Y., Kihara, M., and Sugimoto, S., 2002. DGPS/INS/VMS Integration for High Accuracy Land-Vehicle Positioning, Journal of the Japan society of Photogrammetry and Remote Sensing, vol.41 no.4 pp.77-84.
Manandhar, D., Shibasaki, R., 2002. Auto-Extraction of Urban Features from Vehicle-Borne Laser Data, ISPRS "GeoSpatial Theory, Processing and Application", Ottawa.
Zhao, H., Shibasaki, R., 2001. High Accurate Positioning and Mapping in Urban Area using Laser Range Scanner, Proceedings of IEEE Intelligent Vehicles Symposium, Tokyo.