GISdevelopment.net ---> AARS ---> ACRS 1992 ---> Digital Image Processing A noise reduction method for portable Lidar Echo data using statistical technique Hiroshi Okumura, Tradashi Sugita, Hironori Matsumoto, Nobuo Takeuchi Remote Sensing and Image Research Center, Chiba University I-22, Yayoi-cho, Inage-ku, Chiba-shi, Chiba 263, Japan Abstract: A new noise reduction method for lieder echo data is proposed. This method is based on canonical correlation analysis between two multivariate data groups. GROUP-1 dataset includes moving averaged data of GROUP-1. As the experimental results using both artificial data and actual lidar echo data, the amplitude of noise component in the primary canonical variety of GROUP-1 (result) is reduced more than 30%. Introduction Lidar (Leaser Radar) is a powerful tool for monitoring air pollution, stratospheric and boundary layer, plume dispersion, visibility, and studying atmospheric structure and cloud physics. In this research field, a portable lidar system will be more widely used in the future because of its easy handling and wielder portability. However, the signal-to-noise (S/N) ratio of each echo datum which is acquired with such portable lidar systems is not so high because of its low lasing power in comparison with large scale lidar systems. Lidar echo data includes not only backscatter signal from scatterer but also various kinds of noise components (e.g. shot noise caused by fluctuation of electric current, dark current noise of detector, thermal noise of a resister in the amplifier etc.). The noise reduction methods which are applied for lidar echo data are as follows. accumulation of lidar echo data on the assumption of noise randomness; moving average method; filtering using fast Fourier transform; hysteresis smoothing method. The METHOD (a) is generally using for noise reduction. However, this method is not included in the actual lidar echo data in most cases. Although the METHOD (b) and (c) are very signal. In contrast with this, the METHOD (d) is not so effective against reduction of strong pulse noises. We developed a new noise reduction method based on statistical technique. NORMALS (Noise Reduction method using Multivariate Analysis technique for Lidar echo Signals), in order to overcome these problems. Canonical correlation analysis is applied for noise reduction by the NOMALS. The NROMALS has the advantage of an effective noise reduction without wave form distortion of backscatter signal. In the following chapters, the details of the NORMALS are described and its validity is confirmed by numerical simulation and also by the application to actual lidar echo data. Principle 1 Canonical Correlation Analysis [1], [2] Canonical correlation analysis (CCA) is one of multivatiate statistical methods. The CCA method converts characteristic variates into uncorrelated integrate variates by the same way as in principal component analysis (PCA). While the PCA is applied to one group of p characteristic Variates, the CCA is applied to two characteristic variate groups. The first group (GROUMND-1) and the second group (GROUP-2) consists of s and t (p= s + t, t s) characteristic variates, respectively. Suppose that two characteristic variate groups, X1..Xs and Xs+i. The mean values of these variates are normalized to 0. Consider following linear compounds; Coefficients in Eq.1 and Eq. 2 Ipi and mqi (I, p=1, …., s, j, q=1,….,t), are determined by way of satisfying following conditions; The mean values and the variance values of ui and vj are equal to 0 and 1, respectively, Ui is uncorrelted with u, (i¹i); Vi is uncorrelated with vj (j¹j); ui uncorrelated with vj (i¹j); correlation coefficients exists between uk and vk (k=1,…., s). Integrated variates ui and vi (i=1…….,t) are called canonical variates, and correlation coefficient rk is called i-th canonical correlation. Fig. 1 shows the relation between characteristic variates and canonical variates, schematically. Fig. 1. Characteristic variates and canonical variates(s=t=3) 2 Processing algorithm of the NORMALS In the case where we apply the CCA to noise reduction of lidar echo data, the method for assignment of characteristic variates is important. Here we denote lidar echo signal which is acquired in normal operation by S1, and assume that S1, can be described as: S1 = S + N1 Where S: backscatter signal from scattere, N1: noise component. Since it is difficult to estimate N1, subtraction operation cannot be applied to noise reduction. Then we use lidar echo single which is acquired in non-lasing operation in place of N1, and denote this non-lasing signal by S2. The S2 can be described as: S2=N2 Where N2: noise component. In the case where S1 and S2 are assigned to the characteristic variates of GROUP-1, the Necessary conditions for the assignment of two characteristic variates of GOUPS-2 can be described as follows. the outline wave form data of S is required as the characteristic variate S1' of GROUND-2 The outline wave form data of N1 is required as the characteristic variate S2' of GROUP-2 In the NORMALS, moving averaged S1data S2 data is assigned of S1, and S2, respectively. The schematic processing flow of the NORMALS is shown in Fig. Fig. 2. Flow of processing 3. Experimant and Discussion 1 Simulation Fig. 3 shows an artificial backscatter datum S (made from probability density function of X2 distribution with six degree of freedom) for the simulation. Fig. 4 (a) and (b) show the results by the NORMALS and by the moving average method (the same data as S1). From these results, we see that the noise component is reduced almost completely by the NORMALS. Fig. 4. An artificial A-scope datum(p.d.f of X2 distrubution, deg. of freedom = 6) Fig. 5. Simulation results 2 Actual Lidar Echo Data Fig. 6 (a) and (b) show actual lidar echo data with lasing (S1) and without lasing (S2), respectively. We used the portable YAG [3], [4] lidar system that we have developed for data acquisition. Fig. 7 (a) and (b) show the results by the NORMALS and by the moving average method (the same data as S1). From these results, while the moving average method distorts both the noise component and the wave form of the backscatter signal, the NROMALS only reduces the noise component without distorting the backscatter signal. As the result of measurements, amplitude of noise component and mean square error against ground level were reduced by the NORMALS 35% AND 62%, respectively. Summary The validity of the proposed method, the NORMALS, was confirmed through comparison of the result by moving average method from the view point of the performance of noise reduction and the wave form distortion. As the results of experiments, the NORMALS has the advantage of an effective noise reduction without wave form distortion of backscatter signal. Improvement of the performance is a subject for a future study. Acknowledgements: The authors are grateful to Dr. Koji Kajiwara, Institute of Industrial Science. University of Tokyo. JAPAN, and Mr. Kithsiri Perera, Remote Sensing & Image Research Centre. Chiba University, JAPAN, for their helpful advices and supports. References Rao, C.R. Linear Statistical Inference and Its Applications (2nd edition), John Wiley & Sons (1973) Morrison, D.F.: Multivariate Statistical Methods (3rd edition,) McGraw-Hill (1990) Takeuchi, N., et al. A Portable Lidar Using Diode-Pumped YAG Laser, Proc, of 16th International Laser Radar Conference, 695/698 (1992) Okumura, H. et al.: A High Speed Singal Processing System for a Diode-Pmped YAG Lidar, Proc. Of 16th International Laser Radar Co Fig. 6. Actual lidar data Fig. 7. Processing results.