Processor Evolution for increasing image processing amount

Processor Evolution for increasing image processing amount

Shin-ichi Hanaki, Tsutomu Temma, Kenji Naito
C & C Systems Research Laboratories,Nippon Electric Company, Ltd.
4-1-1 Miyazaki, Takatsu-ku, Kawasaki-city, 213 Japan

Introduction
In the image processing of remotely sensed data, digital image processing is gaining more and more importance. Ditital image processing is advantageous in processing flexibility and stability.

Although general purpose digital computers are improving, they are not sufficiently capable of handling a large amount of image data. Several efforts have been made in regard of image data. Several efforts have been made in regard to both image processing speed up and image processing cost reduction. As a result, stand-alone image processing systems have been developed.

So far, typical remotely sensed data have been those from LANDSAT, for which various kinds of processing techniques and applications have been developed (2)

With the digital processing capability progress, remotely sensed data are expanding both their data amount and variations. For example, digital data processing for Synthetic Aperture Radar (SAR) has appeared as a new digital data processing objective. (3) (4) SAR digital data processing features a huge data processing amount requirement, which has only been possible by using an optical processor. Recently, however, this processing is becoming within the scope of digital processing and several SAR digital data processing demonstrations have been made (3)

This paper first reviews a typical LANDSAT data processing amount per scene, then it discusses SAR digital data processing amount. After briefly analyzing system throughput and processing time, the paper discusses a dataflow image processor TIP and its possible future expansion for handling a large amount of image data.

Digital Image Processing

1 LANDSAT image processing system development
Several LANDSAT image processing and analysis systems are now commercially available. A software library for LANDSAT image processing is also available. e.g. RSIPS-6 on NEC/ACOS computers. In order to speed up processing and reduce image processing cost, hardware oriented stand-along systems have been developed. A block diagram of a stand-along image processing system is shown in Fig. 1. The System “T” configuration, in which image data are store in the image memory (2Mbyte MOSIC memory) and are processed by special hardware image processors. The host mini-computer controls each system component, handles man-machine-communication control and manages data. An image memory and image processor combination handles image data under the host computer control. The computer avoids processing image data either in its own memory or in its arithmetic-logic-unit, so that the entire system should maintain a high efficiency for image data processing. The processed results obtained in the image memory can be displayed on a monitor in order to aid human comprehension of image data and judgment on processing effectiveness. In display mode, the image memory is used as a display refresh memory.

Fig 1 A stand-alone image processing system blockdiagram.

2. LANDSAT data analysis examples
As an example LANDSAT image data will be compared with Japanes national census data. (5)

First the LANDSAT image data are geometrically corrected and the data are resampled to match the census mesh size i.e. 1 x 1 km. The census data, which includes a pair list of both mesh location and population at that location, are rearranged in the LANDSAT PIXEL order. Then it is formed into a complete mesh data series by inserting dummy data into those meshes which lack original population data. A picture element is usually called “pixel”.

Cross-correlation is calculated between the resampled LANDSAT data and rearranged census data.

This data processing amount is estimaged to be approximately 10**9 operations for a typical Landsat scene of 7. 5 x10**6 pixels. Here, the notation 10**6 represents the sixth power of 10.

Operation corresponds to machine instruction on a conventional general purpose mini-computer. It is hard to strictly define operation without specifying a specific machine.

3. SAR digital data processing
SAR (Synthetic Aperture Radar) is a microwave imaging radar system consisting of both an RF-section and a processing section. The RF-section includes antenna, transmitter and receiver on a platform, such as a satellite or airplane. The processing section reconstructs the earth surface image from received radar echo signals.

This discussion will be confined mainly to satellite SAR discussion. A total SAR system block diagram is shown in Fig. 2. The SAR transmitter sends microwave pulse signals to the earth surface from the satellite antenna in the side direction (cross-track) in a narrow radar beam. The SAR receiver receivers its echo signal on the satellite. The obtained echo signals are sent back to a ground station, recorded and processed by processors or correlators to reconstruct the earth surface microwave imagery.

Fig 2 Total SAR system blockdiagram.
NASA employed the Sar technique for SEASAT sensor equipment and launched it into orbit in 1978.

With the SEASAT altitude (800 km), even if the antenna beam is narrowed to as small as 1 degree, the real footprint on the earth surface spreads to about 15km in diameter. The resolution in the range direction is determined by pulse resolvability, for which the chirp radar technique is employed, based on linear frequency modulation.

In order to improve azimuth direction (along-track) resolution, the Doppler radar technique is utilized. It makes use of Doppler frequency shifts caused by range variation with time between satellite and individual objects on the earth’s surface. A Doppler inverse filter is generated, which is applied to the radar echo signal to improve the resolution in the azimuth direction.

Another explanation can be made for the azimuthal data processing. Radar is able to achieve a finer resolution with a longer antenna if both range and frequency stay the same. Therefore, a satellite shold employ as long antenna as possible. It is difficult, however, to carry a long antenna in an actual case. Therefore, the SAR technique has been adopted which employs a rather short antenna and sends back repeated echo signals to ground stations during the satellite flight.

In order to accomplish image reconstruction, several echo signals are summed with adjustment based on the range between antenna and individual objects, which appear as effectively signal phase variations on each each signal. The process effectively synthesize a longer imaginary antenna, which achieves finer resolution than the actual antenna. This is the reason why it is called “Synthetic Aperture Radar”.

The different appearance between the Doppler radar and antenna synthesis is due to the difference in coordinate systems used for describing equations. This difference was pointed out by W. J. Van de Lindt. (6)

4 SAR digital data processing amount
SAR image reconstruction process from the raw SAR echo signals is shown in Fig. 3. Received echo signal data were supplied from one of SEASAT receiving stations in digital computer compatible tapes (CCTs)

Fig. 3 Digital processing flow diagram.
First, range compression is accomplished for a chirped radar signal by chirp matched filter. At this stage, data are arranged in range lines, which should be rearranged into azimuth lines for the latter process. This data rearrangement process is called corner turning. Then, azimuth compression is achieved with an azimth filter generated separately. At this stage, range curvature compensation should be applied, too. Processed results are formed into an image. An example of SEASAT SAR image is shown in Fig. 4, which was reconstructed from SEASAT raw data by NEC, with software developed by NEC on a large ACOS computer. The scene includes a part of Lake Leman and Geneva, Switzerland.

Fig. 4 Reconstructed SEASAT SAR image.
Compressions are carried out by the Fast Fourier Transfor technique, which requires a large amount of floating addition and multiplication operations.

The data processing amount for generating a 100 x 100 km image (4000 x 4000 pixels) is estimated to be about 4 x 10**10 floating operations, which is equivalent to 2.5 5 x10 **2 floating operations per output pixel.

5. Total processing time
As discussed in the preceeding sections, the following processing amounts are required per scene.

1. SEASAT SAR image reconstruction requires an order of 4x10** floating operations. On the other hand, recent computer progress assures processing abilities of

0. 5MIPS With a mini-computer
5-50MIPS With an add-on arrary processor
80-15 MIPS With a super computer.

Here, MIPS stands for million instructions per second. It is by no means easy to make correspondence between MIPS unit and MFLPS (million floating operations per second) unit in amounts to ten instructin steps on a conventional mini computer without floating operation hardware.

Although these super computers have great processing capabilities, they are too expensive. Compared with the stand-along systems described in Section 2.1.

Figure 5 shows the relation between system throughput and processing time required for several data processing amount parameters.

Fig. 5 Relation between throughput and pricessing time.
It we consider SAR digital processing, processor cost should be comparable to the present SAR optical processor cost. With this condition, it is hard to realize a realtime SAR digital processor by the present technology. Hence, compromise will be required between processing speed and processor cost. At the moment, a 500 MFLOPS SAR processor will be reasonable challeng target in 1987. Such a SAR digital processor will be useful for Landsat data processing, too.

Template controlled Image Processor
As an image processor example, a template controlled image processor, called TIP, is explained. It is designed for efficiently processing arry data onsisting of a series of numbers, each of which requires the same operation.

A TIP blockdiagram is illustrated in Fig. 6. The TIP architecture is a result of looking for a flexible pipeline structure. The pipeline technique is used for speeding up an array data processing. Traditional pipeline technique, however, lacks flexiblility, in the sense that one pipeline cannot be directly applicable to other processing categories.

Fig. 6 TIP blockdiagram.
TIP consists of one small main ring and two major rings, an operation ring (OP-ring) and an address ring (AD-ring). Each ring consists of ring bus and several modules connected to the ring bus. Prior to its processing execution, each module is set with an instruction sequence called templeate, which is a part of the total TIP program. Templates are sent from a acontrol unit to modules under the host computer control.

Data are read out from a memory connected to the read module in the AD-ring, which go through the AD-ring to the main ring and then to the op-ring. The op-ring has several modules, as shown in Fig. 6. Each datum on the bus is accompanied by a tag which indicates an operation to which the data shold be applied. The tag is initially set when datum passes through the generator module. Each OP-ring module examines tags shifted synchronously with the clock by its own template to find data that should be processed in that modul and to attach a new tag to the resultant data which are merged into the OP-ring bus data stream.

Data on the ring bus receive necessary operations one by one in this manner and, after circulating the OP-ring to go back through the main ring and the AD-ring to memory from the AD-ring write module.

The AD-ring can generate a constant value data stream, as well as address data, which are used for data read out from memory.

From another viewpoint, the TIP can be regarded as a dataflow processor. With the TIP architectuyre presented in reference (7), hardware is under development, which is expected to have a processing speed of 16 MFLOPS for Fast fourier Transform application.

A Faster Processor
From Fig. 5, it is assumed that a single TIP may require at least 0.7 hours for one 100 x100 km SAR image reconstruction. In order to shorten the SAR image reconstruction processing time, another speeding up technique, i.e. parallel processing time, another speeding up technique, ie.e parallel processing technique, should be employed. One idea is to stack TIPs. Each TIP unit is applied using divided data to achieve a total high throughput by stacked TIP paralled operation. Of course, it should be considered that processin capability, bus capability and memory access neck should be balanced.

In order to realize the stacked paralled TIP, such problems shuld be considered as total equipment cost, implementation, dimension, power consumpution and retaining reliabilities for a complex system. One solution for this is to develop an LSI processor based on TIP and then build a stacked TIP. For such a stacked TIP, the flow diagram in Fig. 3 can be modified. Range line or azimuth line data are divided into N groups, which are processed simultaneously by N independent TIP units to reduce these compression times roughly to 1/N. N represents an integer. The time reduction greatly affects the total SAR image reconstruction time reduction, because compression time mostly governs the total SAR image reconstruction time.

Conclusion
Recently, digital image processing systems are capable of handling popular LANDSAT image data fairly easily. This handling REQUIRES ABOUT 10**9 OPERATIONS FOR A 7.5 x 10**6 pixel image. A new synthetic aperture radar image reconstruction requires about 4x10**10 floating operations for a 1.6x10**7 pixel image.

Although very fast super computers are now available, whose speed reaches up to 150MIPS, they are too expensive from rremotely sensed data processing average. A 500MFLOPS SAR digital processor, whose cost is comparable to the present SAR optical processor cost, will be a reasonable challenge target in 1987.

As an image processor example, a template controlled image processor TIP is explained, with focuses on pipelining technique for speeding up, based on data driven philosophy.

In order to obtain greater processing capabilities for process speeding up, stacking TIP units and a SAR image reconstruction flow diagram for this parallel processing are discussed.

A single TIP is now under construction. Developing a stacked TIP is a future problem.

The authors would like to thank Mr. M. Ogiwara for giving them a chance to prepare this paper. They also thank their colleagues at the C & C systems Research Laboratories.

References

S. Hanaki, An Interactive Image Processing and Analysis System, U.S. Japan Seminar, Tokyo, Japan Oct. 1978.
Proc. Of 13th International Symposium on Remote Sensing of Environment, 1979 ERIM Ann Arbor, Mich.
I. G. Cumming; Digital Processing of Seasat SAR data, 1979 International Conference on Acoustics, Speech and Signal Processing, Washington, D. C. 1979.
H. Nohmit, Synthetic Aperture Radar Digital Data Processing (in Japanes ) 8-3 computer Vision workshop, info. Proc.Soc. Japan, Sept. 1980. Soc. Japan, Sept. 1980.
K. Naito; Correlation Studies between LANDSAT MASS Data and population Density in Japan, 14th Congress, International Socieity of Photogrammetry, Hamburg, Frg, July 1980.
W. J. Van de Lindt; Digital Technique for Generatin synthetic Aperture Radar Image, IBM J. Res. Develop. P. 415, Sept. 1977.
T. Temma; Dataflow Processor for Image Processing, 11th International Symposium Mini and Microcomputers, Monterey, calif. Jan. 1980.